Assessment-led reform: Creating a sustainable culture for WIL

Introduction

This study continues on from a previous mixed-method study investigating Science, Engineering, Technology and Mathematics (STEM) teachers’ experiences in working with others to undertake authentic assessment-led reform. The previous study found that further exploration of the purpose, the criteria and the collaborative mechanisms for enabling academic teams to work with industry to improve authentic, work-related assessments in STEM was needed (Hains-Wesson, Pollard, Kaider, & Young, 2019). This current study responds to this need through an action-research project aimed at on-campus Work Integrated Learning (WIL) reform via an assessment-led approach.

The focus of this study is directed at authentic assessments, designed through a WIL-lens, that have the intention of enhancing employability learning for students. Employability here is understood and agreed to be a set of achievements – skills, understandings and personal attributes – that makes graduates more likely to gain employment and be successful in their chosen occupations, which benefits themselves, the workforce, the community and the economy (Yorke, 2006, p. 8). Employability is about enabling student learning so that they can discern, acquire, adapt and continually enhance the skills, understandings and personal attributes that make them more likely to find and create meaningful paid and unpaid work (Oliver, 2015, p. 59). As such, the study investigates the means by which assessment design could be revised to include an awareness of graduate capabilities important for science graduates, the enhancement of employability learning, and the inclusion of industry perspectives and current practices. Central to the study is the argument that science faculties, like all university disciplines today, need to encourage teaching teams to consider employability assessment in their assessment design (Jorre de St Jorre & Oliver, 2018).

This study hypotheses that, to make a positive impact on employability learning and outcomes, intentionally scaffolded, whole-of-degree WIL-oriented assessment is necessary. The findings gleaned from previous related studies (Young, Palmer, & Campbell, 2017; HainsWesson et al., 2019) indicate that transformation of curricula, to incorporate embedded and scaffolded learning associated with enhanced employability skills, is likely to be challenging both in philosophy and practice. Science academics, in particular, are being encouraged to move beyond delivering discipline specific knowledge and skills and employ scaffolded applications of discipline content combined with the performance of generic capabilities nested in real-world contexts, across an entire program (Lasen, Evans, Tsey, Campbell, & Kinchin, 2018). For some years now, the Office of the Chief Scientist (2013) has called for a rethinking of the way in which knowledge and content-heavy material is delivered. An improved, measurable balance between content knowledge transmission and acquisition and work relevant skills and capabilities (Rayner & Papakonstantinou, 2015), as well as the design of whole-of-degree curricula to include varied WIL experiences (Collis, 2010; Greenbank, 2002) is a challenge currently facing many institutions.

Critically however, building the capacity of academics to design and deliver WIL assessments that are most likely to positively enhance the graduate employability of undergraduate cohorts is required to measure this desired impact. Previous projects (Kaider, Hains-Wesson, & Young, 2017) support that an expansive WIL model (not just placement-based WIL) is necessary. In order to create a sustainable program of employability learning, a mix of on-and off-campus WIL is required. The study demonstrates that teaching teams require understanding of a depth and breadth of WIL that allows them to collaborate with industry to design and deliver WIL assessments in meaningful ways. 

Literature review

Traditionally, WIL has been strongly linked with off-campus work placements (Peach & Gamble, 2011) but now it commonly includes workplace simulations and project work for industry clients (Smith, 2016) in hybrid and on-campus locations (Rowe, Winchester-Seeto, & Mackaway, 2012). In some instances, these WIL experiences provide the benchmarks and standards necessary for allowing graduates to practice professionally (Connor & MacFarlane, 2007). This study contributes to the emerging evidence that for the purpose of scale and sustainability, on-campus/classroom-based WIL experiences provide viable alternatives to placement-based WIL (Sachs, Rowe, & Wilson, 2016).

To be most effective, researchers have posited that WIL should be embedded and scaffolded across the curriculum using appropriate pedagogical strategies (Daniel & Shircore, 2012; Rowe & Zegwaard, 2017). For example, a program could commence with personal, professional and skills development to support WIL, incorporate a range of WIL activities of increasing sophistication and authenticity in the middle years, and culminate in high-level WIL activities in the workplace. Additional placement-based WIL alternatives, such as a simulated placement or an industry-supplied project topic completed on-campus (Peach & Gamble, 2011; Rae, 2007), or micro-placements, hackathons and consultancy services (Kay, Ferns, Russell, & Smith, 2018), to name a few alternatives, should be considered from a whole of program approach to employability learning. However, there are few studies that measure the impact of this whole-of-course approach on graduate and employment outcomes, on course team’s perceptions of the value of WIL, and the likely means for which WIL can be operationalised across a program. 

Programs that provide a spectrum of on-and-off-campus WIL experiences, founded upon purposefully designed curriculum that explicitly integrates theory with work practices (Patrick, Peach, Pocknee, Webb, Fletcher, & Pretto, 2008), are still evolving. In STEM, field-based learning scenarios are common, with a recent rise in placement-based WIL experiences. However, broader WIL approaches (such as applications of classroom-based/on-campus interactions with the people and practices of work) warrant exploration (Young et al., 2017).

Aside from its contribution to theory-practice curricula, WIL has been found to be unique in the way it can contribute positively to graduate employability (Sachs et al., 2016), to students’ job readiness skills (Ferns, 2014) and to applying industry standards (Edwards, Perkins, Pearce, & Hong, 2015). There has also been a recent push to consider employability not as a product (outcome) but as a process, that needs to be taught and learnt. Subsequently, employability learning curriculum frames are needed (Smith, Bell, Bennett, & McAlpine, 2018) as are studies relating to the graduates’ outcomes as a result of employability curriculum initiatives. 

Many universities now include some form of WIL experience in their curricula as a response to enhancing graduate employability (Rowe & Zegwaard, 2017; Sachs et al., 2016). The literature points to universities recognising WIL and its link to employability as a key strategic concern (Senadji, Boman, Allbutt, & Whelan, 2015; Oliver, 2015; Jorre de St Jorre & Oliver, 2018), especially at a time when the dramatic expansion of participation in higher education has led to concerns in the sector about graduate employability (Rayner & Papakonstantinou, 2015). For STEM, greater numbers of graduates, changing economic conditions and employment market characteristics have also seen increasing concern about employment prospects for graduates (Norton & Cakitaki, 2016). For Australian science bachelor graduates, prospects for employment, and particularly for employment in a science role, are a concern (Palmer, Campbell, Johnson, & West, 2018). 

Embedding employability learning is not always straightforward. Academic staff responsible for core units of discipline-specific content may be reluctant, for varied reasons, to teach and assess employability learning. One common view in the sciences is that it is seen as displacing or reducing the primacy of theoretical content knowledge (Abeysekera, 2006); others express a lack of confidence in their ability to competently design and implement employability assessments (Kaider et al., 2017).

Many academic staff, finding themselves with responsibility for WIL, need support and professional development to be able to operate effectively in such a role (Desha & Senadji, 2014; Venables & Tan, 2009; Young, et.al., 2017). Likewise, many employer organisations that are new to offering WIL placements and projects, will need information and support to understand the key academic purposes and processes of WIL, and to be realistic in their expectations of student performance and placement outcomes (Buckley & El Amoud, 2011; Peach & Gamble, 2011). There is growing evidence that students want their degree to be strongly linked to ‘industry-readiness’ and for their program to provide career development support. Additionally, students would welcome greater involvement of industry/community in the design of their employability learning (Jorre de St Jorre & Oliver, 2018). While students will need support in approaching WIL and employability development – as Collis (2010) notes, scaffolding will need to be put into place in order to initiate and support all three stakeholder groups involved – students, academic supervisors, and industry partners (p. 8). The findings from Jorre de St Jorre & Oliver (2018) speak to this point: student attitudes towards the involvement of employers, professionals and recent graduates in the design, delivery and assessment of learning suggest that explicit and prominent involvement of these stakeholders will help to make learning and assessment of graduate employability meaningful (p. 55). 

The central theme of this project was not about specific types of WIL assessment. Rather, it was about understanding the processes required to productively engage stakeholders in enhancing assessment for WIL. However, in all aspects of student learning, including WIL and employability studies, assessment is a key driver of student engagement, and an important tool for measuring student achievement. In terms of WIL assessment, authenticity refers to the degree to which an assessment resembles relevant professional practice (Kaider et al., 2017, p. 158). A useful definition of authentic assessment comes from Gulikers, Bastiaens and Kirschner (2004): an assessment requiring students to use the same competencies, or combinations of knowledge, skills, and attitudes, that they need to apply in the criterion situation in professional life (p. 69). Proximity is also a vital consideration in assessment design – it refers to the degree to which an activity and its assessment occur in the workplace or with practitioners (Kaider et al., 2017). The case study by Jorre de St Jorre & Oliver (2018) highlights the way in which assessment should be used to ensure students are engaging with their employability learning.

Research questions

This study aims to add to the existing knowledge on assessment design in WIL through the following research questions:  

• What contribution can re-framing traditional assessment profiles to include industry oriented and authentic work-related constructs make to the provision of, and evidence of, employability learning? (R1)  
• What contribution can a whole-of-program curriculum-led approach, based on scaffolded WIL-assessment profiles, make to improved graduate outcomes for scientists? (R2)  
• What is likely to be an effective enabler for STEM academics, particularly those that are novice in WIL curriculum development, to take ownership of the re/design and delivery of industry-oriented assessments? (R3) 

The central purpose of the action-research project was to enliven a process for investigating whether industry-oriented assessments were perceived as important for students, industry and academics. Following this, the project needed to investigate if there were indications of importance as translated into the intended and/or actual curriculum. Finally, the project was used to ascertain what provisions would enable enhancements to course-wide approaches to WIL assessment design. 

Research method

Rationale 

One of the Faculty’s strategies was to create sustainable and scalable embedded WIL approaches to all undergraduate degree programs by 2020. Of particular interest was the way in which formative assessments could be designed to include industry perspectives and current practices, with the possibility of highlighting the industry presence on-campus or in simulated ways. This spoke to the overarching aim of the project which was to expand the capacity of teaching teams to strategically involve industry in integrated learning approaches. The driver was to encourage academics to think beyond delivering discipline specific information and skills, to the application of these skills combined with generic capabilities for real-world contexts (Lasen et al., 2018, p. 2).

An initiative was introduced in 2015 to ensure that all undergraduate degree programs had space in which students could undertake a WIL placement. To facilitate this the Faculty introduced a core foundational career education unit to prepare all undergraduates for placement. In 2017, 1200 students within the Faculty undertook a placement-based WIL activity. Moving beyond placement-based WIL, the Faculty aims to use class-based/on campus interactions with industry/community for enabling a sustainable and scalable WIL approach to curriculum that positively impacts the employability of graduates. The Faculty goal was for degrees to provide more than just a penultimate placement activity, but rather, purposefully designed scaffolded authentic WIL assessments that prepare students for placement, as well as alternative placement-based activities (Kay et al., 2018). 

Approach

An action-research project process of plan, act, observe, reflect, plan (and repeat) (Kemmis & McTaggart, 1988) was used to frame the project research. The approach was based on two rounds of workshops with academic staff in two Bachelor of Science degree programs, in parallel with the collection of additional rich and diverse data from the wider stakeholder groups (Berg, 2004). Ethics approval was granted for the mixed-method approach which targeted three key stakeholder groups: academic staff teaching into both degree programs, students undertaking the same degrees, and industry representatives affiliated with the programs and familiar to the Degree Directors. 

The ‘plan’ of action was developed with the intention of sharpening Faculty-wide understanding of current intentional WIL assessment activities and to enhance academic awareness of how to improve on-campus WIL. The focus was on developing relevant and contextualised authentic assessments for improving employability outcomes for graduates. The ‘act’ part of the methodology encompassed the collegial nature of the workshops. All workshop participants were encouraged to contribute equally to discussion and activities during the day. The aim of the workshops was to build the capacity of discipline experts to reimagine their own assessments through a WIL-lens. The workshops involved a flipped classroom approach to ensure all participants were informed equally about the content, to fasttrack discussions, and develop collaborative active learning activities throughout each workshop. 

The ‘observe’ phase involved multiple members of the project team taking notes about what was occurring during the workshop, so that layered perspectives could be gathered and considered. These qualitative findings were recounted during the group ‘reflect’ phase which occurred at the conclusion of both workshops. In-situ, on-action reflections formed the ‘plan (and repeat)’ phase of the action-research process. The qualitative findings from the workshops fed into the self-study. Here the action-research methodology was used to provide a flexible and buoyant means for gaining an evolving and improved understanding of the academic perceptions of WIL assessment. It provided a critical means for investigating the perceptions, interest in, and barriers to, enhancing on-campus assessment activities for students and industry representatives from the two science degrees. 

The total research data collection inventory included archival, survey, interview and observational data (Berg, 2004), and comprised the following elements:  

• A curriculum mapping audit of both degree programs.
• An online survey of students enrolled in both degree programs. 
• An online survey of teaching staff involved in both degree programs.  
• In-person interviews with key industry representatives with a relationship to both degree programs.
• Two teaching team focus groups and two teaching team workshops. 

Additional details for each methodological element follow.

Curriculum mapping 

A desk-top audit mapped all final-year units from the two degrees. There were four mapping exercises undertaken to determine the type, level, breadth and depth of authentic WIL assessments in each unit. The handbook entry for each program was reviewed via a ‘word matching’ process. Keywords were used in the first audit round to determine if the listed assessments were recognisable as WIL assessments. This process involved both an inclusion and exclusion word-matching frame. Assessments that would be omitted included traditional assessments typically found in science curricula, including exams, tests, essays and scientific reports. Keywords used for the first level positive matching included the type of experience (i.e. placement, professional practice, project, internship, fieldwork and case-study), descriptive words explaining the potential engagement with industry (i.e. industry, partner, community, professional, real, real-world), and the learning design (i.e. application, applied, relevant, context, career and reflection/reflective-practice). If the process did not provide direct matches, an appraisal of the unit guide was undertaken to gather more detailed specifics about the assessment type. Finally, the unit outline and assessment guides were analysed for keywords that might indicate if the focus of assessment was proximal or authentic to the workplace, using definitions as proposed by Kaider et al. (2017). 

Mapping prompted the ‘auditor’ to use a new set of criteria to unpack the authentic nature of the assessment. This involved a three dimensional framework: ‘Product, Process and Performance’ developed specifically for this project. The details of how this three dimensional frame was constructed, as well as the subsequent development of a fit-for purpose Work Integrated Learning on-campus Assessment Framework (WAF) will be published in a forthcoming paper (Young, Miller, Campbell, & Palmer, 2019). Relevant to this paper, there were key questions used by the auditor to guide the mapping process. The first question was intended to help the auditor decide on the fidelity of the assessment to real-world tasks: Does the assessment submitted look similar to a common work activity that a graduate would need to produce in a workplace /professional context? The second question was intended to help provide clarity on the resemblance concerning the process of completing the assessment: Does the assessment provide an opportunity for the professional practices of discipline-specific knowledge and skills to be applied and performed in real-world work-contexts? The third question was intended to ascertain if industry were involved in assessing the standards: Is there evidence of a consultation process between academia and industry resulting in a set of standards to judge if the assessment demonstrates the capacity of students to perform to real-world requirements? 

Online survey of students 

The student survey was designed to seek an understanding of the perceptions of undergraduate students in the two degrees towards the presence and value of non-placement WIL activities in their studies. Students enrolled in both degrees were invited to complete an online survey, via an email invitation. Of the 305 students invited to complete the online survey (Degree A n = 92; Degree B n = 213), 47 (15.4%) students responded (Degree A r = 11; Degree B r= 36). The survey was comprised of 14 questions, including Likert scale items and free text response items. The survey data were analysed using SPSS and NVivo to characterise the perceptions of respondents about the presence and value of non-placement WIL activities in their studies. 

Online survey of staff 

Fourteen academics across two academic teaching teams were invited to complete an online pre-project survey that sought their perceptions of the value of non-placement WIL activities in their programs of study. The survey consisted of 16 questions, with response scale items that produced quantitative data and free text response items for producing qualitative data. Due to the small number of staff surveyed, to retain anonymity they were not asked to identify the degree they taught in. Of the 14 staff invited to complete the survey, 10 submitted a response. The survey data were analysed using SPSS and NVivo to characterise the perceptions of the presence and value of non-placement WIL activities in the degrees managed by respondents. 

Focus groups with academic staff 

Two focus groups took place, one at the commencement of the project and one at the completion of the project. The aim of the focus groups was to gain a deeper understanding of the academic perspective of the role of WIL within the Faculty, and how changes could be made to facilitate more WIL opportunities for students. In the first focus group, six academics participated. The focus group employed five stimulus prompts relating to:

• WIL assessment;
• WIL curriculum and the scaffolding of WIL curriculum;
• Embedded WIL;
• Industry engagement and involvement; and
• Understanding our students.

Eight academics participated in the second focus group. The focus group employed three stimulus prompts derived from reviewing the first focus group questions relating to:

• Authentic WIL assessment;
• Signposting the purpose of WIL assessment; and
• Benefits of industry engagement and involvement in on-campus WIL activities and assessments.

Interviews with industry representatives 

In-depth, 45-60 minute individual interviews with eight industry nominees were conducted by the Project Lead for Industry Partnerships. The aim was to gain a better understanding of industry’s current perceptions and their potential ‘appetite’ for engaging in future WIL oncampus plans. The eight questions posed included provision for industry interviewees to provide their insights and experience relating to the work-readiness skills of new graduates entering the workforce. Each interviewee received the questions in advance of the interview to provide context and the opportunity for preparation. The interviews were audio recorded, and transcriptions of the recording were analysed using a combination of thematic content and narrative analysis. 

Workshops 

Two full-day workshops (involving the teaching and project teams) were held during phase one and three of the project. The overarching aim for these was to discuss and introduce possible revisions to assessments to provide stronger signposting of work-context authentic assessments. Discussions in the first workshop were pitched at an introductory level to allow for the development of a shared understanding of WIL assessment as being more than placement-based WIL. For the second workshop, a third teaching team requested to participate. This indicated that the aim of guiding (influencing) and growing the understanding (building professional capacity) of non-placement WIL assessment was gaining momentum (Young et al., 2017). 

Findings 

The aim of the action-research project was to investigate academic, student and industry perceptions of WIL assessment profiles and then influence how academics might change their practices around designing and delivering WIL assessment profiles. Critical here was the role of the academic group as a key participant in the action-research change process – in particular, they were privy to the survey and interview findings. The slippages in intentional assessment design were identified during the mapping outcomes presentation as a means to prompt discussions relating to unit data, as well as student and industry perceptions of WIL, so that during the workshops, the evidence-based approach could be used to tease out potential reasons for cultural change. This aim of cultural change was seen as being ‘in motion’ during the lifecycle of the action-research project, especially as the first focus group indicated that there was still a gap in understanding the definition of WIL. The findings are presented as three themes as they relate to the research questions (R1, R2, R3). The themes are: Reframing assessment profiles relates (R1); Whole-of-program approach (R2); and Enablers of assessment reform (R3). 

Re-framing assessment profiles (R1) 

As with a similar study by Kaider et al. (2017), use of a mapping process as a formal methodological approach was instrumental in providing an evidence-base for discussing the nuances, levels, types and purposes for authentic WIL assessments. The desktop audit and curriculum mapping revealed that the tactic of finding WIL assessments by matching words from the current unit documentation to the authentic WIL assessment criteria and descriptors meant that the documented curricula of both courses did not meet the 2020 Faculty target for authentic assessment. However, this did not necessarily mean that the assessments were not authentic.

The first step involved an audit of the assessments documented in the handbook entry. The second level of auditing involved a review of each unit guide. The final step involved reviewing the unit outline and assessment guides. Matching current documentation to the mapping criteria required subjective interpretations from the reviewer as to whether the unit could be classified as containing WIL approaches to assessment. Of note was the consistent lack of evidence in the assessment documentation that industry input for co-created curricula had been sought. In the first workshop, the participants discussed their individual and collective partnering strategies with industry to design assessments. Discussions of the data analysis during the workshops proved to be the most valuable aspect of this mapping process.

By deliberately linking discussions about changes to process and practice with the mapping outcomes the academics became convinced that, in most instances, the documentation of their curriculum design did not adequately represent the type, level, breadth and depth of authenticity of the assessment tasks. This concurs with a previous study that found mapping can enable shared understandings and lead to increased collegiality and collaboration for assessment change (Kaider, et al., 2017).

The conclusion from the audit was that assessment design through a WIL-lens was at best, ‘hidden’. More importantly though, academics indicated that they were open to receiving assistance to improve their processes for connecting with industry to enhance co-created curricula, that they took accountability for improving their assessment descriptions and that that were beginning to see value in WIL approaches to STEM curriculum design.

The observations of levels of staff participation from the first to the second workshop and the variations in language used from focus group one to two, indicated that a shift to thinking that WIL approaches to assessment may be feasible was becoming evident. An early focus group response revealed a possible limitation in academic perceptions of the ability for WIL approaches to simultaneously occur alongside traditional assessments: …I suspect we don’t see them as WIL opportunities. This is supported by data obtained from the academic staff survey. Academic staff were asked to rank: discipline knowledge and skills; real-world applications in context; employability skills and reflections of the development of those skills; and demonstration of professional performance in practice, in terms of their importance to a whole-of-program approach to WIL. Employability/reflective practice and professional practices were perceived as important – in the range of ‘Essential’ – with an average score of 4.2. Discipline knowledge and skills were seen to be of greater importance (with a mean importance of 4.7/5). The ranked orders within these sections indicate a bias toward discipline knowledge and skills, as opposed to employability and reflective practice. Analyses of the transcribed discussions from the first staff focus group revealed that academic staff were still forming their ideas about WIL in curricula. The phrase ‘I think that…’ was commonly used. Analyses from the second staff focus group revealed that the phrase ‘I think that…’ was used with relatively less frequency and there were positive affirmations relating to the importance of deliberate program planning related to WIL. There were granular discussions around the different needs/characteristics of different degrees in relation to WIL. For example, the environmental science/management degree discussed the possibilities for clearer applied applications with ‘industry’, with specific reference to waste management, as being important when designing and improving curricula. This revealed that a connection to WIL approaches in the lived curricula had been made.

The observed positive engagement by academics in workshops also suggested a positive shift in the shared value of WIL. The majority of the time in workshop one was focused on discussing the purpose of WIL approaches to STEM curricula, the already ‘fairly’ authentic curricula and the student experience and expectation of assessment. An academic noted that students prefer to ‘know the [right] answer’, rather than work with uncertainty. There were useful discussions about core scientific skills and techniques. For example, the difference between traditional lab work where students follow steps and get a result, and open-ended investigations where students have to make a hypothesis, design an experiment, collect data, analyse it, and make inferences from the results. This is noteworthy as WIL is often understood as a vehicle to apply learning. However, the other layers of WIL approaches, such as industry collaboration for co-created curricula; contextualised concepts and practices; integrative learning; and reflective practices for employability learning) were not found. Interestingly, specific discussions on WIL approaches that involve industry collaborations, employability skills in context, and reflection were more robustly discussed in workshop two, and therefore appeared to be better understood by most participants by the end of the project.

This was evidenced during a practical exercise where academics were asked to judge from a random set of six assessment types where they believed the assessment sat on the Traditional versus Authentic assessment spectrum. They were then asked to design one of the assessments to align with an enhanced authentic assessment. This task was similar to a practical exercise that the same group participated in during workshop one. Of note, was that the interest, enthusiasm and ‘buy-in’ from staff was more evident in workshop two, thus suggesting that academics appeared to open to the prospects of WIL approaches to assessment design.

A powerful action, geared to re-frame assessments, came from a presentation led by one of the Degree Directors. Their argument was that despite the team thinking their program content was industry-oriented, excerpts from the handbook, unit and assessment documents revealed that the curriculum documentation did not convey the authenticity of the assessment and the depth and breadth of real-world contexts that were being applied to discipline-specific concepts. The Director’s message was clear, small changes with potential program-wide significance were needed.

The mapping process and the workshops helped to identify where the slippages were occurring. Furthermore, observations of participant behaviour and activity at the second workshop suggested that there are influencers and early adopters as part of both course teams likely to continue to re-shape the framing of assessment to be inclusive of a WIL frame. 

Whole-of-program approach (R2) 

The mapping process was also a useful evidence-based approach for showing academics the hotspots of WIL activity across their program. The academic staff survey identified that discussions relating to the scaffolding and embedding of WIL was in most cases addressed mainly in annual program reviews (n = 4, 40%). The course map therefore provided another layer. The academic (as the agent at the coal-face for enabling assessment change) was seen as pivotal for improved whole-of-program approaches for embedded and scaffolded WIL. One academic provided a comment in response to the importance of identifying WIL opportunities: i.e. map them and make them clear for staff and students. The academic staff survey identified that mapping employability skills was essential for the success of the students and the degree (n = 5, 50%).

The data relating to the student and industry perspectives on embedded and scaffolded WIL were hoped to be enablers of attitudinal change, deemed necessary because of a barrier to improved practice expressed in the first staff survey. Academics were encouraged to consider their student cohort views from a whole-of-program perspective. The student survey findings, presented to the academic group, revealed that students perceive a greater need for WIL education throughout their degrees, especially in the final year. On average, final-year students thought that WIL was more important in their degree than second- and first-year students (p<0.05); with more responses of ‘very important’ for the final-year students, to predominant responses of ‘moderately important’ for first-years (n = 8; 57%). First-year students were least able to recall WIL having been explicitly mentioned in assessment (including those who were unable to judge) (n = 13, 93%) in comparison to second- and final year students (second year n = 7, 44%.; final year n = 11, 65%) (F (2,41) = 5.155, p = 0.01). Likewise, short text responses suggested two themes: that greater awareness of WIL projects and placements from year one and that wider opportunities to engage with potential employers were needed.

When students were asked whether they would value industry professionals grading and providing feedback on some assessments, they indicated this would be useful. The ranking of usefulness increased through the year levels, with 21% (n =3) of first-year students indicating it would be useful, 44% (n = 7) of second-year students, and, final-year students were significantly more likely to indicate that they would find feedback on their assessment most useful (n = 11, 65%, F (2,41) = 3.122, p=0.055). When asked whether they would value firsthand knowledge from industry professionals, students ranked it from moderately to very useful. Overall, second- and final-year students ranked first-hand knowledge higher than first year students.

Responses to the question ‘What would you add to your degree to improve your employability?’ varied between cohorts. Overall, students indicated they would like more: time out in the field; yearly placements/internships (including international experiences); opportunities for networking; more real-world examples; report writing skills; industry-focused coursework; integrated learning experiences within workplaces; further research opportunities; awareness about searching for work placements; and ways to enhance employability. In summary, the student data demonstrate the value of a whole-of-degree approach to embedding and explicit scaffolding of WIL.

Academics were also encouraged to consider the views from industry on assessment collaboration as well as WIL program design. The in-depth industry interviews revealed that there was support by industry for university-based WIL or WIL on-campus. Interviewees from the industry sector perceived the key benefits as completing ‘relevant backburner‘ projects whilst not having to allocate the same level of intensive off-campus resourcing to their engagement amid space and head-count constraints in their organization. All respondents noted that they were ‘resource poor’ (i.e. time, people and financial) and as such, on-campus engagement will need, in the future, to generate positive and demonstrable benefits to their organization.

In terms of work-ready skills, there was a clear and consistent voice from industry interviewees – whilst assuming a degree has provided a foundation of fundamental discipline knowledge, strong interpersonal skills are a fundamental recruitment requirement. Both the industry participants and the teaching teams agreed that a tool that facilitates an engagement based on a 3 S’s approach for improving industry partnerships for WIL on-campus activity would be most useful: synergy of goals; simplicity of engagement; and sustainable relationships. The need for a fit-for-purpose industry engagement framework and methodology for enabling greater depth and breadth of on-campus university-based WIL assessment, related learning activities and opportunities for further off-campus and hybrid WIL scenarios, is clear.

The development of an Industry Framework was a key WIL-on-Campus (WoC) project deliverable (the data relating to industry connections and the elements curated to become part of the final framework and will be detailed in a future publication). It was presented at the second workshop to propel academic behavioural change towards the involvement of industry in assessment design and delivery.

The academics were interested to hear the student and industry stakeholder voice. From the student perspective, their take home message was that students have expressed finding meaning in their curriculum with stronger links between industry and the university. From the industry perspective, their take home message was that industry has a genuine interest in, and support for, stronger and more formal opportunities for university-based WIL, especially as a pathway for quality placement-based WIL experiences. In short, the staff survey summed up a simple, telling and promising point – that most respondents agreed that there could be more done with WIL within their program.

Enablers of assessment-reform (R3) 

The greatest perceived barriers to assessment reform with a whole-of-program approach were underpinned by matters relating to academic workload and resourcing. One academic commented: Authentic WIL experiences take time, resourcing and good planning, and at the moment our team is really stretched in the teaching space. We need more staff. As academics are at the coal face of assessment-reform change, our case study argues that the support and enhancement of WIL-lens assessments are best enabled by focusing on and resourcing initiatives that enable the WIL champions within the course team to influence peers.

This study identified the following set of mechanisms which have been and are likely to be of most benefit for continuing reform: 

• Instil that the central dimension of core bodies of STEM knowledge and skills remain a learning outcome focus of the assessment, but in addition, must be intentionally contextualised to real-world applications and linked to employability learning; 
• Aspire for tri-created assessments (student, industry and academic) that are work related for employability-relevant learning, not for job/industry-specific contexts; 
• Publish the program-specific aim and intended impact of WIL approaches to assessment (i.e. enhancing student employability, improving graduate and employment outcomes); 
• Institutions to invest in the human element of curriculum reform – the academic as the driver of assessment reform at the coal-face. 
• Course-teams to collectively undertake re-imagined assessment design for the purpose of resource sharing as well as sustainable capacity building; 
• Consideration of the course team culture for leveraging those resistant to change to be part of a whole-of-program approach to authentic assessments; 
• Support individual academics to enable sustainable and scalable connections with industry to enable discussion of, and feedback on, current and context-rich industry oriented assessments; 
• Provide a WIL assessment framework as a methodological approach to whole-of program curriculum design and renewal that can be fit-for-purpose at the program level; 
• Provide course teams with a fit-for-purpose framework for enabling the engagement with industry to simplify and sustain whole-of-program WIL approaches in the curricula;
• Provide up to date specific student cohort, industry data and sector-wide employment data as an entry point for discussing ‘other’ stakeholder perspectives on the value of WIL; 
• Develop a set of principles, standards and quality indicators for WIL approaches to assessment to enable the benchmarking, assurance and monitoring of program rationale for delivering on the Higher Education Standards (HES); 
• The inclusion of mandated minimum weightings for authentic and highly proximal assessments; 
• Assure that the systems for measuring and documenting WIL are trusted, easy to follow and enable ‘hidden’ WIL to become a useful data source for evaluation and improvement. 

The overall findings reveal that all stakeholders see the value of assessments that use on-campus locations, that are intentionally embedded and scaffolded across a whole program, and that enable the integration of theory and practice demonstrated through authentic assessment contextualised to STEM. In particular for STEM, because academic staff consider the development and assessment of discipline skills and knowledge with a program as the priority, we assert that the best current vehicle for enhancing graduate employment outcomes is by assessing discipline specific knowledge and skills in conjunction (not separately) with subsets of other employability skills, in-context, using reflective practice approaches focused on professional applications to show integrative learning. This approach still fits with the Chief Scientist’s Five Point Plan for STEM education (Finkel, 2018) that STEM content (‘hard knowledge’ of concepts, facts and principles) must be taught by expert teachers.

Conclusion

This study concludes that academics acknowledge that WIL assessment design is challenging, but accepts evidence from students, industry partners, and the institution, that there is scope and value in including more authentic and proximal elements in assessment design and delivery. Project indicators, from workshops and focus groups suggest that socialisation of WIL has been a successful outcome of the WoC. The project group found that reform of any significant kind cannot begin until teaching teams feel ready, willing and able, so they must be supported to enable change. Academics require top down support from their most senior colleagues to enable course teams to identify and understand the opportunities for reform and implement required changes. Socialisation of shared and recognised value of embedded and scaffolded WIL assessment methods are also an underpinning requirement.

Whilst recognising that such curriculum reform is challenging, this study finds that academics accept that such changes are achievable and, indeed, timely. The implication for this study is that a whole-of-program curriculum-led approach, based on scaffolded WIL-assessment profiles, is a practical way in which course-based evidence can be collected around employability learning and graduate outcomes.

A surprising discovery from this study was that the key to assessment reform may be in the decision to discuss WIL as an approach that is no longer perceived as a loss to the core STEM disciplinary content. The study found that because current WIL approaches to assessment design occur less frequently than the assessment of the core body of science knowledge, for successful WIL assessment reform to occur in a STEM faculty, the combination of the two factors needs to be actively aligned. This finding therefore makes a significant contribution to the STEM sector, as the perceptions of STEM academics may be mirrored elsewhere, and thus warrants further investigation. The caveat to this is that the method applied to collecting and analysing academic perceptions of WIL assessment reform (in particular, the use of focus groups to provide insight into academic perceptions) has an inevitable limitation – that being that the insights gained might not be representative of the different discipline areas outside of science, or indeed in STEM across the faculty.

The authors do however consider that the process of unpacking the challenges and enablers of assessment reform design adopted in this study can be extended into further STEM courses and within other institutions. In turn, a new depth of understanding across the sector on WILlens curricula may contribute to the current limited generalizability of this study that found that our students and academics recognise the importance for the inclusion of employability learning. 

Acknowledgments

A special thank you to the WIL-on-Campus(WoC) project team members for their enthusiasm and efforts in driving this project as well as contributing to the writing and editing of the WoC Report(Young et al.,2018). Thank you also to the Associate Dean Teaching and Learning, Professor Malcolm Campbell (Science, Engineering and Built Environment) who sponsored this project and to Faculty Executive who funded the project.

Declaration of interest statement

The authors hereby declare that they have no pecuniary or other personal interest, direct or indirect, in any matter that raises or may raise a conflict of interest in this study with our other roles and duties within the Faculty.

References

1. Abeysekera, I. (2006). Issues relating to designing a Work-Integrated Learning (WIL) program in an undergraduate accounting degree program and its implications for the curriculum. Journal of Cooperative Education, 7(1), 7–15.
2. Office of the Chief Scientist. (2015). Bridging a gap between study and work. Canberra: Australian Government. Retrieved from http://www.chiefscientist.gov.au/2015/08/media-release-bridging- a-gap-between-study-and-work/
3. Berg, B. L. (2004). Qualitative research methods for the social sciences (Vol. 5). Boston, MA: Pearson.
4. Buckley, J., & El Amoud, L. (2011). Undergraduate Work Placement Programmes in Ireland: Issues and Solutions Cork: Cork Institute of Technology.
5. Collis, C. (2010). Developing work-integrated learning curricula for the creative industries : embedding stakeholder perspectives. LATHE: Learning and Teaching in Higher Education, 2010 (4-1), 3-19. Retrieved from https://eprints.qut.edu.au/48384/
6. Connor, H., & MacFarlane, K. (2007). Work related learning (WRL) in HE: a scoping study York: Higher Education Academy.
7. Daniel, R., & Shircore, M. (2012, 29 October-2 November). Transitioning undergraduate students from law, business, and creative arts towards work integrated learning capstone experiences. Australian Collaborative Education Network National Conference, Geelong.
8. Desha, C., & Senadji, B. (2014, 8-10 December). Holistically approaching curriculum renewal: A case study of the Queensland University of Technology. Australasian Association for Engineering Education Annual Conference, Wellington.
9. Edwards, D., Perkins, K., Pearce, J., & Hong, J. (2015). Work integrated learning in STEM in Australian universities: Final Report. Canberra: Office of Chief Scientist & Australian Council for Educational Research.
10. Ferns, S. (2014). HERDSA Guide: Work Integrated Learning in Curriculum. Australia: Higher Education Research and Development Society of Australasia.
11. Finkel, A. (2018) Chief Scientist’s Five Point Plan for STEM education, July 11, 2018, Retrieved from http://campusmorningmail.com.au/news/chief-scientists-five-point-plan-for-stem-ducation/?utm_campaign=website&utm_source=sendgrid.com&utm_medium=email
12. Greenbank, P. (2002). Undergraduate work experience: an alternative approach using micro businesses. Education + Training, 44(6), 261-270. doi:10.1108/00400910210441392
13. Gulikers, J. T. M., Bastiaens, T. J., & Kirschner, P. A. (2004). A five-dimensional framework for authentic assessment. Educational Technology Research and Development, 52(3), 67. doi:10.1007/bf02504676
14. Hains-Wesson, R., Pollard, V., Kaider, F., & Young, K. 2019, STEM academic teachers’ experiences of undertaking authentic assessment-led reform: A mixed method approach. Studies in Higher Education (CSHE). doi: 10.1080/03075079.2019.1593350
15. Jorre de St Jorre, T., & Oliver, B. (2018) Want students to engage? Contextualise graduate learning outcomes and assess for employability, Higher Education Research & Development, 37(1), 44-57, doi: 10.1080/07294360.2017.1339183
16. Kaider, F., Hains-Wesson, R., & Young, K. (2017). Practical typology of authentic work-integrated learning activities and assessments. Asia-Pacific Journal of Cooperative Education, 18(2).
17. Kay, J., Ferns, S., Russell, L., & Smith, J. (2018) Innovative WIL Models, retrieved from https://acen.edu.au/innovative-models/
18. Kemmis, S., & McTaggart, R., eds. (1988) The Action Research Planner (3rd ed.). Victoria: Deakin University Press.
19. Lasen, M., Evans, S., Tsey, K., Campbell, C. & Kinchin, I. (2018). Quality of WIL assessment in Higher education: a systematic literature review, Higher Education Research & Development, 37(4), 788-804 doi: 10.1080/07294360.2018.1450359
20. Norton, A., & Cakitaki, B. (2016). Mapping Australian higher education 2016 Melbourne: Grattan Institute.
21. Oliver, B. (2015). Redefining graduate employability and work-integrated learning: Proposals for effective higher education in disrupted economies. Journal of Teaching and Learning for Graduate Employability, 6(1), 56–65.
22. Office of the Chief Scientist (2013), Science, Technology, Engineering and Mathematics in the National Interest: A Strategic Approach, Australian Government, Canberra.
23. Palmer, S., Campbell, M., Johnson, E., & West, J. (2018). Occupational Outcomes for Bachelor of Science Graduates in Australia and Implications for Undergraduate Science Curricula. Research in Science Education, 989-1006. doi: 10.1007/s11165-016-9595-x
24. Patrick, C.-J., Peach, D., Pocknee, C., Webb, F., Fletcher, M., & Pretto, G. (2008). The WIL (Work Integrated Learning) report: a national scoping study. Brisbane: Queensland University of Technology.
25. Peach, D., & Gamble, N. (2011). Scoping Work-Integrated Learning Purposes, Practices and Issues. In S. Billett & A. Henderson (Eds.), Developing Learning Professionals: Integrating Experiences in University and Practice Settings (pp. 169-186). Dordrecht: Springer Netherlands.
26. Rae, D. (2007). Connecting enterprise and graduate employability: Challenges to the higher education culture and curriculum? Education + Training, 49(8/9), 605-619. doi:10.1108/00400910710834049
27. Rayner, G., & Papakonstantinou, T. (2015). Employer perspectives of the current and future value of STEM graduate skills and attributes: An Australian study. Journal of Teaching and Learning for Graduate Employability, 6(1), 100-115.
28. Rowe, A., Winchester-Seeto, T., & Mackaway, J. (2012, 29 October-2 November). That’s not really WIL!–building a typology of WIL and related activities. Australian Collaborative Education Network National Conference, Geelong.
29. Rowe, A. D., & Zegwaard, K. E. (2017). Developing graduate employability skills and attributes: Curriculum enhancement through work-integrated learning. Asia-Pacific Journal of Cooperative Education, 18(12), 153-165.
30. Sachs, J., Rowe, A., & Wilson, M. (2016). 2016 Good Practice Report–Work Integrated Learning (WIL) Canberra: Department of Education and Training.
31. Senadji, B., Boman, M., Allbutt, J., & Whelan, K. (2015, 6-9 December). From work placement to employability: A whole-of-course framework. Paper presented at the Australasian Association for Engineering Education Conference, Geelong.
32. Smith, C. D. (2016). The Emergence and Development of Work-Integrated Learning (WIL): Implications for Assessment, Quality and Quality Assurance in Higher Education. In C.-h. C. Ng, R. Fox & M. Nakano (Eds.), Reforming Learning and Teaching in Asia-Pacific Universities: Influences of Globalised Processes in Japan, Hong Kong and Australia (pp. 337-364). Singapore: Springer Singapore.
33. Smith, M., Bell, K., Bennett, D., & McAlpine, A. (2018). Employability in a Global Context: Evolving Policy and Practice in Employability, Work Integrated Learning, and Career Development Learning. Wollongong, Australia: Graduate Careers Australia. doi: 10.6084/m9.figshare.6372506
34. Venables, A., & Tan, G. (2009). Realizing learning in the workplace in an undergraduate IT program. Journal of Information Technology Education: Innovations in Practice, 8, 17-26.
35. Yorke, M. (2006). Employability in higher education: what it is, what it is not. York: Higher Education Academy (p 6).
36. Young, K., Palmer, S., & Campbell, M. (2017). Good WIL hunting: Building capacity for curriculum re-design. Journal of Teaching and Learning for Graduate Employability, 8(1), 215-232. doi: 10.21153/jtlge2017vol8no1art670
37. Young, K., Palmer, S., Campbell, M., Binek, C., Tolson, M., Miller, K., Beech, P., & Krishnan, S. (2018). WIL on Campus (WoC) Report. Final Report. Deakin University, Australia. Unpublished report.
38. Young, K., Miller, K., Campbell, M., & Palmer, S. (2019). A WIL Assessment Framework – the making of the WAF. Manuscript in preparation.

---

This article is governed by: