How to measure success in degree apprenticeships: Should this be different from traditional success measurements?

A blog by Kevin Smith FHEA and Colleen Henderson FHEA.

Limitations of traditional success measures

The motivation for this piece of action research came following an Ofsted visit at the University Centre Quayside, in December 2021. It was evident that the inspection framework was specifically centred on traditional views of student success, such as student retention and classification achievement. While these may have relevance within conventional undergraduate programmes, this seemed rather limited when looking at success within Degree Apprenticeships. The important distinction between these programmes is the involvement of the employer, not only as a sponsor, but also as a beneficiary within the Degree Apprenticeship programme. There was also an ambition to measure the professional and personal success that undergraduates achieved through the Degree Apprenticeship programme.

The research

A review of literature on success measures within Degree Apprenticeships identified a limited array of research into the subject, however there was more success when looking at defining achievement within higher education. Our aim was to try to identify a model of student success, which we could apply to Degree Apprenticeships and consider the impact that this had on both student and employer. We adapted a model developed by York et al (2015) to capture key themes which we considered as essential in measuring success in Degree Apprenticeships (diagram 1).

Diagram 1: Adapted model of academic success in degree apprenticeships

Additional investigation into existing measures of success in UK apprenticeships commissioned by the Department for Business, Innovations and Skills and subsequently the Department for Education produced evaluations of apprenticeships from an apprentice and employer perspective. This provided key themes on which to develop questionnaires to establish the impact of the Degree Apprenticeship programme.

Our focus

For students, we focused on the direct results of the programme on promotion, gaining additional responsibilities, improvements in salary and external employment opportunities. For employers, we focused our questionnaire on productivity, morale, retention, service delivery, creativity and business development.

Positive results

We conducted digital surveys with 19 undergraduates and 9 employers. The results from the questionnaires gave an extremely positive illustration of the impact that the provision delivered by University Centre Quayside has on both apprentices and employers. These outcomes are illustrated in diagrams 2, 3 and 4:

Diagram 2: Opportunities for internal promotion

Diagram 3: Leading to a pay increase

Diagram 4: Employer view

We asked employers to identify the impact that CMDA students have had upon their organisation:

(Questionnaire adapted from Colahan et al, 2014.)

Capturing the essence of student success

Our view is that we should adapt and adopt the York model to consider both the student voice in identifying positive outcomes from participation within the programme and incorporate the employer dimension to measure the impact that degree apprenticeship students have within the business. This will additionally give us a stronger evidence base on which to have discussions with regulatory authorities. By using this model, we feel that we capture the essence of student success within degree apprenticeships, however we are only at the start of this journey.

Next steps

•    As part of the QAA Collaborative Partnerships project we are looking at best practice within Degree Apprenticeships, where there will be opportunities to explore success and impact through this investigation. 
•    To reflect upon the QAA Evidencing Value toolkit and consider how we translate our tangibles and intangibles into a meaningful set of measures.
•    We are seeking partners to establish comparisons, between institutions and programmes to further develop our understanding of modelling and impact.

References

York, Travis T.; Gibson, Charles; and Rankin, Susan (2015) "Defining and Measuring Academic Success, Practical Assessment, Research, and Evaluation: Vol. 20, Article 5. DOI: https://doi.org/10.7275/hz5x-tx03 Available at: https://scholarworks.umass.edu/pare/vol20/iss1/5

Department of Business, Innovation and Skills (2014) “Evaluation of Apprenticeships: Employers” BIS Research Paper No 204. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/635615/bis-14-1207-Apprenticeships-Evaluation-Employers-December-2014.pdf - Accessed 23/01/2023