The Student Experience at Risk: Government Policy in the Measurement and Enhancement of Student Experience

Dr Laura HougazStudy Connections, ACPET

Assoc Prof Mahsood ShahThe University of Newcastle

The Context: Enhancing quality in HE

In recent years the Australian HE sector has come under pressure to demonstrate quality outcomes

Efforts made to improve teaching through professional development, scholarship and research, and evaluation

Review of policies and practices with focus on enhancing learning and teaching – ‘quality of teaching’ evaluation has refocused on ‘quality of student experience’ evaluation.

Increased focus on student evaluations/feedback.

Topic at glance

Student experience matters Measurement of student experience in private higher

education (PHE) Factors influencing student choice in PHE Research on student experience in PHE Government policies on the measurement of student

experience Why is student experience at risk? Way forward

3

Student experience matters

Prominence of student voice Student experience is used to assess teaching quality

• Students’ perceptions are important parameters of the social and psychological aspects of the learning environment

Student experience is used to assess the level of risk Global trend in the use of student feedback to:

• develop and renew courses• review of courses• assessment of teaching and course quality• assessment of support services.

4

Student experience matters

Data is increasingly used in rankings and marketing Data is used on MyUniversity website, Unistats Benchmarking performance of institutions Used in academic staff performance reviews and

promotions Increased diversity of students, thus different expectations

and experience Low satisfaction may cause attrition (quitting or transferring)

Student complaints can harm reputation of institution We can only improve, if we listen and act on feedback.

5

Measurement of student experience in PHE

Australian Graduate Survey (AGS): used by handful of PHE Internal semester based unit and teacher evaluation surveys

are patchy across providers No standard instrument used to measure total student

experience in PHE National Vet Regulator (NVR) learner engagement survey is

used for compliance purpose only, rather than enhancement Use of data and closing the loop is an area where limited work

is being done Ethical issues on data collection Lack of triangulation of survey data (qualitative /quantitative)

along with other academic outcomes measures

6

Factors influencing student choice

Source: Shah, Nair, and Bennett. (2013).

Student Perception

Access and opportunity

Learning environment

Teachers Course design Graduate Success

Reputation Word of mouth Transfer from University to private college due to referral

Pathway to university Stepping stone in tertiary education Ease of entry Location Completing degree via pathway

Small class size Personalised environment Online learning Student profile (mature age in some colleges) Hands on experience

Teachers from industry Easy accessibility One to one interaction with teacher

Duration Practical nature Work experience Flexibility Specialised course relevant to professions

Success of graduates Doing well in industry

Input Processes Outcomes

7

Research on student experience in PHE

Source: Nair, Bennett, and Shah, M. (2012). 8

Government policies: Measurement of student experience

Bradley Review of Australian Higher Education (2008) concluded that a comprehensive set of measures of the quality of teaching and learning should be developed. These should include measures of the student experience and form part of a broader accountability framework focused on the achievement of outcomes.

Recommendation 7

“That the Australian Government require all accredited higher education providers to administer the Graduate Destination Survey, Course Experience Questionnaire and the Australasian Survey of Student Engagement from 2009 and report annually on the findings.” (Bradley Review, 2008, Ch 3.4)

Government policies: Measurement of student experience

More focussed on universities AGS only used in universities, with lack of implementation in

PHE New University Experience Survey (UES) only used in

universities (domestic and onshore U/G) Development of UES did not involve PHE consultation Defeats the idea of “One Tertiary Education” No standard tool to measure offshore student experience -

despite being rated as high risk by TEQSA Lack of benchmarking of student experience No mechanism to ensure comparable experience

10

Why is student experience at risk?

Lack of standard measurement tools used across the HE sector

Data collection, coding, analysis and reporting is inconsistent Lack of benchmarking with all providers Absence of systematic process to identify areas needing

improvement (at national and institutional level) OLT funding for research in learning and teaching does not

support PHE Lack of student engagement in TEQSA assessments Closing the feedback loop – requires improvement The UK is further ahead in the developments in this area

(Higher Education Academy, UK).

11

TEQSA and student experience

Lack of innovation on how to determine if student experience is at risk

Assessment of risk is based on paper trail review Huge reliance on institutional collected and analysed

unit/teacher evaluation data – could be manipulated Current process to identify risk is based on data at overall

institutional level No focus on assessing risk at course, faculty, campus level and

partnership, online delivery mode No focus on assessing the experience of different cohorts of

students (onshore/ offshore/ PG/ HDR, LSES, NESB etc) Lack of student engagement

12

Way forward

Using standard tools such as AGS and UES Benchmarking student experience Need for a standard tool in PHE to measure student

experience at teacher & subject levels Effective use of data in improving overall student experience Building capacity for design, collection, analysis and

reporting of data Culture of closing the loop Use of qualitative data Build internal capacity for QA.

13

Recent publications related to PHE

Shah, M., Nair, S., & Bennett, L. (2013). Factors Influencing Student Choice to Study at Private for-Profit Higher Education Institutions. Quality Assurance in Education, in press.

Shah, M., & Nair, S. (2013). Private for–profit higher education in Australia: Widening Access and Participation and Opportunities for Public-Private Collaboration. Higher Education Research and Development (HERD), 32(5), 820-832.

Shah, M., & Stanford, S. (2013). Quality and Regulation of Australian Tertiary Education: Searching for Sustainable QA Framework, The ACPET Journal for Private Higher Education, 2(1), 24-33.

Shah, M., & Nair, S. (2012). Student Surveys and Feedback: Strategic Solution for all Tertiary Education Institutions. Studies in Learning, Evaluation Innovation and Development, 9(1), 74-83.

Bennett, L., Nair, S., & Shah, M. (2012). The Emergence of Private Higher Education in Australia: The Silent Provider. European Journal of Higher Education, 2(4), 423-435.

Nair, S., Bennett, L., and Shah, M. (2012). Student Experience in Private Higher Education in Australia. The ACPET Journal for Private Higher Education, 1(2), 24.30.

Shah, M., & Nair, S. (2012). A New Dynamic in Australian Higher Education: The Emergence of Private for-profit Higher Education. European Journal of Higher Education, 2(2-3), 307-317. 14

Questions Dr Laura Hougazstudyconnections@gmail.comLaura.Hougaz@acpet.edu.au

Assoc Prof Mahsood Shahmahsood.shah@newcastle.edu.au

15

Thinking of doing PhD with focus on Private Higher Education ? Contact Assoc Prof Mahsood Shah, Newcastle University