A FRAMEWORK FOR MEASURING EQUITY PERFORMANCE IN AUSTRALIAN HIGHER EDUCATION€¦ · A Framework for...

Framework Document

Make tomorrow better. ncsehe.edu.au

Tim Pitman and Paul Koshy, November 2015

A FRAMEWORK FOR MEASURING EQUITY PERFORMANCE IN AUSTRALIAN HIGHER EDUCATION

curtin.edu.au

A Framework for Measuring Equity Performance in Australian Higher Education – Framework Report

National Centre for Student Equity in Higher Education (NCSEHE), November 2015

Contents

Acknowledgements .................................................................................................................................. i

Abbreviations and Acronyms ................................................................................................................ii

Executive Summary ................................................................................................................................ iv

1. Introduction .......................................................................................................................................... 1

2. Defining and Reporting on ‘Equity’ Students .............................................................................. 3

3. Overview of the Equity Performance Framework for Australian Higher Education .......... 6

Tier 1 (Context): Pre-higher education........................................................................................... 8

Tier 2 (Performance): Higher education ........................................................................................ 9

Tier 3 (Outcomes): Post-higher education .................................................................................... 9

Comparisons by State and Territory ............................................................................................ 10

Comparisons with the General Population ................................................................................. 10

Comparisons with Input Levels ...................................................................................................... 11

4. Criteria for Selecting Indicators .................................................................................................... 12

5. Tier 1 (Context): Domains and Indicators.................................................................................. 13

Introduction ........................................................................................................................................ 13

Domain 1: Early childhood development .................................................................................... 14

Domain 2: Primary education ........................................................................................................ 17

Domain 3: Secondary education ................................................................................................... 20

Summary of Tier 1 Indicators:........................................................................................................ 23

6. Tier 2 (Performance): Domains and Indicators ......................................................................... 24

Introduction ........................................................................................................................................ 24

Domain 4: Aspiration towards higher education ...................................................................... 24

Domain 5: Access to higher education ........................................................................................ 28

Domain 6: Achievement in higher education ............................................................................. 33


7. Tier 3 (Outcomes): Domains and Indicators .............................................................................. 37

Introduction ........................................................................................................................................ 37

Domain 7: Graduate outcomes ...................................................................................................... 37


8. Measuring Equity Inputs ................................................................................................................. 42

Domain I: Inputs ................................................................................................................................ 45


National Centre for Student Equity in Higher Education (NCSEHE), November 2015

Summary of the Input Domain: ..................................................................................................... 46

9. Standard Reports .............................................................................................................................. 47

Introduction ........................................................................................................................................ 47

Individual Domain reports .............................................................................................................. 48

Inter-Domain reports ........................................................................................................................ 51

10. Testing the Framework ................................................................................................................ 53

11. Summary .......................................................................................................................................... 59

References ............................................................................................................................................... 62

Appendix A: Definition of Australian higher education equity groups ................................... 65

Appendix B: List of Australian Higher Education Providers, 2013............................................ 66

Appendix C: The Options for Reporting on Equity Inputs .......................................................... 70

Option 1: Creating a distinct Tier for equity inputs.................................................................. 70

Option 2: Integrating equity inputs into the existing Tiers .................................................... 72

Option 3: A separate reporting of equity inputs ....................................................................... 73


National Centre for Student Equity in Higher Education (NCSEHE), November 2015 i

Acknowledgements The authors of this document are Tim Pitman and Paul Koshy of the National Centre for Student Equity in Higher Education (NCSEHE) at Curtin University. They wish to acknowledge Professor Sue Trinidad, Louisa Bowman and Andrew Broadley for their advice and assistance on this project.

The NCSEHE would also like to thank the following key stakeholder groups: the Australian Government Department of Education and Training (the “Department of Education and Training”); the Australian Curriculum, Assessment and Reporting Authority (ACARA); Graduate Careers Australia (GCA); the Productivity Commission; and the Social Research Centre (SRC). In addition, particular thanks go to:

Equity Performance Framework Steering Group Members

• Mr Craig Ritchie and Ms Julie Birmingham, Department of Education and Training;

• Dr Fadwa Al-Yaman, Australian Institute of Health and Welfare (AIHW);

• Dr Andrew Taylor and Dr Yew May Martin, Australian Government Department of Education (Data and Analysis);

• Professor Ian Anderson, Aboriginal and Torres Strait Islander Higher Education Advisory Council;

• Ms Milly Fels and Ms Louise Pollard, Equity Practitioners in Higher Education Australasia (EPHEA);

• Mr Myles Burleigh, Ms Bindi Kindermann and Mr Bruce Caldwell, Australian Bureau of Statistics;

• Ms Sue Fergusson and Dr Mette Creaser, National Centre for Vocational Education Research (NCVER);

• Dr Nathan Cassidy, Universities Australia;

• Mr Larry Davies and Mr Alan Keith, Australian Council for Private Education and Training (ACPET);

and

• Jen Coughran and Catherine Argall, formerly of the Department of Education and Training;

• Elouise White, Department of Education and Training; • Office of Strategy and Planning, Curtin University; and • All participants during the Testing phase of the Framework project.

The NCSEHE also wishes to acknowledge the valuable foundation work undertaken by the Australian Institute for Health and Welfare (AIHW) and the Melbourne Centre for the Study of Higher Education (MCSHE) at The University of Melbourne, as referenced in this report.


National Centre for Student Equity in Higher Education (NCSEHE), November 2015 ii

Abbreviations and Acronyms ACARA Australian Curriculum, Assessment and Reporting Authority

ACER Australian Council for Educational Research

AEDC Australian Early Development Census

AIHW Australian Institute for Health and Welfare

ATAR Australian Tertiary Admission Rank

ATO Australian Taxation Office

BGS Beyond Graduation Survey

CHESSN Commonwealth Higher Education Student Support Number

CSP Commonwealth Supported Place

GCA Graduate Careers Australia

GDS Graduate Destination Survey

GOS Graduate Outcomes Survey

HDR Higher Degree by Research

HEIMS Higher Education Information Management System

HEPP Higher Education Participation Programme

HEPPP Higher Education Participation and Partnerships Programme

LSES Low Socio-Economic Status

MCEETYA Ministerial Council on Education, Employment, Training and Youth Affairs

MCSHE Melbourne Centre for the Study of Higher Education

MFE Measurement Framework for Equity

NAPLAN National Assessment Program – Literacy and Numeracy

NCSEHE National Centre for Student Equity in Higher Education

NESB Non-English speaking background

NUHEP Non-University Higher Education Provider

OECD Organisation for Economic Co-operation and Development

PISA Programme for International Student Assessment


National Centre for Student Equity in Higher Education (NCSEHE), November 2015 iii

QILT Quality Indicators for Learning and Teaching

ROGS Report on Government Services

SER Single Equity Report

SES Socio-Economic Status

SRC Social Research Centre

STAT Special Tertiary Admission Test

TAC Tertiary Admission Centre

TAFE Tertiary and Further Education

TIMSS Trends in International Mathematics and Science Study

UNESCO United Nations Educational, Scientific and Cultural Organization

WINTA Women in Non-Traditional Areas


National Centre for Student Equity in Higher Education (NCSEHE), November 2015 iv

Executive Summary In 2013, the Australian Government Department of Education (“Department of Education”), now the Department of Education and Training (“the Department of Education and Training” or “the Department”), commissioned the National Centre for Student Equity in Higher Education at Curtin University (NCSEHE) to provide further detail on the proposed framework, its structure, overall logic and potential data sources.

The Framework specifically assesses institutional and system-wide performance among higher education providers in terms of the access, participation and performance of six groups of under-represented students who have been specifically identified as equity groups in higher education:

• Low socio-economic status (LSES) students; • Students with disability; • Indigenous students; • Students from regional and remote areas; • Women in non-traditional areas of study (WINTA); and • Students from a non-English speaking background (NESB).

This document, A Framework for Measuring Equity Performance in Australian Higher Education (“the Framework”) describes the resulting work and final recommendation on the form of the Framework and is accompanied by the Data Indicator Dictionary and Forward Plan its operationalisation.

The Framework is intended to:

• provide a set of indicators that will allow the measurement of institutional and system-wide achievement in the higher education sector against the Government’s commitments, targets and goals in relation to equity;

• inform policy through the provision of relevant information to support the development of evidence‐based policy;

• foster informed debate through the provision of key information; • provide a platform which will guide evaluation of programs by government and

activities by Institutions; and • inform practice within, and support equity in, the higher education system.

This final version of the Framework incorporates feedback from the Wider Consultation and Testing phase of the project as well as feedback from the Steering Group. That work confirms support for the existing Framework structure, with some additions to the list of indicators and a clarification of the application of the Framework in relation to institutions and equity groups.


National Centre for Student Equity in Higher Education (NCSEHE), November 2015 v

The Equity Performance Framework for Australian Higher Education The Framework is hierarchical in structure and is comprised of three “Tiers”:

1. Tier 1 (Context): Pre-higher education collections covering pre-primary, primary and secondary education results;

2. Tier 2 (Performance): Higher education collections covering all 129 Australian higher education providers (see Appendix B) and university students at all levels of study; and

3. Tier 3 (Outcomes): Post-higher education collections covering graduates from higher education.

Each tier has related “Domains” and within each domain are the “Indicators” that measure higher education equity performance. Underlying each indicator are data which represent the measurement of that indicator. In addition, there is also an “Input Tier” which contains two indicators on inputs into equity outcomes in Australian higher education.

Figure A provides an overview of the structure of tiers, domains and indicators.

Within and across the tiers and domains, a total of 28 indicators are developed to measure higher education equity performance. In addition, the Framework also reports an “Input Domain”, which includes two indicators, for funding and activity respectively, of inputs into equity in higher education.

The 28 final indicators largely rely on existing data sourced from key (primarily educational) stakeholders who have in place rigorous and systematic data reporting processes and protocols. More specifically:

• Fifteen indicators use existing data and data protocols; • Four indicators rely on existing data and protocols, however the suitability of the data

(for the purposes of the Framework) is not ideal; • Three indicators will require new data and protocols to be developed, principally

because of the transition to a new collection instrument – the Graduate Outcomes Survey (GOS); and

• Six indicators require the development of new data sources and protocols, in addition to relying on existing data with the development of new protocols.

One original indicator, 5.02 (Acceptances by students) as shown in Figure A, has been removed on the grounds that it replicated reporting of another, Indicator 5.03 (Commencements (new students)).

Of the 28 indictors, the 11 in Domains 1 to 3 of Tier 1 relate to broader measurements of preparedness for higher education, indicators in Domains 4 to 6 of Tier 2 report on access and performance in higher education, across all levels of study – sub-bachelor, bachelor, postgraduate coursework (postgraduate) and higher degree by research (HDR). Indicators in Tier 3’s Domain 7 will report data on graduate outcomes.


National Centre for Student Equity in Higher Education (NCSEHE), November 2015 vi

Figure A: Equity Performance Framework for Australian Higher Education

KEY Regular text = Indicator uses existing data and protocols Blue-coloured underlined text = Indicator uses existing data and protocols but data are potentially too weak for the purposes of the Framework Red text shaded grey = Combination: some requirement for data and protocols to be created but also uses existing data but data protocols do not currently exist Red-coloured italics text = Indicator requires data and protocols to be created Black italics text on grey = Removal of indicator from the Framework after testing.

The Input Domain has two indicators: the first records identified Commonwealth resourcing for higher education equity programs; and the second reports on institutional program structures and activities, for instance, the number of schools in a given outreach program. It

TIER 3: OUTCOMES (Post-higher education)

Domain 7

Graduate outcomes

7.01 Graduate earnings (GDS) 7.02 Graduate employment (GDS) 7.03 Graduate further study(GDS) 7.04 Graduate satisfaction (GDS)

TIER 2: PERFORMANCE (Higher education)

Domain 4 Aspirations for higher education

Domain 5 Access to higher education

Domain 6 Achievement in higher education

4.01 Intentions to undertake higher education studies 4.02 Year 12 applications 4.03 Non Year 12 applications

5.01 Offers made to students 5.02 Acceptances by students 5.03 Commencements (new students) 5.04 Enrolments (all students) 5.05 Student Satisfaction (UES) 5.06 Course transitions

6.01 Student retention 6.02 Student success 6.03 Student completion

TIER 1: CONTEXT (Pre-higher education)

Domain 1 Early childhood development

Domain 2 Primary education

Domain 3 Secondary education

1.01 Early childhood development (physical, social and learning)

1.02 Participation in pre-school

2.01 Reading (NAPLAN) 2.02 Numeracy (NAPLAN) 2.03 School attendance

3.01 Reading (NAPLAN) 3.02 Numeracy (NAPLAN) 3.03 School attendance 3.04 Year 12 completion 3.05 ATAR 3.06 Achievement in advanced maths and science (PISA)

INPUTS INTO EQUITY

Domain I Equity Inputs

I.01 Higher education equity funding

I.02 Outreach activities (number of students and organisations reached)


National Centre for Student Equity in Higher Education (NCSEHE), November 2015 vii

is envisaged that data for the Input Domain will be collected via the Single Equity Report (SER), to be managed by the Department of Education and Training.

The Framework will allow the Department of Education and Training to publish aggregated and disaggregated data. It is proposed that the tables will be made publicly available for researchers and other stakeholders to conduct bespoke analysis. Individual Domain reports should be released annually with a more comprehensive, comparative analysis released once every five years.

Conduct of the Wider Consultation and Testing Phase This final version of the Framework Report and the accompanying draft Data Indicator Dictionary and Forward Plan, reflect feedback from the Wider Consultation and Testing phase of this project which was conducted between November 2014 and March 2015 and feedback from the Steering Group on subsequent drafts of the report.

As such, the document catalogues the current structure of the Framework in view of feedback on the design and the results of a preliminary test on the collection of data. Feedback was used with specific regard to:

• Changes in structure to the Framework; • The inclusion of new indicators; and • Preliminary Recommendations on the final reporting format.

The Wider Consultation and Testing phase involved the interrogation of key identified data sources for equity in education in Australia. These are reported below in Figure B.


National Centre for Student Equity in Higher Education (NCSEHE), November 2015 viii

Figure B: Data Sources Investigated During the Testing of the Framework

Data Source Agency Reporting

Format Data Availability Reporting

Tier

Australian Early Development Census (AEDC)

Social Research Centre (SRC)

.xls flat file/ Template

Three yearly; 2015 next year, report in 2016. Context

Report on Government Service (ROGS),

Productivity Commission (PC)

.xls flat file/ROGS website

Annual; ROGS released in March Context

National Assessment Program for Language and Numeracy (NAPLAN)

Australian Curriculum Assessment and Reporting Authority (ACARA)

.xls flat file/NAPLAN

website

Annual; NAPLAN released in December Context

PISA Test

Australian Council for Educational Research (ACER); Reported in ROGS


PISA – 15 year olds test – three yearly; 2015 next year Context

Longitudinal Study of Australian Youth (LSAY)

National Centre for Vocational Education and Research (NCVER) Request

LSAY last cohort in 2009 Context

Higher education applications data

Department of Education and Training (Department)


Annual; finalised in December Context

Student Enrolment and Performance Data Department


Annual; finalised in August Performance

Graduate Destination Survey (GDS)

Graduate Careers Australia (GCA)


2015-16, last GDS survey Outcomes

Beyond Graduation Survey (BGS)




University Experience Survey (UES) SRC

.xls flat file/ Template Annual; March Outcomes

Graduate Outcomes Survey (GOS) SRC


Annual; at design; Commences in 2016 Outcomes

Employer Satisfaction Survey (ESS) SRC

.xls flat file/ Template Unknown; at design Outcomes

Single Equity Report (SER) Department


Outcomes of the Wider Consultation and Testing Phase The outcomes from the Wider Consultation and Testing phase are reported in Figure C below, where this matches each of the 25 identified indicators with the above data source and provides information on the testing outcome in terms of data availability and protocols for collecting data.


National Centre for Student Equity in Higher Education (NCSEHE), November 2015 ix

A number of observations can be drawn from the testing:

• For pre-higher education collections (“Context”), there is little or no comparability in terms of reporting for equity groups as identified in higher education, with the exception of disaggregated reporting on Indigeneity. The identification of socioeconomic status is generally not exactly consistent with that reported in higher education, while definitions of disability and NESB tend to be self-reported. As a result, Indicators in the Context Domain provide some indication of current trends in the system using definitions of equity which are broadly consistent with those in the Framework but which offer no chance of direct comparability.

• Nationally consistent data for Indicators 3.04 (Year 12 completion) and 3.05 (ATAR) are missing at the public reporting level. The first indicator has been somewhat beset by a range of discrepancies in reporting between states and territories, brought about by specific differences across jurisdictions, be it differences in the linkages between secondary and introductory tertiary studies (TAFE) or a lack of coordination in collecting Year 12 completion data in a nationally consistent fashion. Similarly, ATAR data is not easily reported at the national level, except in the context of applications data which is collected from Tertiary Admission Centres (TACs) and institutions by the Department of Education and Training. It is likely that both Indicators 3.04 and 3.05 could be reported upon under separate collection arrangements, ideally with uniform equity definitions applying at both secondary and higher education levels.

• The biggest gap in the data collection occurs in Domain 4 (Aspirations for higher education), specifically in relation to Indicator 4.01 (Intentions to undertake higher education studies), where the closest data source for this measure, the Longitudinal Study of Australian Youth (LSAY), was last undertaken in 2009 by the NCVER, with the 2012 cohort study not be undertaken and considerable doubt being cast on the 2015 study. This creates a gap in the data, which could best be served through the introduction of a new survey instrument to survey Year 10 and 12 students in relation to their aspirations for non-compulsory education and training.

• In regards to Domains 5 (Access to Higher Education) and 6 (Achievement in Higher Education), collection of data from the higher education system is administratively straight-forward from the Data Analysis Branch of the Department of Education and Training, with internal costing being the only issue (~$1,350 for the testing request).


National Centre for Student Equity in Higher Education (NCSEHE), November 2015 x

Figure C: Outcomes from Testing of the Equity Performance Framework for Australian Higher Education

Indicator Data/Agency Data Equity Groups Protocols 1.01 Early childhood development (physical, social and learning) AEDC/SRC Yes All, except

Disability; WINTA AEDC – 3

year census

1.02 Participation in pre-school ROGS/PC Yes LSES; Indigenous; Special Needs. Yes

2.01 Reading (NAPLAN) (Yrs 3 & 5) NAPLAN/ACARA Yes All, except Disability, WINTA Yes

2.02 Numeracy (NAPLAN) (Yrs 3 & 5) NAPLAN/ACARA Yes All, except Disability, WINTA Yes

2.03 School Attendance ROGS/PC Yes Indigenous Yes


3.02 Numeracy (NAPLAN) (Yrs 7 & 9) NAPLAN/ACARA Yes All, except Disability, WINTA Yes

3.03 School Attendance ROGS/PC Yes Indigenous Yes 3.04 Year 12 completion -- No -- -- 3.05 ATAR -- No -- --

3.06 Achievement in advanced maths and science PISA/ACER Yes

LSES; Indigenous; Regions; language

background Yes

4.01 Intentions to undertake higher education studies LSAY/NCVER No -- --LSAY in

abeyance

4.02 Year 12 applications Department Yes LSES; Indigenous; Regions Yes

4.03 Non- Year 12 (non-school leaver) applications Department Yes LSES; Indigenous;

Regions Yes

5.01 Offers made to students Department Yes Yes Yes 5.02 Acceptances by students (removed from Framework) Department -- -- --

5.03 Commencements (new students) Department Yes All Yes 5.04 Enrolments (all students) Department Yes All Yes 5.05 Student Satisfaction (UES) (UES/SRC) Yes No; Likely in future Yes

5.06 Course transitions Department No Possible Yes

6.01 Student retention Department Yes All Yes 6.02 Student success Department Yes All Yes 6.03 Student completion Department Yes All Yes

7.01 Graduate earnings GDS/GCA; GOS/SRC Yes Likely in future Likely

7.03 Graduate satisfaction UES/SRC Yes No; Likely in future Yes

I.01 Higher education equity funding SER/Department Yes No Likely


SER/Department Yes Not Likely Likely

KEY

Regular text = Indicator uses existing data and protocols Blue-coloured underlined text = Indicator uses existing data and protocols but data are potentially too weak for the purposes of the Framework Red text shaded grey = Combination: some requirement for data and protocols to be created but also uses existing data but data protocols do not currently exist Red-coloured italics text = Indicator requires data and protocols to be created


National Centre for Student Equity in Higher Education (NCSEHE), November 2015 xi

Black italics text on grey = Removal of indicator from the Framework after testing.

Key Findings from the Wider Consultation and Testing Phase

The Wider Consultation and Testing phase produced three key findings: 1. Scope for Institutional Reporting: The critical issue in Tiers 2 and 3 of the Framework, and specifically Domains 5 and 6, is the extent to which reporting coverage can be ensured across the entire higher education sector – all 129 higher education providers -- and not just the 38 Table A providers (including the Batchelor Institute of Indigenous Tertiary Education). For reasons of data record confidentiality, the Department of Education and Training does not report student number counts of less than 5, although it does report counts of 0. This means that at even the very aggregated level (equity group performance by institution), non-university higher education providers (NUHEPs) struggle to report meaningful figures. For instance, a simple count of students by equity group in a given institution (see Figure D below) , saw only 16 out of 86 NUHEPs report figures for Indigenous students, implying a student count of less than five – in many cases a count of zero. At even lower levels of analysis, for instance, equity students by field of study by institution, the level of reporting approaches zero, as no institution has enough students to generate a publicly reportable result. Ultimately, many sub-reports for higher education enrolments will only be adequately populated for the 38 Table A providers.

Figure D: Equity Students by Institution, Number of Institutions Reporting More than 5, By Equity Group

Table A

Providers Table B

Providers Table C

Providers NUHEPs Total Total Number of Institutions 38 2 3 86 129 Number of Institutions reporting a student count greater than 5 for an Equity Group – Students from a Non English speaking background 37 2 1 40 80 Students with a disability 37 2 1 48 88 Women in Non-Traditional Area 37 1 2 44 84 Indigenous 38 1 1 16 56 LSES (SA1 measure) 38 2 1 76 117 Regional (2011 ASGS) 38 2 1 66 107 Remote (2011 ASGS) 37 2 1 13 53

In addition, data on graduate earnings and satisfaction can be sourced from the Graduate Destination Survey (GDS) and University Experience Survey (UES) respectively. Testing of the GDS and UES data sets has found that institutional averages for starting salary and student satisfaction are easily reportable at the institutional (Table A) provider level. However, reporting for individual equity groups at each institution is unreliable due to small sample size, although reporting on these groups at the aggregate level is acceptable.


National Centre for Student Equity in Higher Education (NCSEHE), November 2015 xii

2. Scope for Equity Group Reporting: Similarly, as Figure D shows, reporting for two equity groups (Indigenous; Remote) was particularly sparse across the system, due to the relatively small number of students. For this reason, the Framework may be thought of as being particularly applicable to the LSES and Regional groups in particular, with reporting for NESB, WINTA and Students with Disability seeing reasonable institutional response rates. 3. Reporting on Inputs: Input indicators can be sourced from the proposed SER to be collected by the Department of Education and Training. This collection will allow institutions to report on funding (received and spent) and activities in relation to equity programs, although at this point it is focused on outlook programs, with no immediate indication that it will collect data on bridging and enabling programs once students enter the system. However, as this item is still at the design and feedback stages, it is difficult to confirm the availability of data for future collections. Current departmental collections would only partially cover I.01 (Higher education equity funding) and then only for aggregate funding for equity programs, with no further breakdown of expenditure.

Preliminary Recommendations on the Reporting Format Given its wide ambit, the Framework’s reporting template needs to be both amenable to completion but also publicly accessible. While the recommendations are more fully discussed in the accompanying Forward Plan, the outcomes from the Wider Consultation and Testing phase in relation to the Framework’s rollout can be summarised as follows:

1. Reporting Scope: The core of the Framework lies in the indicators reported in Tiers 2 and 3. The Wider Consultation and Testing phase has indicated that reporting for all institutions may not be possible on a consistent basis because of the small number of enrolments in many NUHEPs, which do not allow for disaggregation at the equity group levels at even the total enrolment level, much less the level required of any sub-reports, for instance by Field of Study or Level of Study. For this reason, some consideration will have to be given to the relative merits of a ‘one size fits all’ approach to reporting.

Recommendation 1: The core recommendation is that initial implementation focus on reporting for all institutions on equity groups with larger populations: LSES and Regional Students, with a separate reporting mechanism being developed for Indigenous Students; NESB; WINTA; Students with Disability; and Students from Remote Areas in view of the particular policy requirements and data collection limitations of each.


National Centre for Student Equity in Higher Education (NCSEHE), November 2015 xiii

2. Template Format: The Wider Consultation and Testing phase found that the Framework will source two types of data:

• Pre-higher education data: These data sets provide context for the equity issues confronting higher education policymakers. However, they are generally not easily connected to one another or higher education data via a consistent reporting format. Recommended reporting format: This means that a data flat file is required to report on these Indicators.

• Higher education data: All proposed collection of higher education data can be

sourced directly from, or matched to, the Higher Education Information Management Systems (HEIMS) system. This will allow for consistency in the definition of equity status across these collections and for relatively easy reporting of data at the institutional level. Recommended reporting format: Higher education data can be presented in a data flat file (.xls) or a web-based database page with results generated via query.

It is conceivable that the entire Framework could be reported as a flat file via a web interface, with sections of the institutional reporting being shared with other Department of Education and Training projects, namely the Quality Indicators for Learning and Teaching (QILT) and/or the Datamart project.

Recommendation 2: It is recommended that the reporting on equity indicators follow a similar reporting format for that developed for the institutional reporting under other departmental data releases (e.g. current enrolment data releases by the Department and through QILT).

3. Reporting Timeframe: This is largely determined by institutional reporting, with the previous year’s enrolment usually confirmed and released by August of the following year. For instance, the enrolment data for 2014 will be finalised by August 2015, which enables a compilation of the Framework annual reporting of 2014 by November 2015.

Recommendation 3: Implementation will establish data collection timelines and protocols for an annual collection and reporting of data.


National Centre for Student Equity in Higher Education (NCSEHE), November 2015 1

1. Introduction Equity is a central issue in Australian higher education. This is reflected in bipartisan support for action to address the under-representation of equity groups in Australia, while equity is perhaps the most persistent educational policy issue internationally (Martin, 2009). It has been a policy goal for UNESCO since at least the late 1960s (Faure et al., 1972) and the OECD since the 1970s (Kallen & Bengtsson, 1973).

It is important to state from the outset that membership of an identified equity group does not, in and of itself, constitute disadvantage at the individual level. Further, being identified as a member of one of these groups in no way represents an attempt to disempower persons.

The authors understand some individuals within one or more of the aforementioned equity groups might not consider themselves disadvantaged and might resent being labelled as such. However, the identification of equity and disadvantaged groups is an acknowledgement that under-representation in higher education among members of the identified equity groups leads to socio-economic disadvantage, such as:

• higher levels of unemployment (ABS, 2013a); • lower levels of lifetime earnings

(National Centre for Social and Economic Modelling, 2012; Norton, 2012); and • worse health outcomes (ABS, 2013b).

At the same time it is understood that many causes of disadvantage in higher education occur much earlier in life, and that it is also important to measure whether or not the post-graduation, socio-economic benefits of higher education are being equitably realised.

The Framework for Measuring Equity Performance in Australian Higher Education (“the Framework”) is designed to provide evidence on the access, participation and performance of under-represented students (equity groups) across Australian higher education institutions.

This report outlines the key findings on the content and structure of the Framework, with the accompanying Data Indicator Dictionary and Forward Plan providing further detail on its implementation options.

Structure of this Report

This final version of the Framework Report reflects consultations and testing to date, as well as feedback from the Steering Group, and finalises previous versions of this document, Versions 1.6 and 1.7. The accompanying reports, Data Indicator Dictionary and Forward Plan, provide further detail on indicator definitions and options for rollout respectively.



The remainder of this report is organised as follows:

• Section 2 describes key issues surrounding the definition and reporting of equity students;

• Section 3 provides an overview of the Framework, including the various tiers, domains and indicators;

• Section 4 outlines criteria for selecting indicators;

• Section 5 describes the Tier 1 domains and indicators in greater detail;



• Section 8 outlines the recommended option for the collection and reporting of equity input indicators;

• Section 9 provides an overview of standard reports that will be generated from the Framework;

• Section 10 summarises the findings from the Wider Consultation and Testing phase; and

• Section 11 provides a summary of the Framework.



2. Defining and Reporting on ‘Equity’ Students In Australian higher education policy six groups of under-represented students have been specifically identified as equity groups. These are derived from the equity performance indicators identified by the Review of Equity and General Performance Indicators in Higher Education (the Martin Review) in 1994, (see Martin et al., 1994), and are:

• Low socio-economic status (LSES) students; • Students with disability; • Indigenous students; • Students from regional and remote areas; • Women in non-traditional areas of study (WINTA); and • Students from a non-English speaking background (NESB).

Working definitions for each of these groups are reported in Appendix A of this document.

In the last twenty years, the Federal Government has been regularly reporting on various aspects of the performance of the higher education sector in respect of access, progression and completion of these groups. Recent policy reviews supporting the continued monitoring of equity group achievement in comparison with the general and student populations include the Review of Higher Education (the Bradley Review) in 2008 (Bradley et al., 2008) and the Review of Higher Education Access and Outcomes for Aboriginal and Torres Strait Islander People in 2012 (Behrendt et al., 2012).

Statistics on the progress of equity groups are collected for all higher education providers in Australia, as defined under the Higher Education Support Act (2004). A list of higher education providers included in the framework, the list of approved providers under the Higher Education Support Act, is provided in Appendix B. This list is not static and the Framework discussed here will apply to all new higher education providers.

In initiating this project, the Department of Education and Training indicated retention of these core equity categories in the new Framework, but providing flexibility in the proposed collection to accommodate the inclusion of newly defined equity groups in the future.

This project has therefore emphasised collection of data on existing equity groups with a view to ensuring that data collection can occur for other groups identified in the future. The wider consultation and testing in relation to earlier versions of the Framework confirmed support for the continued collection of data on these equity groups. In addition, the feedback raised the prospect of several additions to this list of equity students, including groups such as mature age students and ‘first in family’ students. To a large extent, these groups are already represented in key equity groups (notably LSES), so for the current reporting purpose, the existing equity groups will form the basis of reports, with other identified sub-groups being reported upon in sub-reports (mature age students) or in future data collections by institutions (first-in-family).



Defining Equity Status There are important issues which complicate the definition and recording of equity group status. They differ in the manner in which they are delineated in higher education. In some instances, equity status is self-reported, such as NESB, Indigenous students, and students with disability. Others, such as LSES and regional and remote students are subject to area measurement, whereby the permanent home address of a student defines their status. Of all equity groups, only WINTA is measured with relative precision. In the case of geographically-defined equity (i.e. LSES, regional and remote) it has lesser effect at the early stages of enrolment, but can affect the quality of data as the student progresses through their studies and, potentially, changes residence.

Indigenous students and students with disability are both entirely self-reported and it is generally assumed that both are under-reported (Australian Curriculum Assessment and Reporting Authority, 2009; Australian Institute of Health and Welfare, 2014). This is a particular issue for students with disability, as there is evidence that younger people are less likely to report disability than older people (Australian Bureau of Statistics, 2011). Furthermore the National Disability Coordination Officer Program’s own advice in this respect is that:

“The decision to disclose is a difficult one. The choice will be different for everyone because they have different experiences and different needs. Disclosing is a personal decision - you are the only one who can make it. Don't give in to pressure to disclose for the sake of other people, you are the one who will live with the positive and negative outcomes” (National Disability Coordination Officer Program, 2011).

The definition of disability under the Disability Discrimination Act 1992 (DDA) is broad and covers a wide range of physical, psychiatric and intellectual disabilities. It also includes disabilities that previously existed but no longer exist, or may exist in the future. However the Australian Bureau of Statistics (ABS), through its Survey of Disability, Ageing and Carers (SDAC), defines disability more simply as “any limitation, restriction or impairment which restricts everyday activities and has lasted or is likely to last for at least six months” (ABS, 2009). As the SDAC is the primary and most reliable source of statistical information for disabilities (Qu, Edwards, & Gray, 2012) it is therefore possible that what is being reported does not fully represent the actual or legal definition of disabled. Problems arise when comparing rates of participation in higher education with proportions of the wider Australian community. Recognising the difficulties in both defining the term and collecting statistical information, the Australian Government has moved to adopt a nationally consistent approach. From 2015, all government and non-government schools in Australia will be required to participate annually in the Nationally Consistent Collection of Data on School Students with Disability. The first data collection took place in selected schools in October 2013 (Thorpe, Shinfield, & Walsh, 2011).

Definitional problems also occur with the LSES category, which attempts to measure socio-economic disadvantage. In Australia, socio-economic status (SES) is measured primarily by



geographical location but could possibly also be measured by household income or use of welfare services. At the time this report was published, the Australian Bureau of Statistics (ABS) had just released its own report linking data from the Census of Population and Housing with Tasmanian government school enrolments and NAPLAN data (ABS, 2014). This provides insights into other aspects of the socio-economic contexts of educational disadvantage, including family demographics, parental education and parents’ professions. This preliminary study suggests that in many senses SES is a broad rubric that sometimes blurs, rather than provides focus to, our understanding of educational disadvantage.

In addition, accuracy in data collection will never be entirely consistent either within or across jurisdictions. To take just two examples:

1. Higher education students apply, and are enrolled, either through tertiary admissions centres (TACs) or directly to/with institutions. In the case of a TAC application, offer and acceptance data are recorded. However, when applying directly to an institution, often it is only the enrolment data that are collected. In many cases, particular programs for groups of equity students require direct application, further distorting this aspect of equity statistics.

2. Data collected on students with disability at the early childhood development stage, through the Australian Early Development Census (AEDC) collection, relies on the teacher’s own assessment of a student’s developmental needs. For this reason, statistics on students with disability are not reported in the AEDC sourced indicators.

As with previous attempts to collect and record higher education equity data, the Framework cannot fully compensate for these issues. However, by focussing on high-quality data sources, jurisdictions and data formats, and by reporting on changes across time or changes to proportional representation, rather than through just the minimal reporting of raw numbers, the effects of these definitional and reporting anomalies can be largely ameliorated.



3. Overview of the Equity Performance Framework for Australian Higher Education

Education is a lifelong learning activity and higher education is just one stage within this continuum (Candy, 2000). Progression throughout formal lifelong learning is a process of recognising, and building upon, prior learning opportunity. Equally, educational disadvantage can be expressed on a similar continuum, existing from the earliest years for many people, through their educational schooling years, including higher education, and beyond. Disadvantage can be exacerbated or ameliorated in the wider community as much as in the higher education sector itself.

For this reason, the three principles underlying the development of the Framework include:

1. Context: The smaller the gap in inequality in pre-tertiary education, the greater the opportunity will be for disadvantaged students to access and achieve in higher education;

2. Performance: It is likely that this inequality will never be fully eradicated and so action must be taken by the higher education sector itself to recognise and alleviate this disadvantage throughout the higher education experience; and

3. Outcomes: If this is done effectively, the post-graduation socio-economic benefits resulting from higher education achievement will be more fairly realised by all students.

Therefore, equity performance in higher education is relative. That is:

• any increase or decrease in the equity ‘gap’ – i.e. the difference in higher education outcomes for equity students and the overall student population – needs to be measured relative to any increase or decrease in the corresponding equity gap that occurs in the pre-tertiary education years; and

• any apparent improvement in equity performance within the higher education sector itself needs to be assessed in relation to future changes in the post-graduation socio-economic benefits realised by these students.

Thus, the Framework is structured around a three tiered format that allows for the monitoring and comparative assessment of equity group performance compared to the Australian population. In addition, the Framework also reports on ‘inputs’ into equity policy, be they Commonwealth funding for institutional programs or the activities and initiatives that flow from that funding.

This is the basis for the Framework in Figure 1. This reflects the structure of the Framework developed in earlier versions of this report and changes which have been adopted in view of the Wider Consultation and Testing phases as outlined in this document.



Figure 1: Equity Performance Framework for Australian Higher Education



Domain 7

Graduate outcomes







5.01 Offers made to students 5.02 Acceptances by students 5.03 Commencements (new students) 5.04 Enrolments (all students) 5.05 Student Satisfaction (UES) 5.06 Course transitions






1.01 Early childhood development

(physical, social and learning) 1.02 Participation in pre-school

2.01 Reading (NAPLAN) 2.02 Numeracy (NAPLAN) 2.03 School attendance

3.01 Reading (NAPLAN) 3.02 Numeracy (NAPLAN) 3.03 School attendance 3.04 Year 12 completion 3.05 ATAR 3.06 Achievement in advanced maths and science (PISA)

INPUTS INTO EQUITY






Tier 1 (Context): Pre-higher education Success in higher education is highly contingent upon prior success in secondary education, which in turn is based upon a child’s social, emotional and intellectual development in primary education. Even at this early stage, a high-quality pre-school education provides a better start to a child’s formal schooling years and can help alleviate the effects of social disadvantage. Transition into and from the early childhood years is extremely important for the child’s future physical, emotional and intellectual development (Sylva, Melhuish, Sammons, Siraj-Blatchford, & Taggart, 2010; Yeboah, 2002).

However in Australia, as in many other countries, there is a gap between the highest and lowest performing students in the pre-tertiary schooling years. In fact, relative to other OECD countries, Australia has a higher-than-average level of inequality in this respect (Jensen, Reichl, & Kemp, 2011). Access to higher education is built upon principles of meritocracy but “taking into account, in making such decisions about the selection of students, educational disadvantages that a particular student has experienced” (Department of the Attorney General, 2003, p. 21). It is therefore necessary to measure the extent of educational disadvantage experienced by students prior to accessing higher education.

At the Tier 1(Context) level, the overall (i.e. academic, social and physical) development of equity students is measured relative to the overall, corresponding student population. Whilst it is understood that higher education institutions have limited scope to shape pre-higher education performance, measuring pre-higher education disadvantage in this manner assists the subsequent measurement of performance of the higher education sector in two key ways:

First, it helps contextualise the sector’s performance. For example, if the educational disadvantage ‘gap’ experienced by Indigenous students (compared to all students) is narrowing, the subsequent performance of the higher education sector – after an appropriate time period - can be contextualised, allowing judgements as to its relative over or underperformance.

Second, areas of focus/activity can be more accurately identified. For example, if the gap in Year 12 performance between LSES and all students remains high, then greater effort may be required to provide alternative pathways to admission to higher education for these students.

Tier 1 has three Domains:

Domain 1 – ‘Early childhood development,’ comprising two performance indicators; Domain 2 – ‘Primary education,’ comprising three performance indicators; and Domain 3 – ‘Secondary education,’ comprising six performance indicators.



Tier 2 (Performance): Higher education Tier 2 (Performance) is where the traditional focus of equity performance measurement lies. As with the current approach, the Framework will continue to measure the relative performance of the six equity groups in terms of offers, enrolments, retention and progression of the six equity groups. However the Framework will provide greater detail on the pathways into higher education used by disadvantaged students, as well as their progression and performance throughout their studies. In terms of measuring student performance, the three key elements of Tier 2 are:

1. Improved measurement and tracking of equity students within and after graduation from the higher education sector;

2. Improved reporting on students belonging to multiple equity groups and levels of study (sub-bachelor; bachelor; postgraduate; and higher degree by research (HDR); and

3. A greater focus on articulation between course levels, for instance, sub-bachelor and undergraduate degree programs.

Tier 2 has three Domains:

Domain 4 – ‘Aspiration,’ comprising three performance indicators; Domain 5 – ‘Access,’ comprising five performance indicators; and Domain 6 – ‘Achievement,’ comprising three performance indicators.

Tier 3 (Outcomes): Post-higher education Equitable access to higher education is a positive outcome only if it results in relative (i.e. to the wider population) socio-economic opportunities and benefits. There is a wealth of research (see for example Department of Employment Education and Training, 1990; Martin, 1994), that demonstrates under-representation in higher education leads to later socio-economic disadvantage.

For example:

• People with higher education qualifications have lower levels of unemployment on average. In 2013, only 2.9% of people with a bachelor degree were unemployed, compared to 4.4% of those with no or lesser qualifications.1

• On average, people with higher education qualifications have higher lifetime earnings.2

1 According to ABS 6227.0 - Education and Work, Australia, May 2013 data (Table 10). 2 Estimations as to its magnitude vary. For example, the National Centre for Social and Economic Modelling (2012) calculates the difference in median gross lifetime income between people with a Year 12 qualification and those with a bachelor degree range from $620,000 for women and $1.1 million for men. Norton (2012) puts it at $800,000 for women and $1.1 million for men.



• People who attain higher education qualifications also tend to have better health, as they are less likely than their counterparts to smoke or drink, and are more likely to meet recommended guidelines for exercise.3

In Tier 3 (Outcomes), indicators will be used to primarily measure the economic benefits of higher education for equity students, as well as the socio-economic benefits associated with further study. Tier 3 has one domain:

Domain 7 – ‘Graduate outcomes,’ comprising four performance indicators.

Comparisons by State and Territory In Australia, higher education policy is complicated by the blurring of responsibilities between the state and territory and federal levels of government. With the exception of the Australian National University, universities are created by, and answerable to, their relevant state or territory government. However, the Federal Government directs most higher education policy, mostly through using its Constitutional funding powers. With regard to equity policy, there is national consistency, primarily through the fairness provisions contained within the Higher Education Support Act (2003). Further, there are no restrictions placed on Australian students applying to study outside their home state or territory. For these and other reasons the Framework is mostly focussed on the national picture.

However, there are important reasons why equity performance should also provide appropriate state and territory comparisons. First, Australian higher education students are relatively immobile. In 2013 more than 85 per cent of higher education applicants applied to institutions in their home state and 93 per cent ultimately accepted a home-state offer of enrolment (Department of Education, 2013). Second, many of the factors affecting disadvantage are geographically contextual including economic conditions and state and territory contributions to pre-tertiary education. Therefore, particularly at the Tier 1 (Context) level, state and territory comparisons are important. However, it is not necessary to explicate this distinction at the Indicator level as it is captured through reporting.

Comparisons with the General Population

Similarly, comparisons between equity groups in education and their representation in the general population are useful, particularly in view of present and future policy targets for equity participation. These can be sourced from references cited below (e.g. ABS; Productivity Commission) for broad social indicators for some of the measures, as well as underlying population shares and participation rates for the general population. Such comparisons will include those between a particular equity group and other equity groups, comparisons with the total student population and ‘general student population’ where this is 3 According to ABS 4102.0 - Australian Social Trends, July 2013 data (“Hitting the books: Characteristics of higher education students”).



defined to include all students with no equity group membership, as well as for identified ‘converse’ groupings, such as:

• LSES – Medium and High SES groups; • Indigenous – non-Indigenous groups; • Regional and Remote — Metro groups; • Students with Disability – without disability groups; • NESB –English speaking background groups; and • WINTA – Women in non-WINTA area groups.

Comparisons with Input Levels In Australia, higher education funding has systematically targeted equity outcomes in institutions through programs such as the Higher Education Participation Programme (HEPP), as well as related funding form the Commonwealth to support institutional programs in equity.

An assessment of equity outcomes in the system needs to be viewed in the context of funding for outreach and access programs at the institutional level. For this reason, the Framework will report a separate ‘Input” domain, with two indicators, one to measure funding for equity group participation and the other to measure associated program development among institutions.



4. Criteria for Selecting Indicators In line with one of the recommendations within the Measurement Framework for Equity (MFE) proposed by the Australian Institute of Health and Welfare, the Indicators informing the Framework meet the ‘SMART’ criteria first proposed by Doran (1981). That is, they are:

• Specific – they all target higher education equity in general, as well as specific, Domains;

• Measurable – they identify the source of the data; • Assignable – to the relevant organisation/jurisdiction; • Realistic – in that they rely on existing data sources that are controlled by reputable

organisations most likely to be in a position to provide data systematically, and over the long term; and

• Time-related – in that the data are regularly reported. These criteria also address key expectations of the Australian Government Department of Education and Training; namely that the proposed framework:

• is focussed on student equity; • is policy-relevant; • draws on existing data and statistics; • recognises that contextual factors are important to policy and performance; • has technical validity and reliability; and • is feasible and practical.

Although not explicitly recommended by the Department of Education and Training in the scope of this project, development of the Framework also took into account the recommendations of the recent Review of Reporting Requirements for Universities, which considered ways to ameliorate the reporting burden placed on universities. A key message from the Review is that both the Department and universities seek ways to enhance and streamline current reporting processes, rather than create new ones (PhilipsKPA, 2012). For this reason, the final set of indicators rely on existing data, and are sourced from key (primarily, educational or statistics collection) organisations who have in place rigorous and systematic data reporting processes and protocols.

Sections 5 to 7 below report on the definition, structure, availability, and testing thereof, for each of the major indicators (and sub-indicators) in Tiers 1 to 3 in the Framework. In each case the outcomes from the testing component of the Wider Consultation and Testing phase are reported (“Outcomes from the Testing”).

More information on the collection of data for each indicator series can be found in the Data Indicator Dictionary.



5. Tier 1 (Context): Domains and Indicators

Introduction Data reported in Tier 1 (Context) provide the critical context for qualifying the performance of the higher education sector in respect to equity. It does this by providing ‘snapshots’ of the extent to which the six equity groups experience educational disadvantage at the pre-tertiary years and whether the gap between their outcomes and those of the wider population are narrowing or widening.

In general, the six identified equity groups differ substantially in terms of their identification at this level of education:

• LSES students – similar definitions across the education hierarchy, however there is a mix of measure types ranging from parental status in pre-higher education reporting to area measures on the basis of permanent home address at the higher education level;

• Students with disability – ‘disability’ is not consistently defined across the educational spectrum, certainly not in a clinical sense, and tends to be self-reported;

• Indigenous students – identification similar to that in higher education, primarily on a self-reporting basis;

• Students from regional and remote areas – based on permanent home address in higher education, but more generally is assigned on the basis of school location elsewhere;

• WINTA – not applicable at the pre-higher education level as this equity group are defined in terms of their higher education enrolment status; and

• NESB – A similar definition is applied at the pre-higher education level, with students classified on the basis of ‘language background’ of their household;

Assessments are made across the first three Domains:

• Domain 1: Early childhood development); • Domain 2: Primary education); and • Domain 3: Secondary education.



Domain 1: Early childhood development Data source(s): Australian Early Development Census (AEDC); Productivity Commission. Number of Indicators: 2


1.01 (Early childhood development)

The Framework sources measures for early childhood development from data collected as part of the Australian Early Development Census (AEDC), a census that helps create a snapshot of early childhood development in Australia. Every three years since 2009, the Australian Government has undertaken a census of all children in their first year of full-time schooling, with the next census due in 2015.

The AEDC is administered by the Social Research Centre (SRC), currently based in Melbourne, but moving to the Australian National University by 2016.

Data for the AEDC is collected from teachers across Australia who complete an online checklist based on their knowledge and observations of the children in their class in their first year of full-time school. The checklist contains approximately 100 questions, which measure five important areas of their early childhood development. These five areas4 are:

• physical health and wellbeing; • social competence; • emotional maturity; • language and cognitive skills; and • communication skills and general knowledge.

4 The AEDC also refers to the areas as ‘domains’. However for the purposes of the Framework the term area will be used, so as to avoid confusion with the Framework’s own Domains.

Number Title What it measures Why it is important 1.01 Early

childhood development

• Physical health and wellbeing • Social competence • Emotional maturity • Language and cognitive skills, including

literacy and numeracy • Communication skills and general knowledge

All five areas are closely linked to the predictors of good education outcomes


• Attendance rates in pre-school education A high rate of participation in primary education, (including the pre-school year) is critical for higher education success



For each area, children are reported as being ‘on track,’ ‘developmentally at risk,’ or ‘developmentally vulnerable.’ The data are collected within the various state and territory jurisdictions and overseen and aggregated by the AEDC, working within the Department of Education and Training. In the 2012 data collection, information was collected on 289,973 Australian children representing 96.5 per cent of children in their first year of formal full-time schooling (Australian Early Developmental Index, 2013).5 AEDC data are collected for individual children and reported at a group level (national, State/Territory or community). Demographic information collected makes the AEDC ideal for inclusion in the Framework, as meaningful comparisons can be made with all higher education equity groups. For example, in the 2013 report it was established that:

• females were less likely to be developmentally vulnerable in one or more areas compared with males;

• Indigenous children were more than twice as likely to be developmentally vulnerable than non-Indigenous children;

• children not proficient in English were more likely to be developmentally vulnerable on all the AEDC areas;

• children who reside in very remote parts of Australia are more likely to be developmentally vulnerable; and

• children living in the most socio-economically disadvantaged Australian communities are more likely to be developmentally vulnerable on each of the AEDC areas (Australian Early Developmental Index, 2013).

Outcomes from the Testing Discussions with the AEDC have established that standard reports for four of the six higher education equity groups can be provided for the Framework. The two exclusions were:

• The WINTA equity group is not applicable at this stage in the framework; and • Disability status being reported by teachers and therefore subject to higher degrees

of subjectivity, data for this category were considered not reliable enough for comparative purposes.

The testing of the sourcing of AEDC data saw the SRC generate the reports required for the Framework, rather than the Department of Education and Training generating the reports itself from the raw data, to minimise, or possibly avoid altogether, the need to change privacy and/or data reporting requirements.

The findings of the testing confirm that national and state/territory collections of AEDC three-yearly data are possible across five defined areas and for all equity groups except WINTA and students with disability.

5 Previously the AEDC was known as the Australian Early Development Index (AEDI) with the name change occurring in 2014. Data collected earlier than this will therefore refer to the AEDI.



1.02 (Participation in Pre-School)

Another measure of early childhood development and opportunity is participation in pre-school education.

Presently, data on pre-school education participation is collected by the Productivity Commission as part of its annual publication, Report of Government Services (ROGS) which includes a volume (Volume B) on childcare, education and training (see Productivity Commission 2014). This includes a measure of the number and proportion of population of children aged 4 and 5 years attending a preschool program in the year before full time schooling, with disaggregation across state and broad equity group – LSES, Indigenous and children with special needs.

Outcomes from the Testing Data for this indicator and its state and territory equivalents can be sourced directly from the Productivity Commission’s reporting of ROGS data. The specific equity groups, defined at this level, which can be reported from this data set are children from low socioeconomic and Indigenous backgrounds, and students with special needs (Disability).

Ongoing annual sourcing of these data can take place directly from public reporting at the ROGS site: http://www.pc.gov.au/research/recurring/report-on-government-services .

http://www.pc.gov.au/research/recurring/report-on-government-services



Domain 2: Primary education Data source(s): Australian Curriculum, Assessment and Reporting Authority (ACARA) Number of Indicators: 3

Number Title What it measures Why it is important 2.01 Reading

performance (NAPLAN)

Achievement in NAPLAN Reading at Year 3 and Year 5 level

Achievement in literacy and numeracy is critical for higher education success

2.02 Numeracy performance (NAPLAN)

Achievement in NAPLAN Numeracy at Year 3 and Year 5 level

2.03 School Attendance

Student attendance rates in Year 3 and 5. School attendance is critical to present and future educational success.


2.01 (Reading Performance (NAPLAN), Years 3 and 5); 2.02 (Numeracy Performance (NAPLAN), Years 3 and 5).

The Australian Curriculum, Assessment and Reporting Authority (ACARA) is responsible for the Australian Curriculum from Kindergarten to Year 12; being divided into Foundation – Year 10 and then the Senior Secondary Curriculum. The Australian Curriculum sets consistent national standards to improve learning outcomes for all young Australians. It sets out, through content descriptions and achievement standards, what students should be taught and achieve as they progress through school.

ACARA also administers the National Assessment Program – Literacy and Numeracy (NAPLAN). The associated reports provide an ideal opportunity for analysis, evaluation and research consistent with the aims of the Framework. ACARA have a defined Research and Data Committee which oversees a well-established data access program. Through ACARA, detailed data regarding the academic performance of all students throughout the primary education years is made regularly available at the commencement of the following calendar year.

The indicators measure two key determinants of academic success:

1. Literacy – focussing on the sub-area of Reading, and excluding Persuasive Writing, Spelling, and Grammar and Punctuation; and

2. Numeracy.

NAPLAN results are reported using five national achievement scales, as well as reporting students as being below, at or above the national minimum standard relative to the year level. NAPLAN results from one year can be compared with those for previous years,



meaning that cross-cohort comparisons are possible. It also allows NAPLAN tests in different years to be reported on the same achievement scale, albeit with some degrees of statistical error.

It has potential to be an effective and valid means of identifying significant differences in achievement across equity groups and the whole population. To minimise, or possibly avoid altogether, the need to change privacy and/or data reporting requirements, ACARA would generate the reports required for the Framework either through its existing public releases, or if possible, the Framework manager could source the data directly from ACARA’s NAPLAN website. Data confidentiality will prohibit some reporting of disaggregated information about equity group performance in NAPLAN, both currently and in the future. This also applies to the relevant Domain 3 indicators.

In addition to the NAPLAN numeracy test, there is an international test of math and science abilities for eight year olds: the Trends in International Mathematics and Science Study (TIMSS) test which is conducted every four years and reported at the national basis in Australia. However, prior to testing it was determined that the NAPLAN test will provide a stronger representation of the performance at the Year 3 level and allow for a disaggregation of outcomes across equity groups.

Outcomes from the Testing ACARA provided the data on both Year 3 and 5 tests for Reading and Numeracy for the following sub-groups for 2014:

• Sex, by State and Territory; • Indigenous Status, by State and Territory; • Language Background Other than English (LBOTE) Status, by State and Territory; • Geolocation, by State and Territory; • Indigenous Students/Non-Indigenous Students, by Geolocation, by State and

Territory; • Parental Education, by State and Territory; and • Parental Occupation, by State and Territory.

These data can also be sourced directly, via search, from ACARA’s website reporting NAPLAN outcomes: http://reports.acara.edu.au/Home/Results.

http://reports.acara.edu.au/Home/Results



2.03 (School Attendance – Primary)

Aside from academic performance, the other core measure of educational preparedness in primary school is that of attendance. The Productivity Commission (2014) reports data sourced from ACARA on average attendance rates for students in Years 1 to 10 in ROGS. While more disaggregated data is not available (state and territory or equity group membership), this collection has potential to be expanded.

Outcomes from the Testing Testing saw the sourcing of data from ROGS on school attendance by Indigenous/non-Indigenous status for all Australian States and Territories. Reporting for other equity groups does not take place.

Ongoing annual sourcing of these data can take place directly from public reporting at the ROGS site: http://www.pc.gov.au/research/recurring/report-on-government-services.




Domain 3: Secondary education Data source(s): Australian Curriculum, Assessment and Reporting Authority (ACARA) Number of Indicators: 6


3.01 (Reading Performance (NAPLAN), Years 7 and 9); 3.02 (Numeracy Performance (NAPLAN), Years 7 and 9).

As with primary education, the rationale for using ACARA and NAPLAN data remains relevant for the secondary education stage.

Outcomes from the Testing ACARA provided the data on both Year 7 and 9 tests for Reading and Numeracy for the following sub-groups for 2014:

• Sex, by State and Territory; • Indigenous Status, by State and Territory; • Language Background Other than English (LBOTE) Status, by State and Territory; • Geolocation, by State and Territory; • Indigenous Students/Non-Indigenous Students, by Geolocation, by State and

Territory; • Parental Education, by State and Territory; and • Parental Occupation, by State and Territory.

Number Title What it measures Why it is important 3.01 Reading

performance Achievement in NAPLAN Reading at Year 7 and Year 9 level

Achievement in literacy and numeracy is critical for higher education success 3.02 Numeracy

performance Achievement in NAPLAN Numeracy at Year 7 and Year 9 level

3.03 School Attendance

Student attendance rates in Year 7 and Year 9. School attendance is critical to present and future educational success.

3.04 Year 12 completion

Proportion of students completing Year 12 studies Prior academic success is the main predictor of higher education participation and achievement 3.05 ATAR Proportion of students achieving high levels of

academic success, as measured by Australian Tertiary Admissions Rank (ATAR)

3.06 Achievement in advanced maths and science (PISA)

As measured by the proportion of students passing advanced maths and science classes at age 15 years – PISA test



These data can also be sourced directly, via search, from ACARA’s website reporting NAPLAN outcomes: http://reports.acara.edu.au/Home/Results.

3.03 (School Attendance – Secondary)

The measure of secondary school attendance was sourced from Productivity Commission data, as per the comparable Domain 2 indicator.

Outcomes from the Testing Data were sourced directly from ROGS on school attendance by Indigenous/non-Indigenous status for all Australian States and Territories. Reporting for other equity groups does not take place.

Ongoing annual sourcing of these data can take place directly from the ROGS site: http://www.pc.gov.au/research/recurring/report-on-government-services.

3.04 (Year 12 Completion)

Transition from Year 12 is the most common pathway into higher education in Australia. For example, in 2014 Year 12 applications represented 55.6 per cent of total applications forwarded through the TACs (Department of Education, 2014). Consideration was also given to reporting Year 10 completion rates. However, it is a requirement in all states and territories that students cannot leave school until they are 17 years old, although from Year 10 this can include options to participate in full-time education, training or employment, or a combination of these activities, until they are 17 years old. Assessing rates of non-full time participation in education from Year 10 would be only a partial way of measuring future higher education participation. However since Year 12 is the more critical stage for articulation into higher education, Year 12 completion rates are the key area for monitoring relative performance. Put another way, measuring Year 10 and Year 12 completions can be viewed as opposite sides of the coin: the former measures equity students less likely to access higher education and the latter measures those students more likely. Therefore, in line with the desire to minimise unnecessary reporting, only Year 12 completion is measured for the Framework.

The Productivity Commission (2014) reports unpublished data from the Department of Education and Training on Year 12 completion rates across states and territories in ROGS.

Outcomes from the Testing Data were sourced directly from ROGS. Data are reported on the basis of by socio-economic status and geolocation at the state and territory level only. Currently, these data are not fully comparable at the national level.

Ongoing annual sourcing of these data can take place directly from the ROGS site: http://www.pc.gov.au/research/recurring/report-on-government-services.






3.05 (ATAR)

Measurement and reporting of ATAR is located in Domain 3, rather than Domain 4 (Aspirations to higher education) or Domain 5 (Access to higher education). For the purposes of the Framework, the ATAR is viewed as an outcome particular to the secondary education sector.

The ATAR is a rank, not a score, in that it shows a student’s ranking relative to all students also sitting relevant Year 12 exams that year. For example, an ATAR of 75 means that the student performed as well as or better than 75 per cent of all Year 12 students that year. As entry into most higher education studies is merit-based, the higher a student’s ATAR, the more likely it is they will be made an offer to undertake higher education studies.

In many cases, an ATAR is also calculated for many non-Year 12 students, based on a variety of academic undertakings including:

• completion or partial completion of other higher education studies; • completion of Year 12 in another country; or • undertaking alternative admission tests, such as the Special Tertiary Admission Test

(STAT).

In these cases, a proxy ATAR is assigned to the student, which is meant to represent their position in the cohort if they had undertaken Year 12 studies. However, this is sometimes imperfect science and in reality proxy ATARs are often treated more as a ‘score’ than an actual ranking. Furthermore, scores are not always made uniformly across or even within states and territories.

Outcomes from the Testing As noted above, ATAR outcomes are not uniformly reported across Australia, especially at the sub-aggregated level. The exception is in relation to Year 12 applications and for that reason, ATAR results by equity group are reported there, but only for the subset of students who attain an ATAR and then apply to university. 3.06 (Achievement in Advanced Maths and Science – PISA)

The wider consultation saw many respondents recommend the inclusion of results for maths and science testing from the PISA (Programme for International Student Assessment) test of maths, science and literacy skills.

The PISA test for scientific, mathematical and literacy skills is an international test administered to 15 year olds on a three yearly basis. Data are reported at the national and jurisdictional level, and by Indigenous status, regional and remoteness classification, socioeconomic status and language background.

The PISA has been adopted as part of the Framework despite its limitations because it provides an international comparator to the NAPLAN results for secondary students –



reflecting both their preparedness for post-compulsory study as well as cumulative primary and secondary education quality in Australia.

The test is managed by the Australian Council for Educational Research (ACER).

Outcomes from the Testing Results from the tests were sourced from ACER at: http://www.acer.edu.au/ozpisa/statistical-tables-pisa-2012. For PISA, national level data were sourced, with sub-national reporting on the basis of Indigenous status, regional and remoteness classification, and socioeconomic status.

Summary of Tier 1 Indicators:

• Overall: Necessarily, the Tier 1 (Context) indicators will report on general trends in the general population with some reference to similarly defined equity groups as those reported for higher education.

• Availability: The Tier 1 indicators are relatively easily sourced from primary data

collections of the SRC (AEDC) and ACARA (NAPLAN) and the secondary data collection of the Productivity Commission (ROGS) and ACER (PISA website).

• Future Access: Access to Tier 1 indicators will require interrogation of two

publicly available data sources – the NAPLAN and ROGS websites – on an annual basis, with access to AEDC triennial reporting (latest year: 2015) can be established with the SRC.

• Data Gaps: Data on Year 12 completion and ATAR collections are not readily

available at the national level on a consistent basis, with sourcing from state and territory institutions.

http://www.acer.edu.au/ozpisa/statistical-tables-pisa-2012



6. Tier 2 (Performance): Domains and Indicators

Introduction Tier 2 (Performance) provides a picture of the performance of the Australian higher education sector itself, both in the context of equity group performance compared with that of the overall student population, but also in terms of geography (state differences), institutional groupings and level of study (sub-bachelor, bachelor, postgraduate and HDR), where possible.

This performance is contextualised against the relative disadvantage already experienced by equity groups of students, as outlined in Section 4. Assessments are made across three Domains:

• Domain 4: Aspirations for higher education; • Domain 5: Access to higher education; and • Domain 6: Achievement in higher education.

Domain 4: Aspiration towards higher education Data source(s): Tertiary Admission Centres (via the Department of Education and Training) Longitudinal Survey of Australian Youth (LSAY) Number of Indicators: 3


4.01 (Intentions to Undertake Higher Education Studies)

Number Title What it measures Why it is important 4.01 Intentions to

undertake higher education studies

Students indicating an interest/desire to undertake higher education studies post-compulsory education

Identifying where/when/why early disengagement with higher education occurs allows for better provision of remedial/support programs

4.02 Year 12 applications

Applications made by Year 12 students to study higher education

Applications are the key source of quantitative data to assess immediate desire to access higher education

4.03 Non-Year 12 (non-school leaver) applications

Applications made by non-Year 12 students to study higher education



Aspirations to higher education do not start in the upper secondary years. In one recent Australian study, it was found that students whose parents wanted them to attend university were four times more likely to complete Year 12 and around 11 times more likely to plan to attend university compared with those whose parents expected them to choose a non-university pathway. Peers also had a strong influence: students whose friends planned to attend university were nearly four times more likely to plan to attend university themselves (Gemici, Bednarz, & Karmel, 2014). This accords with other research (see for example Jacobs & Harvey, 2005).

Unfortunately however, reliable data sources are limited. In Australia, the most reliable source of data tracking higher education aspirations across Domains 2 and 3 is the Longitudinal Survey of Australian Youth (LSAY), which has in the past collected data regarding student expectations and aspirations (e.g. Gemici et al., 2014. above).

Outcomes from the Testing The LSAY survey was found to be in temporary abeyance, with the last study cohort commencing in 2009 at age 15. Due to funding constraints, the LSAY will only be re-activated in the context of PISA collections and only then if funding allows.

Other data sources, such as the ABS’s Survey of Education and Work and state-based school leaver surveys provide transitions information but generally for a limited period of time post-school (ABS, 2014a), with other sources such as the Longitudinal Survey of Australian Children (LSAC) also offering potential as indicator sources.

However, there is potential for these existing sources to include more detailed questions and larger samples to characterise intention to study in a higher education institution.

An alternative is to develop and implement a bespoke aspirational survey that will meet the needs of the Framework. A proposed survey to examine ‘widening participation’ in Australian schools is an example of such an approach. In general terms this could take the form of a nation-wide survey of students at particular times in their lives: such as Years 5, 7 and 10. The two primary forms the survey might take are:

1. Longitudinal Cohort – i.e. following the same students across various years. This would provide actual, rather than inferred aspirational data; that is an individual’s intentions to complete higher education (changing or otherwise) could be tracked over time. The major disadvantage of this approach would be cost: due to attrition significantly large numbers of students would have to be identified at the stage of the first survey, to ensure sufficient numbers of students remained for the final survey.

2. Representational – i.e. selecting different students for each survey. This is a more efficient method of surveying; however actual comparisons over time/surveys are not possible.

The format, timing and organisation responsible for administering such a survey would have to be investigated in further detail and the cost balanced against the relative benefit to the



Framework. The 4.01 indicator is, in a sense, an outlier in the Framework as it measures an intended outcome, not an actual outcome as do the other Indicators (with the exception of Domain 7). That is, it measures an educational outcome expressed, not yet achieved, by an individual. In some cases this expression is false. For example, recent research suggests that at certain life stages, quantitative aspirational information, such as individuals responding to questions regarding their future tertiary intentions, may overstate the reality, expressing an aspiration that they think they should desire, rather than something to which they actually aspire (Parker, Stratton, Gale, Rodd, & Sealey, 2013).

4.02 (Year 12 Applications); 4.03 (Non-Year 12 (non-school leaver) Applications)

Ultimately, the most immediate and reliable source of data information on higher education aspirations that is suitable for the Framework, is the applications data collected by the various tertiary admission centres (TAC), including those institutions who do not use the services of a TAC but who currently submit data to the Department of Education and Training as part of this process. Applications occurring outside the TAC system are more problematic as there is no uniform protocol for institutions to collect and report these – typically only offers, acceptances and enrolments are reported. The planning implementation phase of this project can examine remedies for this issue and advice will be sought from stakeholders in regard to it.

The official TAC collections on applications inform the remaining two Indicators for this Domain – 4.02 (Year 12 Applications) and 4.03 (Non-Year 12 (non-school leaver) Applications). These measure the actual applications made to access higher education in the year following application. Each year, approximately 70 per cent of applicants accept an offer to enrol in higher education (Department of Education, 2013). The actual proportion is even higher, as this excludes students deferring for 12 months. Separation of Year 12 and non-Year 12 applications is important as prior research shows disadvantaged students are disproportionately represented in the non-Year 12 pathway (c.f. Aird, Miller, van Megan, & Buys, 2010; Watson, 2005).

Outcomes from the Testing Data on TAC and Direct applications were sourced from the Department of Education and Training for the following collections in 2014:

• TAC applications by SES and Year 12 status; • TAC applications by Indigenous status and Year 12 status; • TAC applications by Region and Year 12 status; • Direct applications by SES status; • Direct applications by Indigenous status; and • Direct applications by Region.



Data were sourced for Table A providers only, with other higher education providers not reporting through the TACs or on direct applications data to the Department.



Domain 5: Access to higher education Domain 5 is one of the current key focuses of higher education equity performance measurement. Under the proposed Framework, existing Indicators will be strengthened, additional enrolment data will be collected and new inputs will be added. Data source(s): Tertiary admission centres (via the Department of Education and Training) Higher education institutions (via HEIMS) Number of Indicators: 6


5.01 (Offers made to students); 5.02 (Acceptances by students)

Data on offers and acceptances from TACs and direct applications can be sourced from the Department of Education and Training. Feedback from wider consultation and testing recommended that data on Acceptances not be reported as this overlaps too heavily with Commencements. Therefore, the Framework now excludes data on the acceptances of offers via TAC or direct application. Outcomes from the Testing Data on offers from TAC and institutions (Direct) were sourced from the Department of Education and Training for the following collections in 2014:

• TAC offers by SES and Year 12 status; • TAC offers by Indigenous status and Year 12 status;

Number Title What it measures Why it is important 5.01 Offers made to

students Undergraduate offers made to students Offers indicate equity students

are gaining access to higher education

5.02 Acceptances by students

Students accepting undergraduate offers Acceptances indicate students are receiving offers

5.03 Commencements (new students)

New students enrolling in higher education Enrolments indicate actual access into higher education via various pathways 5.04 Enrolments (all

students) All students (re)enrolling in higher education

5.05 Student Satisfaction

All students (re)enrolling in higher education

Indicates initial student satisfaction with university enrolment.

5.06 Course transitions

Students transitioning from one course level to another, for instance, sub-bachelor level to a bachelor level course

Establishes data collection on pathways between post-school education program offerings.



• TAC offers by Region and Year 12 status; • Direct offers by SES status; • Direct offers by Indigenous status; and • Direct offers by Region.

Data were sourced for Table A providers only, because other higher education providers do not report through the TACs or direct applications data to the Department. 5.03 (Commencements (New Students)); 5.04 (Enrolment (All Students))

Currently, the Department of Education and Training releases annual reports and data in two key areas:

1. Undergraduate Applications Offers and Acceptances Publications – providing summaries and data sheets for applications made through TACs as well as applications made directly to universities, including statistics on:

o applications; o offers; o ATAR; and o under-represented groups.

2. Selected Higher Education Statistics – including information on equity groups, namely:

o sector-wide enrolments (commencing and all, for both undergraduate and all students); and

o the same data at the institutional (Table A) level. These data and statistics will be integrated into the Framework with some enhancements. For instance, statistics for equity groups are collected as part of the Undergraduate Applications Offers and Acceptances Publications process (Indicators 5.01 and 5.02). Relevant demographic data are currently collected by TACs and the institutions, and are affected by the same limitations as when the data are collected elsewhere.6 Thus, no new data need be collected; data only needs to be extrapolated from the current sets. It does not seem feasible to include data on offers at other course levels as these are currently not centrally collected. Reporting of equity enrolments by Broad Field of Education will be included in the Framework for all equity groups, not just WINTA, because of the relationship between field of study and graduate earnings. In 2012, for example, the median starting salary for a bachelor-degree graduate was $80,000 for dentistry, $63,000 for engineering and $50,000 for social work (Graduate Careers Australia, 2013). Equity in and through higher education

6 That is, the postcode of the permanent address reported by the applicants is used to derive geo-based equity data; and Indigenous and disability status are self-reported.



relates not only to access in its general sense, but also access to particular courses of study. Finally, to assist with more accurate reporting of equity students in Domains 5 and 6, it is proposed that the Framework take advantage of existing functionality as part of institutional submissions through HEIMS. The primary benefit will be to ameliorate the problems associated with geo-coded equity data. Currently, at each enrolment period a student’s address is updated, including the postcode of their residence. This means as studies progress, the ‘identity’ of many LSES, regional and remote students is lost. However, HEIMS now (as of 2014) records the ‘commencing’ address (including postcode) for a student when a new enrolment is created and this is linked to a student’s Commonwealth Higher Education Student Support Number (CHESSN). This is a much more accurate indication of the student’s equity status, as it does not change throughout their studies. This change will not require institutions to collect any additional data; it represents only a way in which the data are managed and reported through HEIMS. Consequently, more accurate data will underpin Indicators 5.03 and 5.04. Outcomes from the Testing Data on commencements and enrolments of students, by equity group, are currently published by the Department of Education and Training as part of its Higher Education (Students) (Appendix 2 – “Equity Groups”). A specific data request was placed with the Department in regards to sourcing data at the disaggregated level for all higher education providers. Data on commencements and enrolments of students were sourced from the Department of Education and Training for all higher education providers in 2013 (the latest year for which all data was available at the time of testing):

• Commencing Domestic Students by State, Institution and Equity Group; • Commencing Domestic Undergraduate Students by State, Institution and Equity

Group; • All Domestic Students by State, Institution and Equity Group; • All Domestic Undergraduate Students by State, Institution and Equity Group; • Commencing Domestic Students by Broad Field of Education and Equity Group; • Commencing Domestic Undergraduate Students by Broad Field of Education and

Equity Group; • All Domestic Students by Broad Field of Education and Equity Group; • All Domestic Undergraduate Students by Broad Field of Education and Equity

Group; • Commencing Domestic Students by Mode of Attendance and Equity Group; • Commencing Domestic Undergraduate Students by Mode of Attendance and Equity

Group; • All Domestic Students by Mode of Attendance and Equity Group;



• All Domestic Undergraduate Students by Mode of Attendance and Equity Group; • Commencing Domestic Students by Basis of Admission and Equity Group; • Commencing Domestic Undergraduate Students by Basis of Admission and Equity

Group; • All Domestic Students by Basis of Admission and Equity Group; and • All Domestic Undergraduate Students by Basis of Admission and Equity Group.

5.05 (Student Satisfaction)

The best available indicator of student perception of institutional performance in relation to the enrolling experience and preliminary exposure to higher education teaching and interaction is the University Experience Survey (UES). Presently, the UES is used to collect data on student satisfaction in the first year of their enrolment. Outcomes from the Testing The UES provides data at the institutional and equity group levels for the ‘satisfaction’ question in its survey. The testing, occurring through Graduate Careers Australia (GCA), confirmed that results were available at the aggregate level and for larger institutions with a reduced ability to report for smaller institutions both at the aggregate and individual equity group level. 5.06 (Course Transitions)

Focus in higher education is currently being place on course transitions, specifically articulation between course levels. This is especially true to articulation from sub-bachelor programs, although other transitions will gain prominence in different policy contexts. In 2014, the Minister for Education, as part of the 2014-15 Federal Budget submission, advised that both the demand-driven system and HECS-HELP provision would be extended to sub-bachelor places. At the time this document was published, the recommendations were still awaiting Senate consideration. The following assumptions therefore guide the proposed Framework in this respect:

1. The plan to extend the demand-driven system will be expanded to include sub-bachelor places.

2. HECS-HELP will be extended to cover these places for eligible students. 3. The requirement to issue a CHESSN for these new Commonwealth-supported

places will apply. 4. These arrangements will extend to cover all higher education (i.e. Table A and non-

Table A) providers. The sub-bachelor provision is particularly important from an equity perspective, given its function as an alternative pathway to undergraduate studies. For this reason particular attention will need to be paid to sub-bachelor to bachelor articulations within the



Framework. Tertiary education equity policies emphasise the importance of educational pathways that facilitate student transfers from institutions in lower-status, vocationally oriented ‘second’ tiers of tertiary education to institutions in higher status ‘first’ tiers of higher education, particularly universities. This is because students from disadvantaged backgrounds are more likely to go to the former, whereas more privileged students are more likely to go to the latter (Wheelahan, 2009). Critically, transitions to university for disadvantaged students are more likely to result from completing a higher-level VET qualification, but disadvantaged learners tend to enrol in lower-level qualifications (Griffin, 2014). A primary intention of extending the demand-driven system to the sub-bachelor level is to encourage successful transitions by increasing academic preparation for particular students before they enter a bachelor degree course; and providing a lower risk entry point for LSES students (Kemp & Norton, 2014). Future Testing Requirement Should the demand-driven system be extended to include sub-bachelor places, then it is assumed the use of the CHESSN will be similarly extended. This will provide an ideal means of capturing successful transitions from sub-bachelor to bachelor level. Therefore, whilst Indicator 5.05 is reliant on new data and data protocols, based on current processes it can be reasonably assumed that the quality of both will be high.



Domain 6: Achievement in higher education Data source(s): Higher education institutions (via HEIMS) Number of Indicators: 3


6.01 (Student Retention); 6.02 (Student Success); 6.03 (Student Completion)

The transition from pre-tertiary to tertiary education is a challenging one for many disadvantaged students. Research into the first year experience in Australian universities indicated students from rural areas and LSES backgrounds believed they were less-prepared in their first year than the overall population (James, Krause, & Jennings, 2010). Currently the Department of Education and Training publishes data for domestic commencing bachelor students by State and Higher Education Institution on:

• attrition rates; • success rates; • retention rates; and • course completions.

These statistics do not currently provide detail on equity groups, although WINTA equity students can be identified from the course completions publication. However, current data collection and reporting processes should be able to accommodate additional reporting of equity students for the Framework. It is recommended that the reporting on retention rates be adopted for Framework purposes, in place of the attrition rate which is its mirror. At an institutional level, retention and success rates vary considerably across the sector. For instance, in 2012, institutional adjusted success rates ranged from 70.23 to 93.49. For

Number Title What it measures Why it is important 6.01 Student

retention • The proportion of students who commenced

a bachelor course in year(x) who either complete or return in year(x + 1). Adjusted on a match process using both the Student ID and CHESSN.

Retention indicates the extent to which students are successfully matched with courses

6.02 Student success

• The proportion of actual student load (EFTSL) for units of study that are passed divided by all units of study attempted (passed + failed + withdrawn)..

Success indicates the extent to which students perform in their chosen course.

6.03 Student completion

• The proportion of specific cohorts of students completing by level of course, broad field of education

Achievement in higher education is related both to higher education completion and field of study.



the purposes of the Framework therefore, comparisons will be most meaningful at the institutional level. In the past, completion data for equity students has been problematic, due in great part to changes to student addresses over the years taken to complete affecting the validity of geo-coded equity data. The inclusion on student records of fields such as ‘commencing postcode field’ could go some of the way to addressing this issue. Outcomes from the Testing Data on retention, success and completion are published by the Department of Education and Training as part of its Equity Group Performance Data (Based on Current Appendix 5 Reporting). A specific data request was placed with the Department in regards to sourcing data at the disaggregated level for all higher education providers for 2013 (the latest year for which all data are available or 2102 in the case of Retention Ratios): For each of “All Domestic” and “Domestic Undergraduate” students:

• Access Rates for Higher Education Providers, 2013; • Participation Rates for Higher Education Providers, 2013; • Participation Ratios for Higher Education Providers, 2013; • Retention Rates for Higher Education Providers, 2012; • Retention Ratios for Higher Education Providers, 2012; • Success Rates for Higher Education Providers, 2013; • Success Ratios for Higher Education Providers, 2013; and • Attainment Rates for Higher Education Providers, 2013.

A key finding from the testing is that NUHEPs report low student numbers for equity groups, even at the aggregate level. As the Department of Education and Training suppresses numerical counts of 1 to 5 students to ensure confidentiality of student data, this leads to a null reporting for these providers, with many NUHEPs also reporting a zero count which does not have to be suppressed. Figure 2 demonstrates this issue and reports on the number of institutions reporting a student count above 5 for each equity group on the basis of their enrolling status – Table A, B and C providers and the NUHEPs. Among the 38 Table A providers (including the Batchelor Institute), at least 37 providers reported a count at this level across all equity groups (regional and remote are split in this table). By comparison, among the 86 NUHEPs, at most, 76 institutions report a count above 5 for a given equity group (in this case, the LSES group), with only 16 institutions reporting at this level for Indigenous students.



Figure 2: Number of Higher Education Providers Reporting a Student Count for each Equity Group

Table A

Providers Table B

Providers Table C


This problem is exacerbated at lower levels of aggregation, for instance in analyses of student access and participation by “Field of Study”, to the extent that such reporting for NUHEPs can only be undertaken at a level aggregating all institutions into a sectoral estimate.




• Overall: Collections for Tier 2 (Performance) can be broadly classified as: (i) Aspirations (Domain 4) and (ii) Higher Education Performance (Domains 5 and 6), with ideally, comparisons being available between the total population and defined equity groups.

• Availability: Data issues in Tier 2 are as follows:

o Data for Domain 4, Indicator 4.01 (Intentions to undertake higher education studies) is problematic. There is no accepted national survey of high school or general population (mature age students) aspirations for attending university. An established source, the LSAY, saw its last commencing cohort (aged 15) in 2009.

o Data for the other indicators in Domain 4, (4.02 Year 12 applications; 4.03 Non-Year 12 applications) are available. However, data for 4.02 and 4.03 are only available for public higher education providers (Table A and B), with private providers not included in applications data.

o Data for all indicators in Domains 5 and 6 can be readily sourced from internal departmental collections. Data are reported through HEIMS for all relevant higher education providers. Availability becomes an issue for these indicators at the public reporting, where the Department maintains a practice of suppressing student enrolment counts at <5 students. This means that many smaller NUHEPs do not return estimates of equity student enrolment.

• Future Access: There is no reliable data presently available for Indicator

4.01(Intentions to undertake higher education studies). Domain 4 indicators (4.02 and 4.03) and Domain 5 and 6 indicators can be sourced on the basis of data templates developed for the Testing phase.

• Data Gaps: There are two data gaps in Tier 2:

o Indicator 4.01 requires the development of a suitable data source, ultimately through the creation of a suitable survey instrument.

o Data for Indicators 4.02 and 4.03 could be collected for all higher education institutions offering Commonwealth-supported places were obligated to use TACs for central applications and also report on direct applications via HEIMS.



7. Tier 3 (Outcomes): Domains and Indicators

Introduction Tier 3 (Outcomes) measures whether or not the socio-economic advantages associated with higher education are being realised by equity students in the same way they are for the overall student population. At the Tier 3 level assessments are made across one Domain:

• Domain 7: Graduate outcomes.

Domain 7: Graduate outcomes Data source(s): Australian Government Department of Education and Training

Graduate Destination Survey (GDS) / University Experience Survey (UES) – Graduate Careers Australia (GCA); Graduate Outcomes Survey / UES – Social Research Centre (SRC)

Number of Indicators: 4


7.01 (Graduate Earnings); 7.02 (Graduate Employment); 7.03 (Graduate Further Study)

The motivations for engaging in higher education study, and the benefits arising from it, are many and varied. However two of the most universally accepted are:

1. Higher education qualifications result in higher-than-average earnings for the individual.

Number Title What it measures Why it is important 7.01 Graduate

earnings Median salaries of graduates, post-graduation. Higher-than-average

earnings are an expectation of higher education studies

7.02 Graduate employment

Employment outcomes for graduates Higher-than-average employment outcomes are an expectation of higher education studies

7.03 Graduate further study

Further study outcomes for graduates Gauges the level of continuation of higher education studies

7.04 Graduate satisfaction

Qualitative assessment of graduate satisfaction with their overall course experience

Student satisfaction is an important measurement of the quality of the higher education experience



2. Completion of an undergraduate qualification is the general pre-requisite for undertaking further postgraduate study, which in turn leads to improved earnings and socio-economic outcomes for the individual.

Benefits from higher education arise both for the individual student (in terms of higher lifetime earnings) and for the broader society (in terms of lower unemployment and higher productivity). The rationale for government to invest in higher education is that it contributes to a more productive workforce that is skilled and flexible, leading to higher wages and lower unemployment resulting in higher tax revenues, reduced unemployment expenses and improved international competitiveness (National Commission of Audit, 2014). Currently, data do exist via the Graduate Destinations Survey (GDS) and Beyond Graduation Survey (BGS), administered by Graduate Careers Australia (GCA). These surveys provide detail regarding:

• graduate employment outcomes; • previous employment; • continuing study; • work-seeking status; • work-seeking behaviour; • past education; and • key respondent characteristics (e.g. recent qualifications, residency status, etc.).

The GDS is administered approximately four months after graduation on behalf of the Department of Education and Training, with its final year of collection being 2015 (with reporting in 2016), after which it will be replaced by the Graduate Outcomes Survey (GOS) to be managed by the Social Research Centre (SRC). The BGS is administered and managed by the GCA, independently of the GDS. It is sent to students around three years after course completion, although its continuation will depend on ongoing access to the GDS mailing list, as technically the BGS is a follow up survey sent to the GDS list. There are two significant impediments to using the GDS and BGS.

• First, postcode data collected through the survey relate to the graduate’s location at the time the survey was undertaken, not during their studies or while growing up. Therefore it is not possible to generate valid data for LSES, regional and remote students; and

• Second, the number of Indigenous respondents and respondents with disabilities is low.



Based on its 2013 survey, Graduate Careers Australia advise the following sample sizes for various equity groups in the GDS:

Group

Number of survey responses (2013)

General population (Australian domestic bachelor degree (excl. honours) graduates)

39,253

Of these, identifying as Aboriginal or Torres Strait Islander 271 Of these, identifying as having a disability 1,889

The respondent numbers are generally lower still for the BGS as it is a follow-up survey to the GDS, so it has significantly fewer graduates due to attrition in the correspondence list for the original sample as students vacate previous addresses. It may be possible to address the postcode issue by supplying the commencing postcode information to GCA, via the relevant protocols within the Department of Education and Training. However the survey response rates for Indigenous students and students with disability may still be too low for the purposes of the Framework. The GOS is currently at the design and piloting stage, and will be utilised in 2016. It has a number of advantages over the GDS, principally, it is collected electronically which should ensure a faster turnaround time in terms of data provision, and second, it is linked to the HEIMS database via CHESSN, providing for straightforward identification of all equity groups. Testing of the GDS for this project has taken place in view of what will also be possible with the GOS in coming years. Outcomes of the Testing Testing of the GDS data set has found that institutional averages for starting salary, graduate employment, and graduate further study, are reportable at the institutional (Table A) provider level. However, as per the discussion above in relation to sample sizes, generally outcomes at the sub-reporting level, notably for individual equity groups at each institution are unreliable due to small sample size, although GCA’s current reporting on these groups at the aggregate level is acceptable. In relation to the GOS collection, it is likely that given the similar sample size (target) for that survey that an aggregate-level reporting of equity group outcomes is possible, but no reporting on an institutional basis can take place.



However, the GOS does offer the possibility of increasing sample collections for equity groups, as well as defining equity status on the basis of HEIMS data at the institutional level. 7.04 (Graduate Satisfaction) The UES, undertaken since 2012, is the only comprehensive survey of current higher education students in Australia. It provides detailed information on the student experience and satisfaction levels of first and later-year students. Outcomes of the Testing (UES) In line with the findings on the GDS collection, testing of the UES data set has found that institutional averages for student satisfaction are easily reportable at the institutional (Table A) provider level. However, reporting for individual equity groups at each institution is unreliable due to small sample size, although reporting on these groups at the aggregate level is acceptable. A further data source is that of the Employer Satisfaction Survey (ESS), piloted by the Workplace Research Centre at The University of Sydney on behalf of the Department of Education and Training over 2013-14. A report listing the outcomes from the ESS pilot project has been published (Oliver, Freeman, Young, Yu, & Vemma, 2014). The ESS might be able to provide data for the Framework including the employment characteristics of graduates and the relevance of their qualification to their employment. A total of 2,749 graduate interviews and 539 supervisor interviews were completed from four participating universities. Preliminary findings showed “Graduates were even more reluctant to provide their supervisors’ details than originally anticipated” (Oliver et al., 2014, p. 3) and

“Producing robust results with accurate standard errors for reporting purposes at the level of institution by broad field of education will be difficult, requiring a full census of all graduates. Even with a full census, it may only be possible to achieve a sufficient sample size for larger institutions or by compiling results from smaller institutions over several years” (Oliver et al., 2014, p. 4).

Therefore, inclusion of the ESS into the Framework is at this stage premature, though it should be considered as part of the potential to include QILT data more broadly. In September 2014, the Department of Education and Training announced that the SRC had been selected to administer the QILT from 2014 to 2017.




• Overall: The Tier 3 (Outcomes) indicators will be sourced from the current GDS and UES collections to 2016, with the GOS replacing the GDS at that point. These two collections provide data on direct graduate outcomes in terms of employment, earnings, and further study, as well as graduate satisfaction with their institution.

• Availability: Data issues in Tier 3 are as follows:

o GDS/UES collections occur on an annual basis. o The current GDS/UES collection has universal participation among Table

A provider institutions, but very limited responses from other institutions. o Reporting on equity groups in both the GDS and UES collections can take

place at the aggregate level but is unlikely to be of any comparative quality at the institutional level, unless sample sizes are increased.

o The GOS, which will have electronic delivery, among other features, may provide greater scope to include students from a wider range of institutions.

o Equity definition is an issue under the GDS as this is self-reported in the survey and may not necessarily correspond to HEIMS-sourced definitions. The GOS will link to HEIMS via a student’s CHESSN.

• Future Access: Access to the GOS, UES, and prospectively, the ESS is assured

as they are funded through the Department of Education and Training. • Data Gaps: A number of data gaps could be closed with the development of the

new suite of student and employer outcome surveys: o Expansion of collections through the GOS and UES to all higher education

graduates. o The launching of a government-sponsored survey of graduates after three

to five years, along the lines of the current, GCA-managed BGS instrument.



8. Measuring Equity Inputs

The Framework is focussed on equity outcomes; that is the relative performance of disadvantaged students as they progress through compulsory education and higher education, then engage in post-graduation work and further study. At the same time, the provision of equity inputs is critical for improving educational outcomes for these students. Broadly speaking, equity inputs fall into four categories, classified as either funding or programs:

1. Equity-specific funding – for example the proposed Commonwealth Scholarships to provide needs-based scholarships to help meet the costs of education;

2. Equity-enabling funding – for example the Higher Education Loan Program (HELP), which is available to all domestic students, but has been shown to directly improve equity outcomes;

3. Equity-specific programs – for example an outreach program specifically designed to support the higher education aspirations of Indigenous students at the secondary stage of their education; and

4. Equity-enabling programs – for example a Year 12 numeracy development program run for all students in a particular school; or support for Indigenous staff development in higher education.

Examples of all four categories are shown in Figure 3 below. Measuring the performance of equity inputs is far less straightforward than measuring those of equity outcomes. For example, whilst one can say with a high degree of confidence whether enrolments from regional students are increasing over time, it is much more difficult to ascribe the existence of one or multiple equity inputs with a concomitant change in equity outcomes. Further, making a distinction between specific and non-specific funding and programs is often problematic. For example, in Western Australia the State Government recently announced changes to its education funding formula, redirecting approximately $45 million from secondary to primary schools over a five-year transition period. Reasons for the changes included the need to direct more money to Aboriginal and Torres Strait Islander student support and a greater focus on resourcing the early childhood years of education (Collier, 2014). Both are key platforms for eventual higher education equity, but are not labelled equity funding. Further, it is easier to measure equity funding as an input (i.e. its provision) than as an outcome (i.e. what it achieves). This is particularly the case when evaluating the effectiveness of outreach programs. For example, parental support, academic rigour, linkages between educational sectors, and access to technology are all known factors that increase success; however they are less often able to be proven (Swail & Perna, 2001). Furthermore, the bespoke nature of many support programs means that it is difficult to



validate circumstantial evidence by implementing the same program in multiple settings with different populations (Schultz & Mueller, 2006). Ultimately, the uncertain and long-term nature of many support programs means that deriving specific cause and effect behaviour between an initiative and any future improvements in higher education equity is almost impossible.

Figure 3: Types of equity inputs

Equity-specific funding Equity-enabling funding Equity-specific programs Equity-enabling programs

Higher Education Partnerships Programme

Higher Education Loan Program

University outreach programs for prospective equity students

University community outreach programs

Community sourced funding for equity programs and scholarships New Colombo Plan funding

University support programs for current equity students (e.g. bridging courses)

University support programs for current students (e.g. bridging courses)

University bursaries for equity students to finance ongoing course-related expenditures

Support of student organisation initiatives based around equity status

Specific support programs (e.g. support programs for student IT use)

Equity scholarships

Linkages between university and community groups to support equity students

Unique skill based courses (e.g. short-term communications' skills courses)

(Proposed) Commonwealth Scholarships Scheme

Data generation and analytical work to assess equity initiatives

Assistance programs for graduate work placement

Austudy Extra curricula programs for equity students (e.g. sporting programs)

International linkage and overseas study programs

Abstudy Accommodation assistance for equity students

Extra curricula programs for students

Specific infrastructure initiatives for equity students (e.g. infrastructure for students with disability) Accommodation assistance

Student experience surveys

Assistance programs for course-related work placement

However the fact remains that the importance of measuring, reporting and assessing the effectiveness of equity inputs is unquestioned. The issue therefore becomes how the relationship between equity inputs and outcomes is best expressed. Outcomes from Wider Consultation and Testing The approach taken to determine the best sources for input data was two-fold:

• Wider Consultation: Collecting feedback on Version 1.6 of the Framework document (a previous version of this document) from higher education providers and identified stakeholders; and

• Testing: Identifying and collecting data on suitable indicators. Wider Consultation provided few potential sources of data on inputs which didn’t involve separate institutional reporting of input data in equity. Generally, it was found to be difficult to identify specific reporting practices for equity activities alone.



The testing of the Framework focused on the collection of potential input indicators, primarily from data reported to the Department of Education and Training in its collection, Higher Education Publications – Finance Publication (see Department of Education and Training 2015). An application to the Department’s Finance area resulted in the generation of line item data for Table A providers only, which culminated in the production of data for one line item: “equity programmes” which related funding under the Higher Education Participation and Partnerships Programme (HEPPP), now known as the Higher Education Participation Programme (HEPP). In view of this work, the best solution for the collection of input indicators for equity is that of the proposed ‘Single Equity Report’ (SER) as recommended by the PhilipsKPA (2012) in their review and currently being developed and trialled by the Department of Education and Training. The draft elements in the SER include a series of indicators on Commonwealth funding of equity programs in higher education:

• Commonwealth Scholarships Programme; • Commonwealth Scholarship Scheme; • Disability Support Programme; • Higher Education Participation Programme – HEPP (Access and Participation

Funds); • Higher Education Participation Programme - HEPP (Scholarship Stream); and • Indigenous Support Programme.

In addition, information may be collected on a range of activities which are intended to be supported by the funding, including ‘outreach’ activities in schools, tertiary and further education providers (including TAFEs) and other organisations. The SER could form the basis of a broad collection of equity inputs – both funding and activities – with an extension across all higher education providers required to report on equity activities. In its consultation with the sector and the Department of Education and Training, three options were developed for reflecting inputs in the overall Framework:

Option 1: Creating a distinct Tier for inputs; Option 2: Integrating inputs into the existing Tiers; and Option 3: A separate reporting of equity inputs.

An outline of these three options is included in Appendix C of this document.



In practice, Options 1 and 3 are very similar, with the inputs reported in a uniquely defined tier (or domain), either as part of the Framework or in a separate context entirely. Option 2 would involve the joint reporting of both performance and input indicators. The wider consultation on the Framework indicated support among institutions for a separate reporting of input indicators, under either Options 1 or 2, with few preferences for Option 3. A potential input Domain, Domain I, that reports data from the SER for each institution could resemble the following format:

Domain I: Inputs Data source(s): Single Equity Report, Department of Education and Training Number of Indicators: 2


The simplicity of the SER could accommodate the collection of such information from all relevant providers receiving equity funding support, with future extensions creating and implementing relatively straight-forward extensions of these indictors to allow for new funding initiatives or variations on activity programs, for instance the inclusion of student enrolment in enabling and bridging programs.

Number Title What it measures Why it is important

I.01 Higher Education Financial Funding

• Commonwealth Scholarships Programme • Commonwealth Scholarship Scheme • Disability Support Programme • Higher Education Participation Programme

– HEPP (Access and Participation Funds) • Higher Education Participation Programme

– HEPP (Scholarship Stream) • Indigenous Support Programme

Key income support programs for equity in Australian higher education


• Schools • TAFEs • Other Universities • Community Organisations • Other groups

Measures of activity associated with financial inputs.



Summary of the Input Domain:

• Overall: Collection on input indicators at this point can be restricted to data obtained through the SER, as this is the primary collection instrument for Australian institutions. Any expansion in reporting tracking additions to the SER.

• Availability: Data availability for the Input Domain relies solely on the

development and implementation of the SER. • Future Access: Reliant on continued access to the SER. • Data Gaps: The SER could be extended in a number of ways:

o Institutional collection and reporting on equity funding indicators such as:

State government support for institutional equity activities; and Private financial support for institutional equity activities, both in

terms of programs and scholarships. o Institutional collection and reporting on equity activity indicators such as:

Bridging and enabling program enrolments; and Support program development for current university students.



9. Standard Reports

Introduction The general principles underlying the analysis and reporting guidance on Tiers, Domains and Indicators are:

1. At least in the first instance, the Department of Education and Training will be responsible for publishing aggregated and disaggregated data for the entire Framework, including brief commentary and analysis of what the data show.

2. Framework reporting of Domains will not be retrospectively published, however indicative retrospective reporting will occur in the formative years, to help guide the development and implementation of the Framework. Should it be determined that the validity of the retrospective data are appropriate for publication, this can be done at the discretion of the Department of Education and Training.

3. Reports for individual Domains will be published annually, as soon as all available data has been collected and incorporated into standard Framework reports.

4. Reports including inter-Domain analyses will be provided every three years. 5. Stakeholders will be responsible for ‘mixing and matching’ the individual Domain

reports (published annually) for their own purposes. 6. Wherever possible, aggregated and disaggregated data will be provided in formats

that facilitate further stakeholder analysis, such as Portable Document Format (PDF), Excel (XLS) and OpenDocument Spreadsheet (ODS) format.

7. Individual and/or de-identified data, including small (i.e. <5 individuals) data sets will not be published.

8. Specific, tailored requests for data by stakeholders must be made direct to the organisation owning the data, in accordance with the organisation’s own data protocols.



Individual Domain reports Frequency: annually Format: provided in both overview and analysis (i.e. PDF) and statistical table (i.e. XLS, ODS) format. TIER 1


Identifier

Title

Domain (Indicators)

Details

R101 Early childhood development for equity groups

1 (1.01, 1.02)

• Rates and proportional representation of the equity groups in the 5 AEDC areas (all equity groups excluding people with disability and WINTA)

• Includes participation rates in pre-school • Includes State/Territory comparisons • Includes rates of students developmentally vulnerable in one or more

area • Includes rates of students developmentally vulnerable in two or more

areas • Includes information on students experiencing two or more types of

disadvantage that result in statistically significantly different rates - R201 Outcomes in

primary education

2 (2.01 – 2.03)

• Comparative performance of equity groups in NAPLAN at Year 3 and Year 5 level in Reading and Numeracy, including rates of students below minimum national standard (all equity groups excluding people with disability and WINTA)

• Comparative cohort gain (Year 3 to Year 5) for equity groups in Reading and Numeracy

• Includes State/Territory comparisons • Includes comparisons between public and private sectors • School attendance rates

R301 Outcomes in secondary education

3 (3.01 – 3.06)

• Comparative performance of equity groups in NAPLAN at Year 7 and 9 level in Reading and Numeracy, including rates of students below minimum national standard (all equity groups excluding people with disability and WINTA)

• Comparative cohort gain (Year 7 to Year 9) in Reading and Numeracy • School attendance (Indigenous/Non-indigenous only) • Year 12 completion rates (Indigenous/Non-indigenous only)

Comparison of Year 12 completion with the general population • Rates of students enrolled in 1 advanced maths + 2 science subjects

at Year 12 (limited equity group reporting) • ATAR rates by viginitile (5% intervals) • Includes comparisons between public and private sectors • Includes information on students experiencing two or more types of

disadvantage that result in statistically significantly different rates



TIER 2


Identifier

Title

Domain (Indicators)

Details

R401 Intention to access higher education

4 (4.01)

• Proportional rates of equity students intending to undertake higher education studies in the future

• Intentions at defined stages in life (e.g. Year 5, Year 7, Year 10) • Includes comparisons with the general population • Includes information on students experiencing two or more types of

disadvantage that result in statistically significantly different rates • Includes discipline information (i.e. intending to study what) • Includes destination information (i.e. intending to study where)

R402 Applications 4 (4.02, 4.03)

• Rates and proportional representation of equity groups in TAC/other applications

• Includes institutional comparisons • Includes comparisons with the general population • Includes State/Territory comparisons • Includes comparisons by Broad Field of Study • Includes interstate mobility of equity groups • Includes regional-urban mobility of equity groups • For Year 12 applicants, Includes comparisons between public and

private sectors • Includes information on students experiencing two or more types of

disadvantage that result in statistically significantly different rates R501 Offers and

Enrolments 5

(5.01-5.04) • Rates and proportional representation of equity groups in TAC/other

applications, offers and acceptances • Comparative performance of equity groups in enrolments in HE providers • Includes State/Territory comparisons • Includes comparisons with the general student population • Includes institutional comparisons • Includes basis of admission for all enrolments • Includes comparisons by Broad Field of Study • Includes age, gender, and mode of study, load (Part/full time); • Includes level of study – sub-bachelor, bachelor, postgraduate, HDR. • Separates undergraduate enrolments, sub-bachelor enrolments and all

enrolments • Includes information on students experiencing two or more types of

disadvantage that result in statistically significantly different rates R502 Sub-bachelor

transitions for domestic students

5 (5.05)

• Comparative performance of equity groups in sub-bachelor programs • Includes State/Territory comparisons • Includes comparisons with the general population • Includes institutional comparisons • Also successful transition to bachelor programs • Includes comparisons by Broad Field of Study • Includes age, gender, and mode of study, load (Part/full time); • Includes information on students experiencing two or more types of

disadvantage that result in statistically significantly different rates R601 Attrition, success

and completion 6

(6.01-6.03) • Comparative performance of equity groups in higher education retention,

success and completion • Uses a cohort approach (i.e. all students enrolling in year x) at x + 4

years and x + 8 years. • Separates undergraduate enrolments, and all enrolments • Includes institutional comparisons • Includes level of study – sub-bachelor, bachelor, postgraduate, HDR. • Includes information on students experiencing two or more types of




TIER 3

KEY Regular text = Indicator uses existing data and protocols Red-coloured text = Indicator requires data and protocols to be created Blue-coloured text = Indicator uses existing data and protocols but data are potentially too weak for the purposes of the Framework Red text shaded grey = Combination: some requirement for data and protocols to be created but also uses existing data but data protocols do not currently exist

DOMAIN I


Identifier

Title

Domain (Indicator)

Details

R701 Graduate earnings

7 (7.01)

• Median earnings for equity groups 1 and 3 years post completion (AT0) • Includes institutional comparisons • Includes comparisons with the general population • Includes comparisons between narrow fields of study • Includes information on students experiencing two or more types of

disadvantage that result in statistically significantly different rates • Median earnings for equity groups six-months post completion (GDS) • Includes institutional comparisons • Includes comparisons between narrow fields of study • Includes information on students experiencing two or more types of


R702 Graduate satisfaction

7 (7.02)

• Comparative performance of equity groups in higher education satisfaction

• Includes institutional comparisons

Identifier

Title

Domain (Indicator) Details

I.01 Higher Education Financial Funding

• Commonwealth Scholarships Programme • Commonwealth Scholarship Scheme • Disability Support Programme • Higher Education Participation Programme –

HEPP (Access and Participation Funds) • Higher Education Participation Programme -

HEPP (Scholarship Stream) • Indigenous Support Programme

• Key income support programs for equity in Australian higher education


• Schools • TAFEs • Other Universities • Community Organisations • Other groups

• Measures of activity associated with financial inputs.



Inter-Domain reports Frequency: every five years Format: provided in both overview and analysis (i.e. PDF) and statistical table (i.e. XLS, ODS) format Higher education equity performance is relative. That is:

• Any increase or decrease in the equity ‘gap’ – i.e. the difference in higher education outcomes for equity students and the overall student population – needs to be measured relative to any increase or decrease in the corresponding equity gap that occurs in the pre-tertiary education years; and

• Any change in the post-graduation socio-economic benefits realised by equity students, needs to be measured relative to any increase or decrease in the higher education equity gap. Therefore, to contextualise this relative performance, it would be helpful to provide Tier-level reports at regular intervals. Five years is recommended, as it allows sufficient time for cohorts of students to transition between various Domains and Tiers, as well as allow new policy initiatives time to have an impact.

Identifier Title Domain (Indicator)

Details

RT101 Early childhood development

1 (1.01, 1.02)

• the extent to which the gap between various equity groups is increasing or narrowing at the early developmental level

• Includes rates of participation in pre-school • Includes information on students experiencing two or more types of

disadvantage that result in statistically significantly different rates • Includes state/territory comparisons

RT102 NAPLAN literacy and numeracy

2-3 (2.01-2.02; 2.03, 3.01-

3.02)

• the extent to which the gap between various equity groups is increasing or narrowing in NAPLAN reading, numeracy, participation and cohort gain

• School attendance • Includes information on students experiencing two or more types of

disadvantage that result in statistically significantly different rates • Includes state/territory comparisons • Includes public/private sector comparisons

RT103 Year 12 achievement

3 (3.03-3.06)

• the extent to which the gap between various equity groups is increasing or narrowing in Year 12 achievement

• School attendance Year 12 completion • Includes ATAR • Achievement gap in advanced maths and science

RT201a Aspirations 4 (4.01)

• the extent to which the gap between various equity groups is increasing or narrowing in intending to undertake higher education studies in the future

RT201b Aspirations 4 (4.01)

• the extent to which the gap between various equity groups is increasing or narrowing in intending to undertake higher education studies in the future

• Includes information on students experiencing two or more types of disadvantage that result in statistically significantly different rates

• Includes state/territory comparisons • Includes discipline information (i.e. intending to study what) • Includes destination information (i.e. intending to study where)

RT202 Applications 4 (4.02-4.03)

• the extent to which the gap between various equity groups is increasing or narrowing in applying to study higher education


• Includes state/territory comparisons • Includes discipline information (i.e. intending to study what) • Includes destination information (i.e. intending to study where) • Includes institutional comparisons



RT203 Offers, and enrolments

5 (5.01-5.05)

• the extent to which the gap between various equity groups is increasing or narrowing in being made offers, accepting and enrolling in higher education


• Includes state/territory comparisons • Includes discipline information • Includes institutional comparisons • Includes course transitions

RT204 Higher education achievement

6 (6.01-6.03)

• The extent to which the gap between various equity groups is increasing or narrowing in higher education retention, success and completion

• Separates undergraduate enrolments, and all enrolments • Includes institutional comparisons • Includes information on students experiencing two or more types of

disadvantage that result in statistically significantly different rates RT301 Graduate

earnings 7

(7.01) • The extent to which the gap between various equity groups is increasing

or narrowing in graduate earnings • Includes institutional comparisons • Includes discipline comparisons

RT302 Graduate satisfaction

• The extent to which the gap between various equity groups is increasing or narrowing in satisfaction with course experience

• Includes institutional comparisons • Includes discipline comparisons




10. Testing the Framework

Conduct of the Testing The Testing of the Framework took place as part of the Wider Consultation and Testing phase involved the interrogation of key identified data sources for equity in education in Australia. These are reported in Figure 4 below.

Figure 4: Data Sources Investigated During the Testing of the Framework

Data Source Agency Reporting

Format Data Availability Reporting

Tier

Australian Early Development Census (AEDC)

Social Research Centre (SRC)


Three yearly; 2015 next year, report in 2016. Context

Report on Government Service (ROGS),

Productivity Commission (PC)


Annual; ROGS released in March Context

National Assessment Program for Language and Numeracy (NAPLAN)

Australian Curriculum Assessment and Reporting Authority (ACARA)

.xls flat file/NAPLAN

website

Annual; NAPLAN released in December Context

PISA Test

Australian Council for Educational Research (ACER); Reported in ROGS


PISA – 15 year olds test – three yearly; 2015 next year Context

Longitudinal Study of Australian Youth (LSAY)

National Centre for Vocational Education and Research (NCVER) Request

LSAY last cohort in 2009 Context

Higher education applications data

Department of Education and Training (Department)


Annual; finalised in December Context

Student Enrolment and Performance Data Department


Annual; finalised in August Performance

Graduate Destination Survey (GDS)




Beyond Graduation Survey (BGS)




University Experience Survey (UES) SRC

.xls flat file/ Template Annual; March Outcomes

Graduate Outcomes Survey (GOS) SRC


Annual; at design; Commences in 2016 Outcomes

Employer Satisfaction Survey (ESS) SRC


Single Equity Report (SER) Department




Outcomes of the Testing Phase The outcomes from testing are reported in Figure 5 below, where this matches each of the identified Indicators with the above data source and provides information on the testing outcome in terms of data availability and protocols for collecting data. A number of observations can be drawn from the testing:

• For pre-higher education collections, for indicators in Tier 1 (Context), there is little or no comparability in terms of reporting for equity groups as identified in higher education, with the exception of disaggregated reporting on Indigeneity. The identification of socioeconomic status is generally not exactly consistent with that reported in higher education, while definitions of disability and NESB tend to be self-reported. As a result, Indicators in the Context Domain provide some indication of current trends in the system using definitions of equity which are broadly consistent with those in the Framework but which offer no chance of direct comparability.

• Nationally consistent data for Indicators 3.04 (Year 12 completion) and 3.05 (ATAR) are missing at the public reporting level. The first indicator has been somewhat beset by a range of discrepancies in reporting between states and territories, brought about by specific differences across jurisdictions, be it differences in the linkages between secondary and introductory tertiary studies (TAFE) or a lack of coordination in collecting Year 12 completion data in a nationally consistent fashion. Similarly, ATAR data is not easily reported at the national level, except in the context of applications data which is collected from TACs and institutions by the Department of Education and Training. It is likely that both 3.04 and 3.05 could be reported upon under separate collection arrangements, ideally with uniform equity definitions applying at both secondary and higher education levels.

• The biggest gap in the data collection occurs in Domain 4 (Aspirations for higher education), specifically in relation to Indicator 4.01 (Intentions to undertake higher education studies), where the closest data source for this measure, the Longitudinal Study of Australian Youth (LSAY), was last undertaken in 2009 by the NCVER, with the 2012 cohort study not be undertaken and considerable doubt being cast on the 2015 study. This creates a gap in the data, which could best be served through the introduction of a new survey instrument to survey Year 10 and 12 students in relation to their aspirations for non-compulsory education and training.

• In regards to Domains 5 (Access to Higher Education) and 6 (Achievement in Higher Education), collection of data from the higher education system is administratively straight-forward from the Data Analysis Branch of the Department of Education and Training, with internal costing being the only issue (~$1,350 for the testing request).



Figure 5: Outcomes from Testing of the Equity Performance Framework for Australian Higher Education

Indicator Data/Agency Data Equity Groups Protocols 1.01 Early childhood development (physical, social and learning) AEDC/SRC Yes All, except

Disability; WINTA AEDC – 3

year census

1.02 Participation in pre-school ROGS/PC Yes LSES; Indigenous; Special Needs. Yes


2.02 Numeracy (NAPLAN) (Yrs 3 & 5) NAPLAN/ACARA Yes All, except

Disability, WINTA Yes

2.03 School Attendance ROGS/PC Yes Indigenous Yes


3.02 Numeracy (NAPLAN) (Yrs 7 & 9) NAPLAN/ACARA Yes All, except

Disability, WINTA Yes

3.03 School Attendance ROGS/PC Yes Indigenous Yes 3.04 Year 12 completion -- No -- -- 3.05 ATAR -- No -- --

3.06 Achievement in advanced maths and science PISA/ACER Yes

LSES; Indigenous; Regions; language

background Yes

4.01 Intentions to undertake higher education studies LSAY/NCVER No -- --LSAY in

abeyance

4.02 Year 12 applications Department Yes LSES; Indigenous; Regions Yes

4.03 Non- Year 12 (non-school leaver) applications Department Yes LSES; Indigenous;

Regions Yes

5.01 Offers made to students Department Yes Yes Yes 5.02 Acceptances by students (removed from Framework) Department -- -- --

5.03 Commencements (new students)

Department Yes All Yes

5.04 Enrolments (all students) Department Yes All Yes 5.05 Student satisfaction (UES) – Yes Yes Yes 5.06 Course transitions Department No Possible Yes 6.01 Student retention Department Yes All Yes 6.02 Student success Department Yes All Yes 6.03 Student completion Department Yes All Yes

7.01 Graduate earnings GDS/GCA; GOS/SRC Yes Likely in future Likely

7.02 Graduate employment GDS/GCA; GOS/SRC Yes Likely in future Likely

7.03 Graduate further study GDS/GCA; GOS/SRC Yes Likely in future Likely

7.04 Graduate satisfaction UES/SRC Yes No; Likely in future Yes

I.01 Higher education equity funding SER/Department Yes No Likely


SER/Department Yes Not Likely Likely




Key Findings from the Testing

The Testing Phase produced three key findings:

1. Scope for Institutional Reporting: The critical issue in Domains 5 and 6 is the extent to which reporting coverage can be ensured across the entire higher education sector – all 129 higher education providers and not just the 38 Table providers (including the Batchelor Institute). For reasons of data record confidentiality, the Department of Education and Training does not report student number counts of less than 5, although it does report counts of 0. This means that at even the aggregated level (equity group performance by institution), NUHEPS struggle to report meaningful figures. For instance, a simple count of students by equity group in a given institution (see Figure 6 below) , saw only 16 out of 86 NUHEPs report figures for Indigenous students, implying a student count of less than five – in many cases a count of zero. At even lower levels of analysis, for instance, equity students by field of study by institution, the level of reporting approaches zero, as no institution has enough students to generate a publicly reportable result. Ultimately, many sub-reports for higher education enrolments will only be adequately populated for the 38 Table A providers.

Figure 6: Equity Students by Institution, Number of Institutions Reporting More than 5, By Equity Group

Table A

Providers Table B

Providers Table C




In addition, data on graduate earnings and satisfaction can be sourced from the Graduate Destination Survey (GDS) and University Experience Survey (UES) respectively. Testing of the GDS and UES data sets has found that institutional averages for starting salary and student satisfaction are easily reportable at the institutional (Table A) provider level. However, reporting for individual equity groups at each institution is unreliable due to small sample size, although reporting on these groups at the aggregate level is acceptable.

2. Scope for Equity Group Reporting: Similarly, as Figure D shows, reporting for two equity groups (Indigenous; Remote) was particularly patchy across the system, due to the relatively small number of students in both groups. For this reason, the Framework may be thought of as being particularly applicable to the LSES and Regional groups in particular, with reporting for NESB, WINTA and Students with Disability seeing reasonable institutional response rates. 3. Reporting on Inputs: Input indicators can be sourced from the proposed SER collected by the Department of Education and Training. This collection will allow institutions to report on funding (received and spent) and activities in relation to equity programs, although at this point it is focused on outlook programs, with no immediate indication that it will collect data on bridging and enabling programs once students enter the system. However, as this item is still at the design and feedback stages, it is difficult to confirm the availability of data for future collections. Currently, departmental collections would only partially cover I.01 (Higher education equity funding) and then only for aggregate funding for equity programs, with no further breakdown of expenditure.

Preliminary Recommendations on the Reporting Format Given its wide ambit, the Framework’s reporting template needs to be both amenable to completion but also publicly accessible. The Testing phase provides evidence on the key aspects of the reporting:

1. Reporting Scope: The core of the Framework lies in the Indicators reported in Tiers 2 and 3. The Testing phase has indicated that reporting for all institutions may not be possible on a consistent basis because of the small number of enrolments in many NUHEPs, which do not allow for disaggregation at the equity group levels at even the total enrolment level, much less the level required of any sub-reports, for instance by Field of Study or Level of Study. For this reason, some consideration will have to be given to the relative merits of a ‘one size fits all’ approach to reporting.

Recommendation 1: The core recommendation is that initial implementation focus on reporting for all institutions on equity groups with larger populations: LSES and Regional Students, with a separate reporting mechanism being developed for Indigenous Students;



NESB; WINTA; Students with Disability; and Students from Remote Areas in view of the particular policy requirements and data collection limitations of each.

2. Template Format: The Testing phase found that the Framework will source two types of data:

• Pre-higher education data: These data sets provide context for the equity issues confronting higher education policymakers. However, they are generally not easily connected to one another or higher education data via a consistent reporting format. Recommended reporting format: This means that a data flat file is required to report on these Indicators.

• Higher education data: All proposed collection of higher education data can be

sourced directly from, or matched to, the HEIMS system. This will allow for consistency in the definition of equity status across these collections and for relatively easy reporting of data at the institutional level. Recommended reporting format: Higher education data can be presented in a data flat file (.xls) or a web-based database page with results generated via query.

It is conceivable that the entire Framework could be reported as a flat file via a web interface, with sections of the institutional reporting being shared with other Department of Education and Training projects, namely the Quality Indicators for Learning and Teaching (QILT) and/or the Datamart project.

Recommendation 2: It is recommended that the reporting on equity indicators follow a similar reporting format for that developed for the institutional reporting under other departmental data releases (e.g. current enrolment data releases by the Department and QILT).

3. Reporting Timeframe: This is largely determined by institutional reporting, with the previous year’s enrolment usually confirmed and released by August of the following year. For instance, the enrolment data for 2014 will be finalised by August 2015, which enables a compilation of the Framework annual reporting of 2014 by November 2015.

Recommendation 3: Implementation will establish data collection timelines and protocols for an annual collection and reporting of data.



11. Summary

In 2013, the Australian Government Department of Education (“Department of Education”), now the Department of Education and Training (“the Department of Education and Training” or “the Department”), commissioned the National Centre for Student Equity in Higher Education at Curtin University (NCSEHE) to provide further detail on the proposed framework, its structure, overall logic and potential data sources.

The Framework specifically assesses institutional and system-wide performance among higher education providers in terms of the access, participation and performance of six groups of under-represented students who have been specifically identified as equity groups in higher education:

• LSES students; • Students with disability; • Indigenous students; • Students from regional and remote areas; • WINTA students; and • NESB students.

This document describes the resulting work to date on this project: A Framework for Measuring Equity Performance in Australian Higher Education (“the Framework”). It is accompanied by the Framework’s Data Indicator Dictionary and Forward Plan.

Structure of the Equity Performance Framework The Framework is hierarchical in structure and is comprised of three Tiers:

• Tier 1 (Context): Pre-higher education collections covering pre-primary, primary and secondary education results;

• Tier 2 (Performance): Higher education collections covering all 129 Australian higher education providers (see Appendix B) and university students at all levels of study; and

• Tier 3 (Outcomes): Post-higher education covering graduates from higher education.

Each tier has related domains and within each domain are specific indicators that measure higher education equity performance. Underlying each indicator are data which represents the measurement of that indicator.

Figure 7 below (replicating Figure 1 from Section 3) provides an overview of the structure of the Framework’s Tiers, Domains and Indicators.



Figure 7: Equity Performance Framework for Australian Higher Education


Within and across the tiers and domains, a total of 28 indicators are used to measure higher education equity performance. This follows the removal of one Indicator – 5.02 (Acceptances by students) on the grounds that it replicated reporting of 5.03


Domain 7

Graduate outcomes







5.01 Offers made to students 5.02 Acceptances by students 5.03 Commencements (new students) 5.04 Enrolments (all students) 5.05 Student Satisfaction (UES)

5.05 Course transitions







(physical, social and learning)


2.01 Reading (NAPLAN) 2.02 Numeracy (NAPLAN)

2.03 School attendance

3.01 Reading (NAPLAN) 3.02 Numeracy (NAPLAN)

3.03 School attendance

3.04 Year 12 completion

3.05 ATAR 3.06 Achievement in advanced maths and science (PISA)

INPUTS INTO EQUITY






(Commencements (new students)). This structure also includes the Input Domain, which includes two indicators of inputs into equity.

Overwhelmingly, indicators rely on existing data, sourced from key (primarily educational) stakeholders who have in place rigorous and systematic data reporting processes and protocols. More specifically:

• Fifteen Indicators use existing data and data protocols; • Four Indicators rely on existing data and protocols, however the suitability of the data

(for the purposes of the Framework) is not ideal; • Three Indicators (Graduate earnings, employment and satisfaction) will require new

data and protocols to be developed, principally because of the transition to a new collection instrument – the GOS; and

• Six indicators require the development of new data sources and protocols, in addition to relying on existing data with the development of new protocols; and

Of the 28 Indictors, the 11 in Domains 1 to 3 of Tier 1 relate to broader measurements of preparedness for higher education. Indicators in Domains 4 to 6 of Tier 2 report on access and performance in higher education, across all levels of study – sub-bachelor, bachelor, postgraduate coursework (postgraduate) and higher degree by research (HDR). Indicators in Tier 3’s Domain 7 will report data on graduate outcomes.

The indicators included in the Input Domain report on resourcing and activity in equity programs at the institutional level. The first records identified Commonwealth resourcing for higher education equity programs and the second, reports on institutional program structures and activities, for instance, the number of schools in a given outreach program. It is envisaged that data for the Input Domain will be collected via the SER, to be managed by the Department of Education and Training.

The Framework will allow the Department of Education and Training to publish aggregated and disaggregated data. It is proposed that the tables will be made publicly available for researchers and other stakeholders to conduct bespoke analysis. Individual domain reports should be released annually with a more comprehensive, comparative analysis released once every five years.



References

Aird, R., Miller, E., van Megan, K., & Buys, L. (2010). Issues for students navigating alternative pathways to higher education: Barriers, access and equity. Brisbane: Queensland Institure of Technology.

Australian Bureau of Statistics. (2009). Disability, Ageing and Carers, Australia: Summary of Findings, 2009. from http://www.abs.gov.au/ausstats/[email protected]/mf/4430.0

Australian Bureau of Statistics. (2011). 4446.0 - Disability, Australia, 2009. from http://www.abs.gov.au/ausstats/[email protected]/Lookup/4446.0main+features42009

Australian Bureau of Statistics. (2013a). ABS 6227.0 - Education and Work, Australia, May 2013 Retrieved 10 August, 2014, from http://www.abs.gov.au/ausstats/[email protected]/mf/6227.0

Australian Bureau of Statistics. (2013b). Hitting the books: Characteristics of higher education students. Retrieved 10 August, 2014, from http://www.abs.gov.au/AUSSTATS/[email protected]/Lookup/4102.0Main+Features20July+2013

Australian Bureau of Statistics. (2014). 4261.6 - Educational outcomes, experimental estimates, Tasmania, 2006-2013. from http://www.abs.gov.au/AUSSTATS/[email protected]/Lookup/4261.6Main+Features12006-2013?OpenDocument

Australian Curriculum Assessment and Reporting Authority. (2009). National Report on Schooling in Australia 2009 - Aboriginal and Torres Strait Islander education. from http://www.acara.edu.au/reporting/national_report_on_schooling_2009/aboriginal_and_torres_strait_islander_education/enrolment1.html

Australian Curriculum Assessment and Reporting Authority. (2015). National Assessment Program 2015 - 'Results' from:http:/reports.acara.edu.au/Home/Results

Australian Early Developmental Index. (2013). Australian Early Development Index 2012: Summary Report. Canberra: Department of Education.

Australian Institute of Health and Welfare. (2014). Towards a performance measurement framework for equity in higher education. Canberra: Australian Institute of Health and Welfare.

Behrendt, L., Larkin, S., Griew, R. & Kelly, P. (2012). Review of Higher Education Access and Outcomes for Aboriginal and Torres Strait Islander People Final Report. Canberra: Commonwealth of Australia.

Bradley, D., Noonan, P., Nugent, H. & Scales. B. (2008) Review of Australian Higher Education: Final Report. Canberra: Commonwealth of Australia.

Candy, P. (2000). Reaffirming a proud tradition: Universities and lifelong learning. Active Learning in Higher Education, 1(2), 101-125.

Collier, P. (2014). Bold future for schools with new funding model. Retrieved 31/7/2014, 2014, from http://www.mediastatements.wa.gov.au/Pages/StatementDetails.aspx?listName=StatementsBarnett&StatId=8501

Department of Education. (2013). Undergraduate Offers and Acceptances, 2013. Retrieved 9 Ju;y, 2014, from https://education.gov.au/undergraduate-applications-offers-and-acceptances-publications

Department of Education. (2014). Undergraduate Applications and Offers, February 2014. Canberra: Department of Education.

Department of Education and Training. (2015). Higher Education Publications – Finance Publication 2013. Retrieved 10 March, 2014, from https://education.gov.au/finance-publication

Department of Employment Education and Training. (1990). A fair chance for all. Canberra: Australian Governmen Printing Service.

Department of the Attorney General. (2003). Higher Education Support Act 2003. Canberra: Australian Government Printing Service.

Doran, G. (1981). There's a S.M.A.R.T. way to write management's goals and objectives. Management Review, 70(11), 35-36.

Faure, E., Herrera, F., Kaddoura, A., Lopes, H., Petrovsky, A., Rahnema, M., & Ward, F. (1972). Learning to be: The world of education today and tomorrow. London: UNESCO.

http://www.abs.gov.au/ausstats/[email protected]/mf/4430.0

http://www.abs.gov.au/ausstats/[email protected]/Lookup/4446.0main+features42009

http://www.abs.gov.au/ausstats/[email protected]/mf/6227.0

http://www.abs.gov.au/AUSSTATS/[email protected]/Lookup/4102.0Main+Features20July+2013

http://www.abs.gov.au/AUSSTATS/[email protected]/Lookup/4261.6Main+Features12006-2013?OpenDocument

http://www.abs.gov.au/AUSSTATS/[email protected]/Lookup/4261.6Main+Features12006-2013?OpenDocument

http://www.acara.edu.au/reporting/national_report_on_schooling_2009/aboriginal_and_torres_strait_islander_education/enrolment1.html

http://www.acara.edu.au/reporting/national_report_on_schooling_2009/aboriginal_and_torres_strait_islander_education/enrolment1.html


http://www.mediastatements.wa.gov.au/Pages/StatementDetails.aspx?listName=StatementsBarnett&StatId=8501

http://www.mediastatements.wa.gov.au/Pages/StatementDetails.aspx?listName=StatementsBarnett&StatId=8501



Gemici, S., Bednarz, A., & Karmel, T. (2014). The factors affecting the educational and occupational aspirations of young Australians. Adelaide: NCVER.

Graduate Careers Australia. (2013). Graduate Salaries 2012: A report on the earnings of new Australian graduates in their first full-time employment. Melbourne, Australia: Graduate careers Australia.

Griffin, T. (2014). Disadvantaged learners and VET to higher education transitions. Adelaide: NCVER.

Jacobs, N., & Harvey, D. (2005). Do parents make a difference to children’s academic achievement? Differences between parents of higher and lower achieving students. Educational Studies, 31(4), 431-448. doi: 10.1080/03055690500415746

James, R., Krause, K.-L., & Jennings, C. (2010). The first year experience in Australian universities: Findings from 1994 to 2009. Melbourne: Centre for the Study of Higher Education.

Jensen, B., Reichl, J., & Kemp, A. (2011). The real issue in school funding: An analysis of increasing government school expenditure and declining performance. The Australian Economic Review, 44(3), 321-329.

Kallen, D., & Bengtsson, J. (1973). Recurrent education: A strategy for lifelong learning. Washington: OECD.

Kemp, D., & Norton, A. (2014). Review of the demand driven funding system report. Canberra: Department of Education.

Martin, L. (1994). Equity and General Performance Indicators in Higher Education. Canberra: Australian Government Publishing Service.

Martin, M. (2009). On the relationship of external quality assurance and equity: Can they converge on national policy agendas? Quality in Higher Education, 15(3), 251-262.

National Centre for Social and Economic Modelling. (2012). Smart Australians: Education and innovation in Australia. Canberra: National Centre for Social and Economic Modelling.

National Commission of Audit. (2014). National Commission of Audit: 9.13 Higher Education. from http://www.ncoa.gov.au/report/appendix-vol-1/9-13-higher-education.html

National Disability Coordination Officer Program. (2011). Considering higher education? A planning guide for people with disabilities or medical conditions. from http://www.ndco.stepscs.net.au/pdf/Considering%20higher%20education.pdf

Norton, A. (2012). Graduate winners: Assessing the public and private benefits of higher education. from http://grattan.edu.au/static/files/assets/4c182f07/162_graduate_winners_report.pdf

Oliver, D., Freeman, B., Young, C., Yu, S., & Vemma, G. (2014). Employer Satisfaction Suvey: Report for the Department of Education, June 2014. Sydney: The University of Sydney.

Parker, S., Stratton, G., Gale, T., Rodd, P., & Sealey, T. (2013). Higher Education and Student Aspirations: A survey of the adaptive preferences of Year 9 students in Corio, Victoria. Deakin University, Victoria: Centre for Research in Education Futures and Innovation (CREFI).

PhilipsKPA. (2012). Review of Reporting Requirements for Universities. Richmond, Victoria: PhillipsKPA Pty Ltd.

Productivity Commission (2014) Report of Government Services 2014, Volume B: Childcare, education and training from http://www.pc.gov.au/research/recurring/report-on-government-services

Qu, L., Edwards, B., & Gray, M. (2012). Ageing parent carers of people with a disability. Canberra: Australian Institute of Family Studies.

Schultz, J., & Mueller, D. (2006). Effectiveness of programs to improve postsecondary education enrollment and success of underrepresented youth Saint Paul, Minnesota: Wilder Research.

Swail, W., & Perna, L. (2001). A view of the landscape: Results of the national survey of outreach programs College Board Outreach Program Handbook. Washington, DC: Educational Policy Institute.

Sylva, K., Melhuish, E., Sammons, P., Siraj-Blatchford, I., & Taggart, B. (Eds.). (2010). Early childhood matters: Evidence from the Effecttive Pre-school and Primary Education Project. Abingdon, Oxon: Routledge.

Thorpe, J., Shinfield, S., & Walsh, J. (2011). Trial of a model for collecting nationally consistent data on school students with disability. Sydney: PricewaterhouseCoopers.

http://www.ncoa.gov.au/report/appendix-vol-1/9-13-higher-education.html

http://www.ndco.stepscs.net.au/pdf/Considering%20higher%20education.pdf

http://grattan.edu.au/static/files/assets/4c182f07/162_graduate_winners_report.pdf





Watson, L. (2005). Pathways and roadblocks. Challenges for education policy in a changing labour market. Paper presented at the Transitions and Risk; New Directions in Social Policy Conference, Melbourne, Australia.

Wheelahan, L. (2009). Do educational pathways contribute to equity in tertiary education in Australia? Critical Studies in Education, 50(3), 261-275.

Yeboah, D. A. (2002). Enhancing Transition from Early Childhood Phase to Primary Education: Evidence from the research literature. Early Years, 22(1), 51-68. doi: 10.1080/09575140120111517



Appendix A: Definition of Australian higher education equity groups

• Low socio-economic status (LSES) students: Socio-economic status (SES) is assigned to students on the basis of the socio-economic status of the geographical location in which they reside, as identified by ABS statistical area (SA1) or postcode classification. All SA1 areas are ranked on the basis of ABS estimates of the Socio-Economic Index for Areas (SEIFA) of Education and Occupation calculated using census data. LSES students come from the bottom 25% of Australian SA1s (with a postcode backup) in a national ranking.

• Students with disability: Students who self-report a disability to their higher education provider, either at the time of their enrolment or during the course of their studies.

• Indigenous students: Students who self-report as Indigenous to their higher education provider, either at the time of their enrolment or during the course of their studies.

• Students from regional and remote areas: Regional and Remote students are defined as having a permanent home address in an SA1/postcode area that is classified as regional or remote using historic MCEETYA classifications and the Australian Statistical Geography Standard (ASGS).

• Women in non-traditional areas of study (WINTA): Female students who are enrolled in the Natural and Physical Sciences; Information Technology; Engineering and Related Technologies; Architecture and Building; Agriculture, Environmental and Related Studies; Management and Commerce; and the narrow field of Education (Economics and Econometrics).

• Students from non-English speaking backgrounds (NESB): Students from a non-English speaking background who have been resident in Australia for less than ten years.



Appendix B: List of Australian Higher Education Providers, 2013

New South Wales Public Universities (Table A) Charles Sturt University Macquarie University Southern Cross University The University of Sydney University of New England University of New South Wales University of Newcastle University of Technology, Sydney University of Western Sydney University of Wollongong Private Universities (Table B) Not applicable Private Universities (Table C) Not applicable Non-University Higher Education Providers (NUHEPs) Academy of Information Technology Pty Ltd Alphacrucis College Australian College of Physical Education (ACPE) Australian Film, Television and Radio School (AFTRS) Australian Institute of Music Ltd Australian International Conservatorium of Music (AICM) Avondale College of Higher Education Blue Mountains International Hotel Management School Campion College Australia Insearch International College of Management, Sydney (ICMS) Jansen Newman Institute Pty Ltd Kent Institute of Business and Technology Macleay College Moore Theological College Morling College Murdoch Institute of Technology Nan Tien Institute National Arts School National Institute of Dramatic Art (NIDA) Navitas Professional Institute Pty Ltd Raffles College of Design and Commerce SAE Institute and Qantm College Study Group Australia Pty Ltd Sydney College of Divinity Sydney Institute of Business and Technology (SIBT) Sydney Institute of Traditional Chinese Medicine TAFE NSW TCOL (The College of Law) TOP Education Group Pty Ltd Tabor College NSW The JMC Academy Think Education Group UOW College Universal Business School Sydney



Wesley Institute Whitehouse Institute Victoria Public Universities (Table A) Deakin University Federation University Australia La Trobe University Monash University RMIT University Swinburne University of Technology The University of Melbourne Victoria University Private Universities (Table B) MCD University of Divinity Private Universities (Table C) Not applicable Non-University Higher Education Providers (NUHEPs) (NUHEPs) Australian Academy of Design Australian College of the Arts, Collarts Box Hill Institute Carrick Higher Education Chifley Business School Chisholm Institute of TAFE Harvest Bible College Holmes Institute Holmesglen Institute of TAFE John Paul 11 Institute for Marriage and Family, Melbourne La Trobe Melbourne Leo Cussen Centre for Law Marcus Oldham College Melbourne Institute of Business and Technology Melbourne Institute of Technology Monash College Navitas College of Public Safety Northern Melbourne Institute of TAFE Phoenix Institute of Australia Pty Ltd Photography Studies College (Melbourne) Stotts Colleges Tabor College Victoria The Australian Guild of Music Education The Cairnmillar Institute School The Melbourne Institute for Experiential and Creative Arts Therapy (MIECAT) William Angliss Institute Queensland Public Universities (Table A) Central Queensland University Griffith University James Cook University Queensland University of Technology The University of Queensland University of Southern Queensland University of the Sunshine Coast Private Universities (Table B) Bond University



Private Universities (Table C) Not applicable Non-University Higher Education Providers (NUHEPs) Australian Institute of Professional Counsellors Pty Ltd Christian Heritage College Endeavour College Gestalt Therapy Brisbane Jazz Music Institute Queensland Institute of Business and Technology (QIBT) Southbank Institute of Technology TAFE Queensland Western Australia Public Universities (Table A) Curtin University of Technology Edith Cowan University Murdoch University The University of Western Australia Private Universities (Table B) The University of Notre Dame Australia Private Universities (Table C) Not applicable Non-University Higher Education Providers (NUHEPs) Australian School of Management Curtin College Harvest West Bible College Perth Bible College Perth Institute of Business and Technology (PIBT) Polytechnic West Tabor College Perth Vose College of Higher Education South Australia Public Universities (Table A) Flinders University of South Australia The University of Adelaide University of South Australia Private Universities (Table B) Not applicable Private Universities (Table C) Carnegie Mellon University University College London (UCL) Non-University Higher Education Providers (NUHEPs) Adelaide Central School of Art Adelaide College of Divinity Australian Institute of Business Pty Ltd Australian Institute of Management SA Division (AIM SA) Australian Lutheran College Educational Enterprises Australia (Eynesbury College) International College of Hotel Management (ICHM) Kaplan Business School Le Cordon Bleu Australia South Australian Institute of Business and Technology (SAIBT) TAFE SA Tabor Adelaide Tasmania



Public Universities (Table A) University of Tasmania Private Universities (Table B) Not applicable Private Universities (Table C) Not applicable Non-University Higher Education Providers (NUHEPs) Tabor College Tasmania Northern Territory Public Universities (Table A) Batchelor Institute of Indigenous Tertiary Education Charles Darwin University Private Universities (Table B) Not applicable Private Universities (Table C) Not applicable Non-University Higher Education Providers (NUHEPs) Not applicable Australian Capital Territory Public Universities (Table A) The Australian National University University of Canberra Private Universities (Table B) Not applicable Private Universities (Table C) Not applicable Non-University Higher Education Providers (NUHEPs) Canberra Institute of Technology Multi-State Public Universities (Table A) Australian Catholic University Private Universities (Table B) Not applicable Private Universities (Table C) Not applicable Non-University Higher Education Providers (NUHEPs) Australian College of Theology

For more information on types of institutions please refer to: http://www.comlaw.gov.au/Details/C2011C00532/Html/Text#_Toc298408521

http://www.comlaw.gov.au/Details/C2011C00532/Html/Text#_Toc298408521



Appendix C: The Options for Reporting on Equity Inputs

Following consultation with the sector, the Framework can identify three options for reporting “Input” indicators, as follows:

Option 1: Creating a distinct Tier for equity inputs In defining and measuring equity performance in higher education, the AIHW adopts the approach of measuring and reporting a wide range of equity inputs under the rubric of “education system performance” (AIHW, 2014). These include the four categories of inputs as described above i.e. specific and non-specific funding; and specific and non-specific programs and other actions. If this approach were to be incorporated into the Framework, then the Framework might resemble that detailed in Figure C1 below. One issue that needs to be addressed is the amount of specificity that should be applied to defining Indicators. Generally, caution is recommended in being too specific about defining specific Indicators, due to the highly-fluid nature of funding and program provision. However they must be specific enough for meaningful assessments to be made regarding particular initiatives. Much of this can be captured in the reporting phase. Thus a generically-defined Indicator would not need to change if/when a particular program or funding activity ceases, only the way in which the data informing it are reported. The advantages of this approach include:

• A more holistic representation of the factors that influence equity in higher education, including those occurring before and after the actual higher education ‘experience’;

• A better way of measuring equity inputs that occur across, or affect, more than one Domain. For example, outreach programs operated in partnership with communities often target students across a range of ages/years and/or also target their parents.

The disadvantages of this approach include:

• Complexities in defining equity-enabling funding and activities (for example see WA school-funding case-study described above); and

• Difficulties in capturing data recorded in diverse formats and across multiple stakeholders.



Figure C1: Framework incorporating a holistic approach to reporting equity inputs

TIER 4: OUTCOMES

(Post-higher education)

Domain 8 Graduate outcomes

8.01 Graduate earnings 8.02 Graduate satisfaction


Domain 5 Aspirations for higher

education


Domain 7 Achievement in higher

education


6.01 Offers made to students 6.02 Acceptances by students 6.03 Commencements (new) 6.04 Enrolments (all students) 6.05 Course transitions






2.01 Early childhood

development (physical, social and learning)


3.01 Reading performance 3.02 Numeracy performance 3.03 School attendance

4.01 Reading performance 4.02 Numeracy performance 4.03 School attendance 4.04 Year 12 completion 4.05 ATAR 4.06 Achievement in advanced maths and science

TIER 1: INPUTS (System support)

Domain 1: System support 1.01 Equity-specific Commonwealth funding to institutions (e.g. HEPP funding) 1.02 Equity-enabling Commonwealth funding to institutions (e.g. CGS funding) 1.03 Equity-specific Commonwealth funding to students (e.g. Abstudy) 1.04 Equity-enabling Commonwealth funding to students (e.g. HELP loans) 1.05 Equity-specific State funding to institutions (e.g. loadings to support disabled students in primary

education) 1.06 Equity-enabling State funding to institutions (e.g. regional infrastructure building) 1.07 Equity-specific State funding to students (State scholarships for regional students) 1.08 Equity-enabling State funding to students (e.g. public transport subsidies) 1.09 Equity-specific other funding to institutions (philanthropic donations for disadvantaged student

support programs) 1.10 Equity-enabling other funding to institutions (e.g. open donations to institutions) 1.11 Equity-specific other funding to students (e.g. philanthropic scholarships for equity students) 1.12 Equity-enabling other funding to students (e.g. community/volunteer run outreach activities)



Option 2: Integrating equity inputs into the existing Tiers Another possibility is to more fully integrate the Input Indicators into the existing three-tier approach. If this approach were to be incorporated into the Framework, then the Framework might resemble as detailed in Figure C2 below. As with Option 1, it is recommended that a distinction be made between equity-specific and equity-enabling Indicators.

Figure C2: Framework incorporating an integrated approach to reporting equity inputs

The advantages of this approach include:

• A more accurate representation of the factors that influence equity in higher education, including those occurring before and after the actual higher education ‘experience’;

• A more precise means of identifying key data-holders within specific Domains.


Domain 7

Graduate outcomes

7.01 Graduate earnings 7.02 Graduate satisfaction 7.03 Equity-specific funding 7.04 Equity-enabling funding 7.05 Equity-specific programs 7.06 Equity-enabling programs





4.01 Intentions to undertake higher education studies 4.02 Year 12 applications 4.03 Non Year 12 applications 4.04 Equity-specific funding 4.05 Equity-enabling funding 4.06 Equity-specific programs 4.07 Equity-enabling programs

5.01 Offers made to students 5.02 Acceptances by students 5.03 Commencements (new students) 5.04 Enrolments (all students) 5.05 Course transitions 5.06 Equity-specific funding 5.07 Equity-enabling funding 5.08 Equity-specific programs 5.09 Equity-enabling programs

6.01 Student retention 6.02 Student success 6.03 Student completion 6.04 Equity centric support programs 6.05 Equity-specific funding 6.06 Equity-enabling funding 6.07 Equity-specific programs 6.08 Equity-enabling programs






(physical, social and learning) 1.02 Participation in pre-school 1.03 Equity-specific funding 1.04 Equity-enabling funding 1.05 Equity-specific programs 1.06 Equity-enabling programs

2.01 Reading performance 2.02 Numeracy performance 2.03 School attendance 2.04 Equity-specific funding 2.05 Equity-enabling funding 2.06 Equity-specific programs 2.07 Equity-enabling programs

3.01 Reading performance 3.02 Numeracy performance 3.03 School attendance 3.04 Year 12 completion 3.05 ATAR 3.06 Achievement in advanced maths and science 3.07 Equity-specific funding 3.08 Equity-enabling funding 3.09 Equity-specific programs 3.10 Equity-enabling programs



The disadvantages of this approach include: • Complexities in defining equity-enabling funding and activities; • Difficulties in capturing data recorded in diverse formats and across multiple

stakeholders) • Difficulties in ascribing certain Indicators to a particular Domain, when the

funding/program occurs across more than one Domain.

Option 3: A separate reporting of equity inputs

Here, equity inputs would be measured and reported independently of the Framework. The advantages of this approach include:

• A greater focus on outcomes for disadvantaged students within the Framework itself; • Encourages stakeholders to avoid drawing simple cause-and-effect relationships

between particular equity initiatives and broader equity outcomes; • Avoids any misapprehension that all types of equity inputs (both specific and non-

specific) are able to be captured.

The disadvantages of this approach include:

• It potentially marginalises the importance of equity inputs; • It does not provide a sufficiently-detailed representation of equity performance in

higher education; and • By not considering equity inputs at the same time equity outcomes are assessed,

the full range of possibilities for enhancing data collection and management might be overlooked.

A FRAMEWORK FOR MEASURING EQUITY PERFORMANCE IN AUSTRALIAN HIGHER EDUCATION€¦ · A Framework for...

Documents

Transcript of A FRAMEWORK FOR MEASURING EQUITY PERFORMANCE IN AUSTRALIAN HIGHER EDUCATION€¦ · A Framework for...