Research Excellence Framework

This document sets out the general framework for assessment in the 2014 Research Excellence Framework (REF) and provides guidance to UK higher education institutions about making submissions to the 2014 REF. It includes guidance on procedures, the data that will be required, and the criteria and definitions that will apply. The deadline for submissions is 29 November 2013.

This document is for information.

REF 02.2011 July 2011

Assessment framework and guidance on submissions(updated to include addendum published in January 2012)

PageExecutive summary 1Introduction 3

Part 1: Overview of the assessment framework 4

Purpose 4General principles 4Framework for assessment 5Publication of results and submissions 7Equality and diversity 7Timetable 9Key changes since RAE 2008 10

Part 2: Submissions 11

Scope of submissions 11Content of submissions 13The submission process 13Data verification 14Interdisciplinary and collaborative research 15

Part 3: Data requirements and definitions 17

Section 1: Staff details (REF1a/b/c) 17 Individual staff circumstances 20

Section 2: Research outputs (REF2) 21 Amendment to timing of publication 23

Section 3: Impact template and case studies (REF3a/b) 26Section 4: Environment data (REF4a/b/c) 30Section 5: Environment template (REF5) 32

Part 4: Codes of practice on the selection of staff 34

Introduction 34The legislative context 34Guidance to institutions 39

Annexes

Annex A Assessment criteria and level definitions 43Annex B Quality profiles 46Annex C Definitions of research and impact for the REF 48Annex D Units of assessment 49Annex E Timetable 50Annex F Format and page limits for textual parts of submissions 51Annex G Impact case studies: template and guidance 52Annex H Standard data analyses 55Annex I Index of definitions and data requirements 56Annex J List of abbreviations 57

ContentsIn January 2012 updated information was provided for the two sections in bold italic below.

REF 02.2011 1

Executive summaryPurpose1. This document:

• setsouttheframeworkandgenericcriteriaforassessmentinthe2014ResearchExcellenceFramework(REF)

• specifiesthecontent,datarequirementsandrelateddefinitionsforsubmissionstothe2014REF

• guideshighereducationinstitutionsonpolicyandpracticalmattersinpreparingsubmissions.

Key points2. TheREFwillbeaprocessofexpertreview.Expertsub-panelsforeachof36unitsofassessment(UOAs)willcarryouttheassessment,workingundertheleadershipandguidanceoffourmainpanels.

3. InOctober2012thefourUKhighereducationfundingbodieswillinviteUKhighereducationinstitutionstomakesubmissionstothe2014REF.EachsubmissionineachUOAwillcontainacommonsetofdatacomprising:

a. Informationonstaffinpostonthecensusdate,31October2013,selectedbytheinstitutiontobeincludedinthesubmission.

b. Detailsofpublicationsandotherformsofassessableoutputthatselectedstaffhaveproducedduringthepublicationperiod(1January2008to31December2013).

c. Acompletedtemplatedescribingthesubmittedunit’sapproachduringtheassessmentperiod (1January2008to31July2013)toenablingimpactfromitsresearch,andcasestudiesdescribingspecificexamplesofimpactsachievedduringtheassessmentperiod,underpinnedbyexcellentresearchintheperiod1January1993to31December2013.

To

Heads of publicly funded higher education institutions in the UK

Of interest to those responsible for

Research

Reference

REF 02.2011

Publication date

July 2011

Enquiries from staff at UK higher education institutions

E-mail your institutional REF contact. (These are listed at www.ref.ac.uk under Contact.)

Other enquiries

Rebecca Gordge, tel 0117 931 7477, e-mail info@ref.ac.uk

REF 2014: Assessment framework and guidance on submissions

2 REF 02.2011

d. Dataaboutresearchdoctoraldegreesawardedandresearchincomerelatedtotheperiod1August2008to31July2013.

e. Acompletedtemplatedescribingtheresearchenvironment,relatedtotheperiod1January2008to31July2013.

4. Thedeadlineforsubmissionsis29November2013.SubmissionswillbeassessedbytheREFpanelsduringthecourseof2014.ResultswillbepublishedinDecember2014,andwillbeusedbythehighereducationfundingbodiestoinformresearchfundingfromacademicyear2015-16.

Action required5. ThisdocumentisforinformationandtoguideinstitutionsinpreparingandcollectingdataforinclusioninREF2014submissions.Noactionisrequiredbyhighereducationinstitutionsatthisstage.

Further information6. FurtherinformationabouttheREFisavailableatwww.ref.ac.uk.

7. EnquiriesfrommembersofstaffatUKhighereducationinstitutionsshouldbedirectedinthefirstinstancetotheirinstitutionalREFcontact.Thesecontactsforeachinstitutionarelistedat www.ref.ac.ukunderContact.

8. Otherenquiriesshouldbeaddressedto info@ref.ac.uk.

REF 02.2011 3

Introduction9. Thisdocumentsetsouttheframeworkforassessmentandadministrativearrangementsforthe2014ResearchExcellenceFramework(REF).Itspecifiesthedatarequirements,definitionsandcriteriathatwillapply,forsubmissionsbyhighereducationinstitutions(HEIs).Itshouldbereadinconjunctionwiththedocumentssettingoutthepanelcriteriaandworkingmethodsofthefourmainpanels(hereafter,the‘panelcriteria’),whichwewillpublishindraftformforconsultationinJuly2011andinfinalformearlyin2012.

10. Thisdocumentandthepanelcriteriawilltogetherdescribecomprehensivelythedatarequiredinsubmissions,andhowpanelswillusethedataintheirassessments.Wemayissuesupplementstothisguidanceatlaterdatestoclarifypointsofdetailregardingsubmissions,butsuchsupplementswillnotrequestanynewitemsofdata.

11. InOctober2012wewillformallyinviteeligibleHEIstomakesubmissionstothe2014REF.WewilllaunchthesubmissionsystemandprovideaccompanyingtechnicalguidanceinJanuary2013.Thedeadlineforsubmissionsis29November2013.

4 REF 02.2011

Purpose12. TheResearchExcellenceFrameworkisthenewsystemforassessingthequalityofresearchinhighereducationinstitutionsintheUK,andreplacestheResearchAssessmentExercise(RAE),lastconductedin2008.

13. TheREFisconductedjointlybytheHigherEducationFundingCouncilforEngland(HEFCE),theScottishFundingCouncil(SFC),theHigherEducationFundingCouncilforWales(HEFCW)andtheDepartmentforEmploymentandLearning,NorthernIreland(DEL).TheREFismanagedbytheREFteam,basedatHEFCE,onbehalfofthefourUKhighereducationfundingbodies,andisoverseenbytheREFSteeringGroup,consistingofrepresentativesofthefourfundingbodies.Inthisdocument,‘we’referstotheREFteam.

14. TheprimarypurposeofREF2014istoproduceassessmentoutcomesforeachsubmissionmadebyinstitutions:

a. Thefourhighereducationfundingbodiesintendtousetheassessmentoutcomestoinformtheselectiveallocationoftheirgrantforresearchtotheinstitutionswhichtheyfund,witheffectfrom2015-16.

b. Theassessmentprovidesaccountabilityforpublicinvestmentinresearchandproducesevidenceofthebenefitsofthisinvestment.

c. Theassessmentoutcomesprovidebenchmarkinginformationandestablishreputationalyardsticks,forusewithinthehighereducation(HE)sectorandforpublicinformation.

General principles15. TheREFisaprocessofexpertreview.Recentconsultationsaboutreformstotheassessmentframeworkconfirmedwidespreadconfidenceindiscipline-basedexpertreviewfoundeduponexpertjudgement.Tomaintainconfidenceintheassessmentprocessandinthecredibilityoftheoutcomestothosebeingassessed,wehaveappointedpanelsofexpertswhoarecurrentlyorhaverecentlybeenactiveinhighqualityresearch,oritswideruse.Whiletheseexpertswilldrawonappropriatequantitativeindicatorstosupporttheirprofessionaljudgement,expertreviewremainsparamount.

16. TheREFisasingleframeworkforassessmentacrossalldisciplines,withacommonsetofdatarequiredinallsubmissions,standarddefinitionsandprocedures,andassessmentbyexpertpanelsagainstbroadgenericcriteria.Expertpanelswillapplystandardsofassessmentconsistently,workingundertheguidanceoffourmainpanels.Withinthissingleframework,differencesinthenatureofresearchacrossthedisciplinaryspectrummayjustifydifferencesinthedetailedapproachtoassessment.Thereisflexibilityforpanelstodevelopspecificaspectsoftheassessmentcriteriaandtoadoptworkingmethodstoensuretheassessmentissensitivetothesedisciplinarydifferences.Panelswillconsultwiththeircommunitiesandwithinstitutionsindoingso.

17. TheREFhasdevelopedthroughanevolutionaryprocess,buildingonpreviousRAEs.Witheverysuccessiveassessmentexerciseabalancehastobestruckbetweencontinuityanddevelopment.ChangesnewtoREF2014followfromextensivereviewandconsultation,andhavebeenadoptedwhereitisjudgedtheycanbringdemonstrableimprovementswhichoutweighthecostofimplementingthem.

18. ThefollowingprinciplesgoverntheconductoftheREF.TheysettheframeworkinwhichtheREFteamco-ordinatestheexerciseandinwhichthefourmainpanelsand36sub-panelswilldeploytheircollectiveprofessionaljudgementtodraftcriteriaforassessmentandtoassesssubmissions.

a. Equity:Alltypesofresearchandallformsofresearchoutputacrossalldisciplinesshallbeassessedonafairandequalbasis.Panelshavebeeninstructedtodefinecriteriaandadoptassessmentprocessesthatenablethemtorecogniseandtreatonanequalfootingexcellenceinresearchacrossthespectrumofapplied,practice-based,basicandstrategicresearch,whereverthatresearchisconducted;andforidentifyingexcellenceindifferentformsofresearchendeavourincludinginterdisciplinaryandcollaborativeresearch,whileattachingnogreaterweighttooneformoveranother.TheREFaimstoassessalltypesofresearchwithoutdistortingtheactivitythatitmeasuresorencouragingordiscouraginganyparticulartypeofresearchactivity,otherthanproviding

Part 1Overview of the assessment framework

REF 02.2011 5

ageneralstimulustoenhancingtheoverallachievementsoftheUKresearchbase.

b. Equality:HEIsarestronglyencouragedtosubmittheworkofalltheirexcellentresearchers.Toenablethis,institutionsmayreducethenumberofresearchoutputssubmittedforindividualswhosecircumstancesconstrainedtheirabilitytoworkproductivelythroughouttheassessmentperiod.Panelswillassesstheirworkonanequalbasiswithoutanypenaltyforreducingthenumberofsubmittedoutputs.TohelpHEIssubmittheworkofalltheirexcellentresearchersandcomplywithequalitylegislation,wewillrequirethemtodevelop,documentandapplyaninternalcodeofpracticeonthefairandtransparentselectionofstaffforinclusioninREFsubmissions.ThemeasurestosupportequalityanddiversitywithintheREFaresetoutinparagraphs39-43.

c. Transparency: ThecredibilityoftheREFisreinforcedbytransparencyintheprocessthroughwhichdecisionsaremade.Thecriteriaandproceduresthatwillbeappliedintheassessmentwillbepublishedinfull,wellinadvanceofinstitutionsmakingtheirsubmissions.Theoutcomeswillbepublishedinfullanddecision-makingprocesseswillbeexplainedopenly.WeaimtomakeallwrittendocumentsandstatementsabouttheREFclearandconsistent.

19. ThecostandburdenoftheREFshouldbetheminimumpossibletodeliverarobustanddefensibleprocess.PreviousRAEshavebeenhighlycost-effectivegiventhevalueofpublicfundsdistributedthroughtheiroutcomes(includingtheestimatedcosttoHEIs).Forexample,weestimatedthecostsofthe2008RAEinEnglandtobesome0.5percentofthevalueofpublicresearchfundingthatwassubsequentlyallocatedwithreferencetoitsresults.Wewillcontinuetoweightheburdenoninstitutionsagainsttheneedtoensureaccountabilityindisbursingpublicfunds.

Framework for assessment

Units of assessment 20. TheREFwillbeaprocessofexpertreview,withdiscipline-basedexpertpanelsassessingsubmissions

madebyHEIsin36unitsofassessment(UOAs).TheUOAsarelistedinAnnexD.DetaileddescriptorsofeachUOAwillbepublishedinthepanelcriteriastatements.

Submissions21. Institutionswillmakesubmissionsby 29November2013,ineachUOAtheyelecttosubmitin.Eachsubmissionwillcontain,insummary:

a. REF1a/b/c:Informationonstaffinpostonthecensusdate,31October2013,selectedbytheinstitutiontobeincludedinthesubmission.

b. REF2:Detailsofpublicationsandotherformsofassessableoutputwhichtheyhaveproducedduringthepublicationperiod(1January2008to31December2013).Uptofouroutputsmustbelistedagainsteachmemberofstaffincludedinthesubmission.

c. REF3a/b:Acompletedtemplatedescribingthesubmittedunit’sapproachduringtheassessmentperiod(1January2008to31July2013)toenablingimpactfromitsresearch,andcasestudiesdescribingspecificexamplesofimpactsachievedduringtheassessmentperiod,underpinnedbyexcellentresearchintheperiod1January1993to31December2013.

d. REF4a/b/c:Dataaboutresearchdoctoraldegreesawardedandresearchincomerelatedtotheperiod1August2008to31July2013.

e. REF5:Acompletedtemplatedescribingtheresearchenvironment,relatedtotheperiod 1January2008to31July2013.

Expert panels22. Anexpertsub-panelforeachofthe36UOAswillconductadetailedassessmentofsubmissionsinitsUOA.Thesub-panelswillworkundertheleadershipandguidanceoffourmainpanels.Thefourmainpanelswillberesponsiblefordevelopingthepanelcriteriaandworkingmethods,forensuringadherencetothepublishedprocedures,fortheconsistentapplicationoftheoverallassessmentstandards,andforsigningofftheoutcomesoftheassessment.

23. TheexpertpanelswereappointedbythefourUKfundingbodiesthroughanopenprocessofnominations,asdescribedin‘Unitsofassessmentandrecruitmentofexpertpanels’(REF01.2010).

Par

t 1

Ove

rvie

w

6 REF 02.2011

AsweindicatedinREF01.2010,wehavesoughttoensurethatthemembershipofthemainandsub-panelscomprisesindividualswhohaveexperienceinconducting,managingandassessinghighqualityresearch,aswellasexpertswhoarewell-equippedtoparticipateintheassessmentofresearchimpactfromaprivate,publicandthirdsectorperspective.Inappointingthepanels,dueregardwasgiventothedesirabilityofensuringthattheoverallbodyofmembersreflectsthediversityoftheresearchcommunity.

24. Themembershipofeachpanelisat www.ref.ac.ukunderExpertpanels.During2013,additionalassessorswillbeappointedtoextendthebreadthanddepthofexpertiseonthepanels,andtheirdetailswillalsobepublished.

Assessment criteria25. AswithpreviousRAEs,theassessmentprocessisbasedonexpertreview.Eachsub-panelwillexaminethesubmissionsmadeinitsUOA,takingintoaccountalltheevidencepresented.Theywillusetheirprofessionaljudgementtoformanoverallviewabouteachsubmission.Indoingdo,thesub-panelswillassessthreedistinctelementsofeachsubmission,againstthefollowinggenericcriteria:

a. Outputs:Thesub-panelswillassessthequalityofsubmittedresearchoutputsintermsoftheir‘originality,significanceandrigour’,withreferencetointernationalresearchqualitystandards.Thiselementwillcarryaweightingof65 per centintheoveralloutcomeawardedtoeachsubmission.

b. Impact:Thesub-panelswillassessthe‘reachandsignificance’ofimpactsontheeconomy,societyand/orculturethatwereunderpinnedbyexcellentresearchconductedinthesubmittedunit,aswellasthesubmittedunit’sapproachtoenablingimpactfromitsresearch.Thiselementwillcarryaweightingof20 per cent.

c. Environment: Thesub-panelswillassesstheresearchenvironmentintermsofits‘vitalityandsustainability’,includingitscontributiontothevitalityandsustainabilityofthewiderdisciplineorresearchbase.Thiselementwillcarryaweightingof15 per cent.

Panel criteria and working methods26. Allsub-panelswillapplythegenericassessmentcriteriaassetoutinAnnexA,andwilloperateunderacommonassessmentframework.Theywillassesssubmissionsmadeaccordingtothestandarddefinitionsandcommonformatsetoutthisdocument,andwillfollowacommonsetofproceduresinundertakingaspectsoftheirwork(forexampleinmanagingconflictsofinterest).

27. Wewillpublish–initiallyforconsultationinJuly2011andtheninfinalforminearly2012–detailsofthecriteriaandworkingmethodsthatthepanelswillapplywhenassessingsubmissions.Thiswillincludedetailsofthecommonprocedurestobefollowedbyallpanels,andastatementofthecriteriaandworkingmethodsofeachmainpanel.Thesewillexplaininmoredetailhoweachgroupofsub-panelswithinamainpanelwillapplythegenericassessmentcriteria,andsetouttheworkingmethodsofthemainandsub-panelsinundertakingtheirroles.

28. Eachmainpanelwilldevelopacommonsetofcriteriaandworkingmethodsforitsgroupofsub-panels,withdistinctcriteriaorapproachesforparticularsub-panelsonlywherethisisjustifiedbydifferencesinthenatureofresearchinthedisciplinesconcerned.Guidancetothepanelsondevelopingtheircriteriaandworkingmethodsispublishedatwww.ref.ac.ukunderPublications.

29. Thepanelcriteriaandworkingmethodsdocumentswillincludefurtherguidancetoinstitutionsaboutsomeparticularformsofevidencethatwouldbeappropriatetoincludeinthetextualpartsofsubmissions.Theyshouldthereforebereadalongsidethis‘Guidanceonsubmissions’publication:togetherthedocumentswillsetoutcomprehensivelywhatinformationwillberequiredinsubmissions,andhowthepanelswillassessthesubmissions.

Assessment outcomes30. Foreachsubmissionthesub-panelswilldevelopa‘sub-profile’foreachofthethreeelementsoftheassessment(outputs,impactandenvironment).Thesub-profileswillshowtheproportionsofactivityjudgedtomeeteachoffourstarredlevels.ThestarredlevelsaredefinedatAnnexA.

31. Thethreesub-profileswillbecombinedintoanoverallqualityprofile.Anexampleoverallquality

REF 02.2011 7

profileandthemethodforcombiningthesub-profilesisatAnnexB.TheoverallqualityprofileawardedtoeachsubmissionwillbetheprimaryoutcomeoftheREF,tobepublishedinDecember2014.

32. Bypresentingtheoutcomesintheformofqualityprofiles,wewillensurethatpocketsofexcellenceareidentifiedwithintheassessmentoutcomes.

Publication of results and of submissions 33. TheprimaryoutcomeoftheREFwillbeanoverallqualityprofileforeachsubmission,andthesewillbepublishedinDecember2014.Thequalityprofilewillshowtheproportionsofresearchactivityjudgedtomeeteachoffourstarredqualitylevels,instepsof1percent.AnnexBdescribeshowwewillformulateandpresentqualityprofiles,includingtheroundingmethodology.

34. Alongsidethequalityprofile,thefundingbodieswillpublishthefull-timeequivalent(FTE)numberofstaffincludedineachsubmission(submittedstaff).

35. Furtherreportsandfeedbackfromtheexercisewillbeavailableearlyin2015.Weexpectthefeedbacktocomprise:

a. Apublishedreportbyeachmainpanelconfirmingitsworkingmethodsandprovidinganoverviewofitsobservationsaboutthestateofresearch(strengths,weaknesses,vitalityofactivity,andscopeofimpactsachieved)intheareasfallingwithinitsremit.Thesereportswillincludeasectionprovidedbyeachsub-panel.

b. Concisefeedbackoneachsubmissionsummarisingthereasonforthequalityprofileawarded,withreferencetothepublishedcriteriaofthesub-panelthatassessedit.Weexpecttosendthisfeedbackonlytotheheadoftheinstitutionconcerned.Inthecaseofjointsubmissions,wewillprovidethisfeedbackconfidentiallytotheheadsofalloftheinstitutionsinvolved.

c. Theoutput,impactandenvironmentsub-profilesforeachsubmissionthatwerecombinedtoproducetheoverallqualityprofileswillbeprovidedtotheheadsofinstitutions,andthenpublished.

d. Minutesofthesub-panelandmainpanelmeetingsfortheassessmentphaseoftheREF

willbepublishedtoprovideapublicrecordofhowthepanelsconductedtheirbusiness.

e. AreportbytheREFmanager,detailinghowtheprocesswasmanagedinoperationalterms.

f. AreportbytheREFEqualitiesandDiversityPanel,detailingitsworkingmethodsandobservationsaboutimplementingtheequalitymeasuresintheREF.

36. Wewillalsopublishsubmissionsontheinternet,inspring2015.Wewillincludethenamesofselectedstaffandthelistingsofresearchoutputs,butremovepersonalandcontractualdetails.Wewillpresentthesubmitteddataonresearchdoctoraldegreesawardedandresearchincome.Wewillincludesubmittedtextualinformationaboutimpactandtheresearchenvironment.ThesubmissionsoftwarewillincludeafacilityforHEIstoredactthenamesofanystaff,listedoutputs,impactcasestudiesortextualpartsofsubmissionsthatshouldbeomittedfromthepublisheddataforspecificreasons,suchascommercialsensitivityorsecurity.Furtherdetailedguidanceonthiswillbeprovidedinadvanceofthereleaseofthesubmissionssystem.

37. Inreachingtheirjudgements,panelswillnottakeaccountofanyinformationaboutstaffthatarenotselectedforsubmission;alsothepublishedoutcomesoftheREFwillnotincludeanyinformationaboutnon-submittedstaff.SeparatelytotheREFoutcomes,weexpectthattheHigherEducationStatisticsAgency(HESA)willduring2015publishdatabasedonHESAstaffreturnsabouttheproportionofeligiblestaffsubmittedbyeachHEI.Thefundingbodieswillalsopublishananalysisoftheselectionofstaffintermsoftheirequalityanddiversitycharacteristics,attheleveloftheUKhighereducationsectorasawhole.Thisanalysiswillnotidentifyindividualinstitutions.

38. TheresultsoftheREFarenotsubjecttoappeal.Thefundingbodieshaveconsideredcarefullythequestionofappeals,andconcludedthattheabsenceofanappealsprocessdoesnotmaketheassessmentprocessanylessrobust.

Equality and diversity 39. TheUKhighereducationfundingbodiesarecommittedtosupportingandpromotingequalityanddiversityinresearchcareers,andstronglyencourageinstitutionstosubmittheworkofalltheirexcellent

Par

t 1

Ove

rvie

w

8 REF 02.2011

researchers.CompliancewithequalitylegislationisanobligationforHEIs,andthefourUKhighereducationfundingbodieshaveastatutoryobligationaspublicbodiestoadvanceequality.

40. TheREFteamisassistingthefundingbodiesinpromotingequalityanddiversityinresearchcareersandinmeetingtheirstatutoryobligationsinanumberofways:

a. WeareensuringthatallpanelsarebriefedonallequalityandrelevantemploymentlegislationthatwillaffectREF2014.Wehaveinstructedpanelmemberstotakeaccountofequalityissuesthatmayhaveabearingonthevolumeofresearchundertakenandpublishedbysubmittedresearchersinthe2014REF.

b. WearestronglyencouragingHEIstosubmittheexcellentresearchofalltheireligiblestaff.Individualswhosecircumstanceshavesignificantlyconstrainedtheirabilitytoworkproductivelythroughouttheassessmentperiodmaybereturnedwithfeweroutputs,withoutanypenaltyintheassessment.Thisissetoutinparagraphs90-100,andfurtherdetailswillbeprovidedinthepanelcriteriaandworkingmethods.

c. Eachinstitutionmakingasubmissionisrequiredtodevelop,documentandapplyacodeofpracticeonthefairandtransparentselectionofstafffortheirREFsubmissions(seePart4).Guidanceindrawingupacodeofpracticeframesinstitutions’decision-makingprocessesinthecontextoftheprinciplesofequalityandallrelevantlegislation,supportingHEIsinmeetingtheirownobligations.

d. Withineachsubmission,aspartofthedescriptionoftheresearchenvironment,evidencewillberequiredabouthowthesubmittedunitpromotesequalityanddiversity.

e. TheselectionofstafffortheREFwillbemonitoredandanalysedatsectorlevel,andtheresultsofouranalysispublished.

f. TheEqualityChallengeUnit(ECU)willprovidegoodpracticeandtrainingmaterialstosupportinstitutionsintakingaccountofequalityanddiversitywhenpreparingsubmissions.ThesewillbeavailablefromSeptember2011at www.ecu.ac.uk/our-projects/REF.

41. ThesemeasurestopromoteequalityanddiversitythroughtheREFhavebeeninformedbyareviewoftheequalitymeasurestakeninthe2008RAE,andbytheworkoftheREFEqualitiesandDiversityAdvisoryGroup(EDAG)whichwasestablishedspecificallytoadviseusonthedevelopmentofthesemeasuresfortheREF.

42. WewillcontinuetotakeexpertadviceontheimplementationofthesemeasuresthroughouttheREFbyestablishingaREFEqualitiesandDiversityAdvisoryPanel(EDAP).Thispanelwillbeconstitutedduring2012,andincludemembershipdrawnfromthefourmainpanelsandspecialistequalityexpertise.ThetermsofreferenceoftheEDAPareavailableatwww.ref.ac.ukunderEqualityanddiversity.

43. WewillalsoextractandpasstoHESAthefollowingdatatoenableverification:foreachindividualsubmittedasresearchactiveinCategoryA,theirUOA,HESAstaffidentifiercode,anddateofbirth.Hence,HEIsshouldensurethatdecisionsdocumentedinlinewiththeircodeofpracticeareconsistentwithrelevantpartsoftheirannualindividualisedstaffreturntoHESA.The2013-14HESAindividualisedstaffreturnwillrequireHEIstoreturntheREFUOAforalleligibleacademicstaff,includingthosenotselectedforinclusioninREF2014,tofacilitatethefundingbodies’analysisofstaffselection.

REF 02.2011 9

Timetable 44. Thetimetableforthe2014REFisasfollows,andisrepeatedatAnnexE:

March 2010 Publication of ‘Initial decisions’ by the funding bodies on the conduct of the REF (HEFCE Circular letter 04/2010)

July 2010 Publication of ‘Units of assessment and recruitment of expert panels’ (REF 01.2010)

November 2010 Publication of reports on the REF impact pilot exercise

February 2011 Panel membership announced

March 2011 Publication of ‘Decisions on assessing research impact’ (REF 01.2011)

July 2011 Publication of ‘Assessment framework and guidance on submissions’ (REF 02.2011)

End July 2011 Publication of draft panel criteria and working methods for consultation

5 October 2011 Close of consultation on draft panel criteria and working methods

January 2012 Publication of panel criteria and working methods

31 July 2012 Institutions intending to make submissions to the REF submit their codes of practice on the selection of staff

Autumn 2012 Pilot of the REF submissions system

October 2012 Invitation to HEIs to make submissions; invitation to request multiple submissions; and start of survey of submissions intentions

December 2012 Survey of submissions intentions complete and deadline for requests for multiple submissions

January 2013 Launch of submissions systems and accompanying technical guidance

31 July 2013 End of assessment period (for research impacts, the research environment, and data about research income and research doctoral degrees awarded)

Mid 2013 Appointment of additional assessors to panels

31 October 2013 Census date for staff eligible for selection

29 November 2013 Closing data for submissions

31 December 2013 End of publication period (cut-off point for publication of research outputs, and for outputs underpinning impact case studies)

Throughout 2014 Panels assess submissions

December 2014 Publication of outcomes

Spring 2015 Publication of submissions, panel overview reports and sub-profiles

Par

t 1

Ove

rvie

w

10 REF 02.2011

Key changes since RAE 2008 45. Therearedifferencesbetweenthe2014REFandthe2008RAEconcerningboththeoverallassessmentframeworkandthedetaileddatarequirementsanddefinitions.ThereforeHEIsshouldnotrelyontheirknowledgeoftheguidanceanddatarequirementsinpreviousresearchassessmentexercises,andshouldrefertotheguidanceassetoutinthisdocument.

46. Wedrawparticularattentiontothefollowingchanges:

a. TheUOAsandsub-panelshavebeenreducedfrom67to36,andthemainpanelsfrom15to4.

b. ThereisgreaterconsistencyintheassessmentprocessacrossallUOAs,includingstandardweightingsbetweenthethreeelementsofassessment(outputs,impactandenvironment)andstandardisationofanumberofcriteria,datarequirementsandprocedures.Panelcriteriaandworkingmethodshavebeendevelopedatmainpanellevel,withinputfromsub-panels(ratherthanatsub-panellevelwithoversightbymainpanels).

c. ThedefinitionsofeligibleCategoryAandCategoryCstaffhavebeenrevised.CategoryBandCategoryDstaffcannolongerbesubmitted.

d. Somesub-panelswillmakeuseofcitationinformation,providedbytheREFteamonaconsistentbasis,asadditionalinformationabouttheacademicsignificanceofresearchoutputs.

e. TheREFincludesanexplicitelementtoassessthenon-academicimpactofresearch.ThisisdescribedinPart3Section3.

f. ‘Esteem’isnolongerincludedasadistinctelementintheassessment.

g. Theapproachtoassessingtheresearchenvironmenthasbeenrevised.Itwillbebasedonastructuredtemplatefortextualinformation,andasignificantlyreducedsetofstandarddatarequirements.ThesearedescribedinPart3,Sections4and5.

h. ThemeasurestopromoteequalityanddiversityinresearchcareershavebeenstrengthenedandwillbeappliedconsistentlyacrossUOAs,asoutlinedatparagraphs39-43,paragraphs90-100andPart4.

i. Additionalassessorswillbeappointedtoextendthebreadthanddepthofexpertiseonsub-panelsduringtheassessmentphase.Theassessors’roleisdifferentfromthatofspecialistadvisersinthe2008RAE,andtheywillparticipatefullyindevelopingthesub-profiles.Outputswillnotbereferredtospecialistadvisers,withtheexceptionofoutputssubmittedinlanguagesthatthesub-panelisunabletoassess.

j. Theoutcomesoftheassessment–theoverallqualityprofiles–willbepublishedinstepsof 1percentratherthan5percent.

REF 02.2011 11

Part 2Submissions

Scope of submissions47. EachHEImaysubmitinanyofthe36UOAslistedatAnnexD.AnHEIwillnormallymakeonesubmissionineachUOAitsubmitsin,andonlyexceptionallywillthisbewaived.Suchexceptionsaresetoutinparagraph50.

48. Asubmissioncomprisesacompletesetofdataaboutstaff,outputs,impactandtheresearchenvironment,returnedbyanHEIinanyofthe36UOAs(asdescribedinPart3).Asubmissionprovidesevidencetothesub-panelabouttheactivityandachievementsofa‘submittedunit’.Asubmittedunitmeansthegrouporgroupsofstaffincludedinasubmission,andbyextension:

• theirresearch(producedduringtheREFpublicationperiod)

• thestructuresandenvironmentthatsupportedtheirresearchanditsapplicationorimpact(duringtheassessmentperiod)

• researchrelatedtothatUOAandundertakenwithintheinstitution(since1January1993),whichunderpinssubmittedimpactcasestudies.

49. Asubmittedunitmay,butneednot,comprisestaffwhoworkwithinasingledepartmentorotherorganisationalunitintheHEI.AsubmittedunitmaycomprisestaffwhoworkinmultipleorganisationalunitsintheHEI.TheresearchofasubmittedunitmustrelateprimarilytotheareasofresearchsetoutinthedescriptoroftheUOAinwhichitissubmitted.TheUOAdescriptorswillbepublishedinthepanelcriteriastatements.

Multiple submissions50. InstitutionswillnormallymakeonesubmissionineachUOAtheysubmitin.Theymayexceptionally,andonlywithpriorpermissionfromtheREFmanager,makemorethanonesubmission(multiplesubmissions)inthesameUOA.Theseexceptionsare:

a. WhereaninstitutioninvolvedinajointsubmissionwishestomakeanadditionalindividualsubmissioninthesameUOA.

b. MultiplesubmissionstoSub-panel28(ModernLanguagesandLinguistics)willbepermittedwhereonesubmissionisinCelticStudiesandtheotherinModernLanguagesandLinguistics.

ThishasbeenagreedinrecognitionofthespecialculturalsignificanceofCelticStudiesinpartsoftheUK,andtheparticularlegalstatusoftheWelshlanguageinWales.

c. WhereHEIsmergeafter1July2011theymayseekpermissiontomaketwoseparatesubmissionsinalloftheUOAsinwhichtheywishtosubmit,ifforexampletheyanticipatedifficultyinachievingacademiccohesionbetweenthemergerdateandthesubmissiondate.PermissionisunlikelytobegrantedtosuchHEIstomakeseparatesubmissionsonlyinsomeoftheUOAsinwhichtheywishtosubmit.IntheeventthatHEIsmergedpriorto1July2011,themergedHEIshouldnormallymakeonesubmissiononlytoeachUOA.

d. Whereasub-panelconsidersthereisacaseformultiplesubmissionsinitsUOA,giventhenatureofthedisciplinescovered,theinstitutionmayrequestamultiplesubmission.Thefollowingproceduresapply:

i. Theinstitutionwillneedtomakeaconvincingcasethat:

• thebodiesofresearchtobelistedineachproposedsubmissionfallwithinthescopeoftheUOAbutareclearlyacademicallydistinctfromeachother,and

• theresearchenvironmentsofeachproposedsubmittedunitareclearlyseparateanddistinct,withoutsignificantoverlapintheirresearchorstaffingstrategies,infrastructure,facilitiesorotheraspectstobedescribedinthetextualpartsofsubmissions.

ii. TheREFpanelswillindicateintheircriteriastatementswhethertheyexpecttoreceivesuchrequestsformultiplesubmissions,giventhenatureofthedisciplinescovered,andtheymaysetadditionalcriteriathatwillneedtobesatisfied.

iii. TheREFmanagerwilldecideonallsuchrequestsinconsultationwithchairsoftherelevantmainandsub-panels.Inconsideringtheserequests,administrativeconvenienceofthesubmittinginstitution,oritspreferenceforseparateassessmentoutcomes,willnotbefactors.

Par

t 2

Su

bm

issi

on

s

12 REF 02.2011

51. AnyHEIthatwishestomakemultiplesubmissionsinrelationtoanyoftheexceptionssetoutabovewillneedtorequestpriorpermissionfromtheREFmanager.InOctober2012wewillinviterequeststomakemultiplesubmissionsandresponseswillberequiredbyDecember2012.Asidefromtheexceptionmentionedinparagraph50c,aseparateapplicationwillberequiredforeachUOAinwhichtheHEIwishestomakemultiplesubmissions.AllapplicationswillbejudgedbytheREFmanager,inconsultationwiththerelevantmainandsub-panelchairs.

52. Eachsubmissionwillbeawardedasingleoverallqualityprofile.WhereasinglesubmissionincludesdistinctorganisationalunitsorareasofresearchandwheretheREFsub-panelconsidersitappropriate,thesub-panelwillprovidefeedbacktotheheadofinstitutionrelatingtothedistinctunitsorareasofresearch.

Joint submissions53. WeencouragejointsubmissionsinaUOAbytwoormoreUKinstitutions,wherethisisthemostappropriatewayofdescribingresearchtheyhavedevelopedorundertakencollaboratively.Themethodforjointsubmissionsisdescribedinparagraphs54-58andisdrivenbytwoconsiderations:

a. Panelsshouldreceivejointsubmissionsintheformofaunifiedentity,enablingthemtoassessajointsubmissioninthesamewayassubmissionsfromsingleinstitutions.

b. TheREFteammustbeabletoverifydatainajointsubmissionthroughtheHEIstowhichthedatarelates.

54. Purelyforadministrativepurposes,oneHEIneedstobeidentifiedastheleadintermsofmanagementanddatasecurityofajointsubmission.TwoelementsoftheREFdata(REF3a/b:Impacttemplateandcasestudies;andREF5:Environmenttemplate)willbesubmittedbytheleadHEIonbehalfofalltheotherHEIsinthejointsubmission.EachHEIinvolvedinthejointsubmissionwillsubmitseparateREFdatainthefollowingforms:REF1a/b/c(Staffdetails),REF2(Researchoutputs)andREF4a/b/c(Environmentdata).

55. Inlinewiththesesubmissionarrangements,thesubmissionsystemwillincludethefacilityforHEIs

involvedinjointsubmissionstogive‘View’and‘Edit’permissionstotheotherHEIsinvolvedintherelevantUOA.Inorderforpanelstobeabletojudgethejointsubmissionlikeasinglesubmission,theREFteamwillaggregatethedataforeachHEIsothatpanelscanreceiveandassessitasacoherentwhole.

56. Thefollowingrulesapply:

a. Panelswillassessthejointsubmissionastheywouldasinglesubmission,andtheoutcomewillbeasinglequalityprofile.ThequalityprofileforajointsubmissionwilllisttheHEIsinvolvedinalphabeticalorder,irrespectiveofwhichHEItooktheadministrativeleadinmakingthesubmission.

b. PanelswillprovideconfidentialfeedbackonjointsubmissionstotheheadsofalltheHEIsconcerned;butthepanelsandtheREFteamwillnotcommentspecificallyonthecontributionbyanindividualHEItotheoverallqualityprofile.

c. InlinewithageneralREFrulethatnoindividualmaybesubmittedasCategoryAresearchactiveinmorethanonesubmissionunlesstheyholdafractionalemploymentcontractwithmorethanoneHEI(seeparagraph78f),noindividualcanbesubmittedinajointsubmissionand in asubmissionfromoneHEIunlesstheyholdtwoseparateemploymentcontractswithtwodifferentHEIs.

d. InstitutionsinvolvedinajointsubmissionthatwishtomakeanadditionalindividualsubmissioninthesameUOAwouldnormallybepermittedtodoso.

57. FollowingtheconclusionoftheREF,eachoftheUKfundingbodieswilldistributeresearchfundinginlinewithitsindividualfundingformula.Assumingthat,asinformeryears,theFTEofstaffsubmittedwillbeanelementinthoseformulae,thefundingbodiesenvisageusingtheactualFTEofstaffsubmittedbyeachHEIinvolvedinajointsubmission,unlesstheHEIsinvolvedproposeadifferent,agreedpercentagesplitoffundingatthetimeofsubmission.

58. Furtherguidanceonthetechnicalprocedureformakingjointsubmissionswillaccompanythetechnicalguidanceonhowtousethesubmissionssystem.

REF 02.2011 13

Content of submissions 59. Eachsubmissionwillcontainthecoredataoutlinedinsub-paragraphsatoefollowing.FurtherdetailsaresetoutinPart3.(TheREFnumberingreferstothenameoftheformsinwhichthedatawillbecollected.)

a. Staff details (REF1a/b/c):Informationonindividualsinpostonthecensusdate (31October2013)andselectedbytheinstitutionforinclusioninthesubmission(REF1a);individualstaffcircumstancesforthosesubmittingfewerthanfouroutputs(REF1b);anddetailsofCategoryCstaff(REF1c).

b. Research outputs (REF2):Detailsofuptofourresearchoutputsproducedbyeachmemberofsubmittedstaffduringthepublicationperiod (1January2008to31December2013).

c. Impact template and case studies (REF3a/b): Acompletedtemplatedescribingthesubmittedunit’sapproachduringtheassessmentperiod (1January2008to31July2013)toenablingimpactfromitsresearch(REF3a);andcasestudiesdescribingspecificexamplesofimpactsachievedduringtheassessmentperiod,underpinnedbyexcellentresearchintheperiod1January1993to31December2013(REF3b).

d. Environment data (REF4a/b/c):Dataonresearchdoctoraldegreesawardedeachyearintheperiod1August2008to31July2013(REF4a);theamountsandsourcesofexternalresearchincomeforeachyearintheperiod1August2008to 31July2013(REF4b);andtheamountofresearchincome-in-kindforeachyearintheperiod 1August2008to31July2013(REF4c).

e. Environment template (REF5):Informationabouttheresearchenvironmentrelatedtotheperiod1January2008to31July2013.

60. Wehavesoughttoalignthedatarequirementsasfaraspossiblewithdatareportedtootheragencies,orusedforotherpurposes:

a. ThedatarequirementsrelatingtoresearchoutputswillbecompatiblewiththeCommonEuropeanResearchInformationFormat(CERIF).InstitutionswillbeabletoimportdataintotheREFsubmissionsysteminvariousfileformats,enablingthemtouseexistinginternaldataforpreparingREFsubmissions.

b. ThedefinitionsofdataonresearchdoctoraldegreesawardedandonresearchincomehavebeenalignedasfaraspossiblewithdefinitionsusedinHESAdatareturns.WewillprovideinstitutionswithHESAdatathatcanbeusedinpreparingsubmissions.InstitutionswillbeabletoallocatetheHESAdatatoREFUOAs;ortheymaypreparetheirREFdatafrominternalsystems.Ineithercase,wewillusetheHESAdataforvalidationpurposes.

c. WewillarrangefortheResearchCouncilsandthehealthresearchfundingagencieslistedinparagraph172toprovideinstitutionswithrelevantdataaboutresearchincome-in-kindforuseinpreparingsubmissions,andwewillusethesedataforvalidationpurposes.

61. Intheircriteriastatements,REFpanelsmayrequireadditionalspecificdatawherethisisnecessaryfortheassessmentandtheburdenoninstitutionsisjustified,tobeincludedinthetextualpartsofthesubmission(REF3aandREF5).Thepublishedcriteriastatementswillalsoprovidegeneralguidanceaboutwhatkindsofinformationtoprovideinthetextualpartsofsubmissions.

The submission process

Method of submission62. Forthe2014REF,wearedevelopingsoftwaretocollectsubmissionsfromHEIs,onbehalfofthefourfundingbodies.Thissubmissionsystemwillbuildonthesystemusedinthe2008RAE.Itwillbeaweb-basedapplicationusingadatabasehostedatHEFCE,andwillbetheonlywayHEIscanmakeasubmissiontotheREF.ApilotversionofthesystemwillbemadeavailabletoHEIsinautumn2012.ThefinalversionwillbeavailableinJanuary2013.

63. DataentrywillnotbelimitedtodirectentryonscreenandwillalsoallowHEIstoimportdatainvariousfileformats,includingXMLfiles(theXMLschemaswillbeavailableatwww.ref.ac.ukunderSubmissionsguidance).Allcommunicationsbetweenthewebserverandclientmachinesininstitutionswillbeencrypted.AccesstothedatabasebeforethesubmissiondatewillnotbepermittedtoanyoneotherthanauthorisedpersonnelwithineachHEIandtoHEFCE’ssystemadministrators.

Par

t 2

Su

bm

issi

on

s

14 REF 02.2011

Access to research outputs64. Alloutputsthatarelistedinsubmissionswillbemadeavailabletotherelevantsub-panel,asfollows:

a. Forjournalarticlesandconferenceproceedings,thesubmissionmustincludeaDigitalObjectIdentifier(DOI)whereveravailable,toenabletheREFteamtosourcetheseoutputsfromthepublishers.WherewecannotsourceparticularoutputsfromthepublisherstheHEImustprovideanelectroniccopy.

b. Forallothertypesofoutput,theHEImustprovideanelectroniccopywhereverthisisavailable;orwhereitisnot,aphysicaloutputorappropriateevidenceoftheoutput.

Survey of submission intentions65. InOctober2012wewillundertakeasurveyofHEIsabouttheirsubmissionintentions,toassistwithplanningoftheassessmentphaseandwithidentifyingareaswheretheappointmentofadditionalassessorswouldbedesirable.ThroughthesurveywewillaskHEIstoidentifywhichUOAstheyintendtomakesubmissionsin,andforeachsubmissiontoindicate:

• thelikelyvolumeofstaff

• themainareasofresearchandimpacttobeincludedinthesubmissionandthelikelyvolumeofworkineacharea(thiswillneedtobeinsufficientdetailforpanelstounderstandthebreadthanddepthofexpertiserequiredfortheassessment,andinparticulartoinformtherecruitmentofadditionalassessors)

• thelikelyvolumeofworktobesubmittedinlanguagesotherthanEnglish.

66. Responsestothesurveywillnotbebinding,butinstitutionsshouldnotethattheinformationwillbeveryhelpfultotheREFteamandpanelsinplanningtheirworkandespeciallytounderpintherecruitmentandselectionofassessors.ResponseswillberequiredbyDecember2012.

Data verification67. AllinformationprovidedbyHEIsinsubmissionstothe2014REFmustbecapableofverification.Wewillcheckaproportionofsubmittedinformationfromeachinstitutionasamatterofcourse.Panelmemberswillalsobeaskedtodrawattentiontoany

datathattheywouldlikeustoverify,andthesedatawillbeinvestigated.HEIsshouldthereforebeabletoprovidejustificationforallinformationsubmitted.

68. Wehavealignedthedataonresearchdoctoraldegreesawardedandresearchincomeasfaraspossiblewithotheravailabledatasets,includingHESAreturnsandinformationheldbytheResearchCouncils.AsdescribedinPart3Section4,theREFsubmissionsystemwilllimittheextenttowhichsuchdatasubmittedbyanHEIcanexceedtheirpriorreturnstoHESA.WhilewerecognisethatthebasisofreturnsforREFandotherdatasetsmaydifferandexactmatchesmightnotbepossible,wewillinvestigateinstanceswherethereappeartobesignificantdifferencesbetweensubmitteddataandotherreturns.

69. Eachsubmittedimpactcasestudyshouldincludedetailsofexternalsourcesofinformationthatcouldcorroborateclaimsmadeabouttheimpactofthesubmittedunit’sresearch.Wewillauditaproportionofcasestudiesandinvestigatethesesourceswhererequestedbysub-panels.

70. WewillissueafullerstatementontheverificationarrangementsforREFdataatalaterdate.

71. WhereanHEIisunabletoprovidejustificationforanypieceofinformationcontainedinitssubmission,thatinformationwillbeexcludedfromassessment.Thefundingbodieswillconsiderwhatfurtheractiontotakeinanycasewhereseriousdiscrepanciesarefound.

72. WeareconsciousofthepotentialadditionalworkloadverificationmaycauseHEIs,andaimtominimisethis.Tothisend,datacheckingandverificationwillnormallybeconductedbycorrespondencewiththeREFteam.IfHEIswishtomakeitavailable,ourverificationteammayalsoseektorelyonanyrelevantinternalauditworkthatanHEIhasundertakeninpreparingorsubmittingREFdata.MembersoftheREFteamandotherofficersofthefundingbodiesreservetherighttovisitHEIstoverifysubmissioninformation.

Data protection73. Wewillcollect,storeandprocessallinformationsubmittedbyHEIstotheREFinaccordancewiththeDataProtectionAct1998.InformationwillbeprocessedforthepurposesofconductingandevaluatingtheREF.Informationmaybesharedwithotherorganisations

REF 02.2011 15

tofacilitatethis,andwillbesharedwithpanelchairs,members,assessors,secretariesandobservers,whoareallboundbyconfidentialityarrangements.Asstatedinparagraph43,wewillextractandpasssomeinformationtoHESAtoenabledataverification.Wewillalsopublishpartsofsubmissionsontheinternet(asdescribedinparagraph36).WewilluseinformationfromHEIstomonitorthediversityofstaffselectedfortheREF.HEIsshouldensurethatindividualswhoseworkisincludedintheirsubmissionsareawareoftheseuses,includingthepublicationofsubmissions.

Interdisciplinary and collaborative research 74. AnunderpinningprincipleoftheREFisthatalltypesofresearchandallformsofresearchoutputacrossalldisciplinesshallbeassessedonafairandequalbasis,includinginterdisciplinaryandcollaborativeresearch.TherehavebeenconcernsthattheassessmentofinterdisciplinaryresearchhaspresentedchallengesinpreviousRAEs,duetothediscipline-basedstructureoftheUOAs,orthatcollaborativeresearchhasnotbeenencouragedduetothecompetitivenatureoftheexercise.

75. Anumberofenhancementstotheproceduresforassessinginterdisciplinaryresearchwereintroducedinthe2008RAE,andtheREFwillbuildonthese:

a. GiventhebroaderUOAs(thesehavebeenreducedfrom67to36)theREFsub-panelswillingeneralincludeabroaderrangeofexperts,withtheexpertisetoassessawiderrangeofresearch.Wehavespecificallysoughttoincludepanelmemberswithinterdisciplinaryexperts.

b. Followingthesurveyofsubmissionintentionswewillrecruitadditionalassessors,includingpeoplewithinterdisciplinaryexpertise,toextendthedepthandbreadthofpanels’expertisetoundertaketheassessment.Assessorswillparticipatefullyinassessingresearchoutputsandimpacts.Wewillseektoappointassessorstoworkwithmorethanonesub-panel,wheretherearestrongcross-disciplinaryconnectionsbetweenparticularsub-panels.

c. Asubmittinginstitutionwillbeabletoidentifythoseoutputswhichitconsiderstobeinterdisciplinary,todrawthistothepanels’attention.

d. Whileasubmissionwillnormallybeassessedonlybythesub-panelfortheUOAinwhichitissubmitted,mechanismswillberetainedtocross-referpartsofsubmissionstoothersub-panelsforadvicewheretherelevantmainandsub-panelchairsadvisethatthisisnecessary1:

i. Specificoutputsorimpactcasestudiesmaybecross-referredforadvice;entiresubmissionsmaynot.

ii. BoththesubmittingHEIandthesub-panelreceivingthesubmissionmaymakearequesttocross-referpartsofsubmissions.Inallcases,therelevantmainandsub-panelchairswilladvisetheREFmanagerwhowilldecideontherequests.

iii. Whilemainpanelswilloverseetheprocessformanagingcross-referralrequests,cross-referralsmayalsobemadetosub-panelsoutsideofthemainpanelareaofthereceivingUOA.

iv. Wherepartsofsubmissionsarecross-referred,advicewillbesoughtandgivenonthebasisoftheassessmentcriteriafortheUOAinwhichtheworkwasoriginallysubmitted.Theoriginalsub-panelwillspecifythescopeofadvicethatitisseeking:thismayrangefromadviceontheoverallqualityofoutputsorimpactofcasestudies,toadviceonspecificfeaturesofthoseoutputsorcasestudies.Theoriginalsub-panelwillretainresponsibilityforrecommendingthequalityprofile.

76. TheREFwillsupportcollaborativeresearchthroughthefollowingarrangements:

a. WeencouragejointsubmissionsinaUOAbytwoormoreUKinstitutions,wherethisisthemostappropriatewayofdescribingtheresearchtheyhavedevelopedorundertakencollaboratively.(Seeparagraphs53-58.)

b. Outputsthatareco-authoredorco-producedbystafflistedinmorethanonesubmission(whetherwithinthesameHEIorfromdifferentHEIs)maybelistedinanyorallofthosesubmissions,andwillbeassessedonanequalfootingtoanyotheroutput.(Thepanelcriteriawillprovide

1 In addition to cross-referring parts of a submission between sub-panels for advice to inform the assessment, parts of submissions may be made available to other sub-panels for the purposes of calibration exercises. Calibration procedures will be described in more detail in the panel criteria and working methods documents.

Par

t 2

Su

bm

issi

on

s

16 REF 02.2011

guidanceonwhetheranoutputco-authoredbystaffwithinasinglesubmissionmaybelistedagainstmorethanonememberofstaffwithinthatsubmission.)

c. Whereasubmittedimpactisunderpinnedbycollaborativeresearch,eachsubmittingunitwhoseresearchmadeadistinctandmaterialcontributiontotheimpactmaysubmitthatimpact.

d. Withintheenvironmenttemplate,institutionsshouldprovideinformationabouthowtheysupportinterdisciplinaryandcollaborativeresearch,andpanelswillgiveduecreditwherethesearrangementshaveenhancedthevitalityandsustainabilityoftheresearchenvironmentorthesubmittedunit’scontributiontothewiderresearchbase.

REF 02.2011 17

Part 3Data requirements and definitions

Part 3 Section 1: Staff details (REF1a/b/c)77. EachHEImustdecidewhichindividualstoselectforsubmission,inaccordancewithitsinternalcodeofpractice(seePart4).Staffselectedforsubmissionmustbelistedinoneofthetwopossiblecategories,AorC.

Category A staff 78. CategoryAstaffaredefinedasacademicstaffwithacontractofemploymentof0.2FTEorgreaterandonthepayrollofthesubmittingHEIonthecensusdate(31October2013),andwhoseprimaryemploymentfunctionistoundertakeeither‘researchonly’or‘teachingandresearch’2.

79. Regardlessoftheirjobtitle,allstaffwhosatisfythedefinitionatparagraph78,alongwiththesupplementarycriteriainparagraphs79-81,areeligibleasCategoryAstaff:

a. Staffwhoholdinstitutional/NHSjointappointmentsareeligibletobereturnedasCategoryA.ThesestaffshouldbereturnedwithanFTElessthan1.0,reflectingtheircontractofemploymentwiththeinstitution.

b. Pensionedstaffwhocontinueinsalariedemploymentcontractedtocarryoutresearchandmeetthedefinitionatparagraph78areeligibletobereturnedasCategoryAstaff.

c. AcademicstaffwhoareonunpaidleaveofabsenceoronsecondmentonthecensusdateandarecontractedtoreturntonormaldutiesuptotwoyearsfromthestartoftheirperiodofabsenceorsecondmentareeligibletobereturnedasCategoryA,providedthatanystaffrecruitedspecificallytocovertheirdutiesarenotalsolistedasCategoryA.

d. AcademicstaffwhoareemployedbythesubmittingHEIandbasedinadiscretedepartmentorunitoutsidetheUKareeligibleonlyiftheHEIdemonstratesthattheprimaryfocusoftheirresearchactivityonthecensusdateisclearlyanddirectlyconnectedtothesubmittingunitbasedintheUK.Staffwhoseconnectioncannotbedemonstratedtothe

satisfactionoftheREFmanager,asadvisedbytherelevantpanel,willbediscountedfromtheassessmentandremovedfromtheREFdatabase.

e. Staffabsentfromtheir‘home’institutionbutworkingonsecondmentascontractedacademicstaffatanotherUKhighereducationinstitutiononthecensusdate,maybereturnedbyeitherorbothinstitutions.Insuchacasetheindividualandbothinstitutionsconcernedshouldagreehowthereturnistobemade.TheirtotalFTEmaynotexceedtheircontractedFTEwiththeirmainemployer.

f. Otherthanindividualsonsecondmentonthetermsdescribedinsub-paragraphe,anindividualmayonlybereturnedasCategoryAbymorethanoneHEIiftheyhaveacontractwithandreceiveasalaryfrommorethanoneHEI.Insuchcases:

i. ThetwoHEIsmustensurethatthetotalFTEvalueoftheindividualsumstonomorethanthelowerof1.0ortheindividual’stotalcontractedFTEduties.IfanyindividualisreturnedinsubmissionswithacontractedFTEthatsumstomorethan1.0,theREFteamwillrectifythisthroughverification,andwillapportiontheFTEtoeachHEIpro-ratatotheindividual’scontractedFTEateachHEI.

ii. Thesameresearchoutputsmay,butneednotbe,listedineachsubmission.

g. Noindividualmaybereturnedinmorethanonesubmission,exceptasdescribedatsub-paragraphseandf.Whereanindividualholdsajointappointmentacrosstwoormoresubmittingunitswithinthesameinstitution,theHEImustdecideononesubmissioninwhichtoreturntheindividual.

h. Staffwhosesalaryiscalculatedonanhourlyordailybasisareeligibleonlyiftheymeetthedefinitionatparagraph78andonthecensusdatehaveacontractofemploymentofatleast0.2FTEperyearoverthelengthoftheircontract.

2 These are staff returned to the HESA Staff Collection with an activity code of ‘Academic Professional’ (currently identified as code ‘2a’ in the ACT1, ACT2 or ACT3 fields) and an academic employment function of either ‘Research only’ or ‘Teaching and research’ (currently identified as codes ‘2’ or ‘3’ in the ACEMPFUN field). Revised guidance on the coding of these staff in HESA returns will be issued following the review of the HESA staff record, which is due to conclude in September 2011.

Par

t 3

Sta

ff

18 REF 02.2011

i. StaffwhoholdmorethanonecontractfordifferentfunctionswithintheHEI,areeligibleifoneofthosecontractssatisfiesthedefinitionofCategoryAstaffatparagraph78.SuchstaffshouldbereturnedwithanFTEthatisnogreaterthanthatofthequalifyingcontract.

Research assistants

80. Researchassistantsareindividualswhoareonthepayrollofandholdacontractofemploymentwiththeinstitution.Theyareacademicstaffwhoseprimaryemploymentfunctionisdefinedas‘researchonly’.Theyareemployedtocarryoutanotherindividual’sresearchprogrammeratherthanasindependentresearchersintheirownright(exceptinthecircumstancesdescribedinparagraph81).TheyareusuallyfundedfromresearchgrantsorcontractsfromResearchCouncils,charities,theEuropeanUnion(EU)orotheroverseassources,industry,orothercommercialenterprises,buttheymayalsobefundedfromtheinstitution’sownfunds.IndividualswhomeetthisdefinitionmaybedescribedinHEIs’gradingstructuresassomethingotherthanresearchassistant(forexampleresearchassociate,assistantresearcher).

81. Researchassistants,asdefinedinparagraph80,arenoteligibletobereturnedtotheREFunless,exceptionally,theyarenamedasprincipalinvestigatororequivalentonaresearchgrantorsignificantpieceofresearchworkonthecensusdateandsatisfythedefinitionofCategoryAstaffinparagraph78.ResearchassistantsmustnotbelistedasCategoryAstaffpurelyonthebasisthattheyarenamedononeormoreresearchoutputs.

Category C staff82. CategoryCstaffaredefinedasindividualsemployedbyanorganisationotherthananHEI,whosecontractorjobrole(asdocumentedbytheiremployer)includestheundertakingofresearch,andwhoseresearchisprimarilyfocusedinthesubmittingunitonthecensusdate(31October2013).

83. CategoryCstaffmaybeemployedbytheNHS,aResearchCouncilunit,acharityorotherorganisationexceptforanHEI.SubmittedoutputsbyCategoryCstaffwillinformthequalityprofilesawardedtosubmissions,butthesestaffwillnotcontributetothevolumemeasureforfundingpurposes.Forclarity,thefollowingarenoteligibletobereturnedasCategoryCstaff:

a. AnystaffemployedbytheHEI,includingvice-chancellorsorheadsofHEIs;HEIstaffonnon-academiccontracts,includingthoseworkinginuniversitymuseumsandlibraries;orretiredstaffwhoarestillactiveinresearch.(Wheretheysatisfythedefinitionatparagraph79ior,forretiredstaff,paragraph79b,thesestaffareeligibletobereturnedasCategoryAstaff.)

b. Visitingprofessors,fellowsandlecturersemployedbyotherHEIs.

Research staff data requirements (form REF1a)84. ThefollowingdataarerequiredonallstaffwhomtheHEIhasselected:

a. HESAstaffidentifier(CategoryAstaffonly).Thisisforverificationandequalopportunitiesmonitoringpurposes.

b. Staffreferencecode:acodedeterminedbytheHEI.

c. Initials.

d. Surname.

e. Dateofbirth(CategoryAstaffonly).

f. Categoryofstaff(AorC)onthecensusdate.

g. ContractedFTEonthecensusdate(CategoryAstaffonly).TheminimumFTEthatmaybereportedis0.2.

h. ForHEFCW-fundedinstitutionsonly:Whethertheindividualisaresearchfellow3(CategoryAstaffonly).

3 HEFCW-funded institutions must indicate which of the staff they submit meet the definition of research fellow that is set out in this footnote. This information is requested for funding purposes. A research fellow holds a specific fellowship award on the basis of their own research record or research proposals. The fellowship award must be to a named individual in recognition of independent research they have undertaken or proposed, must include a significant element of external funding and must follow a process of expert review (including competitive review) involving an input from outside the institution. Such fellowships include Research Council fellows (senior, advanced or postdoctoral) and Royal Society research fellows and professors. Staff on an HEI-funded or awarded fellowship, even with external referees involved in the selection process, may not be identified as a research fellow for REF purposes.

REF 02.2011 19

i. Whethertheindividualisanearlycareerresearcher(ECR),asdefinedinparagraph85.AllsubmittedstaffwhomeetthisdefinitionmustbeidentifiedasECRs,regardlessofhowmanyoutputsarelistedagainsttheirname.

j. Date(day,monthandyear)ofstartingasacademicstaffattheinstitution,ifbetween 1January2008and31October2013(CategoryAonly).

k. Iftheindividualisonafixed-termcontract,secondment,orperiodofunpaidleavethestartandenddates(day,monthandyear)ofthecontract,secondmentorperiodofunpaidleave(CategoryAonly).Staffonrollingcontractsoraseriesofrenewablefixed-termcontractswillberegardedasfixed-termforthispurpose,althoughinstitutionsmaywishtodrawattentiontotheiruseofrollingcontractsinthetextualpartoftheirsubmissions,especiallywhereafixed-termcontracthasanexpirydatesoonafterthecensusdate.

l. ForCategoryAstaffbasedinadiscreteunitoutsidetheUK,detailsoftheconnectionbetweentheirresearchactivityandthesubmittedunitintheUK(seeparagraph79d).

m. Anyresearchgroupsthattheindividualbelongsto,whererelevantanduptoamaximumoffour.Thisisnotamandatoryfield.Somesub-panelsmayaskHEIstodescriberesearchgroupsinREF5(theenvironmenttemplate),butneitherthepresencenorabsenceofresearchgroupsisassumed.

Early career researchers

85. EarlycareerresearchersaredefinedasmembersofstaffwhomeetthecriteriatobeselectedasCategoryAorCategoryCstaffonthecensusdate,andwhostartedtheircareersasindependentresearchersonorafter1August2009.ForthepurposesoftheREF,anindividualisdeemedtohavestartedtheircareerasanindependentresearcherfromthepointatwhich4:

a. Theyheldacontractofemploymentof0.2FTEorgreater,whichincludedaprimaryemploymentfunctionofundertaking‘research’

or‘teachingandresearch’,withanyHEIorotherorganisation,whetherintheUKoroverseas,and

b. Theyundertookindependentresearch,leadingoractingasprincipalinvestigatororequivalentonaresearchgrantorsignificantpieceofresearchwork.(Amemberofstaffisnotdeemedtohaveundertakenindependentresearchpurelyonthebasisthattheyarenamedononeormoreresearchoutputs.)

86. ThefollowingdonotmeetthedefinitionofanECR(thislistisnotexhaustive):

a. Staffwhofirstactedasanindependentresearcherwhileatapreviousemployer–whetheranotherHEI,businessorotherorganisationintheUKorelsewhere–before1August2009,withacontractof0.2FTEorgreater.

b. Staffwhofirstactedasanindependentresearcherbefore1August2009andhavesincehadacareeroutsideofresearchoranextendedbreakfromtheirresearchcareer,beforereturningtoresearchwork.Suchstaffmayreducethenumberofoutputssubmittedaccordingtoparagraph92a.iv(careerbreaks).

c. ResearchassistantswhoareineligibletobereturnedtotheREF,asdefinedinparagraphs 80-81.

87. ECRsmaybesubmittedwithfewerthanfouroutputswithoutpenaltyintheassessment,asdescribedinparagraphs90-100andinthepanelcriteriaandworkingmethodsdocuments.Regardlessofwhetherornottheyaresubmittedwithfewerthanfouroutputs,allstaffincludedinasubmissionwhomeetthedefinitionofanECRmustbeidentifiedasECRsinthesubmission.ThisistoenablethefundingbodiestoanalysetheselectionratesforECRsacrossthesectorasawhole,aspartofourwideranalysisofselectionrates.Toenablethisanalysis,theHESAstaffreturnfor2013-14willincludeafieldforHEIstoidentifyallacademicstaffon‘research’or‘teachingandresearch’contractswhomeettheREFdefinitionofanECR.

4 Main Panel A will provide further details in its criteria document about how junior clinical academics meet this definition.

Par

t 3

Sta

ff

20 REF 02.2011

Individual staff circumstances

Amendment: Following consultation on the

draft panel criteria, the arrangements concerning

maternity, paternity and adoption leave have

been amended. Paragraphs 88-95 are replaced

by paragraphs 64-91 of the ‘Panel criteria and

working methods’ (REF 01.2012) and are

reproduced below.

64. Uptofourresearchoutputsmustbelistedagainsteachmemberofstaffincludedinthesubmission.Amaximumoffouroutputsperresearcherwillprovidepanelswithasufficientselectionofresearchoutputsfromeachsubmittedunituponwhichtobasejudgementsaboutthequalityofthatunit’soutputs.ConsultationsonthedevelopmentoftheREFconfirmedthatthisisanappropriatemaximumvolumeofresearchoutputsforthepurposesofassessment.

65. Asakeymeasuretosupportequalityanddiversityinresearchcareers,inallUOAsindividualsmaybereturnedwithfewerthanfouroutputswithoutpenaltyintheassessment,wheretheirindividualcircumstanceshavesignificantlyconstrainedtheirabilitytoproducefouroutputsortoworkproductivelythroughouttheassessmentperiod.Thismeasureisintendedtoencourageinstitutionstosubmitalltheireligiblestaffwhohaveproducedexcellentresearch.

66. HEIsareallowedtolistthemaximumoffouroutputsagainstanyresearcher,irrespectiveoftheircircumstancesorthelengthoftimetheyhavehadtoconductresearch.AminimumofoneoutputmustbelistedagainsteachindividualsubmittedtotheREF.

67. Inordertoprovideclarityandconsistencyonthenumberofoutputsthatmaybereducedwithoutpenalty,therewillbeaclearly defined reductioninoutputsforthosetypesofcircumstanceslistedatparagraph69a.Circumstancesthataremorecomplexwillrequireajudgementabouttheappropriatereductioninoutputs;thesearelistedatparagraph69b.Arrangementshavebeenputinplacefor complex circumstancestobeconsideredonaconsistentbasis,asdescribedatparagraphs88-91.

68. Whereanindividualissubmittedwithfewer

thanfouroutputsandtheydonotsatisfythecriteriadescribedatparagraphs69-91below,any‘missing’outputswillbegradedas‘unclassified’.

69. CategoryAandCstaffmaybereturnedwithfewerthanfouroutputswithoutpenaltyintheassessment,ifoneormoreofthefollowingcircumstancessignificantlyconstrainedtheirabilitytoproducefouroutputsortoworkproductivelythroughouttheassessmentperiod:

a. Circumstanceswithaclearly defined reductioninoutputs,whichare:

i. Qualifyingasanearlycareerresearcher(onthebasissetoutinparagraph72andTable1below).

ii. Absencefromworkduetoworkingpart-time,secondmentsorcareerbreaks(onthebasissetoutinparagraphs73-74andTable2below).

iii. Qualifyingperiodsofmaternity,paternityoradoptionleave(onthebasissetoutinparagraphs75-81).

iv. OthercircumstancesthatapplyinUOAs1-6,asdefinedatparagraph86.

b. Complex circumstancesthatrequireajudgementabouttheappropriatereductioninoutputs,whichare:

i. Disability.Thisisdefinedin‘guidanceonsubmissions’Part4,Table2under‘Disability’.

ii. Illhealthorinjury.

iii. Mentalhealthconditions.

iv. Constraintsrelatingtopregnancy,maternity,paternity,adoptionorchildcarethatfalloutsideof–orjustifythereductionoffurtheroutputsinadditionto–theallowancesmadeinparagraph75below.

v. Othercaringresponsibilities(suchascaringforanelderlyordisabledfamilymember).

vi. Genderreassignment.

vii.Othercircumstancesrelatingtotheprotectedcharacteristicslistedatparagraph190of‘guidanceofsubmissions’orrelatingtoactivitiesprotectedbyemploymentlegislation.

REF 02.2011

Clearly defined circumstances 70. Whereanindividualhasoneormorecircumstanceswithaclearlydefinedreductioninoutputs,thenumberofoutputsthatmaybereducedshouldbedeterminedaccordingtothetablesandguidanceinparagraphs72-86below.Allsub-panelswillacceptareductioninoutputsaccordingtothisguidanceandwillassesstheremainingnumberofsubmittedoutputswithoutanypenalty.

71. InREF1b,submissionsmustincludesufficientdetailsoftheindividual’scircumstancestoshowthatthesecriteriahavebeenappliedcorrectly.Thepanelsecretariatwillexaminetheinformationinthefirstinstanceandadvisethesub-panelsonwhethersufficientinformationhasbeenprovidedandtheguidanceappliedcorrectly.Thepanelsecretariatwillbetrainedtoprovidesuchadvice,onaconsistentbasisacrossallUOAs.Wherethesub-paneljudgesthatthecriteriahavenotbeenmet,the‘missing’output(s)willberecordedasunclassified.(Forexample,anindividualbecameanearlycareerresearcherinJanuary2011butonlyoneoutputissubmittedratherthantwo.Inthiscasethesubmittedoutputwillbeassessed,andthe‘missing’outputrecordedasunclassified.)

Early career researchers

72. Earlycareerresearchersaredefinedinparagraphs85-86of‘guidanceonsubmissions’.Table1setsoutthepermittedreductioninoutputswithoutpenaltyintheassessmentforearlycareerresearcherswhomeetthisdefinition.

Table 1: Early career researchers: permitted reduction in outputs

Date at which the individual Number of outputs first met the REF definition of may be reduced by an early career researcher: up to:

On or before 31 July 2009 0

Between 1 August 2009 1 and 31 July 2010 inclusive

Between 1 August 2010 2 and 31 July 2011 inclusive

On or after 1 August 2011 3

Absence from work due to part-time working, secondments or career breaks

73. Table2setsoutthepermittedreductioninoutputswithoutpenaltyintheassessmentforabsencefromworkdueto:

a. part-timeworking

b. secondmentsorcareerbreaksoutsideofthehighereducationsector,andinwhichtheindividualdidnotundertakeacademicresearch.

Table 2: Part-time working, secondments or career breaks: permitted reduction in outputs

Total months absent between 1 January 2008 and 31 October 2013 due Number of outputs to working part-time, may be reduced by secondment or career break: up to:

0-11.99 0

12-27.99 1

28-45.99 2

46 or more 3

74. TheallowancesinTable2arebasedonthelengthoftheindividual’sabsenceortimeawayfromworkinginhighereducation.Theyaredefinedintermsoftotalmonthsabsentfromwork.Forpart-timeworking,theequivalent‘totalmonthsabsent’shouldbecalculatedbymultiplyingthenumberofmonthsworkedpart-timebythefull-timeequivalent(FTE)notworkedduringthosemonths.Forexample,anindividualworkedpart-timefor30monthsat0.6FTE.Thenumberofequivalentmonthsabsent=30x0.4=12.

Qualifying periods of maternity, paternity or adoption leave

75. Individualsmayreducethenumberofoutputsbyone,foreachdiscreteperiodof:

a. Statutorymaternityleaveorstatutoryadoptionleavetakensubstantiallyduringtheperiod1January2008to31October2013,regardlessofthelengthoftheleave.

Par

t 3

Sta

ff

REF 02.2011

b. Additionalpaternityoradoptionleave*lastingforfourmonthsormore,takensubstantiallyduringtheperiod1January2008to31October2013.

76. Theapproachtothesecircumstancesisbasedonthefundingbodies’consideredjudgementthattheimpactofsuchaperiodofleaveandthearrivalofanewchildintoafamilyisgenerallysufficientlydisruptiveofanindividual’sresearchworktojustifythereductionofanoutput.Thisjudgementwasinformedbytheconsultationondraftpanelcriteria,inwhichanoverwhelmingmajorityofrespondentssupportedsuchanapproach.

77. Thefundingbodies’decisionnottohaveaminimumqualifyingperiodformaternityleavewasinformedbythesector’sclearsupportforthisapproachintheconsultation;recognitionofthepotentialphysicalimplicationsofpregnancyandchildbirth;andtheintentiontoremoveanyartificialbarrierstotheinclusionofwomeninsubmissions,giventhatwomenweresignificantlylesslikelytobeselectedinformerRAEexercises.

78. Thefundingbodiesconsideritappropriatetomakethesameprovisionforthoseregardedasthe‘primaryadopter’ofachild(thatis,apersonwhotakesstatutoryadoptionleave),astheadoptionofachildandtakingofstatutoryadoptionleaveisgenerallylikelytohaveacomparableimpactonaresearcher’sworktothatoftakingmaternityleave.

79. Asregardsadditionalpaternityoradoptionleave,researcherswhotakesuchleavewillalsohavebeenawayfromworkandactingastheprimarycarerofanewchildwithinafamily.Thefundingbodiesconsiderthatwhereresearcherstakesuchleaveoverasignificantperiod(fourmonthsormore),thisislikelytohaveanimpactontheirabilitytoworkproductivelyonresearchthatiscomparabletotheimpactonthosetakingmaternityorstatutoryadoptionleave.

80. Whiletheclearlydefinedreductionofoutputsduetoadditionalpaternityoradoptionleaveissubjecttoaminimumperiodoffourmonths,shorterperiodsofsuchleavecanbetakenintoaccountasfollows:

a. Byseekingareductioninoutputsundertheprovisionforcomplexcircumstances,forexamplewheretheperiodofleavehadanimpactincombinationwithotherfactorssuchasongoingchildcareresponsibilities.

b. Bycombiningthenumberofmonthsforshorterperiodsofsuchleaveincombinationwithotherclearlydefinedcircumstances,accordingtoTable2.

81. Anyperiodofmaternity,adoptionorpaternityleavethatqualifiesforthereductionofanoutputundertheprovisionsinparagraph75abovemayinindividualcasesbeassociatedwithprolongedconstraintsonworkthatjustifythereductionofmorethanoneoutput.Insuchcases,thecircumstancesshouldbeexplainedusingthearrangementsforcomplexcircumstances.

Combining clearly defined circumstances

82. Whereindividualshavehadacombinationofcircumstanceswithclearlydefinedreductionsinoutputs,thesemaybeaccumulateduptoamaximumreductionofthreeoutputs.Foreachcircumstance,therelevantreductionshouldbeappliedandaddedtogethertocalculatethetotalmaximumreduction.

83. WhereTable1iscombinedwithTable2,theperiodoftimesince1January2008upuntiltheindividualmetthedefinitionofanearlycareerresearchershouldbecalculatedinmonths,andTable2shouldbeapplied.

84. Whencombiningcircumstances,onlyonecircumstanceshouldbetakenintoaccountforanyperiodoftimeduringwhichtheytookplacesimultaneously.(Forexample,anindividualworkedpart-timethroughouttheassessmentperiodandfirst

* ‘Additional paternity or adoption leave’ refers to leave of up to 26 weeks which is taken to care for a child where the person’s spouse, partner or civil partner was entitled to statutory maternity leave or statutory adoption leave, and has since returned to work. The term ‘additional paternity leave’ is often used to describe this type of leave although it may be taken by parents of either gender. For the purposes of the REF we refer to this leave as ‘additional paternity or adoption leave’.

REF 02.2011

metthedefinitionofanearlycareerresearcheron1September2009.Inthiscasethenumberofmonths‘absent’duetopart-timeworkingshouldbecalculatedfrom1September2009onwards,andcombinedwiththereductionduetoqualifyingasanearlycareerresearcher,asindicatedinparagraph83above.)

85. Whereanindividualhasacombinationofcircumstanceswithaclearlydefinedreductioninoutputsandcomplexcircumstances,theinstitutionshouldsubmitthesecollectivelyas‘complex’sothatasinglejudgementcanbemadeabouttheappropriatereductioninoutputs,takingintoaccountallthecircumstances.Thosecircumstanceswithaclearlydefinedreductioninoutputsshouldbecalculatedaccordingtotheguidanceabove(paragraphs72-84).

Other circumstances that apply in UOAs 1-6

86. InUOAs1-6,thenumberofoutputsmaybereducedbyuptotwo,withoutpenaltyintheassessment,forthefollowing:

a. CategoryAstaffwhoarejuniorclinicalacademics.ThesearedefinedasclinicallyqualifiedacademicswhoarestillcompletingtheirclinicaltraininginmedicineordentistryandhavenotgainedaCertificateofCompletionofTraining(CCT)oritsequivalentpriorto31October2013.

b. CategoryCstaffwhoareemployedprimarilyasclinical,healthorveterinaryprofessionals(forexamplebytheNHS),andwhoseresearchisprimarilyfocusedinthesubmittingunit.

87. Theseallowancesaremadeonthebasisthatthestaffconcernedarenormallysignificantlyconstrainedinthetimetheyhaveavailabletoundertakeresearchduringtheassessmentperiod.Thereductionoftwooutputstakesaccountofsignificantconstraintsonresearchwork,andisnormallysufficienttoalsotakeaccountofadditionalcircumstancesthatmayhaveaffectedtheindividual’sresearchwork.Wheretheindividualmeetsthecriteriaatparagraph86,andhashadsignificantadditionalcircumstances–foranyofthereasonsatparagraph69–theinstitutionmayreturnthecircumstancesas‘complex’withareductionofthreeoutputs,andprovideajustificationforthis.

Complex circumstances 88. Wherestaffhavehadoneormorecomplexcircumstances–includingincombinationwithanycircumstanceswithaclearlydefinedreductioninoutputs–theinstitutionwillneedtomakeajudgementontheappropriatereductioninthenumberofoutputssubmitted,andprovidearationaleforthisjudgement.

89. Asfarasispracticable,theinformationinREF1bshouldprovideanestimate–intermsoftheequivalentnumberofmonthsabsentfromwork–oftheimpactofthecomplexcircumstancesontheindividual’sabilitytoworkproductivelythroughouttheassessmentperiod,andstateanyfurtherconstraintsontheindividual’sresearchworkinadditiontotheequivalentmonthsabsent.AreductionshouldbemadeaccordingtoTable2inrelationtoestimatedmonthsabsentfromwork,withfurtherconstraintstakenintoaccountasappropriate.ToaidinstitutionstheEqualityChallengeUnit(ECU)willpublishworkedexamplesofcomplexcircumstances,whichwillindicatehowthesecalculationscanbemadeandtheappropriatereductioninoutputsforarangeofcomplexcircumstances.Thesewillbeavailableatwww.ecu.ac.uk/our-projects/REFfromFebruary2012.

90. AllsubmittedcomplexcircumstanceswillbeconsideredbytheREFEqualityandDiversityAdvisoryPanel(EDAP),onaconsistentbasisacrossallUOAs.ThemembershipandtermsofreferenceoftheEDAPareavailableatwww.ref.ac.ukunderEqualityanddiversity.TheEDAPwillmakerecommendationsabouttheappropriatenumberofoutputsthatmaybereducedwithoutpenaltytotherelevantmainpanelchairs,whowillmakethedecisions.Therelevantsub-panelswillthenbeinformedofthedecisionsandwillassesstheremainingoutputswithoutanypenalty.

91. Toenableindividualstodisclosetheinformationinaconfidentialmanner,informationsubmittedaboutindividuals’complexcircumstanceswillbekeptconfidentialtotheREFteam,theEDAPandmainpanelchairs,andwillbedestroyedoncompletionoftheREF(asdescribedin‘guidanceonsubmissions’,paragraphs98-99).

Par

t 3

Sta

ff

21 REF 02.2011

Individual staff circumstances data requirements (form REF1b)96. Foreachmemberofstaffreturnedwithfewerthanfouroutputs,submissionsmustincludethefollowinginformationinREF1b:

a. Staff with clearly defined circumstances (maximum200words):ForECRs,institutionsmuststatethedateatwhichtheindividualbecameanearlycareerresearcher(meetingthedefinitionatparagraph85);providebriefdetailsoftheirresearchcareerhistory,specificallyidentifyingthepointatwhichtheybecameanindependentresearcher,andthenumberofoutputsreturned.Forstaffwithotherclearlydefinedcircumstances,institutionsmustprovidebriefdetailsaboutthenatureofthecircumstance(s),theirtimingandduration,acalculationofthetotalabsenceovertheperiod1Jan2008to31Oct2013,andthenumberofoutputsreturned.

b. Staff with complex circumstances(maximum300words):Institutionsmust:

• describethenatureandtimingofthecircumstances

• explaintheeffectsontheindividual’scontractedworkinghoursorabilitytofulfiltheircontractedworkinghours

• explainanyothereffectsontheindividual’sabilitytoworkproductively

• provideacalculationforthereductioninoutputsandthenumberofoutputsreturned.

WerecommendthatinpreparingREF1binstitutionsusethetemplateforcomplexcircumstancesavailableonwww.ecu.ac.uk/our-projects/REF.

97. TheinformationreturnedinREF1bforanytypeofcircumstancesmustbebasedonverifiableevidence.

98. InformationsubmittedinformREF1bwillbekeptconfidentialtotheREFteamandthepanelmembers(forclearlydefinedcircumstances)andtheEDAPandmainpanelchairs(forcomplexcircumstances),whoareallsubjecttoconfidentialityundertakingsinrespectofallinformationcontainedinsubmissions.REFsub-panelswillknowthattherearecomplexcircumstancesandwillreceiveadecisionabouttheappropriatenumberofoutputsto

reducewithoutpenalty,butwillnothaveaccesstofurtherinformationaboutthecircumstances.Thesearrangementswillenableindividualstodisclosetheinformationinaconfidentialmanner,andenableconsistenttreatmentofcomplexcircumstancesacrosstheexercise.

99. InformationsubmittedinREF1bwillbeusedonlyforthepurposesofassessingtheREFsubmissioninwhichitiscontained,willnotbepublishedatanytimeandwillbedestroyedoncompletionoftheREF.

100. ItistheresponsibilityoftheHEItoensurethattheinformationinREF1bissubmittedincompliancewiththeDataProtectionAct1998andallotherlegalobligations.

Category C staff details (form REF1c) 101. ForeachindividualreturnedasCategoryC,informationisrequiredinREF1ctodemonstratethattheysatisfythedefinitionofCategoryCstaffatparagraph82,asfollows(maximumof200words):

• Thenameoftheiremployer,jobtitleandspecificresearchresponsibilities.

• Evidencethattheirresearchisprimarilyfocusedinthesubmittingunit.

102. Institutionswillneedtobeabletoprovide,ifaudited,documentedevidenceabouttheiremployment,andthatresearchisincludedintheircontractordocumentedjobdescription.

103. Ifasub-panelisnotsatisfiedbytheevidenceprovidedinREF1cand/orinresponsetoanaudit,itmaytakeaccountofthisinassessingthatindividual’scontributiontothesubmission,ortheindividualmayberemovedfromthesubmission.

104. TheinformationinREF1cwillbepublishedaspartofthesubmission(seeparagraph36).

Part 3 Section 2: Research outputs (REF2)Eligibility definitions for research outputs

(paragraphs 112-113 have been updated)

105. SubmissionsmustincludeuptofouritemsofresearchoutputlistedagainsteachCategoryAorCstaffmemberincludedinthesubmission.Eachoutputmustbe:

a. Theproductofresearch,brieflydefinedasaprocessofinvestigationleadingtonewinsights,

REF 02.2011 22

effectivelyshared.(ThefulldefinitionofresearchforthepurposesoftheREFisatAnnexC.)

b. Firstbroughtintothepublicdomainduringthepublicationperiod,1January2008to 31December2013or,ifaconfidentialreport,lodgedwiththebodytowhomitisconfidentialduringthissameperiod(seeparagraphs 111-113).

c. Producedorauthoredsolely,orco-producedorco-authored,bythememberofstaffagainstwhomtheoutputislisted,regardlessofwherethememberofstaffwasemployedatthetimetheyproducedthatoutput.

106. Inadditiontoprintedacademicwork,researchoutputsmayinclude,butarenotlimitedto:newmaterials,devices,images,artefacts,productsandbuildings;confidentialortechnicalreports;intellectualproperty,whetherinpatentsorotherforms;performances,exhibitsorevents;workpublishedinnon-printmedia.AnunderpinningprincipleoftheREFisthatallformsofresearchoutputwillbeassessedonafairandequalbasis.Sub-panelswillnotregardanyparticularformofoutputasofgreaterorlesserqualitythananotherperse.

107. Reviews,textbooksoreditedworks(includingeditionsoftextsandtranslations)maybeincludediftheyembodyresearchasdefinedinAnnexC.EditorshipsofjournalsandotheractivitiesassociatedwiththedisseminationofresearchfindingsshouldnotbelistedasoutputonREF2.

108. Wheretwoormoreresearchoutputslistedagainstanindividualinasubmissionincludesignificantmaterialincommon(forexample,ajournalarticlethatalsoappearsasachapterinabook)thesub-panelmaydecidetoassesseachoftheseoutputsonlyintermsofthedistinctmaterialincludedineach,orjudgethattheyshouldbetreatedasasingleoutputiftheydonotcontainsufficientlydistinctmaterial.

109. Theses,dissertationsorotheritemssubmittedforaresearchdegreeincludingdoctoralthesesmaynotbelisted.Otherassessablepublisheditemsbasedonresearchcarriedoutforaresearchdegreemaybelisted.

110. HEIsmaynotlistastheoutputofastaffmemberanyoutputproducedbyaresearchassistant

orresearchstudentwhomtheysupervised,unlessthestaffmemberco-authoredorco-producedtheoutput.

Timing of publication

111. Therelevantdatefordeterminingwhetherornotanoutputwasproducedwithinthepublicationperiod,andhenceiseligibleforsubmission,willbethedateatwhichthesubmittedoutputfirstbecamepubliclyavailable(or,forconfidentialreports,waslodgedwiththerelevantbody).Wherethisisneartothestartortheendofthepublicationperiod(1January2008and31December2013respectively)andtheactualdateatwhichitbecamepubliclyavailableisnotclear,wemayrequireHEIstosubmitevidenceofthedateitbecamepubliclyavailable.Inparticular:

a. Wherethedateofimprintonapublicationliesoutsidethepublicationperiodbuttheactualdateofappearanceiswithinthepublicationperiod,evidenceoftheactualdateofappearancewillberequiredfordataverificationpurposes,suchasaletterfromthepublisher.

b. Outputsexpectedtobemadepubliclyavailablebetweenthesubmissiondateandtheendofthepublicationperiod(thatis,between 29November2013and31December2013)shouldbeflaggedinsubmissions;whereonlysomeofthedatarequirementsforthoseoutputscanbesupplied,wewillrequirefulldetailstobesubmittedby31January2014.HEIsmayhavetophysicallysubmitanyoutputsoflaggedforverificationpurposes.Anitemexpectedtobebroughtintothepublicdomainafter 31December2013shouldnotbesubmitted,evenifithasbeenacceptedforpublication.

c. Forwebcontentandelectroniccorpora,HEIswillneedtomaintainproofofthedateatwhichtheitembecamepubliclyavailableandofitscontentatthatdate,forexampleadate-stampedscannedorphysicalprintoutorevidencederivedfromweb-sitearchivingservices.

d. Fornon-textoutputs,suchasperformances,wewillrequireevidenceofwhentheoutputwasdisseminatedinthepublicdomain.

Par

t 3

Ou

tpu

ts

23 REF 02.2011

Amendment: The original definitions at

paragraphs 112-113 have been superseded

by paragraphs 43-44 in the ‘Panel criteria’,

reproduced below.

43. AnoutputfirstpublishedinitsfinalformduringtheREFpublicationperiodthatwas‘pre-published’duringcalendaryear2007–whetherinfullinadifferentform(forexample,an‘onlinefirst’articleorpreprint),orasapreliminaryversionorworkingpaper–iseligibleforsubmissiontotheREF,providedthatthe‘pre-published’outputwasnotsubmittedtothe2008RAE.

44. Otherthantheexceptiondescribedinparagraph43above,anoutputpublishedduringtheREFpublicationperiodthatincludessignificantmaterialincommonwithanoutputpublishedpriorto1January2008iseligibleonlyifitincorporatessignificantnewmaterial.Inthesecases:

a. Thepanelmaytaketheviewthatnotalloftheworkreportedinthelistedoutputshouldbeconsideredashavingbeenissuedwithinthepublicationperiod;andifthepreviouslypublishedoutputwassubmittedtothe2008RAE,thepanelwillassessonlythedistinctcontentoftheoutputsubmittedtotheREF.

b. Submissionsshouldexplainwherenecessaryhowfaranyworkpublishedearlierwasrevisedtoincorporatenewmaterial(seeparagraph127of‘guidanceonsubmissions’).

114. IfanHEIcannotmakeavailablearequestedoutputorprovideevidenceofitspublicationwithinthepublicationperiodthatitemwillberemovedfromthesubmissionandthe‘missing’outputawardedagradeofUnclassified.Therewillbenoopportunitytosubmitasubstituteitem.

Confidential reports

115. Confidentialreportsincludeanyitemproducedforandlodged,inthepublicationperiod,withacompany,governmentbodyorotherresearchsponsor(s),butwhichhasnotbeenpublishedbecauseofitscommercialorothersensitivity.AconfidentialreportmayonlybesubmittediftheHEIhaspriorpermissionfromthesponsoringorganisationthatthe

outputmaybemadeavailableforassessment.HEIswillconfirmpermissionhasbeensecuredwhentheymakesubmissions.IftheREFteamrequestsaconfidentialreportforassessmenttheHEImustmakeitavailable.

116. Allpanelmembers,advisers,observersandothersinvolvedintheassessmentprocessareboundbyaconfidentialityagreement.Therefore,itispossibleforHEIstosubmitconfidentialreportswithoutcompromisinganydutyofconfidentialityuponthem.Theremaybemainorsub-panelmemberswhoHEIsbelievewouldhaveacommercialconflictofinterestinassessingconfidentialreports.HEIswillberequiredtonamesuchindividualswhenmakingsubmissions.

117. Outputsidentifiedbyinstitutionsasconfidentialwillnotbelistedaspartofthepublishedsubmissions.

Data requirements for outputs (form REF2)118. Foreachoutputlisted,enoughinformationshouldbegiventoenabletheREFteamandpanelstodeterminepreciselywhatisbeinglisted,whetheritisaproductofsoleormultipleauthorshiporproduction,inwhatformitexistsandwhereitmaybefound.Thefollowingarerequiredforeachoutput:

a. Output number:sequentiallyfromonetofourforeachindividualreturned.Thisnumberisforadministrativeconvenienceofreferencingonly.ThesubmissionssystemwillalsoenableHEIstoenteranoutputreferencecode,determinedbytheHEI.

b. Year of output:thecalendaryearinwhichtheoutputbecamepubliclyavailable.

c. Type of output:Outputsshouldbecategorisedintothefollowingbroadtypes(therewillbeanumberofspecificdatarequirementsincommonforeachoutputtype;furtherdetailsofthesewillbeprovidedinduecourse):

i. Books(orpartsofbooks).

ii. Journalarticlesandconferencecontributions.

iii. Physicalartefacts.

iv. Exhibitionsandperformances.

v. Otherdocuments.

vi. Digitalartefacts(includingwebcontent).

vii.Other.

d. Title of the output:iftheoutputhasnotitleadescriptionisrequired.

REF 02.2011 24

119. Eachofthefollowingisrequiredwhere applicabletotheoutput:

a. Co-authors: thenumberofadditionalco-authors.

b. Interdisciplinary research:aflagtoindicatetothesub-paneliftheoutputembodiesinterdisciplinaryresearch.

c. The research grouptowhichtheresearchoutputisassigned,ifapplicable.Thisisnotamandatoryfield,andneitherthepresencenorabsenceofresearchgroupisassumed.

d. Request for cross-referral:arequesttothesub-paneltoconsidercross-referringtheoutputtoanothersub-panelforadvice(seeparagraph75d).

e. Request to ’double weight’ the output:foroutputsofextendedscaleandscope,thesubmittinginstitutionmayrequestthatthesub-panelweightstheoutputastwo(seeparagraphs123-126).

f. Additional information:Onlywhererequiredintherelevantpanelcriteria,abriefstatementofadditionalinformationtoinformtheassessment(seeparagraph127).

g. A brief abstract, for outputs in languages other than English(seeparagraph128-130).

Co-authored/co-produced outputs

120. Forco-authoredoutputs,thenumberofotherauthorswillberequired.Regardlessofthenumberofauthorslistedonanoutput,aco-authoredoutputlistedagainstanindividualmemberofstaffwillcountasasingleoutputintheassessment.Co-authored/co-producedoutputswillnotbecountedpro-rata.

121. Wheretwoormoreco-authorsorco-producersofanoutputarereturnedindifferentsubmissions(whetherfromthesameHEIordifferentHEIs),anyorallofthesemaylistthesameoutput.

122. Furtherguidancewillbepublishedinthepanelcriteriadocumentsabout:

a. Whetheradditionalinformationisrequiredaboutthecontributionoftheindividualmemberofstafftoaco-authoredoutput(seeparagraph127b);and,ifso,howthepanelswilltakeaccountofthisinformationwhen

undertakingtheassessment.

b. Whetheraco-authoredoutputmaybelistedagainstmorethanonememberofstaffreturnedwithinthesamesubmission.

Double-weighted outputs

123. Institutionsmayrequestthatoutputsofextendedscaleandscopebedouble-weighted(countastwooutputs)intheassessment.Panelswilldescribeintheircriteriastatementshowtheywilljudgeifanoutputisofsufficientscaleandscopetomeritdouble-weightingintheassessment.Institutions’requestsfordouble-weightingmustbeaccompaniedbyastatementofupto100wordsexplaininghowthescaleandscopeoftheoutputsatisfiesthesecriteria.

124. Nosingleoutputmaybecountedasmorethandouble-weighted(twooutputs).Giventhatamaximumoffouroutputsmustbelistedagainsteachmemberofstaff,submissionsmayrequestthatnomorethantwooutputslistedagainstanindividualmemberofstaffshouldbedouble-weighted.

125. Whererequestinganoutputtobedouble-weighted,thesubmittinginstitutionmustreducethenumberofoutputslistedagainstthatmemberofstaffbyone(unlessa‘reserve’outputispermitted,asdescribedinparagraph126).Thesub-panelswilldecidewhethertodouble-weighteachoutputthathasbeensorequested,accordingtothepublishedcriteria.Thisdecisionwillbeseparatetothepanel’sjudgementaboutthequalityofthatoutput.Wherethepaneldecidestodouble-weightanoutput,itwillcountastwooftheindividualmemberofstaff’smaximumoffouroutputs.Wherethepaneldoesnotacceptthecasefordouble-weighting,itwillcountthesubmittedoutputasasingleoutput,andgradethe‘missing’outputasUnclassified(unlessa‘reserve’outputispermitted).

126. InsomeUOAs,wherethepanelsconsiderthatsuchanapproachisjustifiedintheirdisciplines,institutionsmayincludea‘reserve’outputwitheachoutputrequestedfordouble-weighting.TheseUOAswillbeidentifiedinthepanelcriteriastatements.IntheseUOAs,a‘reserve’outputwillonlybeassessedintheeventthatthepaneldoesnotaccepttherequestfordoubleweighting.

Par

t 3

Ou

tpu

ts

25 REF 02.2011

Additional information

127. IntheircriteriadocumentspanelsmayrequestadditionalinformationrelatingtooutputsifrequiredfortheassessmentofresearchintheirUOA.Thepublishedpanelcriteriawillstatewhich,ifany,ofthefollowingtypesofadditionalinformationarerequired,andprovidefurtherdetailsaboutthenatureoftherequiredinformation,andtheassociatedwordlimitsuptoamaximumof300words.Additionalinformationshouldonlybesubmittedifspecificallyrequestedintherelevantpanelcriteria,otherwiseitwillbedisregardedbythepanel.Anyadditionalinformationprovidedshouldnotbeusedtovolunteeropinionsaboutthequalityofanoutput.Panelsmayrequestanyofthefollowing:

a. Detailsabouttheresearchquestions,methodologyormeansofdissemination,wherethesearenotdescribedwithintheoutputitself.Thisappliestopractice-basedoutputs,forexample,anexhibition,performanceorartefact.

b. Factualinformationaboutthesignificanceoftheoutputwherethisisnotevidentwithintheoutput(forexample,iftheoutputhasgainedexternalrecognition,ledtofurtherdevelopmentsorhasbeenapplied).Citationdatamaynotbeincludedintheadditionalinformation.Wheresub-panelsmakeuseofcitationdataasadditionalinformationabouttheacademicsignificanceofoutputs,asdescribedinparagraphs131-134,thecitationdatawillbeprovidedtopanelsbytheREFteamonaconsistentbasis.

c. Wheretheoutputincludessignificantmaterialpublishedpriorto1January2008,detailsofhowfartheearlierworkwasrevisedtoincorporatenewmaterial(seeparagraph113).

d. Forco-authoredorco-producedoutputs,detailsofthecontributionoftheindividualsubmittedresearchertotheoutput.

Outputs in languages other than English

128. ForresearchoutputsinalanguageotherthanEnglish(includingoutputssubmittedinthemediumofWelsh),ashortabstractinEnglishshouldbeprovidedtodescribethecontentandnatureofthework(maximum100words).AseparatefieldforeachoutputinRA2willbeavailableforthis.Panelswillusethisabstracttoidentifyappropriateexternal

specialistadviserstowhomtheworkmaybereferred.Theabstractsthemselveswillnotformthebasisforassessment.Workmaybereferredtoexternalspecialistadvisersonlywherepanelmembersandassessorsareunabletoassessanoutputinthelanguageinwhichitissubmitted.

129. InthecaseofresearchoutputsinthemediumofWelsh,thespecialistadviser(s)willnormallybepairedwithadesignatedpanelmemberwithwhomtheywilldiscusstheadviceprovided.Ifasub-panelreceivesasubstantialvolumeofresearchoutputsinthemediumofWelsh,thespecialistadviser(s)willbeinvitedtoattendoneormoreofthepanelmeetingsduringtheassessmentphase.TheseprovisionsaremadeinrecognitionoftheparticularlegalstatusoftheWelshlanguageinWales.

130. TherequirementforanabstractiswaivedforoutputssubmittedinUOA28iftheoutputisproducedinanyofthelanguageswithintheremitofthatUOA;andforanyotherUOAsthatindicateintheircriteriastatementsthattheyareabletoassessoutputsinthatlanguage.

Citation data131. Somesub-panelswillconsiderthenumberoftimesthatanoutputhasbeencited,asadditionalinformationabouttheacademicsignificanceofsubmittedoutputs.Thosepanelsthatdosowillcontinuetorelyonexpertreviewastheprimarymeansofassessingoutputs,inordertoreachroundedjudgementsaboutthefullrangeofassessmentcriteria(‘originality,significanceandrigour’).Theywillalsorecognisethesignificanceofoutputsbeyondacademiawhereverappropriate,andwillassessalloutputsonanequalbasis,regardlessofwhetherornotcitationdataisavailableforthem.

132. Panelswillstateintheircriteriadocumentsiftheywillmakeuseofcitationdata,andifso,providefurtherdetailsabouthowtheywillmakeuseofthedatatoinformtheirassessments.Inusingsuchdatapanelswillrecognisethelimitedvalueofcitationdataforrecentlypublishedoutputs,thevariablecitationpatternsfordifferentfieldsofresearch,thepossibilityof‘negativecitations’,andthelimitationsofsuchdataforoutputsinlanguagesotherthanEnglish.Panelswillalsobeinstructedtohavedueregardtothepotentialequalityimplicationsofusingcitationdataasadditionalinformation.5

5 See ‘Analysis of data from the pilot exercise to develop bibliometric indicators for the REF: The effect of using normalised citation scores for particular staff characteristics’ (HEFCE 2011/03). This is available via www.hefce.ac.uk under 2011 publications.

REF 02.2011 26

133. Wheresub-panelsmakeuseofcitationdata,itwillbemadeavailabletothemasfollows:

a. TheREFteamwillprocureasinglesourceofcitationdatathatprovidesagoodlevelofcoverageacrossallUOAsinwhichthesub-panelswillmakeuseofsuchdata.

b. OutputsenteredontotheREFsubmissionsystembyHEIswillbematchedbytheREFteamand/orourcontractorsagainstthisdatabase,usingDOIsandotherbibliographicdataenteredontothesubmissionssystembyHEIs.Institutionswillbeabletoverifythesematchesthroughthesubmissionssystem,andtoviewthecitationcountsinthesameformthattheywillbeprovidedtopanels(althoughwewillcontinuetocountcitationsmadeafterthesubmissiondeadline,toprovidepanelswithup-to-dateinformation).

c. ForallmatchedoutputssubmittedbyHEIsintherelevantUOAs,theREFteamand/orourcontractorwillprovideREFpanelswithacountofthenumberoftimestheoutputhasbeenciteduptoafixedpointintime,atthestartoftheassessmentphase.

134. Allsub-panelsthatmakeuseofcitationdataintheassessmentwillhaveaccesstothedataprovidedonaconsistentandtransparentbasis;submissionsmaynotincludedetailsofcitationswithinanystatementsofadditionalinformationforoutputs.

135. Wewillprovidefurtherdetailsaboutthesourceofthecitationdatainautumn2011,followingthecompletionofaprocurementexercise.

136. Thosepanelsthatusecitationinformationwillcontinuetorelyonexpertreviewastheprimarymeansofassessment.ThefundingbodiesdonotsanctionorrecommendthatHEIsrelyoncitationinformationtoinformtheselectionofstafforoutputsforinclusionintheirsubmissions.Institutionsshouldselectandsubmitoutputsthatintheirjudgementreflecttheirhighestqualityresearchinrelationtothefullrangeofassessmentcriteria(originality,significanceandrigour),andinaccordancewiththeircodesofpracticefortheselectionofstaff(seePart4),havingdueregardtothepotentialequalityimplicationsofusingcitationdata(seefootnote5).

Access to submitted outputs137. TheREFteamwillattempttosourceallsubmittedjournalarticlesandconferenceproceedingsinelectronicformatdirectlyfromthepublishers.WewillthereforerequirethesubmissionofaDOInumberwhereverpossibleforthesetypesofoutput.

138. Forallotheroutputtypes,andwhereweareunabletosourcejournalarticlesandconferenceproceedingsfromthepublishers,wewillrequireinstitutionstomakeavailableeither:

• Theoutputinelectronicformat,whereveravailable.

• Ifnotavailableinelectronicformat,aphysicalcopyoftheoutputorappropriateevidenceoftheoutput.

139. Furtherdetailsofthemethodofsubmissionwillaccompanythepilotversionofthesubmissionsystemsoftwareinautumn2012.

Part 3 Section 3: Impact template and case studies (REF3a/b)Definition of impact for the REF140. ForthepurposesoftheREF,impactisdefinedasaneffecton,changeorbenefittotheeconomy,society,culture,publicpolicyorservices,health,theenvironmentorqualityoflife,beyondacademia(assetoutinparagraph143).

141. Impactincludes,butisnotlimitedto,aneffecton,changeorbenefitto:

• theactivity,attitude,awareness,behaviour,capacity,opportunity,performance,policy,practice,processorunderstanding

• ofanaudience,beneficiary,community,constituency,organisationorindividuals

• inanygeographiclocationwhetherlocally,regionally,nationallyorinternationally.

142. Impactincludesthereductionorpreventionofharm,risk,costorothernegativeeffects.

143. ForthepurposesoftheimpactelementoftheREF:

a. Impactsonresearchortheadvancementofacademicknowledgewithinthehighereducationsector(whetherintheUKorinternationally)areexcluded.(Thesubmitted

Par

t 3

Imp

act

27 REF 02.2011

unit’scontributiontoacademicresearchandknowledgeisassessedwithinthe‘outputs’and‘environment’elementsofREF.)

b. Impactsonstudents,teachingorotheractivitieswithinthesubmittingHEIareexcluded.

c. Otherimpactswithinthehighereducationsector,includingonteachingorstudents,areincludedwheretheyextendsignificantlybeyondthesubmittingHEI.

144. Impactswillbeassessedintermsoftheir‘reachandsignificance’regardlessofthegeographiclocationinwhichtheyoccurred,whetherlocally,regionally,nationallyorinternationally.TheUKfundingbodiesexpectthatmanyimpactswillcontributetotheeconomy,societyandculturewithintheUK,butequallyvaluetheinternationalcontributionofUKresearch.

145. TheREFpanelswillprovidefurtherguidanceinrelationtothekindsofimpactthattheywouldanticipatefromresearchintheirUOAs;thisguidancewillnotberestrictive,andanyimpactthatmeetsthegeneraldefinitionatAnnexCwillbeeligible.

Submission requirements for impact146. TheREFaimstoassesstheimpactofexcellentresearchundertakenwithineachsubmittedunit.Thiswillbeevidencedbyspecificexamplesofimpactsthathavebeenunderpinnedbyresearchundertakenwithintheunitoveraperiodoftime,andbythesubmittedunit’sgeneralapproachtoenablingimpactfromitsresearch.Thefocusoftheassessmentistheimpactofthesubmittedunit’sresearch,nottheimpactofindividualsorindividualresearchoutputs,althoughtheymaycontributetotheevidenceofthesubmittedunit’simpact.

147. Eachsubmissionmustinclude:

a. A completed impact template (REF3a): describingthesubmittedunit’sapproach,duringtheassessmentperiod(1January2008to31July2013),toenablingimpactfromitsresearch.

b. Impact case studies (REF3b):describingspecificimpactsthathaveoccurredduring

theassessmentperiod(1January2008to31July2013)thatwereunderpinnedbyexcellentresearchundertakeninthesubmittedunit.TheunderpinningresearchmusthavebeenproducedbythesubmittingHEIduringtheperiod1January1993to31December20136.

148. Panelswillassessalltheevidenceprovidedinthecompletedimpacttemplate(REF3a)andthesubmittedcasestudies(REF3b),andwillinitiallyformanimpactsub-profileforeachsubmissionbyattributingaweightingof20percenttotheimpacttemplate(REF3a)and80percenttothecasestudies(REF3b).Panelswillapplytheirexpertjudgmentbasedonalltheinformationprovidedintheimpacttemplateandcasestudies,beforeconfirmingtheimpactsub-profiles.

Impact template (form REF3a)149. Submissionsmustincludeacompletedimpacttemplate,describingthesubmittedunit’sapproachduringtheassessmentperiod(1January2008to31July2013)tosupportingandenablingimpactfromresearchconductedwithintheunit.Thisinformationisintendedtoenableamoreholisticandcontextualisedassessmentofimpactthanwouldbepossiblefromcasestudiesalone,throughtheprovisionof:

• contextfortheindividualcasestudies(thoughpanelswillrecognisethatcasestudiesareunderpinnedbyresearchoveratimeframethatislongerthantheassessmentperiod,andthatindividualcasestudiesmaythereforenotrelatedirectlytotheapproachsetoutintheimpacttemplate)

• additionalinformationaboutawiderrangeofactivitywithinthesubmittedunitanditscapacityforimpact,thanmaybecapturedinthecasestudies.

150. Theinclusionoftheimpacttemplatealsoprovidesamechanismfortheassessmenttotakeaccountofparticularcircumstancesofaunitthatmayhaveconstraineditsselectionofcasestudies(forexamplewhereitisanewdepartment,orwherethefocusofitsresearchmayhavelimitedopportunitiesforapplication).

6 The end of the period for the underpinning research (31 December 2013) extends beyond the end of the period for the impact (31 July 2013). This is to align with the end of the publication period for outputs, and recognises that research may have had impact prior to the publication of the outputs. Also, the start of the period for underpinning research may be extended, exceptionally, to 1 January 1988 for some UOAs. Any such exceptions will be published in the panel criteria and working methods documents.

REF 02.2011 28

151. Theimpacttemplatewillseekinformationoneachofthefollowing:

• context

• theunit’sapproachtoimpactduringtheperiod2008-2013

• strategyandplansforsupportingimpact

• therelationshipbetweentheunit’sapproachtoimpactandthesubmittedcasestudies(recognisingthatindividualcasestudiesmaynotrelatedirectlytotheapproach).

152. TheimpacttemplaterecognisesthatthesubmittedunitmaynothavehadaspecificstrategyforimpactinplaceduringtheREFassessmentperiod,andthereforeenablessubmissionstodescribetheirapproachtoimpactduringtheassessmentperiodaswellastheirdevelopmentofastrategyandplansforthefuture.

153. Panelcriteriastatementswillprovidefurtherguidanceonthekindsofinformationandevidenceexpectedwithineachsectionoftheimpacttemplate.Panelswillassesstheimpacttemplateintermsoftheextenttowhichtheunit’sapproachisconducivetoachievingimpactsofreachandsignificance.

154. Thecompletedimpacttemplateshould:

a. Focusprimarilyontheapproachtakenbythesubmittedunittoachievingimpactfromitsresearch–nottheapproachoftheHEIasawhole.However,partofthesubmittedunit’sapproachcouldincludeastatementofhowithasmadeuseofinstitutionalresourcesandinfrastructure,andalignedwithawiderHEIstrategy.

b. Notrepeatdetailedevidencethatisincludedincasestudies,thoughthecompletedimpacttemplatecouldrefertosubmittedcasestudies.

c. Includeevidenceandspecificdetailsorexamplesofthesubmittedunit’sapproach,ratherthanbroadgeneralstatements.

155. Completedimpacttemplatesmustbesubmittedaccordingtotheguidanceonformattingandpagelimits,setoutinAnnexF.

Impact case studies (form REF3b)

Number of case studies in a submission

156. Thenumberofcasestudiesrequiredineachsubmissionwillbedeterminedbythenumber(FTE)ofCategoryAstaffreturnedinthesubmission,assetoutinTable1.Ifasubmissionincludesfewerthantherequirednumberofcasestudies,agradeofunclassifiedwillbeawardedtoeachrequiredcasestudythatisnotsubmitted.Submissionsmaynotincludemorethantherequirednumberofcasestudies.

Table 1: Number of case studies required in submissions

Number of Category Required number of case A staff submitted (FTE) studies

Up to 14.99 2

15 – 24.99 3

25 – 34.99 4

35 – 44.99 5

45 or more 6, plus 1 further case study per additional 10 FTE

157. Submissionswillnotbeexpectedtoprovideimpactcasestudiesthatarerepresentativeofthespreadofresearchactivityacrossthewholesubmittedunit.Institutionsshouldselectthestrongestexamplesofimpactthatareunderpinnedbythesubmittedunit’sexcellentresearch,andshouldexplainwithintheimpacttemplatehowtheselectedcasestudiesrelatetothesubmittedunit’sapproachtoenablingimpactfromitsresearch.

Eligibility definitions for case studies

158. Eachcasestudymustprovidedetailsofaspecificimpactthat:

a. MeetsthedefinitionofimpactfortheREFinAnnexC.

b. Occurredduringtheperiod1January2008to31July2013(seeparagraph159).

c. Wasunderpinnedbyexcellentresearchproducedbythesubmittingunitintheperiod1January1993to31December2013(seeparagraphs160-161andfootnote6).

Par

t 3

Imp

act

29 REF 02.2011

159. Casestudiesmustdescribeimpactsthatoccurredspecificallywithintheperiod1January2008to31July2013.Theimpactsmayhavebeenatanystageofdevelopmentormaturityduringthisperiod,solongassomeeffect,changeorbenefitmeetingthedefinitionofimpactatAnnexCtookplaceduringthatperiod.Thismayinclude,forexample,impactsatanearlystage,orimpactsthatmayhavestartedpriorto1January2008butcontinuedintotheperiod1January2008to31July2013.Casestudieswillbeassessedintermsofthereachandsignificanceoftheimpactthatoccurredonlyduringtheperiod1January2008to31July2013,andnotintermsofanyimpactpriortothisperiodorpotentialfutureoranticipatedimpactafterthisperiod.

160. Tobeeligibleforassessmentasanimpact,theimpactdescribedinacasestudymusthavebeenunderpinnedbyexcellentresearchproducedbythesubmittingunit,duringtheperiod1January1993to31December2013(seefootnote6).Eachcasestudymustdescribetheunderpinningresearch,includereferencestooneormorekeyresearchoutputs,provideevidenceofthequalityofthatresearch,andexplainhowthatresearchunderpinnedorcontributedtotheimpact.FurtherguidanceontheinformationrequiredincasestudiesisatAnnexG.Thefollowingdefinitionsapply:

a. ‘Researchproducedbythesubmittingunitintheperiod1January1993to31December2013’meansthatstaffcarriedoutresearchwithinthescopeoftherelevantUOAdescriptor,whileworkinginthesubmittingHEI(evenifthosestaffhavesinceleft).Thisresearchmustbeevidencedbyoutputsreferencedinthecasestudy,publishedbetween1January1993and31December2013.Thestaffmay,butneednot,havebeenselectedforapreviousRAEorthe2014REF.Theresearchoutputsmay,butneednot,havebeensubmittedtoapreviousRAEorthe2014REF.IfstaffemployedbythesubmittingHEIonthecensusdateconductedalloftheresearchunderpinninganimpactbeforejoiningtheinstitution,thesubmittingHEImaynotsubmittheimpactofthisresearch.(Inthiscase,theinstitutionwherethestaffconductedtheresearchmaysubmittheimpact.)

b. ‘Excellentresearch’meansthatthequalityoftheresearchisatleastequivalenttotwostar:

‘qualitythatisrecognisedinternationallyintermsoforiginality,significanceandrigour’.EachcasestudymustincludereferencestooneormorekeyresearchoutputsthatunderpinnedtheimpactandwereproducedbythesubmittingHEI,andevidenceofthequalityoftheresearchasrequestedintherelevantpanelcriteriadocuments.Panelswillconsidertheevidenceofresearchquality,andmayreviewoutputsreferencedinacasestudy.Apanelwillgradeasunclassifiedacasestudyifitjudgesthattheunderpinningresearchoutputsarenotpredominantlyofatleasttwostarquality.

c. ‘Underpinnedby’meansthattheresearchmadeadistinctandmaterialcontributiontotheimpacttakingplace,suchthattheimpactwouldnothaveoccurredorwouldhavebeensignificantlyreducedwithoutthecontributionofthatresearch.Eachcasestudymustexplainhow(throughwhatmeans)theresearchledtoorcontributedtotheimpact,andincludeappropriatesourcesofinformationexternaltotheHEItocorroboratetheseclaims(seeAnnexG).Wherethepaneljudgesthatthesubmittedunit’sresearchdidnotmakeadistinctandmaterialcontributiontotheimpact,thecasestudywillbegradedasunclassified.

161. Therearemanywaysinwhichresearchmayhaveunderpinnedimpact,includingbutnotlimitedto:

a. Researchthatcontributeddirectlyorindirectlytoanimpact.Forexample,asubmittedunit’sresearchmayhaveinformedresearchinanothersubmittedunit(whetherinthesameoranotherHEI),whichinturnledtoanimpact.Inthiscase,bothsubmittedunitsmayshowthattheirresearchmadeadistinctandmaterialcontributiontotheimpact.

b. Researchembodiedinoneormoreoutputs,conductedbyoneormoreindividuals,teamsorgroups,withinoneormoresubmittedunits,thatledtoorunderpinnedanimpact.Morethanonesubmittedunit(withinthesameHEIorindifferentHEIs)mayincludethesameimpactwithintheirrespectivecasestudies,solongaseachsubmittedunitproducedexcellentresearchthatmadeadistinctandmaterialcontributiontotheimpact.

REF 02.2011 30

c. Impactson,forexample,publicawareness,attitudes,understandingorbehaviourthatarosefromengagingthepublicwithresearch.Inthesecases,thesubmittingunitmustshowthattheengagementactivitywas,atleastinpart,basedonthesubmittedunit’sresearchanddrewmateriallyanddistinctlyuponit.

d. Researchersthatimpactedonothersthroughtheprovisionofprofessionaladviceorexperttestimony.Insuchacase,thesubmittingunitmustshowthattheresearcher’sappointmenttotheiradvisoryrole,orthespecificadvicegiven,wasatleastinpartbasedonthesubmittedunit’sresearchanddrewmateriallyanddistinctlyuponit.

e. ResearchthatledtoimpactthroughitsdeliberateexploitationbytheHEIorthroughitsexploitationbyothers.ThesubmittingHEIneednothavebeeninvolvedinexploitingtheresearch,butmustshowthatitsresearchmadeadistinctandmaterialcontributiontotheimpact.

162. InstitutionsmustsubmitimpactcasestudiesintheappropriateUOAs.ImpactsfromresearchundertakenatthesubmittingHEImaybesubmittedeitherintheREFUOAthatrelatestotheunderpinningresearch,or,ifthisdiffers,totheREFUOAthatrelatestothestaffwhoconductedtheresearch.

Case study submission requirements (form REF3a)

163. Submittingunitsarerequiredtosubmitcasestudiesusingagenerictemplate.Thetemplate,annotatedwithguidance,isatAnnexG.Thetemplatehasbeendeveloped,throughtheimpactpilotexercise,toenablesubmittingunitsinallUOAstoclearlyexplainanddemonstratetheimpactoftheirresearchthroughanarrativethatincludesindicatorsandevidenceasappropriatetothecasebeingmade,andinaformatthatissuitableforpanelstoassessthem.

164. The onus is on submitting units to provideappropriate evidence within each case study of theparticularimpactclaimed.TheREFpanelswillprovideguidance, in the panel criteria documents, about the

kindsofevidenceandindicatorsofimpacttheywouldconsider appropriate to research in their respectiveUOAs, but this guidance will not be exhaustive.

Part 3 Section 4: Environment data (REF4a/b/c)165. TheREFpanelswillformanenvironmentsub-profilebyassessingtheinformationsubmittedinREF5(theenvironmenttemplate),informedbythedatasubmittedinREF4a/b/c,asdescribedinthissection.Whenweprovidesubmissionstosub-panels,wewillsupplyastandardanalysisofthequantitativedatasubmittedinREF4a/b/c,inrespectofeachsubmissioninthatUOA,andaggregatedforallsubmissionsinthatUOA,aslistedinAnnexH.PanelswillconsiderthesedatawithinthecontextoftheinformationprovidedinREF5,andwithinthecontextofthedisciplinesconcerned.Panels’criteriastatementswillindicatehowthedataanalyseswillbeusedininformingtheassessmentoftheresearchenvironment.

Research doctoral degrees awarded (REF4a)166. Eachsubmissionmustincludethenumberofresearchdoctoraldegreesawarded7ineachacademicyear(1August–31July)2008-09to2012-13tostudentssupervisedwithinthesubmittedunit.

167. TheREFteamwillprovidetoinstitutionsdatacollectedbyHESAonthenumbersofresearchdoctoraldegreesawarded,tohelpinpreparingsubmissions.WewillprovidedatacollectedbyHESAforacademicyears(1August–31July)2008-09,2009-10,2010-11and2011-12,atboththelevelofacademiccostcentre,andby2008RAEUOA(basedonthesupervisors’2008RAEUOA).TheREFteamwillnotprovidedatafor2012-13,andinstitutionswillberequiredtocompletethedataforthatyearbasedontheirowndatasourcesorthroughtheirreturnstoHESAforthatyear.

168. Inpreparingtheirsubmissions,institutionsshouldallocatethesedatatotherelevantREFUOAstheyaresubmittingin;ortheymaypreparetheirdatafrominternalsystemsusingHESAdefinitions.In

Par

t 3

En

viro

nm

ent

7 These are students returned in the HESA Student Record whose qualification awarded is recorded as ‘Doctorate degree obtained primarily through advanced supervised research written up as a thesis/dissertation’ or as a ‘New Route PhD’ (currently identified as codes ‘D00’ and ‘D01’ respectively in the QUAL field).

31 REF 02.2011

eithercasetheREFsubmissionsystemwilllimittheextenttowhichthetotalnumberofdoctoraldegreesawardedthatissubmittedbytheinstitutionasawholecanexceedthetotalreportedbytheinstitutiontoHESAineachacademicyearandacrosstheperiod2008-09to2011-12.

169. TheREFteamwillcomparethedatasubmittedtotheREFwiththeHESAdata,andthiswillinformourselectionofsubmissionstobeaudited.Ifaudited,aninstitutionwillneedtoexplainanysignificantvarianceswiththetotalssubmittedtoHESA,andtoshowhowtheyhaveallocateddatatotheappropriateUOAsintheirREFsubmissions.

170. SomeREFpanelsmay,byexception,stateintheirpanelcriteriadocumentsthattheyrequiredataaboutthenumbersofpostgraduateresearch(PGR)studentsregistered,inadditiontothenumberofresearchdoctoraldegreesawarded.Wherethisisthecase,submissionsshouldincludesuchdatawithintherelevantsectionoftheenvironmenttemplate(REF5),andnotonREF4a.

Research income (REF4b) 171. Eachsubmissionmustincludedataonthesubmittedunit’sexternalresearchincomeforeachacademicyear2008-09to2012-13,accordingtoHESAdefinitionsofresearchincomeintheFinanceStatisticsReturn(FSR)table5b.Incomewillbelistedagainstthefollowingsources:

a. BISResearchCouncils,TheRoyalSociety,BritishAcademyandTheRoyalSocietyofEdinburgh.

b. UK-basedcharities(opencompetitiveprocess).

c. UK-basedcharities(other).

d. UKcentralgovernmentbodies,localauthorities,healthandhospitalauthorities(forsubmissionsinUOAs1-6,incomefromspecificbodiesthatfundhealthresearchshouldbereportedinaseparateline–seeparagraph172).

e. UKindustry,commerceandpubliccorporations.

f. EUgovernmentbodies.

g. EU-basedcharities(opencompetitiveprocess).

h. EUindustry,commerceandpubliccorporations.

i. EUother.

j. Non-EU-basedcharities(opencompetitiveprocess).

k. Non-EUindustry,commerceandpubliccorporations.

l. Non-EUother.

m. Othersources.

172. ForsubmissionsinallUOAsinMainPanelA(UOAs1-6),researchincomeawardedthroughopencompetitionfromthefollowingbodiesshouldbeexcludedfrom‘UKcentralgovernmentbodies’(sub-paragraphdabove)andreturnedinaseparateline:

• theNationalInstituteofHealthResearch(NIHR)

• theChiefScientistOffice,ScottishGovernmentHealthandSocialCareDirectorates

• theWelshGovernment’sNationalInstituteforSocialCareandHealthResearch(NISCHR)

• HealthandSocialCareResearch&Development(HSCR&D),NorthernIreland.

Thesedatawillbereportedtopanelsalongsideincomefrom‘BISResearchCouncils’(sub-paragraphaabove),reflectingthecompetitivenatureofsuchincome.

173. TheREFteamwillprovidetoinstitutionsresearchincomedatacollectedbyHESAinFSRtable5bforacademicyears(1August–31July)2008-09,2009-10,2010-11and2011-12,atthelevelofacademiccostcentre,bysourceofincome.For2012-13,theREFteamwillnotprovidethedataandinstitutionswillberequiredtocompletethedataforthisyearbasedontheirowndatasourcesorthroughtheirreturnstoHESAforthatyear.

174. Inpreparingtheirsubmissions,institutionsshouldallocatethesedatatotherelevantUOAstheyaresubmittingin;ortheymaypreparetheirdatafrominternalsystemsusingthesamedefinitionsasHESAFSRtable5b.IneithercasetheREFsubmissionsystemwilllimittheextenttowhichthetotalincomesubmittedbytheinstitutiontotheREFcanexceedthetotalsreportedbytheinstitutiontoHESAineachyear(totalforallsources)andacrosstheassessmentperiodasawhole,byeachsource.TheREFteamwillcomparethedatasubmittedtotheREFwiththeHESAdata,andthiswillinformourselectionof

REF 02.2011 32

submissionstobeaudited.Ifaudited,aninstitutionwillneedtoexplainanysignificantvarianceswiththetotalssubmittedtoHESA,andtoshowhowtheyhaveallocateddatatotheappropriateREFUOAs.

175. WhenreportingincometotheHESAFSRinthepast,institutionsinScotlandandWalesmaynothavesplittheirresearchincomefromUKcharitiesaccordingtowhetherornotthecharityawardedthegrantthroughanopen,competitiveprocess.ThesplitwasonlycompulsoryforinstitutionsinEnglandandNorthernIrelandforfundingpurposes.However,forthepurposesofREFsubmissions,institutionsinScotlandandWalesshouldseparateouttheircharityresearchincomeaccordingtowhetherornotthecharityawardedthegrantthroughanopencompetitiveprocess,usingtheguidancesetoutintheFSRguidancenotesforthecorrespondingyear.

Ineligible income

176. WewillexcluderesearchincomethatispassedontootherinstitutionsororganisationsaspartofacollaborativeprojectorsubcontractedworktotheextentthatthisisreflectedintheHESAdata.Webelievethatthevastmajorityofthisincomewouldhavebeenexcludedfromthe2008-09HESAdata.For2009-10onwards,anextrarowaddedtotheresearchgrantsandcontractstablewillenableustoidentifyandexcludethisincome.

Other notes

177. Whereagrantorcontractisheld,orworkconducted,acrossmorethanoneUOAitshouldbedividedbetweensubmissionsindifferentUOAsaccordingtothewaytheincomehasbeenused.Researchprojectswhicharefundedfromseveralsourcesshouldhavetheirincomeallocatedundertherespectiveheadingstoreflecttheactualsourceoftheincome.

Research income-in-kind (REF4c)178. TheestimatedvalueofResearchCouncilfacilitytimeallocatedthroughpeerreviewandusedbyresearchersatsubmittedunitswillbeprovidedtoinstitutionsforuseinpreparingsubmissionsandshouldbereturnedinREF4c.ForsubmissionsinUOAs1-6,wewillalsoprovidetheestimatedvalueofequivalentincome-in-kindfromthehealthresearchfundingbodieslistedinparagraph172.

179. TheleadResearchCouncilresponsibleforaccessandfundingofanamedfacilitywillsupplydataonthevalueoftheseallocationstotheHEIsconcernedandtotheREFteam.WeexpectthattheResearchCouncilsandthebodieslistedinparagraph172willsupplydatarelatingtotheperiod1August2008to31July2013,andthesewillbemadeavailabletoHEIsinsufficienttimetobeincludedintheirsubmissions.Wewillconsultwiththesebodiesonthearrangementsandwillissuefurtherguidance,includingconfirmationoftheprovisionaldatesforthesupplyofdatatoHEIs.

180. Theincome-in-kinddatawillbereportedtopanelsalongsideresearchincomefrom‘BISResearchCouncils’andwillbeidentifiedinaseparateline.

181. TheREFteamwillcomparethedataonincome-in-kindprovidedbytheResearchCouncilsandthebodieslistedinparagraph172withthosesubmittedtotheREF,andthiswillinformourselectionofsubmissionstobeaudited.Ifaudited,aninstitutionwillneedtoexplainanysignificantvariancesandtoshowhowtheyhaveallocateddatatotheappropriateREFUOAs.

182. SomeREFpanelsmay,byexception,stateintheircriteriadocumentsthattheyrequiredataaboutotherspecificsourcesofincome-in-kind.Wherethisisthecase,submissionsshouldincludesuchdatawithintherelevantsectionoftheenvironmenttemplate(REF5)andnotonREF4c.

Part 3 Section 5: Environment template (REF5)183. Informationisrequiredabouttheresearchenvironmentforeachsubmittingunitrelatingtotheperiod1January2008to31July2013.EachsubmissionmustincludeasinglecompletedREF5form,consistingofthefollowingsections:

• Overview.

• Researchstrategy.

• People,including:

– Staffingstrategyandstaffdevelopment.

– Researchstudents.

• Income,infrastructureandfacilities.

• Collaborationandcontributiontothediscipline.

Par

t 3

En

viro

nm

ent

33 REF 02.2011

184. Whereasubmissionincludesstafffromdistinct‘departments’orotherorganisationalunits,thesubmissionshouldexplainthisandanydistinctiveaspectsoftheresearchenvironmentsoftheseorganisationalunits,withineachsectionoftheenvironmenttemplate.Thereisnoexpectationthattheenvironmentelementofasubmissionwillrelatetoasingledepartmentorcoherentorganisationalunit.

185. DetailedguidanceontherequirementsforthecontentofREF5willbeprovidedinthepanelcriteriadocuments.TheywillstateanyquantitativeindicatorsthatshouldbeincludedinREF5andwillindicatethenatureoftheevidencethatshouldbeprovidedabouthowthesubmittedunitpromotesequalityanddiversity.ThecriteriastatementswilldescribehowpanelswillusetheinformationinformREF5togetherwiththedatainformsREF4a/b/cinassessingthesubmissionstoformtheenvironmentsub-profiles.Inallcases,eachsectionoftheenvironmenttemplate(excludingtheOverview)willbesignificantininformingtheenvironmentsub-profile,andpanelswillprovidefurtherdetailsoftheirrelativeimportance.

186. Completedenvironmenttemplatesmustbesubmittedaccordingtotheguidanceonformattingandpagelimits,setoutinAnnexF.

REF 02.2011 34

Introduction187. ThepurposeoftheguidanceinPart4istosupportinstitutionsinpromotingequalityanddiversitywhenpreparingsubmissionstotheREF,throughdrawingupandimplementingacodeofpracticeonthefairandtransparentselectionofstaff.Thiswillaidinstitutionsinincludingalltheireligiblestaffinsubmissionswhoareconductingexcellentresearch,aswellaspromotingequality,complyingwithlegislationandavoidingdiscrimination.

188. Eachinstitutionmakingasubmissionisrequiredtodevelop,documentandapplyacodeofpracticeonselectingstafftoincludeintheirREFsubmissions.Onmakingsubmissions,theheadofinstitutionwillberequiredtoconfirmadherencetothiscode.Thefundingbodiesrequirethatinstitutions’codesofpracticebesubmittedtotheREFteamby31July2012.TheEqualityandDiversityAdvisoryPanel(EDAP)willexaminetheseinadvanceofthesubmissiondeadline,andallinstitutions’codeswillbepublishedwiththerestofthesubmissionsattheendoftheassessmentprocess.

189. WehavedevelopedtheguidanceinPart4withadvicefromtheREFEqualitiesandDiversityAdvisoryGroup(EDAG),anddrawingonareviewofthe2008RAEcodesofpractice.ItisintendedtoassistHEIsindrawingupacodeofpracticethatframestheirdecision-makingprocessesinrelationtotheREF2014inthecontextoftheprinciplesofequalitiesanddiversity,andallrelevantlegislation.ForthoseinstitutionsthatdonotalreadyhaveanequalitycodegoverningtheirREFpreparations,itofferssuggestionsonprocedurestheymightconsiderincluding.FurthersupportondevelopingandapplyingacodeofpracticewillbeontheECUweb-sitewww.ecu.ac.uk/our-projects/REFfromSeptember2011.

The legislative context190. TheEqualityAct2010harmonisedandconsolidatedpreviousanti-discriminationlegislation.TheActcoverstheprotectedcharacteristicsof:

• age

• disability

• genderreassignment

• marriageandcivilpartnership

• pregnancyandmaternity

• race

• religionorbelief

• sex

• sexualorientation.

191.AswellasprohibitingdirectdiscriminationtheActprohibitsindirectdiscrimination–followingapolicythat,althoughappliedequallytoeveryone,isharderforthosewithaprotectedcharacteristictocomplywith.IndirectdiscriminationisnotabreachoftheActifitisaproportionatemeansofachievingalegitimateaim.Directdiscriminationonthegroundsofagewillnotbeunlawfulifitisaproportionatemeansofachievingalegitimateaim.

192. Withtheexceptionsofmarriageandcivilpartnershipandpregnancyandmaternity,protectionfromdiscriminationextendstopeoplewhoareperceivedtohaveorareassociatedwithsomeonewhohasaprotectedcharacteristic.Forexample,ifaresearcheristreatedlessfavourablybecausetheycarefortheirdisabledparentthatcouldbeunlawfuldisabilitydiscrimination.

193. Similarlytopreviouslegislation,itislawfultotreatadisabledpersonmorefavourablythananondisabledperson,andpublicbodiesincludingHEIsarerequiredtomakereasonableadjustmentsfordisabledpeople.

194. TheEqualityActplacesrequirementsonthefundingbodiesaspublicsectororganisationsandonHEIsaspublicsectororganisationsandemployers.MostoftheAct,asitrelatestopublicfunctionsandemployment,cameintoforceinOctober2010.TheActcoversEngland,ScotlandandWales.Apartfromminorprovisions,NorthernIrelandisnotcoveredbytheAct.

195. ThepublicsectorequalitydutyoftheActcameintoforceinApril2011.Underthepublicsectorequalityduty,thehighereducationfundingbodiesandHEIsinEngland,ScotlandandWales,incarryingouttheirfunctions,musthavedueregardtotheneedto:

• Eliminatediscrimination,harassment,victimisationandanyotherconductthatisprohibitedbyorundertheAct.

• Advanceequalityofopportunitybetweenpersonswhosharearelevantprotectedcharacteristicandpersonswhodonotshareit.

Part 4Codes of practice on the selection of staff

Par

t 4

Co

des

of

pra

ctic

e

35 REF 02.2011

• Fostergoodrelationsbetweenpersonswhosharearelevantprotectedcharacteristicandpersonwhodonotshareit.

(Inthiscontexta‘relevant’protectedcharacteristicisoneotherthanmarriageandcivilpartnership.)

196. ThescopeofequalitylegislationinNorthernIrelandissimilartotheEqualityAct.Inaddition,itisunlawfultodiscriminateagainstpeopleonthegroundsofpoliticalopinion.Section75oftheNorthernIrelandAct1998alsoplacesastatutoryobligationontheDepartmentforEmploymentandLearningNorthernIrelandandHEIsinNorthernIrelandincarryingouttheirpublicfunctionstohavedueregardtotheneedtopromoteequalityofopportunity:

• betweenpersonsofdifferentreligiousbelief,politicalopinion,racialgroup,age,maritalstatusorsexualorientation;

• betweenmenandwomengenerally;

• betweenpersonswithadisabilityandpersonswithout;and

• betweenpersonswithdependantsandpersonswithout.

Funding bodies’ legal responsibilities197. Inordertodemonstratecompliancewiththepublicsectorequalitydutyandsection75oftheNorthernIrelandAct,thehighereducationfundingbodiesneedtoconsiderandunderstandtheimpactoftheirpoliciesonequality.ThefundingbodieshaveconsideredtheequalityimpactoftheRAEinthedevelopmentoftheREF,andequalityhasbeenembeddedintoallrelevantelementsoftheREF.Thefundingbodieswillanalysetheselectionratesbystaffcharacteristicsatsectorleveltoinformtheirfuturework.

HEIs’ legal responsibilities198. Asbothemployersandpublicbodies,HEIsneedtoensurethattheirREFproceduresdonotdiscriminateunlawfullyagainstindividualsbecauseofage,disability,genderidentity,marriageandcivilpartnership,race,religionorbelief,sexorsexualorientationorbecausetheyarepregnantorhaverecentlygivenbirth.Inaddition,inNorthernIreland,HEIsmustensurethattheirproceduresdonotdiscriminateonthegroundsofpoliticalopinion.

199. WhendevelopingtheirREFprocedures,HEIswillalsoneedtobemindfulthatunderthefixed-termemployeeandpart-timeworkersregulations,fixed-termemployeesandpart-timeworkershavetherightnottobetreatedbyanemployeranylessfavourablythantheemployertreatscomparableemployeesonopencontractsorfull-timeworkers.Therelevantregulationsare:

• Part-timeWorkers(PreventionofLessFavourableTreatment)Regulations2000

• Part-timeWorkers(PreventionofLessFavourableTreatment)Regulations(NorthernIreland)2000

• Fixed-termEmployees(PreventionofLessFavourableTreatment)Regulations2002

• Fixed-termEmployees(PreventionofLessFavourableTreatment)Regulations(NorthernIreland)2002.

200. Aspublicsectororganisations,inordertoshowcompliancewithrequirementsofthepublicsectorequalitydutyoftheEqualityAct2010,HEIsinEngland,ScotlandandWalesneedtoconsiderandunderstandtheeffectoftheirREFpoliciesonequality.EqualitieslegislationinNorthernIrelandandWalesplacesaspecificdutyonHEIstoconductequalityimpactassessmentsonnewandexistingpolicies.Consequently,thefundingbodiesrequireallHEIstoconductequalityimpactassessmentsontheirpoliciesforselectingstafffortheREF.

Summary of legislation201. Asummaryoftheequalitylegislationwithwhichinstitutionshavetocomplygenerally,andwhichtheyshouldtakeintoaccountwhenpreparingREF2014submissionsisincludedinTable2.Panelchairs,membersandsecretarieshavereceivedabriefingaboutthislegislation(see‘EqualitybriefingforREFpanels’availableatwww.ref.ac.ukunderPublications).ThebriefinginstructsthemtodevelopworkingmethodsandassessmentcriteriathatencourageHEIstosubmittheworkofalloftheirexcellentresearchers,includingthosewhoseabilitytoproducefouroutputsorworkproductivelythroughouttheassessmentperiodhadbeenconstrainedforreasonscoveredbyequalitylegislation.

REF 02.2011 36

Table 2: Summary of equality legislation

Age All employees within the higher education sector are protected from unlawful age discrimination in employment under the Equality Act 2010 and the Employment Equality (Age) Regulations (Northern Ireland) 2006. Individuals are also protected if they are perceived to be or if they are associated with a person of a particular age group. (These provisions in the Equality Act 2010 are partially in force, but should be fully in place by April 2012.)

Age discrimination can occur when people of a particular age group are treated less favourably than people in other age groups. An age group could be for example, people of the same age, the under 30s or people aged 45-50. A person can belong to a number of different age groups.

Age discrimination will not be unlawful if it is a proportionate means of achieving a legitimate aim. However, in the context of the REF, the view of the funding bodies is that if a researcher produces excellent research an HEI will not be able to justify not submitting them because of the their age group.

It is important to note that early career researchers are likely to come from a range of age groups. The definition of early career researcher used in the REF (see paragraph 85) is not limited to young people.

HEIs should also note that given developments in equalities law in the UK and Europe, the default retirement age will be abolished from 1 October 2011 in England, Scotland, Wales and Northern Ireland.

Disability The Equality Act 2010, the Disability Discrimination Act (1995) (Northern Ireland only) and the Disability Discrimination (Northern Ireland) Order 2006 prevent unlawful discrimination relating to disability. Individuals are also protected if they are perceived to have a disability or if they are associated with a person who is disabled (for example, if they are responsible for caring for a disabled family member).

A person is considered to be disabled if they have or have had a physical and/or mental impairment which has ‘a substantial and long-term adverse effect on their ability to carry out normal day-to-day activities’. Long-term impairments include those that last or are likely to last for at least 12 months.

Cancer, HIV, multiple sclerosis and progressive/degenerative conditions are disabilities too, even if they do not currently have an adverse effect on the carrying out of day-to-day activities.

The definition of disability is different in Northern Ireland in that a list of day-to-day activities is referred to. There is no list of day-to-day activities for England, Scotland and Wales but day-to-day activities are taken to mean activities that people, not individuals, carry out on a daily or frequent basis.

While there is no definitive list of what is considered a disability, it covers a wide range of impairments including:

• sensoryimpairments

• impairmentswithfluctuatingorrecurringeffectssuchasrheumatoidarthritis,depressionandepilepsy

• progressiveimpairments,suchasmotorneuronedisease,musculardystrophy,HIVandcancer

• organspecificimpairments,includingrespiratoryconditionsandcardiovasculardiseases

• developmentalimpairments,suchasautisticspectrumdisordersanddyslexia

• mentalhealthconditionssuchasdepressionandeatingdisorders

• impairmentscausedbyinjurytothebodyorbrain.

It is important for HEIs to note that people who have had a past disability are also protected from discrimination, victimisation and harassment because of disability.

Equality law requires HEIs to anticipate the needs of disabled people and make reasonable adjustments for them. Failure to make a reasonable adjustment constitutes discrimination. If a disabled researcher’s impairment has affected the quantity of their research outputs, they may be submitted with a reduced number of outputs (see paragraphs 90-100 and the panel criteria).

Par

t 4

Co

des

of

pra

ctic

e

37 REF 02.2011

Gender The Equality Act 2010 and the Sex Discrimination (Northern Ireland) Order 1976 reassignment protect from discrimination trans people who have proposed, started or completed a

process to change their sex. Staff in HE do not have to be under medical supervision to be afforded protection because of gender reassignment and staff are protected if they are perceived to be undergoing or have undergone gender reassignment. They are also protected if they are associated with someone who has proposed, is undergoing or has undergone gender reassignment.

Trans people who undergo gender reassignment will need to take time off for appointments and in some cases, for medical assistance. The transition process is lengthy, often taking several years and it is likely to be a difficult period for the trans person as they seek recognition of their new gender from their family, friends, employer and society as a whole.

The Gender Recognition Act 2004 gave enhanced privacy rights to trans people who undergo gender reassignment. A person acting in an official capacity who acquires information about a person’s status as a transsexual may commit a criminal offence if they pass the information to a third party without consent.

Consequently, staff within HEIs with responsibility for REF submissions must ensure that the information they receive about gender reassignment is treated with particular care.

Staff whose ability to work productively throughout the REF assessment period has been constrained due to gender reassignment may be submitted with a reduced number of research outputs (see paragraphs 90-100 and the panel criteria). Information about the member of staff will be kept confidential as described in paragraph 98.

Marriage and civil Under the Equality Act 2010 and the Sex Discrimination (Northern Ireland) Order partnership 1976 as amended, individuals are protected from unlawful discrimination on the

grounds of marriage and civil partnership status. The protection from discrimination is to ensure that people who are married or in a civil partnership receive the same benefits and treatment in employment. The protection from discrimination does not apply to single people.

In relation to the REF HEIs must ensure that their processes for selecting staff do not inadvertently discriminate against staff who are married or in civil partnerships.

Political opinion The Fair Employment and Treatment (Northern Ireland) Order 1998 protects staff from unlawful discrimination on the grounds of political opinion.

HEIs should be aware of not making any judgements about the selection of staff for REF submissions based on their political opinion.

Pregnancy and Under the Equality Act 2010 and the Sex Discrimination (Northern Ireland) Order maternity 1976 women are protected from unlawful discrimination related to pregnancy and

maternity.

Consequently researchers who have taken time out of work or whose ability to work productively throughout the assessment period because of pregnancy and/or maternity, may be submitted with a reduced number of research outputs, as set out in paragraphs 90-100 and in the panel criteria documents.

In addition, HEIs should ensure that female researchers who are pregnant or on maternity leave are kept informed about and included in their submissions process.

For the purposes of this summary it is important to note that primary adopters have similar entitlements to women on maternity leave.

REF 02.2011 38

Race The Equality Act 2010 and the Race Relations (Northern Ireland) Order 1997 protect HEI staff from unlawful discrimination connected to race. The definition of race includes colour, ethnic or national origins or nationality. Individuals are also protected if they are perceived to be or are associated with a person of a particular race.

HEIs should be aware of not making any judgements about the selection of staff for REF submissions based on their race or assumed race (for example, based on their name).

Religion and The Equality Act 2010 and the Fair Employment and Treatment (Northern Ireland) Order belief including 1998 protect HEI staff from unlawful discrimination to do with religion or belief. Individuals non-belief are also protected if they are perceived to be or are associated with a person of a particular religion or belief.

HEIs should be aware of not making any judgements about the selection of staff for REF

submissions based on their actual or perceived religion or belief, including non-belief. ‘Belief’ includes any structured philosophical belief with clear values that has an effect on how its adherents conduct their lives.

Sex (including The Equality Act 2010 and the Sex Discrimination (Northern Ireland) Order 1976 breastfeeding protect HEI staff from unlawful discrimination to do with sex. Employees are also and additional protected because of their perceived sex or because of their association with someone paternity and of a particular sex. adoption leave) The sex discrimination provisions of the Equality Act explicitly protect women from less favourable

treatment because they are breastfeeding. Consequently the impact of breastfeeding on a women’s ability to work productively will be taken into account, as set out in paragraph 90-100 and the panel criteria documents.

From 3 April 2011, partners of new mothers and secondary adopters will be entitled to up to 26 weeks of additional paternity and adoption leave. People who take additional paternity or adoption leave will have similar entitlements to women on maternity leave and barriers that exist to taking the leave, or as a result of having taken it, could constitute unlawful sex discrimination. Consequently researchers who have taken additional paternity and adoption leave may be submitted with a reduced number of outputs, as set out in paragraphs 90-100 and in the panel criteria documents.

HEIs need to be wary of selecting researchers by any criterion that it would be easier for men to comply with than women, or vice versa. There are many cases where a requirement to work full-time(orlessfavourabletreatmentofpeopleworkingpart-timeorflexibly)hasbeenheldtodiscriminate unlawfully against women.

Sexual The Equality Act 2010 and the Employment Equality (Sexual Orientation) Regulations orientation (Northern Ireland) 2003 protect HEI staff from unlawful discrimination to do with sexual

orientation. Individuals are also protected if they are perceived to be or are associated with a person who is of a particular sexual orientation.

HEIs should be aware of not making any judgements about the selection of staff for REF submissions based on their actual or perceived sexual orientation.

Welsh Language The Welsh Language Act 1993 places a duty on public bodies in Wales to treat Welsh and English on an equal basis. This is reinforced by the provisions of the Welsh Language (Wales) Measure 2011.

The arrangements for the assessment of outputs in the medium of Welsh by the REF panels are set out in paragraphs 128-130.

Par

t 4

Co

des

of

pra

ctic

e

39 REF 02.2011

Guidance to institutions202. ItisarequirementoftheREFthateachsubmittinginstitutionestablishesacodeofpracticeontheselectionofstaffforREFsubmissions.Theguidancehereprovidesacommonframeworkforinstitutions’developmentofpoliciesandprocedureswithintheREF,butdoesnotgenerateobligationsbeyondthosethatHEIswillinanycaseneedtoaddress.ItistheresponsibilityofHEIstoensurethattheircodesofpractice,andthemannerinwhichtheyparticipateintheREF,arelawful.

203. InstitutionsthatconductmockREFexercisesmightconsiderusingthemasanopportunitytoapplytheirdraftcodeandrefineitfurther.Whereexternaladvisorsareusedinsuchexercises,institutionsshouldbriefthemabouttheirdevelopingREFcodesofpracticeand,inparticular,provideguidanceabouttheappropriatelevelofdetailedcommentbyexternaladvisersonindividuals.Mockexercisesshouldincludeequalityimpactassessmentsaspartoftheprocess,andtheimplicationsoftheseshouldbeconsideredwhenpreparingthefinalsubmission.

Principles204. Eachinstitution’scodeofpracticeshoulddemonstratefairnesstoitsstaffbyaddressingthefollowingprinciples:

a. Transparency:AllprocessesfortheselectionofstaffforinclusioninREFsubmissionsshouldbetransparent.Codesofpracticeshouldbedrawnupandmadeavailableinaneasilyaccessibleformatandpublicisedtoallacademicstaffacrosstheinstitution,includingonthestaffintranet,anddrawntotheattentionofthoseabsentfromwork.Wewouldexpecttheretobeaprogrammeofcommunicationactivitytodisseminatethecodeofpracticeandexplaintheprocessesrelatedtoselectionofstaffforsubmission.Thisshouldbedocumentedinthecode.Weencourageinstitutionstopublishtheircodesofpracticeontheirexternalweb-site,andtheywillbepublishedbytheREFteamaspartofthesubmissions.

b. Consistency:Itisessentialthatpolicyinrespectofstaffselectionisconsistentacrosstheinstitutionandthatthecodeofpracticeisimplementeduniformly.Thecodeofpractice

shouldsetouttheprinciplestobeappliedtoallaspects/stagesoftheprocessatalllevelswithintheinstitutionwheredecisionswillbemade.

c. Accountability:Responsibilitiesshouldbeclearlydefined,andindividualsandbodiesthatareinvolvedinselectingstaffforREFsubmissionsshouldbeidentifiedbynameorrole.Codesshouldalsostatewhattrainingthosewhoareinvolvedinselectingstaffwillhavehad.Operatingcriteriaandtermsofreferenceforindividuals,committees,advisorygroupsandanyotherbodiesconcernedwithstaffselectionshouldbemadereadilyavailabletoallindividualsandgroupsconcerned.

d. Inclusivity: Thecodeshouldpromoteaninclusiveenvironment,enablinginstitutionstoidentifyalleligiblestaffwhohaveproducedexcellentresearchforsubmissiontotheREF.

Staff and committees205. StructuraldifferencesbetweenHEIsmeanthatthemethodofdevelopingsubmissionsandthepositionsofindividualsresponsibleforselectionwillnotbeuniformacrossthesector.Theproceduresforidentifyingdesignatedstaff(evenaseniorofficersuchastheprovice-chancellorofresearch)andestablishingcommitteesresponsibleforselectingstaffshouldbeclearlydocumentedinthecodeofpractice,asshouldtheirtermsofreference.

206. StaffwithsuchresponsibilitiesmustbeprovidedwithtrainingonequalityanddiversitywhichhasbeentailoredtotheREFprocesses.ThecodeshoulddetailtheequalitytrainingthatdesignatedpersonswilleitherundertakeorhaveundertakensincetheintroductionoftheEqualityAct2010,orthelevelofunderstandingoftheissuestheywillberequiredtoattain.

207. Werecommendthattrainingincludecasestudiesthatareusedtoexploreissuessuchastheimplicationsofdealingwithpersonalcircumstancesintheprocessofselectingstaffforinclusioninthesubmission.(TrainingmaterialwillbeavailableontheECUweb-sitewww.ecu.ac.uk/our-projects/REF.)

208. Cleardefinitionsofeachperson’srolewithintheselectionprocessmustbeprovided,includingtherationalefortheirroleandwheretherolefitsintotheinstitutionalmanagementframework.

REF 02.2011 40

209. WhereacommitteeorcommitteeshavedesignatedREFresponsibilities–whetheritisatdepartmental,faculty,UOAorcentrallevel–theseshouldbedetailedinthecodeofpractice,including,foreachcommittee:

• howthecommitteehasbeenformed

• itsmembership

• thedefinitionofitspositionwithintheadvisoryordecision-makingprocess

• thestepstakentoensurethatmembersarewellinformedabouttheirownandtheinstitution’slegalobligationsregardingequality.

210. Thefollowingdetailsshouldbeprovidedaboutitsmodeofoperation:

• thecriteriathatitwilluseincarryingoutitsfunctions

• themethodbywhichthesecriteriaarecommunicated

• thetimescaleforselectingstaff

• themethodandtimescaleinwhichfeedbackwillbeprovidedinrespectofthedecisionsmade

• theappealsmechanism.

211. Wherecommitteesconsiderreportsfromothercommitteesordesignatedstaffreportingtothem,thebasisofthediscussionmustbeclear,andrecordsmustbekept.Whenindividualperformanceisdiscussedandtheindividualisabsent,committeesshouldbemadeawareofallthefactsrelatingtotheindividual.

Equality impact assessment212. ThefundingbodiesrequireallHEIstoconductanequalityimpactassessment(EIA)ontheirpolicyandproceduresforselectingstafffortheREF.WhilethefundingbodiesrecognisethatthereisnolongeraprescribedprocessforconductinganEIA,itshouldbeathoroughandsystematicanalysistodeterminewhethertheinstitution’sstaffselectionpolicyfortheREFmayhaveadifferentialimpactonparticulargroups.Itshouldinformtheinstitution’scodeofpracticeandbekeptunderreviewassubmissionsareprepared.

213. EIAsshouldbeinformedbyananalysisofdataonstaffwhoareeligibleforselectioninrespectofalltheprotectedcharacteristicsforwhichdataareavailable.Theanalysisshouldcoveralleligiblestaff.

Thefundingbodiesrecognisethatitisbestpracticetouseinformationgainedfromengaging,consultingorinvolvingstafffromprotectedgroupstoinformanEIA.(Consultationisarequirementofsection75oftheNorthernIrelandAct1998andengagementisarequirementoftheWelshspecificdutiesoftheEqualityAct2010.EngagementisalsokeytoshowingdueregardtotherequirementsofthepublicsectorequalitydutyinEnglandandScotland.)

214. HEIswhoconductmockexercisescanusetheprocesstoinformtheirEIA,andHEIscanalsoconsiderinformationonequalityfrompreviousRAEsinidentifyingpossiblebarrierstoparticipationandopportunitiestoadvanceequality.

215. TheEIAshouldbereviewedatkeystagesoftheselectionprocess,toensurethatanynecessarychangestopreventdiscriminationorpromoteequalityaretakenpriortothesubmissiondeadline.ExamplesofappropriatepointstoreviewtheEIAare:

• whenidentifyingeligiblestaffwhoarelikelytobeselected

• whenconsideringappeals

• whenpreparingthefinalsubmission.

216. EIAsshouldenableHEIstoidentifywherediscriminationmayinadvertentlyoccurwithintheirREFprocesses.TheywillalsoenableHEIstoidentifywhereaparticularpolicyorpracticehasapositiveimpactontheadvancementofequality.WherepotentialdiscriminationisidentifiedHEIswillneedtojustifythepolicyorpracticewithintheconstraintsofthelawortheywillneedtotakeactionstochangethepolicyorpractice.Ifaparticularpolicyorpracticeisfoundtohaveapositiveimpactonequality,HEIscanseektoapplyittootherareasoftheirREFwork.

217. HEIsmayconsiderundertakingimpactassessmentsatthelevelofUOAsaswellasattheinstitutionallevel,forexampleiftheyperceiveimbalanceinparticularUOAs.

218. ThefundingbodiesexpectHEIstopublishtheirequalityimpactassessmentsafterthesubmissionshavebeenmade,asamatterofgoodpractice.Thepublishedinformationshouldincludetheoutcomesofanyactionstakentopreventdiscriminationoradvanceequality.PublicationisalegalrequirementinNorthernIrelandandinWaleswhereapolicyorpracticeisregardedashavingasignificantimpact.

Par

t 4

Co

des

of

pra

ctic

e

41 REF 02.2011

Disclosure of individual staff circumstances219. GuidancehasbeenproducedonhowREFpanelswilldealwithindividualcircumstancesthatconstrainedanindividual’sabilitytoproducefouroutputsorworkproductivelythroughouttheassessmentperiod.Theinstitution’scodeofpracticeontheselectionofstaffmustdrawattentiontothisguidanceandbeclearonhowsuchcircumstancesshouldbedeclaredbystaff.

220. Institutionsshouldhaverobustprocedurestoenablestafftodisclosetheircircumstanceswithanappropriatedegreeofconfidentiality.Particularregardshouldbehadtothedisclosureofsensitiveissuessuchasongoingillnessormentalhealthconditions.Werecommendthatthisisconductedproactively:insteadofrelyingonindividualscomingforward,allstaffpotentiallyeligibleforselectionshouldbeaskedtocompleteaformabouttheirindividualcircumstances.Toenableindividualstodisclosecircumstancesinconfidence,institutionsshouldconsidermanagingthisprocesscentrally.

221. Examplesofcomplexindividualstaffcircumstances(seesub-paragraph92bforanexplanationofcomplex)willbeavailableontheECUweb-sitewww.ecu.ac.uk/our-projects/REF,indicatingtheappropriatereductioninoutputsforparticularcircumstances.Werecommendthatinstitutionsuseoradaptthetemplateusedfortheseexamples,whenpreparingsubmissions.

222. Ateachselectionstage,institutionsareencouragedtoformallymonitortheprocessforidentifyingindividualswhosecircumstancesmightneedspecialconsideration,andtoevidencedecisionsandactions.

223. Theinstitutionalcodeofpracticemustincludealistofcircumstancesthatwillbetakenintoaccount,andthemechanismsbywhichpanelsandtheinstitutionwilltakethemintoaccount.Theinstitution’sapproachmustbeconsistentwiththerangeofcircumstancesandproceduressetoutinboththeguidanceonsubmissionsandpanelcriteriastatements,andmustbestandardacrossalldepartments.

Fixed term and part-time staff224. InthelightoftheFixed-termandPart-timeRegulations(seeparagraph199),considerationwillalsoneedtobegiventohowtheinstitutioncan

demonstratetheimplementationofequalityforthoseonfixed-term(relativetoopen)andpart-time(relativetofull-time)contracts.

225. Thecodeofpracticeshouldthereforeincludeastatementabouthowtheinstitutionsupportsitsfixed-termandpart-timestaff,includingcontractresearchstaff,inrelationtoequalityanddiversity.

Joint submissions226. Institutionsmakingjointsubmissionsmaywishtomaketheircodeofpracticeavailabletocollaboratinginstitutions.Inanycase,theyshouldensurethatjointdecision-makingacrossinstitutionsdoesnotcompromisetheiradherencetotheirrespectivecodesofpractice.

Feedback and appeals227. Appropriateandtimelyproceduresshouldbeputinplacetoinformstaffwhoarenotselectedofthereasonsbehindthedecision,andforappeals.Appealsproceduresshouldallowmembersofstafftoappealaftertheyhavereceivedthisfeedback,andforthatappealtobeconsideredbytheHEIbeforethefinalselectionismade.Theindividualsthathandleappealsshouldbeindependentofthedecisionsaboutselectingstaffandshouldreceiveappropriatetraining.

228. Thecodeofpracticeshouldincludedetailsoftheappealsprocedures.Institutionsshouldconsidercarefullybeforedecidingtouseexistingcomplaintreportingmechanisms,andshouldonlydosoiftheseareappropriateforthepurposeandcanbeconcludedpriortotheREFsubmissiondeadline.

Submitting codes of practice229. CodesofpracticeshouldbesubmittedtotheREFteamonorbefore31July2012.TheEDAPwillexaminethecodesandadvisetheUKfundingbodiesontheiradherencetothisguidance,priortothefinalsubmissiondeadline.IftheEDAPadvisesthemthataninstitution’scodeofpracticedoesnotadheretothisguidancethefundingbodieswilltakeappropriateaction.

230. Allsubmittedcodesofpracticewillbepublishedaspartofinstitutions’submissions,aftertheconclusionoftheREF.

231. ThefundingbodiesplantoundertakeanevaluationofthecodesofpracticeaftertheREFhasconcluded.AswiththeRAE,weanticipatethiswillidentifyareasofgoodpractice.

REF 02.2011 42

Useful resources232. Furtherinformation,includingthetextoflegislationandexamplesofgoodpracticecanbeaccessedthroughECU’sweb-siteat www.ecu.ac.uk/our-projects/REF,from September2011.

Par

t 4

Co

des

of

pra

ctic

e

43 REF 02.2011

1. Thisannexsetsoutthegenericcriteriaforassessingsubmissionsandthedefinitionsofthestarredlevelsintheoverallqualityprofilesandeachofthesub-profiles(foroutputs,impactandenvironment).

2. Sub-panelswillusetheirprofessionaljudgementtoformtheoverallqualityprofiletobeawardedtoeachsubmission,takingintoaccountalltheevidencepresented.Theprimaryoutcomeoftheassessmentwillbeanoverallqualityprofileawardedtoeachsubmission,showingtheproportionofthesubmissionthatmeetseachstarredlevelintheprofile.

3. Thedefinitionsofthestarredlevelsintheoverallqualityprofilearebelow.

Table A1: Overall quality profile: Definitions of starred levels

Four star Quality that is world-leading in terms of originality, significance and rigour.

Three star Quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence.

Two star Quality that is recognised internationally in terms of originality, significance and rigour.

One star Quality that is recognised nationally in terms of originality, significance and rigour.

Unclassified Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment.

4. Foreachofthethreeelementsoftheassessment–outputs,impactandenvironment–sub-panelswilldevelopasub-profile,showingtheproportionofthesubmissionthatmeetseachoffourstarredqualitylevels.Theassessmentcriteriaandthedefinitionsofthestarredlevelsforthesub-profilesaresetoutbelow.

Table A2: Outputs sub-profile: Criteria and definitions of starred levels

The criteria for assessing the quality of outputs are ‘originality, significance and rigour’.

Four star Quality that is world-leading in terms of originality, significance and rigour.

Three star Quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence.

Two star Quality that is recognised internationally in terms of originality, significance and rigour.

One star Quality that is recognised nationally in terms of originality, significance and rigour.

Unclassified Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment.

Annex AAssessment criteria and level definitions

REF 02.2011 44

Table A3: Impact sub-profile: Criteria and definitions of starred levels

The criteria for assessing impacts are ‘reach and significance’:

• Inassessingtheimpactdescribedwithinacasestudy,thepanelwillformanoverallviewaboutits‘reachand significance’ taken as a whole, rather than assess ’reach and significance‘ separately.

• Inassessingtheimpacttemplate(REF3a)thepanelwillconsidertheextenttowhichtheunit’sapproachdescribed in the template is conducive to achieving impacts of ‘reach and significance’.

Four star Outstanding impacts in terms of their reach and significance.

Three star Very considerable impacts in terms of their reach and significance.

Two star Considerable impacts in terms of their reach and significance.

One star Recognised but modest impacts in terms of their reach and significance.

Unclassified The impact is of little or no reach and significance; or the impact was not eligible; or the impact was not underpinned by excellent research produced by the submitted unit.

Table A4: Environment sub-profile: Criteria and definitions of starred levels

The research environment will be assessed in terms of its ‘vitality and sustainability’. Panels will consider both the ‘vitality and sustainability’ of the submitted unit, and its contribution to the ‘vitality and sustainability’ of the wider research base.

Four star An environment that is conducive to producing research of world-leading quality, in terms of its vitality and sustainability.

Three star An environment that is conducive to producing research of internationally excellent quality, in terms of its vitality and sustainability.

Two star An environment that is conducive to producing research of internationally recognised quality, in terms of its vitality and sustainability.

One star An environment that is conducive to producing research of nationally recognised quality, in terms of its vitality and sustainability.

Unclassified An environment that is not conducive to producing research of nationally recognised quality.

5. Thefourmainpanelswillexplaininmoredetail,withintheirstatementsonthepanelcriteriaandworkingmethods,howtheirgroupofsub-panelswillapplytheassessmentcriteriaandinterprettheleveldefinitionsindevelopingthesub-profiles.

6. Themethodforcombiningthesub-profilesintotheoverallqualityprofileisexplainedatAnnexB.

Ann

exes

45 REF 02.2011

Notes on the criteria and definitions of the starred levels7. ‘World-leading’qualitydenotesanabsolutestandardofqualityineachunitofassessment.

8. ‘Worldleading’,‘internationally’and‘nationally’inthiscontextrefertoqualitystandards.Theydonotrefertothenatureorgeographicalscopeofparticularsubjects,nortothelocusofresearchnoritsplaceofdissemination.Forexample,researchwhichisfocusedwithinonepartoftheUKmightbeof‘worldleading’standard.Equally,workwithaninternationalfocusmightnotbeof‘worldleading,internationallyexcellentorinternationallyrecognised’standard.

9. Thecriterionof‘reach’forimpactsdoesnotreferspecificallytoageographicscale.Sub-panelswillconsideranumberofdimensionstothe‘reach’ofimpactsasappropriatetothenatureoftheresearchanditsimpacts.Forexample,animpactlocatedwithinoneregionoftheUKmightbejudgedas‘outstanding’(gradedasfourstar).Equally,animpactwithinternationalreachmightnotbejudgedas‘outstanding’,‘veryconsiderable’or‘considerable’.

10. Theprofileforasubmissionthatcontainsnoresearchwhichmeetstheonestarthresholdwillbe100percentunclassified.Asubmissionthatcontainsnoresearch(thatis,noworkthatmeetsthedefinitionofresearchfortheREF)willnotbeawardedaqualityprofile.

REF 02.2011 46

1. Theoverallqualityprofileswillshowtheproportionofresearchactivityinasubmissionjudgedtomeetthedefinitionsateachstarredlevel.Theoverallqualityprofileswillbepublishedinstepsof1percent.TableB1showsoverallqualityprofilesfortwofictionaluniversities.

Table B1 Sample overall quality profiles*

Unit of FTE Category A Percentage of research activity in the submission judged assessment A staff submitted to meet the standard for: four three two one unclassified star star star star

University X 50.45 18 41 25 16 0

University Y 65.2 12 32 45 10 1

* The figures are for fictional universities. They do not indicate expected proportions.

2. Sub-panelswillproducetheoverallqualityprofilesbyassessingthreedistinctelementsoftheassessment–researchoutputs,impactandtheresearchenvironment–toproduceasub-profileforeachelement.Thethreesub-profileswillbeaggregatedtoformtheoverallqualityprofileforthesubmission,witheachelementweightedasfollows:

• Outputs:65percent

• Impact:20percent

• Environment:15percent.

3. Theroundingmethodologydescribedinparagraphs4-7ofthisannexwillbeusedtoproducetheoverallqualityprofiles.Inrecommendingtheoverallqualityprofilestothemainpanels,sub-panelswillconfirmthat,intheirexpertjudgement,theoverallqualityprofileisafairreflectionoftheresearchactivityinthatsubmission,andthattheirassessmenthastakenaccountofallthedifferentcomponentsofthesubmission.

Figure B1 Building a quality profile: a worked example

Ann

exes

Annex BQuality profiles

47 REF 02.2011

Rounding 4. Thesub-profileswillbecombinedusingtheweightsinparagraph2ofthisannex.Acumulativeroundingprocesswillthenbeappliedtothecombinedprofile,toproduceanoverallqualityprofile.Thismethodologywillensurethattheoverallqualityprofileforanysubmissionwillalwayssumto100percentandtoavoidtheunfairconsequencesthatsimpleroundingcanproduce.

Worked example 5. UsingtheexampleinFigure1,firstcalculatetheinitialoverallprofile,thatis,thesumoftheweightedsub-profilesforoutputs,environmentandimpact.

Starred levels

4* 3* 2* 1* u/c

Outputs 12.8 32.8 43 11.4 0

Environment 0 40 40 20 0

Impact 20 45 35 0 0

Weighted

65% 8.3 21.3 28 7.4 0

15% 0 6 6 3 0

20% 4 9 7 0 0

Initial profile 12.3 36.3 41 10.4 0

6. Cumulativeroundingworksinthreestages:

a. Theinitialprofileis:

4* 3* 2* 1* u/c

12.3 36.3 41 10.4 0

b. Stage1:Calculatethecumulativetotals(forexamplethecumulativetotalat3*orbetteris12.3+36.3=48.6).

4* 3* or better 2* or better 1* or better u/c or better

12.3 48.6 89.6 100 100

c. Stage2:Roundthesetothenearest1percent(roundingupifthepercentageendsinexactly0.5).

4* 3* or better 2* or better 1* or better u/c or better

12 49 90 100 100

d. Stage3:Findthedifferencesbetweensuccessivecellstogivetheroundedprofile.So,forexample,thepercentageallocatedto2*isthedifferencebetweenthecumulativetotalat2*orbetter,minusthecumulativetotalat3*orbetter(90-49=41).

4* 3* 2* 1* u/c

12 37 41 10 0

7. Cumulatingthetotalstheotherway(roundingdownifthepercentageendsinexactly0.5)givesexactlythesameanswer.

REF 02.2011 48

Definition of research for the REF1. ForthepurposesoftheREF,researchisdefinedasaprocessofinvestigationleadingtonewinsights,effectivelyshared.

2. Itincludesworkofdirectrelevancetotheneedsofcommerce,industry,andtothepublicandvoluntarysectors;scholarship8;theinventionandgenerationofideas,images,performances,artefactsincludingdesign,wheretheseleadtoneworsubstantiallyimprovedinsights;andtheuseofexistingknowledgeinexperimentaldevelopmenttoproduceneworsubstantiallyimprovedmaterials,devices,productsandprocesses,includingdesignandconstruction.Itexcludesroutinetestingandroutineanalysisofmaterials,componentsandprocessessuchasforthemaintenanceofnationalstandards,asdistinctfromthedevelopmentofnewanalyticaltechniques.Italsoexcludesthedevelopmentofteachingmaterialsthatdonotembodyoriginalresearch.

3. Itincludesresearchthatispublished,disseminatedormadepubliclyavailableintheformofassessableresearchoutputs,andconfidentialreports(asdefinedatparagraph115inPart3,Section2).

Definition of impact for the REF4. ForthepurposesoftheREF,impactisdefinedasaneffecton,changeorbenefittotheeconomy,society,culture,publicpolicyorservices,health,theenvironmentorqualityoflife,beyondacademia(assetoutinparagraph7).

5. Impactincludes,butisnotlimitedto,aneffecton,changeorbenefitto:

• theactivity,attitude,awareness,behaviour,capacity,opportunity,performance,policy,practice,processorunderstanding

• ofanaudience,beneficiary,community,constituency,organisationorindividuals

• inanygeographiclocationwhetherlocally,regionally,nationallyorinternationally.

6. Impactincludesthereductionorpreventionofharm,risk,costorothernegativeeffects.

7. ForthepurposesoftheimpactelementoftheREF:

a. Impactsonresearchortheadvancementofacademicknowledgewithinthehighereducationsector(whetherintheUKorinternationally)are excluded.(Thesubmittedunit’scontributiontoacademicresearchandknowledgeisassessedwithinthe‘outputs’and‘environment’elementsofREF.)

b. Impactsonstudents,teachingorotheractivitieswithinthesubmittingHEIareexcluded.

c. Otherimpactswithinthehighereducationsector,includingonteachingorstudents,areincludedwheretheyextendsignificantlybeyondthesubmittingHEI.

Ann

exes

Annex CDefinitions of research and impact for the REF

8 Scholarship for the REF is defined as the creation, development and maintenance of the intellectual infrastructure of subjects and disciplines, in forms such as dictionaries, scholarly editions, catalogues and contributions to major research databases.

49 REF 02.2011

Main panel Unit of assessment

A 1 Clinical Medicine

2 Public Health, Health Services and Primary Care

3 Allied Health Professions, Dentistry, Nursing and Pharmacy

4 Psychology, Psychiatry and Neuroscience

5 Biological Sciences

6 Agriculture, Veterinary and Food Science

B 7 Earth Systems and Environmental Sciences

8 Chemistry

9 Physics

10 Mathematical Sciences

11 Computer Science and Informatics

12 Aeronautical, Mechanical, Chemical and Manufacturing Engineering

13 Electrical and Electronic Engineering, Metallurgy and Materials

14 Civil and Construction Engineering

15 General Engineering

C 16 Architecture, Built Environment and Planning

17 Geography, Environmental Studies and Archaeology

18 Economics and Econometrics

19 Business and Management Studies

20 Law

21 Politics and International Studies

22 Social Work and Social Policy

23 Sociology

24 Anthropology and Development Studies

25 Education

26 Sport and Exercise Sciences, Leisure and Tourism

D 27 Area Studies

28 Modern Languages and Linguistics

29 English Language and Literature

30 History

31 Classics

32 Philosophy

33 Theology and Religious Studies

34 Art and Design: History, Practice and Theory

35 Music, Drama, Dance and Performing Arts

36 Communication, Cultural and Media Studies, Library and Information Management

Annex DUnits of assessment

REF 02.2011 50

March 2010 Publication of ‘Initial decisions’ by the funding bodies on the conduct of the REF (HEFCE Circular letter 04/2010)

July 2010 Publication of ‘Units of assessment and recruitment of expert panels’ (REF 01.2010)

November 2010 Publication of reports on the REF impact pilot exercise

February 2011 Panel membership announced

March 2011 Publication of ‘Decisions on assessing research impact’ (REF 01.2011)

July 2011 Publication of ‘Assessment framework and guidance on submissions’ (REF 02.2011)

End July 2011 Publication of draft panel criteria and working methods for consultation

5 October 2011 Close of consultation on draft panel criteria and working methods

January 2012 Publication of panel criteria and working methods

31 July 2012 Institutions intending to make submissions to the REF submit their codes of practice on the selection of staff

Autumn 2012 Pilot of the REF submissions system

October 2012 Invitation to HEIs to make submissions; invitation to request multiple submissions; and start of survey of submissions intentions

December 2012 Survey of submissions intentions complete and deadline for requests for multiple submissions

January 2013 Launch of submissions systems and accompanying technical guidance

31 July 2013 End of assessment period (for research impacts, the research environment, and data about research income and research doctoral degrees awarded)

Mid 2013 Appointment of additional assessors to panels

31 October 2013 Census date for staff eligible for selection

29 November 2013 Closing data for submissions

31 December 2013 End of publication period (cut-off point for publication of research outputs, and for outputs underpinning impact case studies)

Throughout 2014 Panels assess submissions

December 2014 Publication of outcomes

Spring 2015 Publication of submissions, panel overview reports and sub-profiles

Ann

exes

Annex ETimetable

51 REF 02.2011

Format1. TemplatesforREF3a,REF3bandREF5willbeprovidedtoinstitutionsinWord.CompletedtemplatesandcasestudiesmustbesubmittedasPDFdocuments.PDFdocumentsmustbeaccessibletoscreenreadingtechnology(ratherthanscanneddocuments).Theymustadheretothefollowing:

• Arialfont,11point(minimum)

• singlelinespacing(minimum)

• 2cmmargins(minimum).

2. Completedtemplatesmayincludeformatting(boldorunderlinedtext,headings,lists,andsoon),tablesandnon-textcontent,solongastheguidanceonmaximumpagelimitsandonminimumfontsize,linespacingandmarginwidthsareadheredto.

Page limits3. Allcasestudies(REF3b)willbelimitedtofour pages,includingallreferences(exceptforthepersonaldetailsofthecorroboratingsourceslistedinsection5ofthecasestudytemplate,whichwillbecollectedinaseparateformtomakethemeasiertoomitfrompublication).

4. Themaximumpagelimitsfortheimpacttemplate(REF3a)andtheenvironmenttemplate(REF5)willdependonthetotalFTECategoryAstaffincludedinthesubmission,accordingtoTableF1.

Annex FFormat and page limits for textual parts of submissions

Table F1: Page limits for REF3a and REF5

Number of Category A Page limit for Page limit for staff in the submission impact template environment template (FTE) (REF3a) (REF5)

1 – 14.99 3 7

15 – 24.99 3 8

25 – 34.99 3 9

35 – 44.99 4 10

45 – 54.99 4 11

55 – 74.99 4 12

75 or more 5, plus 1 further 13, plus 1 further page per additional page per additional 60 FTE 20 FTE

REF 02.2011 52

1. Thisannexprovidesthetemplateforimpactcasestudies,annotatedwithguidanceabouttheinformationrequiredineachofitssections.ThisshouldbereadalongsidethedefinitionsandeligibilitycriteriaforimpactcasestudiesinPart3,Section3ofthemaindocument,andalongsidethepanelcriteriastatements.ThecasestudytemplateforuseinpreparingsubmissionswillbeprovidedinWord,alongwithtemplatesforREF3aandREF5,ontheREFsubmissionsystem.

2. Eachcasestudyshouldincludesufficientlyclearanddetailedinformationtoenablepanelstomakejudgementsbasedontheinformationitcontains,withoutmakinginferences,gatheringadditionalmaterial,followingupreferencesorrelyingonmembers’priorknowledge.Referencestoothersourcesofinformationwillbeusedforverificationpurposesonly,notasameansforpanelstogatherfurtherinformationtoinformjudgements.

3. Eachcompletedcasestudytemplatewillbelimitedto four pages inlength(seeAnnexF).Withintheannotatedtemplatebelow,indicativeguidanceisprovidedabouttheexpectedmaximumlengthlimitofeachsection,butinstitutionswillhaveflexibilitytoexceedthesesolongasthecasestudyasawholeremainsnolongerthan four pagesandtheguidanceonformattingatAnnexFisadheredto.

Impact case study template (REF3b)

Title of case study:

1. Summary of the impact (indicative maximum 100 words)

Thissectionshouldbrieflystatewhatspecificimpactisbeingdescribedinthecasestudy.

2. Underpinning research (indicative maximum 500 words)

This section should outline the key research insights or findings that underpinned the impact, and provide details of what research was undertaken, when, and by whom. References to specific research outputs that embody the research described in this section, and evidence of its quality, should be provided in the next section.

Details of the following should be provided in this section:

• Thenatureoftheresearchinsightsorfindingswhichrelatetotheimpactclaimedinthecasestudy.

• Anoutlineofwhattheunderpinningresearchproducedbythesubmittedunitwas(thismayrelatetooneormore research outputs, projects or programmes).

• Datesofwhenitwascarriedout.

• Namesofthekeyresearchersandwhatpositionstheyheldattheinstitutionatthetimeoftheresearch(where researchers joined or left the HEI during this time, these dates must also be stated).

• Anyrelevantkeycontextualinformationaboutthisareaofresearch.

Ann

exes

Annex GImpact case study template and guidance

53 REF 02.2011

3. References to the research (indicative maximum of six references)

This section should provide references to key outputs from the research described in the previous section, and evidence about the quality of the research.

Include the following details for each cited output:

• Author(s).

• Title.

• Yearofpublication.

• Typeofoutputandotherrelevantdetailsrequiredtoidentifytheoutput(forexamplejournaltitleandissue).

• Detailstoenablethepaneltogainaccesstotheoutput,ifrequired(forexample,aDOIorURL),orstatingthat the output is listed in REF2 or can be supplied by the HEI on request.

All outputs cited in this section must be capable of being made available to panels. If they are not available in the public domain or listed in REF2, the HEI must be able to provide them if requested by the REF team.

Evidence of the quality of the research must also be provided in this section. Guidance on this will be provided in the panel criteria documents. Where panels request details of key research grants or end of grant reports, the following should be provided:

• Whothegrantwasawardedto.

• Thegranttitle.

• Sponsor.

• Periodofthegrant(withdates).

• Valueofthegrant.

4 Details of the impact (indicative maximum 750 words)

This section should provide a narrative, with supporting evidence, to explain:

• howtheresearchunderpinned(madeadistinctandmaterialcontributionto)theimpact

• thenatureandextentoftheimpact.

The following should be provided:

• Aclearexplanationoftheprocessormeansthroughwhichtheresearchledto,underpinnedormadeacontributiontotheimpact(forexample,howitwasdisseminated,howitcametoinfluenceusersorbeneficiaries, or how it came to be exploited, taken up or applied).

• Wherethesubmittedunit’sresearchwaspartofawiderbodyofresearchthatcontributedtotheimpact(for example, where there has been research collaboration with other institutions), the case study should specify the particular contribution of the submitted unit’s research and acknowledge other key research contributions.

• Detailsofthebeneficiaries–whoorwhatcommunity,constituencyororganisationhasbenefitted,beenaffected or impacted on.

• Detailsofthenatureoftheimpact–howtheyhavebenefitted,beenaffectedorimpactedon.

• Evidenceorindicatorsoftheextentoftheimpactdescribed,asappropriatetothecasebeingmade.

• Datesofwhentheseimpactsoccurred.

REF 02.2011 54

5. Sources to corroborate the impact (indicative maximum of 10 references)

This section should list sources external to the submitting HEI that could, if audited, provide corroboration of specific claims made in the case study. Sources provided in this section should not be a substitute for providing clear evidence of impact in section 4; the information in this section will be used for audit purposes only.

This section should list sufficient sources that could, if audited, corroborate key claims made about the impact of the unit’s research. These could include, as appropriate to the case study, the following external sources of corroboration (stating which claim each source provides corroboration for):

• Reports,reviews,weblinksorotherdocumentedsourcesofinformationinthepublicdomain.

• Confidentialreportsordocuments(iflisted,thesemustbemadeavailablebytheHEIifaudited).

• Individualusers/beneficiarieswhocouldbecontactedbytheREFteamtocorroborateclaims*.

• FactualstatementsalreadyprovidedtotheHEIbykeyusers/beneficiaries,thatcorroboratespecificclaimsmade in the case study and that could be made available to the REF team by the HEI if audited*.

* Where the sources are individuals who could be contacted or have provided factual statements to the HEI, the submitted case study should state only

the organisation (and, if appropriate, the position) of the individuals concerned, and which claim(s) they can corroborate. Their personal details (name,

position, contact details) must be entered separately on the REF submission system and not on REF3b. Details of a maximum of five individuals may be

entered for each case study; these data will not be published as part of the submission.

Ann

exes

55 REF 02.2011

Whenweprovidesubmissionstosub-panels,wewillsupplyastandardanalysisofthedatasubmittedin REF4a/b/c,andsomeofthedatasubmittedinREF1a,inrespectofeachsubmissioninthatUOA,andaggregatedforallsubmissionsinthatUOA(seeparagraph165).Theitemslistedbelowwillbeprovidedtopanels.

Summary of each submission within a UOA1. Totalnumberofsubmittedstaff,bycategoryandECRstatus(headcount).

2. TotalnumberofCategoryAsubmittedstaff(FTE).

3. Totalnumberofoutputssubmitted.

4. Numbersofsubmittedstaff(headcount)withamaximumof1,2,3or4researchoutputs(fourseparatetotals).

5. Numberofresearchdoctoraldegreesawarded,byyear.

6. Numberofresearchdoctoraldegreesawardedpersubmittedstaff9,byyear.

7. Researchincomebyyear,totalacrossallsources.

8. Researchincomepersubmittedstaff(seefootnote9)byyear,totalacrossallsources.

9. Researchincomebysource,totalacrossallyears(seePart3Section4foralistofsources).

10.Researchincomepersubmittedstaff(seefootnote9)bysource,totalacrossallyears.

UOA summary 1. Totalnumberofsubmittedstaff,bycategoryandECRstatus(headcount).

2. TotalnumberofcategoryAsubmittedstaff(FTE).

3. Totalnumberofoutputssubmitted.

4. Averagenumberofresearchdoctoraldegreesawarded,byyear.

5. Averagenumberofresearchdoctoraldegreesawardedpersubmittedstaff(seefootnote9),byyear.

6. Averageresearchincomebyyear,totalacrossallsources.

7. Averageandmedianresearchincomepersubmittedstaff(seefootnote9)byyear,totalacrossallsources.

8. Averageresearchincomebysource,totalacrossallyears.

9. Averageandmedianresearchincomepersubmittedstaff(seefootnote9)bysource,totalacrossallyears.

Annex HStandard data analyses

9 Two versions of this indicator will be shown: one using headcount of both Category A and Category C staff as the denominator; the second using FTE of only Category A staff as the denominator.

REF 02.2011 56

Thisannexprovidesanindexofdefinitionsusedinthisguidance,andofdescriptionsofthedatarequirements.

Definitions Paragraph

Assessmentcriteria 25;AnnexA

Assessmentperiod 21;59

Censusdate 21;59;78;82

Earlycareerresearcher(ECR) 85-86

Eligiblestaff(CategoryA) 78-81

Eligiblestaff(CategoryC) 82-83

Expertpanel/sub-panel/mainpanel 22-24

Impact 140-143;AnnexC4-7

Individualstaffcircumstances 92

Jointsubmission 53-58

Multiplesubmission 50-51

Output/researchoutput 105-110

Publicationperiod 21;59;111

Researchassistant 80-81

Qualityprofile/sub-profile 30-33

Starredleveldefinitions AnnexA

Submission 21;48;59

Submittedunit 48

Underpinningresearch(forimpactcasestudies) 160-161

Unitofassessment(UOA) 20;AnnexD

Data requirements Paragraph

REF1a:Informationonstaffincludedinthesubmission 84

REF1b:Individualstaffcircumstances 96 (forthosesubmittingfewerthanfouroutputs)

REF1c:DetailsofCategoryCstaff 101

REF2:Detailsofresearchoutputs 118

REF3a:Impacttemplate 149-155

REF3b:Impactcasestudies 156-157;163-164;AnnexG

REF4a:Dataonresearchdoctoraldegreesawarded 166-168

REF4b:Dataonresearchincome 171-177

REF4c:Dataonresearchincome-in-kind 178-182

REF5:Environmenttemplate 183-186

Annex IIndex of definitions and data requirements

Ann

exes

TM

ENVIRONMENTALLY FRIENDLY

DEL Department for Employment and Learning, Northern Ireland

DOI Digital Object Identifier

ECU Equality Challenge Unit

ECR Early career researcher

EDAG The REF’s Equalities and Diversity Advisory Group

EDAP The REF’s Equalities and Diversity Advisory Panel

EIA Equality impact assessment

EU European Union

FSR Financial Statistics Return

FTE Full-time equivalent

HE Higher education

HEFCE Higher Education Funding Council for England

HEFCW Higher Education Funding Council for Wales

HEI Higher education institution

HESA Higher Education Statistics Agency

NIHR National Institute for Health Research

RAE Research Assessment Exercise

REF Research Excellence Framework

SFC Scottish Funding Council

UOA Unit of assessment

Annex JList of abbreviations

Northavon House Coldharbour Lane Bristol BS16 1QD

tel 0117 931 7477 e-mail info@ref.ac.ukwww.ref.ac.uk