Information Management in Higher Education Institutions · schulen in Bayern (computer-based...
Transcript of Information Management in Higher Education Institutions · schulen in Bayern (computer-based...
Der folgende Text wird über DuEPublico, den Dokumenten- und Publikationsserver der UniversitätDuisburg-Essen, zur Verfügung gestellt.
Diese auf DuEPublico veröffentlichte Version der E-Publikation kann von einer eventuell ebenfallsveröffentlichten Verlagsversion abweichen.
Randhahn, Solveig:
Information Management in Higher Education Institutions - Training on Internal QualityAssurance Series | Module 4
In: Training on Internal Quality Assurance Series
DOI: http://dx.doi.org/10.17185/duepublico/43225
URN: urn:nbn:de:hbz:464-20170215-102130-9
Link: http://duepublico.uni-duisburg-essen.de/servlets/DocumentServlet?id=43225
Lizenz:
Dieses Werk kann unter einer Creative Commons Namensnennung - Nicht kommerziell - KeineBearbeitungen 4.0 International Lizenz genutzt werden.
Information Management in Higher Education Institutions
Solveig Randhahn
Training on Internal Quality Assurance Series | Module 4Solveig Randhahn and Frank Niedermeier (Eds.)
With financial support from the
Imprint
This e-publication is part of the Training on Internal Quality Assurance Series which is also published as paperback (ISBN: 978-3-7345-7691-1) and distribut-ed in book shops worldwide. More information is available at http://www.trainiqa.org
Author: Solveig Randhahn
Editors: Solveig Randhahn and Frank Niedermeier
Reviewers: Karl-Heinz Stammen, Frank Niedermeier
Edition: First edition
Layout: Nikolaj Sokolowski, Randi Ramme
Publisher: DuEPublico, Duisburg/Essen, Germany
DOI: 10.17185/duepublico/43225
Copyright © 2017 Solveig Randhahn
This book is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0). https://creativecommons.org/licenses/by-nc-nd/4.0/
Please cite the use of our course book series in presentations, trainings, papers etc. according to scientific standards. You can cite this book as:
Randhahn, S. (2017). Information Management in Higher Education Institutions. Module 4. In Randhahn, S. & Niedermeier, F. (Eds.) Training on Internal Quality Assurance Series. Duisburg/Essen: DuEPublico. Retrieved from: http://dx.doi.org/10.17185/duepublico/43225
Acknowledgment
Our modules and books have been prepared and written in a joint effort of the University of Duisburg-Essen
and the University of Potsdam under the DIES (Dialogue on Innovative Higher Education Strategies) Programme
conducted by the German Academic Exchange Service (DAAD) and the German Rectors’ Conference (HRK) with
funds from the Federal Ministry for Economic Cooperation and Development (BMZ). We take this opportunity
to thank the DIES Programme and all the partners from Africa, Europe and Southeast Asia who were involved
in the development process and express our deepest gratitude for the received support, without which the
modules and course books would not have been possible to realise.
We want to further express our sincere gratitude for the most valuable support and contributions from the
involved partners
Autorité Nationale d’Assurance Qualité de l’Enseignement Supérieur (ANAQ-Sup), Sénégal,
ASEAN Quality Assurance Network (AQAN),
ASEAN University Network (AUN),
Association of African Universities (AAU),
Conseil Africain et Malgache pour l’Enseignement Supérieur (CAMES),
European Association for Quality Assurance in Higher Education (ENQA),
National Accreditation Board (NAB), Ghana,
National Council for Tertiary Education (NCTE), Ghana,
National Universities Commission (NUC), Nigeria,
Southeast Asian Ministers of Education Organization Regional Centre for Higher Education and Develop-
ment (SEAMEO RIHED),
United Nations Educational, Scientific and Culutral Organization (UNESCO),
University of Professional Studies Accra (UPSA), Ghana
and especially from Prof. Dr. Shahrir Abdullah, Richard Adjei, Prof. Dr. Goski Bortiorkor Alabi, Prof. Dr. Bassey
Antia, Prof. Dr. Arnulfo Azcarraga, Gudrun Chazotte, Assoc. Prof. Dr. Tan Kay Chuan, Kwame Dattey, Prof. Dr.
Ong Duu Sheng, Prof. Zita Mohd. Fahmi, Mae Fastner, Assoc. Prof. Dr. Nantana Gajaseni, Robina Geupel, Josep
Grifoll, Juliane Hauschulz, Dr. Pascal Hoba, Dato’ Syed Hussein, Benjamin Jung, Prof. Abdel Karim Koumare,
Dr. Vipat Kuruchittham, Prof. Dr. Chiedu Mafiana, Prof. Dr. Duwiejua Mahama, Barbara Michalk, Prof. Dr. Le
Quang Minh, Nguyen My Ngoc, Johnson Ong Chee Bin, Concepcion V. Pijano, Prof. Dr. Philipp Pohlenz, Sonja
Pohlmann, Dr. Suleiman Ramon-Yusuf, Dr. Sylvia Ruschin, Dr. Chantavit Sujatanond, Dr. Oliver Vettori and Marc
Wilde.
The authors
Dr. Solveig RandhahnFaculty of Social Sciences
University of Duisburg-Essen, Germany
https://www.uni-due.de/gesellschaftswissenschaften/
Dr. Solveig Randhahn is Managing Director of the Faculty of
Social Sciences at the University of Duisburg-Essen (UDE) in
Germany. She studied Political Science, Spanish Philology and
Economic Policies at the University of Münster. She received
her PhD in Political Science, doing research on education and
social policy in Germany. Furthermore, she is a certified expert
in Education and Science Management.
Dr. Solveig Randhahn shows a wide range of work experience in
the field of quality management at higher education institutions.
She was responsible for the Service and Information Centre at
the Institute of Political Science and worked at the Department of
Quality Development in Teaching and Learning at the University
of Münster. Afterwards, she was employed at the University
of Applied Sciences in Aachen, coordinating the accreditation
processes of the University and advising the University leadership
in terms of higher educationpolicies in teaching and learning.
In January 2014, Dr. Randhahn took over the responsibility as
manager of the TrainIQA project (Training on Internal Quality
Assurance in West Africa), coordinated by the Centre for Higher
Education Development and Quality Enhancement (CHEDQE) at
UDE. The project aimed at developing capacity in the field of
internal quality assurance (IQA) in higher education institutions
by providing hands-on workshops for quality assurance officers
from higher education institutions in the West African region.
In March 2016, Dr. Randhahn switched to the Faculty of Social
Sciences as Managing Director. In addition, she was elected as
Vice-Dean for teaching and learning at the faculty.
List of Abbreviations
AAA
BA
BSC
CB
CEUS
CHE
EUNIS
HEI
IUCEA
MA
MSRE
PDCA
PhD
QA
Annual Academic Achievements
Bachelor
Balanced Scorecard
Course Book
Computerbasiertes Entscheidungsunterstützungssystem für die Hoch-
schulen in Bayern (computer-based management tool for the institu-
tions of higher education in Bavaria, Germany)
Centre for Higher Education
European University Information Systems
Higher Education Institutions
Inter-University Council for East AfricaUniversity of Duisburg-Essen
Master
Federal Ministry of Science, Research and Economy
Plan-Do-Check-Act or Plan-Do-Check-Adjust
Doctor of Philosophy
Quality Assurance
8
Table of ContentsIntroduction to the Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Introduction to Information Management at Higher Education Insitutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1 Introduction to Information Management at Higher Education Institutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.1 Why Should Higher Education Institutions Collect Data? . . . . . . . . . . . . . . . . . . 13
1.2 Characteristics of an Information Management System . . . . . . . . . . . . . . . . . . . . 16
Translation of Higher Education Objectives Into Numbers . . . . . . . . 30
2 Translation of Higher Education Objectives Into Numbers: Quantitative and Qualitative Indicators . . . . . . . . . . . . . . . . . . . . . . 31
2.1 Meaning and Function of Quantitative and Qualitative Indicators . . . . . . . . . . . 31
2.2 Determination and Operationalisation of Quantitative and Qualitative Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.3 Using Indicators – Key Aspects to be Considered . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.4 Challenges of Using Quantitative and Qualitative Indicators . . . . . . . . . . . . . . . . 39
Reporting: Presentation and Communication of Data and Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3 Reporting: Presentation and Communication of Data and Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.1 Definition of Reporting Objectives for Different Target Groups . . . . . . . . . . . . . . 45
3.2 Content of Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.3 Organisational Conditions for Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
9
Elaborated Information Systems - Examples for Data Sharing . . . . 52
4 Elaborated Information Systems – Examples for Data Sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.1 Case Study of the ETH Zurich: Annual Academic Achievements Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.2 Case Study of the University of Vienna: Course Controlling . . . . . . . . . . . . . . . . 55
4.3 Unidata – Facts and Figures at the Push of a Button – A Case Study from Austria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
10
Preface
Introduction to the Module
Prerequisites for the Module Learnershaveabasicunderstandingandknowledgeofdifferentqualitymanagementapproaches(e.g.pro-
cess-andevaluation-based)inthehighereducationcontext(seecoursematerialModule1),
theyareabletousethePDCA-cycleasasystematicapproachtomanagingquality(seecoursematerial
Modules1and2),
theyhavebasic theoreticalknowledgeof thenewpublicmanagementapproachand its challenges for
highereducationinstitutions(HEI)(seecoursematerialModule1).
Intentions of the Module Establishingsystematicqualityassurancestructuresathighereducation institutionsrequiresawiderange
ofdecision-makingprocessesbydifferentstakeholders.Toimplementthederivingmeasuresandactivities
effectivelyandefficientlyandaccordingtothequalityobjectivesofthehighereducationinstitution,dataand
informationandtheirappropriatecirculationarenecessary.
Thismodulegivesanintroductiontothebasicdiscussionaboutinformationmanagementsystemsathigher
educationinstitutions.Itanalysesthequestionwhyuniversitiescollectdataanddiscusseskeycharacteristics
of informationmanagementsystemsatuniversities.Basedonthis, thecoursebook introducestheuseof
quantitativeandqualitativeindicatorsasameansofmeasuringandassessingobjectives.Itexplainshowto
determineandoperationaliseindicators,howtocriticallyreflectonthemandhowtousetheminaresponsi-
bleandappropriateway.ItpresentstheBalancedScorecardasamethodicalmanagementapproachtodeal
withindicatorsathighereducationinstitutions.
Furthermore,thecoursebookgivesan introductiononhowtoestablishadata-basedreportingsystemat
highereducationinstitutions.Itdealswiththeobjectivesofdifferentstakeholdergroupsandassesseshowto
considertheseappropriatelyinareportingsystem.Itgivesaninsightonthekeyconditionstobeconsidered
whengeneratingreports.
Finally,thecoursebookpresentsvariousexamplesofhowhighereducationinstitutionsdealwithinformation
byestablishingdifferent(technical)structuresandproceduresofcampus-widedatasharingandreporting..
11
dealwith informationthat is relevant forplanningandcontrollingwithregardtoqualitydevelopment/
assurance/management,
developinternaldataandinformationchannels,consideringtherespectivetechnicalandstructuralframe-
workofhighereducationinstitutions,
defineandoperationalisequantitativeandqualitativeindicatorsathighereducationinstitutions,
recogniseandconsideropportunitiesandlimitationsofquantitativeandqualitativeindicatorsasmeasures
forqualityassuranceofprocessesathighereducationinstitutions,
developandmanagereportingsystemsfordifferenttargetgroupsbasedonatransparentsetofinternal/
externalcriteria.
On successful completion of the module, you should be able to…
Chapter 1: Introduction to Information Management at Higher Education Insitutions
12
1 IntroductiontoInformationManagementat HigherEducationInstitutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.1 WhyShouldHigherEducationInstitutionsCollectData? . . . . . . . . . . . . . . . . . . 13
1.2 CharacteristicsofanInformationManagementSystem . . . . . . . . . . . . . . . . . . . . 16
identifythereasonsforhighereducationinstitutionstocollectdata,
recogniseanddifferentiatethelinkagesbetweeninformationmanagementandcontrollingprocessessuch
asplanning,managingormonitoring,
identifyelementarycharacteristicsofaninformationmanagementsystemandtodeducesystematicsteps
todealwithinformationatHEI(e.g.gatheringandacquisitionofinformationneeds,processingandstorage
ofinformationaswellascommunicationchannelsofinformation).
On successful completion of this chapter, you should be able to…
Chapter 1
Introduction to Information Management at Higher Education Insitutions
Chapter 1: Introduction to Information Management at Higher Education Insitutions
13
1 Introduction to Information Management at Higher Education Institutions
1.1 Why Should Higher Education Institutions Collect Data?
Establishingsystematicinstitutionalqualityassurancestructuresrequiresabroadvarietyofinformationthat
isfundamentaltoenabledecision-making,communicationandorganisationalprocessesbetweendifferent
stakeholdersandtherealisationofactivities.
Informationcanbedefinedaspurposefuldatathat isrelatedtoaproblemandthat isusedtoachievean
objective(Wittmann1980).Wecantalkaboutknowledgewhenpeoplestarttoputinformationintoamean-
ingfulcontext(Gladen2003,2).
InformationisnecessaryforallorganisationalconcernsandobjectivesofaHEI:foreasingandoptimisingdeci-
sion-makingprocesses,forplanninganddevelopingrealisticsettings,forreportingandqualitydevelopment,
andwithitenhancingtheinstitutionalefficiencyandeffectiveness(Saupe1981).Highereducationresearcher
J.FrederickVolkweinsystemisesthesestrategicandoperativeobjectivesintofivefundamentalconcernsof
highereducationinstitutions(Volkwein1999):
1. Expensesforhighereducation(shortageoffinancialfunding)
2. Requiringanefficientmanagementandincreasingproductivityatthesametime
3. Effectivenessandsurplusvalueofhighereducationinstitutions(competitionandrighttoexistwithout
thenecessitytoproduceoutputwithregardtocontents)
4. Accesstohighereducationinstitutions(increasingnumberofstudentsasajustificationforadditional
funding)
5. Reporting
These fundamental concerns go alongwith various, constantly changingwhile simultaneously increasing
demandsforinformation.Thequestionishowhighereducationinstitutionscanrecognise,determine,pro-
ceedand,finally,copewiththeseinformationdemandsefficientlyandeffectivelyinthelightofavailablestaff,
materialandtechnical resources.Forexample, todeterminetheavailablecapacitiesofyour institutionto
establishanotherstudyprogramme,youhavetoconsiderandcalculatetheplannednumberofstudents,the
numberoflecturerswhoareavailable(intermsofworkinghours),aswellastheresultingcostsforstaffand
infrastructure.
Inorder tobeable todealwith such informationdemands,highereducation institutionshave started to
establishintegrateddata-basedinformationsystems.Thesearebaseduponexistingeconomicalapproaches
Information as“purpose- fulknowledge” (Wittmann 1980)
Chapter 1: Introduction to Information Management at Higher Education Insitutions
14
forbusinessstrategiesandmanagementconcepts.Usingup-to-datedataand informationtechnologiesat
highereducationinstitutionsshouldcontributetoeffectiveandefficientprocessesinthehighereducation
organisation.
Inthiscontext,data canbedefinedasasetofqualitativeand/orquantitativevariablesthatbecomeinfor-
mationbyinterpretation.Dataarearesultofmeasurementsandcanbevisualisedbyusingtables,graphsor
images.Hence,datacanbeunderstoodasanabstractconceptfromwhichinformationandthenknowledge
arederived(BostonUniversity2015;DWBI2014;seealsoModule2).
Methodicalinformationmanagementservesaccountabilityandreportingpurposesintheinternalandexter-
nalcontextofhighereducation.Itcreatesperformanceandcosttransparencyandthereforeprovidesacen-
tralcontributionforqualityassuranceinresearch,teachingandsupportingservices:awell-establishedinfor-
mationsystemserves the formulationof institutionalobjectivesandthereforethe facilitationandoptimi-
sationofdecision-makingprocesses fora sustainable strategicplanning inhighereducation (Saupe1981;
Küpper,Friedl,Hofmann,Hofmann&Pedell2013).
“An information system can be understood as a coordinated arrangement of staff, organisational
and technical elements that provides decision-makers with purposeful knowledge for their task
fulfilment.”
(Eberhardt, 67 in Frese 1992)
Thekeypurposesofinformationmanagementincludeacloselinkagetomanagerialaccountingprocessesat
HEI.
Asaprimaltaskofmanagerialaccounting,wecanconsidertheoverallcoordinationofthemanagementsys-
temofahighereducationinstitution:
“Management must deal with the dynamics of change and provide coordination for the overall
system.”
(Kast & Rosenzweig 1974, 620 in Horváth 2011, 8)
AccordingtoHorváththemanagementsystemconsistsoffivesubsystems:planning,accounting,information
supply,organisationandhumanresourcemanagement(Hórvath2011,8;Küpperetal.2013,636).Concern-
ingtheinformationsupply,managerialaccountinghastocoordinateandaligntheaforementionedsubsys-
temswithregardtotheinformationneedsofdecision-makers.Ontheonehand,thisincludesthecoordina-
tionwithintheinformationsystem–thecollectionofnecessarydata,itssystematisation,storageand,finally,
itsallocation.Ontheotherhand,thisincludesthetransmissionofdatatotheaforementionedsubsystemsof
themanagementsystembysuitablereportingsystems.
Datacreate suitable
information fordecision-
making processes
Linkage between
information management
andmanagerial accounting
Chapter 1: Introduction to Information Management at Higher Education Insitutions
15
Figure 1 Layer model for higher education institutions (Tropp 2002, 2)
Thedesignofinformationsystemsisorientedtowardstworeferencelevels.Theverticallevelreferstosuch
levelsathighereducationinstitutions,wheredecisionsaremadeandtasksarecarriedout,i.e.thetopman-
agement,faculties,institutesandchairs.Thehorizontallevelreferstothecoreprocessesofhighereducation,
i.e. research, teachingand services. These includevarious informationneeds thatgoalongwithdifferent
requirementsregardingthewayofsystematisationandallocationofinformation.Dependingonthelevelof
centralisedandde-centraliseddecision-makingprocessesbetweenthetopmanagement,faculties,institutes
andchairs,multi-dimensionalinformationsystemsareneeded(Küpperetal.2013,636).
Theincreasingcomplexityanddiversityofinformationleadtoverydifferentscopesofperformanceofthese
informationsystemsamonghighereducationinstitutions.Coreprocessesofthesocalled“studentlifecycle”1,
thatarefrequentlymanagedthroughprofessionalinformationtechnologies,aresuchasthefollowing:
application,assessmentandadmissionprocesses
studentadministration
planningandmanagementoflectures(universitywidecourseschedule,generalandindividualcourse
schemes,registrationandderegistrationofstudentsfromcourses/exams)
managementoflecturehallbooking
examinationmanagement(e.g.examregistrationandderegistration,transcriptofrecords,recognition,
archivalstorageoffinalexamination)
managementoforganisationaldata(buildingandlecturehallplans,e-mailandphoneindex)
1Thestudent-life-cycleincludesallrelevantactivitiesandfieldsforstudents,lecturersandadministratorsthathavetobecon- sideredduringtheacademiceducationprocess:e.g.application-->admission-->teachingandlearning-->assessment--> graduation-->alumni.
Chapter 1: Introduction to Information Management at Higher Education Insitutions
16
Higher education institutions have started to integrate thiswidely ramified IT-landscape in complex data
warehousesystems.
Comingbacktothequalitymanager,wecanaskwhichareasofsuchacomplexdatasystemarerelevantto
her/him.Focussingonteachingandlearning,wecanthinkofaprofessionaldatamanagementofprocesses
suchasinternalandexternalevaluationsonfacultylevelorthehighereducationinstitutioningeneral,tracers
studies,oralsostaffdevelopmentinteaching.
Further Reading
Taylor,J.(2014).Informingordistracting?Guidingordriving?Theuseofperformanceindicatorsin
highereducation.InMenon,M.,Terkla,D.,Gibbs,P.(Ed.),Using data to improve higher education.
Research, policy and practice.Rotterdam:SensePublishers.
HigherEducationFundingCouncilforEngland(HEFCE)(2011).Performance indicators in higher ed-
ucation. First report of the performance indicators steering group (PISG).London:HEFCE.
Balasubramanian,K.(2009).ICTs for higher education. Background paper from the commonwealth
of learning.Paris:UNESCO;WorldConferenceonHigherEducation;CommonwealthofLearning.
1.2 Characteristics of an Information Management System
Whyshouldqualitymanagerscareaboutinformationmanagement?–Basically,qualitymanagershaveacon-
sultativefunctionwithregardtodifferentdecision-makingprocessesathighereducationinstitutions,beiton
managementlevel,onorganisational/administrationleveloronfacultylevel.Therefore,theyneedtobeable
togatherinformationrequirementscorrectlyandanalyseandevaluatethecollecteddataandinformation
accurately.
Examplesoftargetsinaninformationmanagementsystem,forwhichqualitymanagerscanplayakeysup-
portingrolecanbethefollowing:
Definition of Data Warehouse
“Adatawarehouseisacopyoftransactiondataspecificallystructuredforqueryingandreporting.”
Source: (Kimball 2002)
Chapter 1: Introduction to Information Management at Higher Education Insitutions
17
Defineandestimateinformationneedsrequiredforcertaindecision-makingprocesses.
Prepareunderstandableandinterpretabledatafortherespectivetargetgroupsandavoidcontradictions.
Interfacefunctionwithregardtoinformationdistribution,inordertohelptoclosecommunicationand
informationgapsamongsendersandaddressees.Thatmeans,theycanexplainandclarifywhichinfor-
mationisavailableforwhichissues,orwhoneedswhichpartoftheexistingdataandinformation.
Supportforreading,analysingandinterpretingdatamaterial,consideringtherespectiveparticularcon-
text.
Contributetodevelopingmoretransparencyabouthowtheinformationflowsofahighereducationinsti-
tutionworkaccordingtodefinedqualitycriteria.
Insomehighereducationinstitutionsthesetargetscanbecloselyrelatedtomanagerialaccounting.2Toavoid
overlappingactivitiesbutachieveaneffectivetargetallocation,youshoulddefineandcoordinatetherespec-
tiveresponsibilitiesbetweenaqualitymanagerandaunitformanagerialaccountingclearly.
Takingthisintoconsideration,thewholefieldofinformationmanagementcontainsenoughquestionstobe
discussedinapropertrainingcourse.Thisiswhyinthiscoursebookwehavetolimitourfocusonsomepar-
ticularaspects.Inshort,wewillfocusonthelinkagesbetweeninformationandqualitymanagementandthe
roleofqualitymanagers.
Thecoursebookgivesanintroductiontomanagementrelevantdataandinformationwhichahigheredu-
cationinstitutionneedsforimproving,assuringandmanagingqualityinthecoreprocessesofteachingand
learning,researchandservices.Therefore,itgivesanoverviewonthekeycharacteristicsofinformationman-
agementsystemsanddiscussesthecriteriathatarenecessarytodevelopasystematiccollection,analysisand
interpretationofdataandinformationaccordingtotheneedsandrequirementsofspecifictargetgroups.3
Basedonthis,yougettoknowthemostimportantessentialstoassessandjudgeinhowfarstrategicand
operativeobjectivesofqualityassurancehavebeenreached.Youwill learnaboutthechallengesofdefin-
ingquantitativeandqualitative(keyperformance)indicators(Chapter2.1),howtocollectandanalysethem
(Chapter2.2),aswellashowtodealwithresistanceagainstdataandinformationandtoachieveacceptance
(Chapter2.3and2 .4).
AccordingtoHorváthamethodicalinformationmanagementsystemcanbestructuredintothethreefollow-
ingphases(Hórvath2011,308etseq.)
I. IdentifyinginformationneedsandgatheringrawmaterialatHEI
II. Datacollection,processingandanalysis
III.Datadissemination(workflowsbetweendisseminatorandreceiver)
2ForfurtherinformationonmanagerialaccountingandtherelationtoinformationanalysisseeDemski(198,2008). 3 MoreinformationonthisissuecanbefoundinCB2aswell.
Chapter 1: Introduction to Information Management at Higher Education Insitutions
18
I. Identifying information requirements and gathering raw material at Higher Education Institutions
Tobeabletogather,classify,processandreportdataandinformationinaninformationsystem,firstofallyou
havetofindoutabouttherespectiveinformationrequirements.Decision-makersathighereducationinstitu-
tionshavedifferentinformationneedsaccordingtotheirrespectivestrategicobjectivesandtargets(seeTable
1).Theseinformationneedshavetobedefinedclearlyandunambiguouslytobeabletodeducesystematic
andeffectivedatacollectionanddistribution.
Information requirements can be defined as “the type, amount and quality of informationwhich a deci-
sion-makerneedstofulfilher/histargets”4(Koreimann1976,6;Gladen2003,4).
Wecandifferbetweenobjectiveandsubjectiveinformationneeds.Objectiveinformationrequirementsrefer
totheamountofinformationwhichissetinafactualcontexttosolveaproblem.Subjectiveneedsarethe
informationwhichadecision-makerconsiderstoberelevantforher/histargets(Küpper2013,218).
Basedonthis,aconcreteinformationdemandgenerallyincludesbothsubjectiveandobjectiveinformation
requirements.Veryoftendecision-makersarenotsufficientlyawareoftheirsubjectiveinformationneedsor
cannotformulatethemappropriately.Itmayalsohappenthattheyevenwanttohidetheirrealinformation
requirements(Nusselein2002,3).
Figure 2 Gathering information based on needs, supply and demand (translated based on Picot & Frank 1988, 608 in Hórvath 2011, 311)
4OwntranslationfromGermanintoEnglish.
Objective andsubjective
information requirements
Chapter 1: Introduction to Information Management at Higher Education Insitutions
19
Thefollowingtableillustratespossibleinformationneedsofdifferentstakeholders,referringtostructuralcon-
ditions,resourcesorprocessesinteachingandlearningwhichcancomeupwhenestablishingqualityassur-
ancestructuresatanhighereducationinstitution.
Subject-matter Examples for information sources Examples for information requirements
Structuralframeofresearchandteaching
(National)lawonhighereduca-tion
StrategicplansofaHEI
Strategicplansoffaculties
HEIconstitutionandregulations
Examinationregulations
Regulationsfordoctoral
degreesandhabilitation
IstherearegulatoryobligationtoestablishaQA-unit?Ifso,whichrequirementshavetobefulfilled?
WhichobjectivesshallbeachievedwiththeQA-unit?(E.g.annualevaluationofstudypro-grammes;establishmentandcoordinationofqualitycyclesinteachingandlearning)
Whichinformationhastobedocumentedinanexaminationregulationtocomplywithinternal/externalqualitystandards?
ResourcesofaHEI(staff,facilities)
Dataonavailableresourcesandcashflows
Staffingperprofessor Third-partyfundsperprofessor Overviewonavailablestaffandresourcesatfaculties
Whoprovideswhichamountoffinancialresourc-esfortheset-upofaQA-unitandforwhatperi-od?Forwhichpurposescantheseresourcesbeused?(E.g.facilities,staff,IT)
WhatisthenumberofqualifiedstaffavailablefortheQA-unit,andforwhatperiod?
Whichadditionalqualityassuranceactivitiescanberealisedbasedonthird-partyfunds(e.g.additionallectures,tutorials,mentoringpro-grammes)?
Processman-agementofteachingandlearning
Input/outputdataofthepro-cessteachingandlearning(aggregationonprogrammelevel)
Dataoninternationalisation Qualityofgraduates Detaileddataonteachingandlearning(e.g.coursescheme,assessment,mentoring)
Capacitiesofprofessorshipinteachingandlearning
Whichdataisavailableonthenumberofappli-cationsperplaceinaprogramme,thenumberofstudents/graduatesperprogramme,thedrop-outratioetc.?Isthisdataconsistentwithinter-nal/externalqualityrequirements?Whichaddi-tionaldatamightbenecessary?
Howmanyincomingandoutgoingstudentsarethereonfaculty/programmelevel?
Isthereanyinformationavailableonthegradu-atesandtheircareerpaths?
Whichinterdisciplinarycoursesdowehave? Scopeofregularcoursesofferedperpro-gramme?Numberofparticipantsperlecture?
Numberofprofessorsperprogramme?Mentor-ingratioperprogramme?
Table 1 Information sources and requirements from different stakeholders (adapted from Nusselein 2002)
According to thedifferent subject-mattersmentioned in the table, theprioritiesof the listed information
requirementsdifferdependingontherespectivetargetgroup.Focussingonthestrategicframeinresearch
and teaching, forexample,a vice-chancellorneedsother information thanadeanoradeanof students.
Chapter 1: Introduction to Information Management at Higher Education Insitutions
20
Theformerisespeciallyinterestedinstrategicplanningofthewholehighereducationinstitutionandconsid-
ersinformationaboutstrategicplanningonfacultylevel.Adeanofstudentshowever,isresponsibleforteach-
ingandlearning,focussingespeciallyonexaminationandprogrammeregulations.Yetforadean,information
onregulationsofthedoctorateorpost-doctorallecturequalificationsmightbemorerelevant.
Informationthatreferstothefinancial resourcesandcashflowsareespeciallyrelevantforthechancellor
(understoodasheadofadministration)whoisresponsibleforthebudgetofahighereducationinstitution.
However,theinformationrequirementsofthevice-chancellororthesenatemightfocusondataaboutstaff-
ingorthird-partyfundsperprofessorwhichcanbeusedasquantitativeindicatorsforresearchperformance.
Amongothers,theyneedthis informationforprofessorialappointmentprocedures.Afacultyneedsmore
detailedindicatorssuchastheavailablestaffingorfinancialresourcesofthefaculty.
Focussingontheprocessofteachingandlearning,thetopmanagementisusuallyinterestedininput/output
dataonprogrammelevel(e.g.numberofapplication,students,graduates,drop-outratioperprogramme).
Furthermore,dataoninternationalisationandthequalityofthegraduatesisrelevantinordertoanalyseand
interpretthesuccessofastudyprogramme.Deansofstudentsneedinformationthatdifferentiatesinmore
detailbetweenthewholestudyprocesses(e.g.dataontheorganisationofassessment,courses,andproce-
duresofrecognition).Finally,achancellorneedsdatatobeabletodeterminetherequiredresources(capac-
ities)inteachingandlearning.
Qualitymanagersshouldknowallthesedifferentperspectivesandtherespectiveinformationrequirements.
Basedonthis,theycancontributetothedistributionofinformationtothosewhoeffectivelyneedthem,but
alsosupportdecision-makingprocessesondifferentinstitutionallevels.
Questions & Assignments
1. Pleasestudythetableandthementionedexamplesofinformationrequirementsagain.Lookingat
yourown institution,whichof these informationrequirementsdothedecision-makersprioritise
andwhy?
2. Arethereanyadditionalinformationrequirementswithregardtoqualityimprovementinteaching
andlearningatyourhighereducationinstitution?Forwhomandwhy?Pleasegiveexamples.
Howcanqualitymanagersfindoutaboutthesedifferentinformationneedswithoutonlyraisingassumptions
orhypotheses?Therearedifferentwaysofgatheringinformationrequirements,whichcanbeseparatedinto
inductiveanddeductiveprocedures (Küpper2001,145). Inductivemethods focusontheconditionsofan
organisationasthefundamentforinformationrequirements.Basedonthis,youparticularlyidentifyinforma-
tionsupplyaswellassubjectiveinformationneeds.Examplesformethodologicalapproachesaresuchasthe
analysesoforganisationaldocumentsanddataorananalysisoftheorganisationorasurveybasedoninter-
viewsorquestionnaires.Deductivemethodsidentifyinformationinasystematicway:Basedonthestrategic
objectivesofanorganisation,theytrytofindoutabouttheobjectiveinformationneeds(Küpperetal.2013,
222;Nusselein2002,3).
Determination ofinformation requirements
basedon inductiveand
deductive procedures
Chapter 1: Introduction to Information Management at Higher Education Insitutions
21
Togainamorecomprehensivepictureoftheinformationneeds–thatisbothobjectiveandsubjectiveinfor-
mationneeds–itisrecommendedtocombineboththeinductiveanddeductiveapproach.Thefollowing
procedureofananalysisofinformationneedsmayserveasanexample:
Integrated concept of an information needs analysis
Figure 3 Based on the project “Computer-based management tool for the institutions of higher education in Bavaria” (CEUS) (Nusselein 2002, 4)
Description of Figure 3:
Theorganisationanalysis focusesontherespectiveunitsofanorganisationunddeterminestargetsand
decision-makingcompetencesoftherespectivedecision-makers(inthecaseofhighereducationinstitu-
tionssuchas(vice)chancellor,highereducationboard,senate,chancellor,dean,deanofstudents).5
Theresultsoftheorganisationalanalysesarethebasisforthesubsequentinterviewswiththeabove-men-
tioneddecision-makers.Theinterviewshavetwopurposesinparticular:First,theycompletetheobjective
targetprofilebyaddingsubjectivelyconsideredtargets(seeabove,No.1);second,theygiveinformation
aboutthesubjectivelyconsideredinformationrequirementsforthedefinedsetoftargets.
Thedeductiveanalysisgathersobjective informationrequirementsandwith it completes thesubjective
informationneedsgainedbytheinterviews.
Followingthis,theresultsaretestedwithanothersurveybytheabove-mentioneddecision-makers.Based
onaquestionnaire,theyshallevaluateandnarrowdowntheinformationrequirementsaccordingtoprior-
ities(Küpper1997,133).IntheprojectCEUS,theoutlineofthequestionnairewasbasedontheaforemen-
tionedsubject-matters:a)structuralconditions,b)resources,c)processplanninginteachingandlearning,
d)processplanninginresearch(Nusselein2002,5).
Inaconcludingworkshopthesurveyresultsarediscussedwiththedecision-makersagain.Ifnecessary,fur-
theradaptionsoftheinformationneedsaretobeeffected.6
5 Thetypesofdecision-makersmaydifferdependingontheorganisationalstructureandhavetobeadaptedaccordingly.6 IntheCEUSprojectthismethodofgatheringinformationneedswasrealisedatseveralhighereducationinstitutions.Basedonthis itwaspossibletoachieveasufficientandcomparabledatabasis.
Chapter 1: Introduction to Information Management at Higher Education Insitutions
22
Thedescribedwayofgatheringinformationrequirementsexemplifiestheprocedureatvarioushighereduca-
tioninstitutionsinGermany.Itisimportanttokeepinmindthatduetodifferentstructuralconditionsindiffer-
entcountriesandinstitutions,thedescribedmethodtoanalyseinformationrequirementshastobeadjusted,
dependingontheinternalandexternalparticularitiesofahighereducationinstitution.
Dependingonthepurposesoftheinformationistobeused,thecollectionofdatahastobemoreaggregated
ormoredetailed.Consideringtheabove-mentionedexamplesofinformationrequirementsofa(vice)chan-
cellor,achancellororrepresentativesfromfaculties,itcanbeconcludedthatthedetail-leveloftheprovid-
edinformationincreaseswithadecreasinghierarchylevel.Viceversa,theaggregationlevelofinformation
increasesfromthelowesttothetophierarchylevel.Inordertoprovidecomparabledataandinformationon
alllevels,theaggregationofinformationshouldalwaysrefertoacommonandstandardiseddata-basis(Eber-
hardt2003,73).
Furthermore,itcanbeconcludedthatingeneralitisnotpossibletocoverallinformationneeds.Establish-
inganddevelopingastructuredinformationsystemathighereducationinstitutionscanhelptocloseorat
leasttoreducethesegaps.Therefore,oneofthekeychallengesisnotknowingexactlywhichunitsofahigher
educationinstitutionprovidepromisinginformationsourcesandhowtoconnectandusetheseinformation
sourcesforthewholeinstitution.Sometimesthatisbecausetherespectiveinvolvedpartiesdonotwishsuch
“connections”.Sometimes,collectingspecificinformationneedsisjustnotpossible,beitbecauseofalackof
time,beitduetotechnicalrestraints,orbecausethereisnotenoughstafffortheprocessing.
Consideringthis,acontrollerwhoisresponsibleforgatheringinformation,firstofallhastoanswerthefol-
lowingquestions:
Doesmyinstitutionprovidetheinformationneeded?
Whichpossibilitiestogatherinformationtheinstitutiondoesnotyetprovideexist?
Howmuchtimeandeffortdoesittaketoprovidethisinformationandwhocandoit?
Whichqualitycriteriacanbeguaranteedfortheinformationtobeneededwithregardtobeingcomplete,
timely,comparableetc.(seeTable2)
Questions & Assignments
1. Howdoyouproceedwhengatheringinformationatyourinstitution?Whoisresponsibleforthis
task?
2. Howfardoestheprovidedinformationmeettheneedsofthetargetgroups?
3.Whichchallengesareyouconfrontedwithwhencollectingdataatyourinstitution?
Chapter 1: Introduction to Information Management at Higher Education Insitutions
23
II. Data collection, processing and analysis
Havingcollectedthenecessarydatafortherespectiveinformationneeds,thisdatanowhastobeevaluated
andanalysedinatransparentandunderstandableway.Generallythisisdonebystafflocatedataunitfor
managerialaccounting.Butwithregardstodataanalysisaccordingtodefinedqualitycriteria(seeTable2),it
isrecommendabletoinvolvethequalitymanageraswell.Additionally,she/hecanhelptoillustratethetech-
nicaldatainsuchawaythattherespectivetargetgroupisabletoread,understandandinterpretitcorrectly.
Themaintaskforqualitymanagerswhoareresponsiblefortheevaluationandanalysesofdataandinforma-
tionistocheckwhichcharacteristicsanidealinformationshouldhavetosatisfythedesiredinformationneeds
asmuchaspossible(Hórvath2011,298etseqq.).Thisprocessofevaluationandanalysesincludesvarious
challenges.
Averycommonproblemis, forexample,thatdata isnotcurrent,butretrospective,that it istoodetailed
andextensive,orthatitisinconsistentandcontradictory.Basedontheserestrictions,thedatadoesnotgive
enoughsignificantinformationontherespectiverequirements.
Workingagainsttheserestrictionsandachievinggreaterprecisionofthecollecteddatawithregardtothe
respectiveinformationneeds,somecriteriaofsuccessshouldbeevaluated.
Thefollowingtableshowsexamplesofkeycriteriaofsuccesswhenevaluatingandanalysingdataandinfor-
mation.Itincludessomeimportantquestionsthatshouldbeansweredwhencheckingthesecriteria.
Criteria of success for data collection
Questions to be clarified Phrase to memorise
Typeofdata Isitquantitativeorqualitativedata? Whichinformationdoesthedatagive? Isthedatasignificantlyvaluable?
Thedataiscategorisedclearlyintoquantitativeorqualitativecategories.Thesignificanceofthedataisclearandcanbenamed.
Degreeofcom-pression
Arethereanyduplicationsthatcanbereduced? Howtoaggregateandsummarisedata?
Asmuchdataasnecessary,aslittledataaspossible.
Timelinessofdata Isthedataup-to-date? Istheperiodofdatacollectionandthereportingperiodcongruenttotherespectiveissueofinter-est?
Theperiodofdatacollectionreferstotherelatedissueofinterest.Theperiodofdatacollectionmatchesthereportingperiod.
Chapter 1: Introduction to Information Management at Higher Education Insitutions
24
Criteria of success for data collection
Questions to be clarified Phrase to memorise
Layout Whichlayoutisappropriateforthetargetgroup?(e.g.writtenreport;tablesummary;graphic/visualisedlayout)
Doesthelayouttransfertheneededinformation? Doesthelayoutincludeasystematicandreada-bleoutline?
Thelayoutisappropriatefortheneedsofthetargetgroup.
Problem-solvingrelevance
Whichinformationvaluedoesthedatahaveforthetargetgroup?
Whichindicatorprovesthisvalueandwhodecidesaboutthisindicator?
Thecollecteddataisvaluablewithregardtotheissueofinterest.
Priorityandcol-lectionfrequency
Whenisthedataneededandwho/whatdecidesaboutthistimeframe?
Whatisthefrequencyofdatacollectionandreporting?
Whichconsequenceshavetobeconsideredwithregardtothescopeofdataevaluationandanaly-sesresultingfromshort-termorlong-terminfor-mationneeds?
Istheperiodofdatacollectioncoordinatedwiththedateofprovision?
Whichcontrolmechanismscanbeconsideredrespectivetotheavailabletime?
Isthecollectionfrequencysufficienttoachievesignificantinformationfromthedata?
Theperiodofdatacollectioniscoordinatedwiththedateofprovision.Thefrequencyofdatacollec-tionissufficienttoproducesignificantinformation.
Purposeofuse Isthedataonlyusedforonepurposeordoesitservevariouspurposes?
Doesthepurposerequireaspecialformofdataevaluationandanalyses?
Checkifdatacanbeusedfordifferentpurposes.
Amount Whichdataisrequiredtodelivertheinforma-tionneededfromtherespectivetargetgroupandwhichnot?
Howdetailedshoulddatabetodelivercertaininformation?
Thelevelofdetailandtheamountofdatamatchestheissuesofinterestandinforma-tionneededfromthetargetgroup.Basedonfiltering,comprehen-sionandcanalisationofdata,youshouldproducesignificantandunderstandableinforma-tion.
Chapter 1: Introduction to Information Management at Higher Education Insitutions
25
Criteria of success for data collection
Questions to be clarified Phrase to memorise
Accuracy Whatisthelevelofaccuracyofthecollecteddata?
Doesthedatadelivercoherentandconsistentinformationordoesitincludecontradictoryordifferingpossibilitiesofinterpretation?Ifso,howfardoesthisreducethevalueofthegainedinfor-mation?
Reducecontradictoryformsofinterpretation,butproduceclearandunambiguousinfor-mationfromthedata.
Reliability Whatisthedatasource?Isthedatasourcereli-ablewithregardtotransparency,methodologyandmeasurability?
Thecollecteddataisobtainedfromareliabledatasource.
Measurability/plausibility
Whichcriteriahavebeendefinedtomeasurethedata?
Arethesecriteriatransparentandunderstanda-ble?
Defineclearandunderstanda-blecriteriaofmeasurability.
Costs Whichfinancial,stafformaterialcostsresultfromcollecting,analysingandreportingdata?
Clarifythecostsfordatacollec-tion,analysesandreporting.
Data-protection Whataretheproceduresofdocumentingandsavingdata?
Whichdataprotectionruleshavetobeconsid-eredwithregardtodataaccess?
Clarifyregulationsandpro-ceduresofdocumentingandsavingdata.
Communicationprocesses
Whichcommunicationflowsarenecessaryforcollecting,analysingandusingdata?
Whoisinvolvedindatacollectionandanalyses? Whohastobeinformedaboutthedatacollec-tionandanalysesandhow?
Arethesecommunicationflowsclearandtrans-parenttoallinvolvedstakeholders,andtowhatextentaretheyputintopractice?
Coordinateanddefinecom-municationchannelstocollect,analyseandusedata.
Table 2 Criteria of success for data collection
Questions & Assignments
Theseniormanagementofyourinstitutionwantsallfacultiestohandinareportaboutthecurrent
successoftheirstudyprogrammes.
1.Howdoyoureportthesuccessofstudyprogrammesatyourinstitution?
2.Whichinformationneedsdoyouconsidertoberelevantinthisregard?
3.Whichcriteriaofsuccessareimportanttobeconsideredinthedatacollectionprocess?
Chapter 1: Introduction to Information Management at Higher Education Insitutions
26
III. Distribution of information
Aftercollectingandanalysingthedatathegainedinformationistobedistributedtotherespectiveaddress-
eesviareportingsystems.Thedesignofthesereportingsystemscanvarydependingonthetypeandthe
amountofinformation,aswellasthetargetgroupanditsobjectives.InChapter4wewilllearnaboutthe
reporting-issueinmoredetail.Therefore,thischapterwillonlygiveanoverviewontherequirementsofan
informationmanagementsystemwithregardtodistribution.
The key element of distributing information is the relation between the sender of information and the
addresseeandthequestionofhowtotransfertherelevantinformationappropriately.Thismeans,aninfor-
mationsenderhastoknowtowhomshe/hehastodelivertheinformationandinwhichform.Atthesame
time,addresseesofinformationshouldknowhowtoread,understandandusethereceivedinformationfor
thearticulatedneeds(Hórvath2011,354etseq.).
Suchcoordinationisnoteasytoachieveinpracticebutincludesvariouschallenges.Forexample,producers
ofinformationoftendonotknowsufficientlywhotheaddresseeofthecollecteddataisandwhatthedatais
neededfor.Ontheotherhand,forinformationusersitmightbeunclearwhichinformationcanbeprovided,
howtoreadandanalysecollecteddataconsideringtherespectivecontext.
Figure 4 Types of interferences during the process of information distribution (adapted from Küpper et al. 2013, 241)
Dealingwiththesechallenges,qualitymanagerscanplayanimportantrolebybeingacommunicationlinkage
betweenthedifferentstakeholdersandunitsofahighereducationinstitution.Theycanrevealcommunica-
tionandinformationgapsbetweensendersandaddresseesofinformationandreducethembyclarifyingthe
contentofthespecificdatainanunderstandablewayforthetargetgroups.Indoingsotheycontributeto
achievingmoretransparencyandworkinginformationflowsathighereducationinstitutions.
Chapter 1: Introduction to Information Management at Higher Education Insitutions
27
Incidence Case: Students Newsletter
Theresultofasurveyatthe“African-University”wasthatthestudentsfeel insufficiently informed,
beitwithregardtoorganisationalproceduresandrelevantdeadlinesoftheirstudiesorwithregard
tocurrentdevelopmentsinresearch.Thevice-chancellorforacademicsaskedthequalitymanagerin
chargetodevelopanewsletter.Thepurposeofthisnewsletterwastoinformregularly(e.g.quarter-
ly)aboutrelevantorganisationalissues,deadlinesandfixeddates,newservices,orotherissuesthat
mightbeofinterest.Sinceanewsletteriscloselyrelatedtothetargetsofthedepartmentforpublic
relations, thequalitymanager informedthedepartmentabout thiswork task. Indoingso,healso
wantedtofindouthowfarthepublicrelationscolleagueswereabletosupporthimwithregardto
developinganddistributingthenewsletter.Aftertalkingtoeachother,thequalitymanagerdecided
topublishthenewsletterbothasaprintversionandasanonlinepdf-versionontheuniversityhome-
pagetoreachasmanyuniversitymembersaspossible.Thepublicrelationscolleaguesofferedtocare
fortheplacementofthedocumentonthewebsiteandtosendasufficientnumberofprintedcopies
toeachfacultyandunit.Furthermore,thequalitymanageraskedacolleaguefromthedepartmentof
dataandinformationmanagementtocreateamailinglist.Inthefuture,interesteduniversitymem-
berscansubscribetothismailinglistandwillreceivethenewsletterautomatically.
Concerningthecontentdesignofthenewsletter,thequalitymanagerwantstoproceedaccordingto
thefollowingoutline:
1. Didyoualreadyknowabout…?
Informationaboutinterestingevents
Importantdatesanddeadlines
Currentresearchprojectsattheuniversity
Miscellaneous
2. LibraryServices
3. ICTServices
4. Haveyoualreadyread?–Newpublicationsfromresearchersoftheuniversity
5. Portraitofauniversitymember(shortinterviewwith5-6questions)
Thequalitymanagerisveryenthusiasticabouthisprojectactionplanforthepublicationofthenews-
letter and already very excited about feedback from the students and the other universitymem-
bers.
Chapter 1: Introduction to Information Management at Higher Education Insitutions
28
Further Reading
Alter,S.(1996).Information systems: A management perspective(2ndedition).MenloPark:
BenjaminCummingsPub.Co.
Questions & Assignments
Your vice-chancellor of academics asks you to develop a newsletter for the lecturers at your
university.
1.Whatmightbeinterestingandrelevantinformationforlecturers?Howandbywhomcouldyou
gathertheseinformationneeds?
2.Whichstepsdoyouhavetoconsidertodesignanddistributethisnewsletter?
Whichchallengesshouldbeconsideredinthisregard?Whichcriteriaofsuccessareimportanttobe
consideredinthedatacollectionprocess?
Chapter 1: Introduction to Information Management at Higher Education Insitutions
29
Chapter 2: Translation of Higher Education Objectives Into Numbers
30
2 TranslationofHigherEducationObjectivesInto Numbers:QuantitativeandQualitativeIndicators . . . 31
2.1 MeaningandFunctionofQuantitativeandQualitativeIndicators . . . . . . . . . . . 31
2.2 DeterminationandOperationalisationof QuantitativeandQualitativeIndicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.3 UsingIndicators–KeyAspectstoBeConsidered . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.3.1 RequirementstoDefineIndicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.3.2 TheBalanced-Scorecard–AnInstrumenttoMonitorIndicators . . . . . . . . . . . . . 36
2.4 ChallengesofUsingQuantitativeandQualitativeIndicators . . . . . . . . . . . . . . . . 39
differentiatekeyfunctionsofusingquantitativeandqualitativeindicators,
determineandoperationalisequantitativeandqualitativeindicatorsbydeterminingcentralparameters
suchasthesample,thereferenceperiodorthenumericalvalue,
considerkeyconditionswhenusingquantitativeandqualitativeindicators(e.g.trade-offsbetweenrele-
vantandnon-relevantdata,validityofdata,sensitisationofthetargetgroup,expenditureincostandtime,
dataprotection),
dealwiththeconceptoftheacademicbalancedscorecard.Basedonthis,participantsareabletotranslate
HEIstrategiesintoobjectivesandfindsuitableindicatorstomeasureaperformanceleveltobereachedin
adefinedperiod.
On successful completion of this chapter, you should be able to…
Chapter 2
Translation of Higher Education Objectives Into Numbers
Chapter 2: Translation of Higher Education Objectives Into Numbers
31
2 Translation of Higher Education Objectives Into Numbers: Quantitative and Qualitative Indicators
2.1 Meaning and Function of Quantitative and Qualitative Indicators
Inthepreviouschaptersyouhavelearnedthatthepurposesofinformationsystemsareto
supportdecision-makingprocesses,
achievetransparencyonstructuralprocesses,
increaseefficiencyandeffectivenessoftheprocessesathighereducationinstitutions.
Indicatorsplayanimportantroletoreachtheseobjectives.Theirtaskistosummariseaquantitativemeasur-
ablesituationandtoidentifyrelevantfactsandcorrelationsinasimpleandcondensedform.(Küpper2013,
476).
Focussingonhighereducationinstitutionsmeansmakinganyactivitiesreferringtodecision-making,organi-
sationalorplanningprocessestransparent.Theygiveaquantitativeoverviewaboutthestatusquoatahigher
educationinstitution.Indicatorsreducecomplexityandaggregateinformation,whichmeansthattheyinform
aspreciselyandbrieflyaspossibleaboutperformances.Indoingso,theyhelptoachieveanadequateinfor-
mationsupplyforhighereducationmanagement:theyallowanalysingthestatusquoaswellastoevaluating
theoutcomesofthespecificcoursesofactions.Fromaninternalperspectivetheyareafundamentalbasisof
managementandrelateddecision-makingprocesses.Fromanexternalperspective,highereducationinsti-
tutionscanbemeasured,compared(e.g.rankings)andevenmanaged(e.g.targetagreementswiththemin-
istry)basedonperformanceindicators.Basedonthis,indicatorsarealsocloselyrelatedtothequalityassur-
ancesystemofahighereducationinstitution.
If indicatorsareusedtodescribeperformancesorthesuccessofdefinedobjectivesofahighereducation
institution,weoftenusetheterm“keyperformanceindicators“or“performanceindicators“.Accordingto
theAnalyticQualityGlossary,
“Performance indicators are data, usually quantitative in form, that provide a measure of some
aspect of an individual’s or organisation’s performance against which changes in performance or
the performance of others can be compared.”
(Harvey 2004-14)
It shouldbeconsidered, thatalthoughperformance indicatorshavea relativelyprecisemeaning, there is
atendencytousethistermforanystatisticaldatarelatedtotheactivitiesofhighereducationinstitutions,
whetherornotitreallyreferstoperformanceorsuccess(Harvey2004-14).
Performance Indicators
Chapter 2: Translation of Higher Education Objectives Into Numbers
32
Consideringthis,qualitymanagersshouldbeabletounderstandmeaningandfunctionof(performance)indi-
cators,usethemcorrectlyandexplainthemappropriatelytotherespectivetargetgroups.
AccordingtoGladen(2003,11)keyfunctionsofindicatorscan...
describecomplexandoperationalissues,structuresandprocessesinarathersimpleway,
guarantyacomprisingandquickoverview,
serveleadershipforspecificanalyses,
serveleadershipforcurrentplanning,decision-makingandmanagerialaccounting,
enableinformationreleasebyaggregationandselection,
describecriticalfactorsofsuccessandshortagesinthemanagementsystem.
2.2 Determination and Operationalisation of Quantitative and Qualitative Indicators
Indicatorscanbedescribedwiththreekeyparameters:
1. Theobject/target,theyaredescribing(what?).
2. Thetimeframe,whichtheyreferto(dateorperiod?).
3. Adefinednumericalvalueforquantification(howmuch?).
Indicatorscanbedifferentiatedintoquantitativeandqualitativeindicators.Quantitativeindicatorsdescribe
issuesandsituationswithaclearlydefinednumber.Basedonthereductiontothesubstantialsignificance,
existingindividualinformationiscondensedtoanobservableandmeasurablematteroffact(Gladen2003,
12).
Examplesincludeavailablethird-partyfundsofafaculty,numberofstudentsinacertainprogramme,number
ofPhDstudentsperprofessor,availableacademicstaffofafaculty,drop-outstudentsratiosetc.
“Qualitativeindicatorsareproxyparameters,whosecharacterorvaryingvaluehelpstoconcludethechar-
acterorvaryingvalueofanotherimportantparameter”(translatedfromGladen2003,15).Thatmeansthat
theydonotdescribedirectlymeasurablevariables,buttheyserveasasubstitutewhichiseasiertobemeas-
ured.Basedonthiswecananalyseperformancesthatcannotbequantifiedormeasureddirectly.Forexam-
ple,ifafacultyorachairwantstodescribeitsresearchperformancelevel,theyconsidervariousquantitative
indicatorssuchasnumberofpublications,patents,successfuldoctoratesortheamountofraisedthird-party
funds.Thesumoftheseindicatorsissupposedtohelpratingtheresearchperformance.
Theproblemofusingqualitative indicators is that theyonlyhavea limitedvalidity,becausethecause-ef-
fectrelationshipbetweentheoriginalandthesubstitutingindicatorisonlybasedonassumptions,butnot
Quantitative Indicators
Qualitative indicators
Chapter 2: Translation of Higher Education Objectives Into Numbers
33
onexactdescriptions.Thismeansthatcause-effectrelationshipscanbebiasedormono-causalandwithit
incomplete(Küpper2013,480).7Thiscanprovokecontradictionswithregardtotheanalysisandinterpreta-
tionoftherespectivedata,asisshowninthefollowingexample:
Theseniormanagementofahighereducation institutionwantstoknowwhichthemostsuccessfulstudy
programmesof their faculties are. Therefore, theydefine thequantitative indicator “numberof achieved
degrees”.Viewedinisolation,thisindicatorisdefinitelyvalidsinceitdescribeswhatitismeanttodescribe
–thesuccessofstudyprogrammes,whichismirroredintherespectivenumberofdegrees.Nevertheless,
ifnotusedadequately,this indicatorcanentailwrongincentivesorundesiredside-effects.Forexample,a
target-settingbasedonthisindicatorcouldinducefacultiestoneglectexistingcriteriatopassfinalexamsin
ordertobeabletoachieveasmanysuccessfuldegreesaspossible.
Theexampleshowsthatwehavetobecarefulandmustdefineindicatorsdeliberatelywhenusingthemfor
managementpurposes(alsoconsiderChapter2.3.2focussingontheBalancedScorecard(BSC)).
Ifasuccessfulstudyprogrammeisnotonlydefinedbythenumberofgraduatesbutalsobyfulfillingprevi-
ouslydefinedminimumrequirementsinteachingandlearning,thismeansdifferentiatingandconcretising
consideredparametersinamorequalitativeway.Forexample,todescribeasuccessfulstudyprogrammewe
canconsiderevenmorequantitativeindicatorsthataresummarisedtoaqualitativeindicator(e.g.mentoring
student’sratio,drop-outstudent’sratio,numberofrepetitionoffinalexamsortheaveragetimeneededto
completeadegree).
Similarly,wecanrefertosuccessfulresearch:Thesuccessofascientificexperimentdependsonvariousinflu-
encingparameters,whicharesearcheroftenisnotabletocontrol.Thatmeans,weneedindicatorsthatare
abletoreduceinformationasymmetriesinsuchawaythattheaddressee(e.g.theseniormanagement)is
abletoconcludeonthefactualresearchactivitiesoftheresearcher.
Therefore, data cannot only be analysed quantitatively, but their qualitative characteristics and possible
resultingeffectshavetobeconsideredaswell.
Further Reading
Dealingwithnationalteachingperformanceindicators–thefollowingarticlegivesanexamplefrom
Australia:
Barrie,S.,&Ginns,P.(2007).Thelinkingofnationalteachingperformanceindicatorstoimprove-
mentsinteachingandlearninginclassrooms. Quality in Higher Education,13(3),205-286.
7Youfindmoreinformationonhowtodealwiththeissue“validity”inModule2,Chapter5.4.
Chapter 2: Translation of Higher Education Objectives Into Numbers
34
Qualitymanagerscanplayanimportantroleinthiscontext.Theycanuncovercontradictionswhenusingindi-
cators,theycanmaketransparentandunderstandablecause-effectrelationships,andtheycanshowdeci-
sion-makerswaysofdealingwiththemappropriately.Todoso,qualitymanagersshouldknowandbeableto
dealwiththekeyrequirementsofindicators.Therefore,thefollowingchaptergivesanintroduction.
2.3 Using Indicators – Key Aspects to Be ConsideredThischapterdescribeskeyrequirementstobeconsideredtodefinevaluableindicators.Furthermore,wewill
gettoknowtheBalancedScorecardasanexampleofaninstrumenttouseanddealwithindicators.Thechap-
tertiesinwiththediscussionaboutthemethodologicalrealisationofsurveysinModule2.
2.3.1 Requirements to Define IndicatorsThefollowingfactorsshouldbeconsideredwhenaimingatdefiningpreciseindicators(Hórvath2011,542et
seq.;Tropp2002,57etseqq.)
1. Each indicator needs a concrete purpose
Tobesignificantanindicatorneedsaconcretepurposeandoneorseveral(butnotarbitraryselected)
addressees.
Tobeabletouseindicatorsforseveralpurposes,theyhavetobedefinedanddifferentiatedexactly.
Datacollection,thatisnecessarytodefineanindicator,hastoberelatedappropriatelytothepurposeof
theindicator.
Formalrequirements(e.g.law/politicalrequirements),whicharerelevantfordefininganindicator,have
tobeconsidered.
Key questions to be answered:
Whatisthesignificanceoftheindicator?
Whichnumericalvaluetranslatesthissignificance?
Whichinformationdoesthisnumericalvaluetakeintoaccountandwhichnot?
Whichformalrequirementshavetobeconsidered?
2. Validity of data: No quantitative data without additional qualitative information
Indicatorshavetobecontrolledwithregardtotheirvaliditytoavoidwrongincentivesorunexpected/
undesirableside-effects(seeexampleonsuccessfuldegrees).
Key questions to be answered:
Whatarethecontinualdatasourcesandwhocollectsthemtodefineanindicator?
Whataresuitablereferencevalues(benchmarks)tocontrolthevalidityofanindicator?
3. Trade-off between relevant and non-relevant data and information
Providedhigh-qualityvalidity,thescopeofdatatodefineindicatorsshouldbereducedasmuchaspossi-
ble.Anoverloadedlevelofdetailcanevenhinderstrategicmanagement.
Reductionofdatacollectionthatisnotrelevantforthedefinitionofindicatorsandwithitavoid“data
graveyards”.
Chapter 2: Translation of Higher Education Objectives Into Numbers
35
Key questions to be answered:
Whichdataisnecessarytodefineacertainindicatorandwhichisnot?
Arethereanyirrelevantdatathatkeepbeingconsideredunnecessarily?
4. Considering feedback
Thenumericaldatashouldbealignedtotherealityoftheaffectedstakeholdersandevaluatedwith
regardtocontradictions.
Atthesametime,theaffectedstakeholderscanbeprovidedwiththeevaluatedandanalyseddatatobe
consideredforfurtheractionsanddevelopments.
Key questions to be answered:
Doesthecollecteddatareflectreality?
Arethereanyconstraints?
Dotheselectedindicatorsprovideanyadditionalbenefitsforimprovementandenhancement?
5. No isolated measurements
Whencollecting,analysinganddocumentingdata,itshouldnotbedoneinisolationbutcomparable
parametersshouldbeconsidered(e.g.descriptionofabsolute,relativeandaccumulatednumbers).
Datatobeusedtodefineindicatorsshouldbecollectedcontinuouslyoveralongerperiodinsteadofonly
onceandinisolation.Byconsideringalongerperiodthesignificanceofindicatorsincreasesanditfacili-
tatesamoreexactjudgementofaverageperformancelevels.
Key questions to be answered:
Whatisthedateofreferenceandtheperiodofreferenceforthedefinedindicator?
Inwhichintervalshouldtheindicatorsbelookedat?
6. Expenditure in cost and time
Collecting,analysingandpublishingdataandinformationrequiresfinancial,staffandalsomaterial
recourseswhichhavetobecalculatedintime.
Timeneededtogatherinformationistobecalculatedintimeandtobecoordinatedwithpossibledead-
lineswhichhavetobeconsidered.
Key questions to be answered:
Whichexpendituresonresources(staff,finances,IT-system,material)havetobeconsidered?
Whatisthetimeframetosubmittherequireddataandinformation?
Whatisthecost/benefit-ratiowithregardtoexpenditureofresourcesandtimeandtheadditional
benefitoftheprovidedinformation?
7. Data protection
Thecollectedandanalyseddataaretreatedresponsiblyandaccordingtogivendataprotectionguide-
lines.
Key questions to be answered:
Dodataandinformationcomplywiththerespectivedataprotectionguidelinesinforce?
Whathastobedonetomeetpersonaldataprotectionrightsandtoavoidmisuse?
Chapter 2: Translation of Higher Education Objectives Into Numbers
36
8. Sensitisation of the target group to use edited data reports
Informingthegroupofaddresseesabouthowtointerpretindicatorsandwhattousethemfor.
Key questions to be answered:
Istheinformationoftheindicatortransparenttothegroupofaddressees?
Whichinformationdoesthegroupofaddresseesneedtobeabletousetheindicatorsappropriately?
Further Reading
Chalmers,D.(2008).Teaching and learning quality indicators in australian universities. Outcomes of
higher education: Quality relevance and impact.Paris:ProgrammeonInstitutionalManagementin
HigherEducation.
2.3.2 The Balanced-Scorecard – An Instrument to Monitor Indicators
Indicatorsthataredefinedunderstandablyandcomprehensivelycancontributetoreduceinformationasym-
metriesbetweendifferenttargetgroups.Theyspecifytherespectivedefinedobjectivesandthusfacilitate
thecoordinationofnecessaryprocessestoreachtheseobjectives(Küpper2013,500).Thiscanbecarriedout
eitherverticallyacrossthedifferenthierarchicallevelsofahighereducationinstitution,aimingatmanaging
itsmultipleunits (e.g.with targetperformanceagreements),orhorizontally tomanagedifferentdomains
basedondefinedtargetsforthesedomains(e.g.orientationofstudyprogrammesoninternationalstudents).
Oneexampleofaninstrumenttomonitorindicatorsathighereducationinstitutionareindicatorsystems.An
indicatorsystemis
“an arrangement of indicators in a systematic way, which means that the individual indicators
are linked in a meaningful way, that they complement each other, and that they are aligned to an
overriding common objective.”
(translated from Tropp 2002, 3 et seq.)
Questions & Assignments
1. Whichparticular conditionsdoesyour institutionhave toconsiderwhendealingwithdataand
information?Whichchallengesdosuchconditionscomewith?
Indicator system
Chapter 2: Translation of Higher Education Objectives Into Numbers
37
Besides indicator systems, another strategicmanagement instrument is theBalanced Scorecard,which is
increasinginpopularityathighereducationinstitutions.
ABSCfacilitatesthelinkbetweenstrategicplanningandoperationalprocessestorenderperformanceassess-
ment.Otherthanindicatorsystems,aBSCisnotbasedonapredefinedsetofindicators,butenablesamore
precisechoiceofindicatorsfortherespectiveobjectiveswhicharetobeoperationalised.Therefore,aBSC
isveryusefulinmonitoringcomplexitiesandorganisationalparticularitiesofahighereducationinstitution,
suchasuncleartechnologiesofperformanceassessment,ambiguousandcomplextargetstructures,differing
memberships,staffexpertise,hierarchiesororganisationbasedonknowledge(Scheytt2007).
ABSCcancontributesignificantlytoachievemoretransparencyandclarityaboutthestrategicobjectivesof
ahighereducationinstitution.Basedonthis,suitableorganisationalprocessescanbedevelopedinorderto
reachthesedefinedobjectivescanbedeveloped(Röbken2003,4).
Theterm“balanced”signifiesthattheperspectivesthatarerelevanttorealiseastrategyareequallyweighted
inthescorecard(Kaplan/Norten,inRöbken2003).AccordingtoKaplanandNorten,typicalperspectivestobe
consideredinaBSCarethefollowingfour8:
1.customer
2.learningandgrowth(humanresourcesandorganisationaldevelopment)
3.financial
4.internalprocesses
Consideringtheseperspectives,wecandefineindicatorsforthestrategicobjectivesanddeterminetargetval-
uesthathelptomeasurehowfartheseobjectiveshavebeenreached.
Duetothebalancedconsiderationofthementionedperspectives,theBSC-approachtriestocopewiththe
challengingtaskofcomprisingdifferingcontextsandinfluencingfactorsofsubject-mattersandofanalysing
andinterpringoutcome-linkagesmoretransparentlyandclearly(Scheytt2007).
AccordingtoKaplanandNorton(1996)theimplementationofaBalancedScorecardcanbebasedonfivekey
steps(Scheytt2007):
1 . Definitionofthedifferentperspectiveswhichareoffundamentalimportancetothehighereducation
institution.ThesecandifferfromtheabovementionedeconomicalBSCmodel.
2. Deductionofobjectives,whichareparticularlyimportanttofollowthestrategicplan(operationalisation
ofobjectives).
3. Definitionofindicators,whichinformaboutcontent,extentandtimeframetoreachtheobjectivesand
thushelptomanagetheorganisationalprocessesofperformanceassessment.
8Theseperspectivescanbeadaptedtotherespectiveneedsofaninstitution.
ABSC translates thevision andthe strategyofa higher education institution intocoherent objectivesand indicators
Chapter 2: Translation of Higher Education Objectives Into Numbers
38
4. Definitionoftargetvaluesbasedoninfluencingparameterstobereachedinacertainperiod(e.g.one
year).
5. Definitionofinitiatives/activitiestoberealisedinordertoreachtheobjectivesduringadefinedperiod.
Figure 5 System of a Balanced Scorecard (adapted from Scheytt 2007)
Themanagedprocessing,asitisdescribedintheillustration,istheparticularcharacteristicoftheBSC,alsoto
bedistinguishedfromotherconceptsofperformancemanagementsuchasindicatorsystems.Suchprocess
orientationfacilitatesthediscussionabouttarget-performancecomparisons:Onetheonehand,thecurrent
statusisdefinedbyanalysingthequestions“whodoeswhat,when,whereandhow?”Ontheotherhand,tar-
getvaluesandthequestionwhohastobeinvolvedandwhichinformationistobeneededfromwhomand
tillwhen(Scheytt2007)
Deducingindicatorsforthetotal“objectivehierarchy”ofahighereducationinstitutionaimsatguaranteeing
congruencebetweenthedifferentobjectivesandatcoordinatingstrategicplanningwiththeorganisational
processesofdailyperformanceassessment.Basedonthis,theBSCcansupportcommunicationprocesses
betweenthedifferentdepartmentsandstaffbydevelopingaframeworkthatenablesacontinuousprocess
ofself-evaluationandorganisationallearning(Röbken2003,4).Thisincludesaimingcontinuouslyatquality
enhancementandwithitestablishingandsystemisinginternalqualityassurancestructures.
Chapter 2: Translation of Higher Education Objectives Into Numbers
39
2.4 Challenges of Using Quantitative and Qualitative Indicators
Whilecompaniesgenerallyhaveonebigstrategictargettobeachievedbyallemployees,athighereduca-
tioninstitutionswecanfinddifferentlooselycoupledtargetsystems,whicharenotrelevantforallmembers
oftheinstitutionbutonlyforpartialgroups.Thedifferentfaculties,theseniormanagement,aswellasthe
administrationofahighereducationinstitutioncanhaveratherdiffering,sometimesevenconflictingtargets
withdifferentpriorities.Forexample,aprofessorwhoisdoingresearchmightbeparticularlyinterestedin
gainingsufficientthird-partyfundstobeabletodoresearch.Atthesametime,fortheseniormanagement
third-partyfundsofferapossibilitytobalancebudgetdeficits.Furthermore,theystrengthenexternalinsti-
tutionalprofiling.Meanwhile,alecturermightbeespeciallyinterestedinadequateresourcestobeableto
facilitategoodteachingandlearningconditions.Thelatterisalsoakeyconcernofthestudentswhowantto
completetheirstudiessuccessfully.
Accordingtothis,anotherchallengetodealwithistheformulationofobjectives.Whatlevelofclarityand
precisiondoobjectivesneedinordertobemeasurable?Andwhichamplescopecantheyhavetoenable
a broadflexibilitywith regard to their design and implementationaccording to the academic freedom in
researchandteaching.
Basedonthis,anotherobstaclewhendefiningandusingindicatorsisthattheycannotbedefinedforseveral
objectivesatthesametime,butonlyforoneconcreteobjective.Duetothissingle-sidedfocus,itmayoccur
thatcausalitiesbetweendifferentobjectivesarenotconsideredandwithitentailcontradictoryorevenwrong
interpretationsfortakingfurtheractions.UsingaBSC,requiresconsideringsuchcausalitieswhencombining
differentindicatorsforanobjective.
Theproblemofcontradictoryconclusionscanalsobeaconsequenceofdifferentunderstandingsaboutindi-
catorsandtheirassumedprioritylevels.Thefollowingmetaphoricalcomparisoncouldhelptoillustratethis
problem:whentalkingaboutapples,wecanassumethatonepersonconsidersanappletobebig,sourand
Further Reading
Kaplan,R.S.(2011).StrategicperformancemeasurementandmanagementinNonprofitOrganiza-
tions.Nonprofit Management & Leadership, 11(3),353–370.
Kaplan,R.S.,&Norton,D.P.(1993).Puttingthebalancedscorecardtowork.Harvard Business Re-
view,71(5),134-147.
Kaplan,R.S.,&Norton,D.P.(1996).Usingthebalancedscorecardasastrategicmanagementsys-
tem.Harvard Business Review, 74(1),75-85.
Chapter 2: Translation of Higher Education Objectives Into Numbers
40
green,meanwhileanotherthinksaboutsmall,crispandredapples.Translatingthistothehighereducation
contextmeans, forexample, that“goodteaching”at the facultyofmathematicscanbecharacteriseddif-
ferentlythanatthefacultyofsocialsciences9.Also,internationalpublicationstobeusedasanindicatorof
researchqualitycanberatherimportantinonefaculty,whileinanothertheyarenotasrelevant.
Thesedifferingunderstandingshavetobeconsideredandclarifiedwhendefiningindicators.Onlythen,are
weabletoachieveacommonbasisfortheiranalysisandinterpretationandcanthereforeavoidtheapple
comparisonbecomingacomparisonofapplesandpears.
Anotherprecedingchallengeisthathighereducationinstitutionsneedanoverarchingstrategyasabasisto
defineanduseindicators.Whatwecanobserveisthatstrategiesonlyexistonpaper,buttheydonotplaya
rolewithregardtooperationalisingprocessesandactivities.Ifhighereducationinstitutionswanttodealwith
indicators,strategicplanningisanobligatoryrequirement–itisthestrategythatistranslatedintoconcrete
operationalisedtargets(e.g.basedonaBSC)thataremeasuredbasedonappropriateindicators.Thatmeans,
theessentialprerequisiteforintroducingaBalancedScorecardisahighereducationinstitutiondetermining
itsstrategicorientation,documentingitandmakingittransparentamongthewholeorganisation,forexam-
plebydevelopingstrategicplansoninstitutionalorfacultylevel.
Furthermore,whenusingindicatorsdifferentcomparisondimensionshavetobeconsidered:forexample,for
internalpurposesindicatorsareoftenusedtocomparedatainahistoricaltimeframe.Thatmeans,theymon-
itorcertaindevelopmentsduringagivenperiodoftimeandserveasabasisforfutureperformancelevelsto
beachieved,andwhicharenegotiated,e.g.viatarget-performanceagreements(Röbken2003).Forexternal
purposes,indicatorscansupportthecomparisonofhighereducationinstitutions(orafaculty,aunitetc.)in
termsofrankingsorbenchmarking.
Focussingonthevalidityofindicatorsanotherchallengeisthatveryoftentheycannotbecontrolledcom-
parably,which lead to furtherdiffering interpretation frameworks. For example, higher education institu-
tionscanhardlyinfluenceinput-parametersbecausetheycannotinfluencetheprovisionofresources.This
changeswhenwelookatprocess-parameters:toensureandenhancethequalityofteachingandlearning,
weshouldnotonlyconsidertheprovidedresources,butfocusonaspectssuchascurriculumdesign,didac-
tics,programmeandassessmentmanagement,planningstudentinfrastructure,evaluationofchairsorother
teachingunits.
Thementionedchallengesindicatethatdealingwithindicatorsinvolvesahighworkloadandexpenditureof
time.Themorecomplicatedthemethodsandtechniquesforthedatacompilation,themoreriskofanincom-
pleteandnonpermanentdata-collection,andwithitindicatorsthatareneitherrelevantnorsignificant.Con-
sideringthis,wealsohavetoquestiontheintendedbenefitscomparedtotheintroducedcosts.Tocountervail
thisproblem,itisimportanttoreflectwhichdata-collectionmethodsandwhichdataisalreadyavailableto
describehighereducationprocesses,whichadditionalinformationmightbeusefulandtowhatextentexten-
sionsoradaptionsoftheexistingdata-systemmightbepossibleanduseful.
9Forfurtherexplanationsonhowtooperationalisethequalityof“goodteaching”pleaseconsiderModule2,Chapter5.2.
Chapter 2: Translation of Higher Education Objectives Into Numbers
41
Furthermore,highworkloadwithregardtocollectingdataandthe followingdocumentationandcommu-
nicationflowscanresult inopposingandnegativeattitudesamongstaff.Toreducesuchoppositions, it is
veryimportanttoexplainandcommunicatetheadditionalbenefitandthepurposeoftheintroductionofan
instrumentliketheBSCorotherindicatorsystemsforahighereducationinstitution.
ThedescribedchallengesbringoutthenarrowlimitsofusingaBSCandindicatorsasameanstoimprovepro-
cessesandactivitiesthatservetoachievecertainobjectives.Wehavetokeeptheselimitsinmindandshould
notunderestimatethem,sinceitmightbecomeevenmoreproblematicandcomplicated,whencontradic-
tionsarenotclarified,butcontinuouslyproceeded.Inthiscase,theexpectedbenefitofworkingwithindica-
torsasaninstrumenttosystematiseandmanageprocesseswouldbenotberealised.
Consideringthis,whenoperationalising indicatorswecontinuouslyhavetocheckwhichindicatorcanpro-
videwhichcontributionandhowrelevantthiscontributioniswithregardtoachievingtheintendedstrategic
objective.
Comingbacktoqualityassuranceprocessesathighereducationinstitutions,qualitymanagersplayanimpor-
tantroleindealingwiththeabove-mentionedchallenges.Theycanhelptodefineappropriateindicatorsfor
thekeyprocesses,teachingandresearch.Furthermore,theyshouldrevealbothopportunitiesandalsolimits
ofusingindicatorsandmakethemtransparenttotherespectivetargetgroups.Basedonthis,theycanfacili-
tateacoordinatedandadequateinformationfundamentfordecision-makingprocesses.
Challenges when dealing with (performance) indicators
Example
AtHEItherearedifferentstakeholderswithmul-tiple,sometimescontradictoryobjectives.
Topmanagement:getthird-partyfundsforrea-sonsofcompetitionandcompensationofbudgetdeficits.
Professor:getsthird-partyfundstodomoreresearch.
Anindicatorcannotrepresentmultipleobjectives
butonlyonedefinedobjective.
Theindicator“third-partyfunds”ofafacultyreferstotheallocationofthird-partyfundsatafaculty.Itdoesnotrefertoresearchquality.
Adefinedstrategyisaprerequisitetouseabal-anced-scorecard.
HEIstrategy:toincreasetheinternationalisationofteachingandlearning.
Indicators: Numberofinternationalstudyprogrammes Numberofinternationalcollaborativeresearch
projects
Thecomparabilityofindicatorsmaydiffer(e.g.dependingontheirlongitudinalorinter-organi-sationaluse)
Longitudinaluse:comparedatawithregardtothedevelopmentofstudyprogrammesoveracertainperiod.
Inter-organisationaluse:comparetwofacultieswithregardtothenumberofgraduates.
Chapter 2: Translation of Higher Education Objectives Into Numbers
42
Challenges when dealing with (performance) indicators
Example
Theinfluenceonthevalidityofindicatorsmaydiffer(e.g.inputvs.processindicators)
Indicatorsforthequalityofteachingandlearn- ing:
Inputindicators:resourceallocationthatis determinedbyexternalstakeholders(e.g.min- istry).HEIcanhardlyinfluencetheamountof resourceallocation.
Processindicators:curriculumdesign,didactics, managementofassessment/studentinfrastruc- ture,evaluationetc.HEIcaninfluencethequali- tyoftheseindicators.
Therelationbetweenexpenditureoftimewhencollectingdatafortheindicatorsandtheeffectsshouldbebalanced.
Whichadditionalinformationdoweexpectfrom students’drop-out-rates?Dowegetmoreinfor- mationthanwhatwealreadyknow?Isthis informationworthinvestingtimeonrespective datacollection?
Whichinformation/dataalreadyexistandwhichadditionalinformation/datashould/couldbeaddedoradjusted?
Both,administrationandfacultycollectdata aboutstudentswhogoabroadduringtheirstud- ies.Itshouldbecheckedinhowfarthesenum- bersarecoherenttoeachotherand/orcanbe matched.
Whichnotionsofresistanceamongstaffhavetobeconsidered?
Staffresistanceduetooverlappingresponsibil- ities
Table 3 Challenges of (performance) indicators
Chapter 2: Translation of Higher Education Objectives Into Numbers
43
Chapter 3: Reporting: Presentation and Communication of Data and Information
44
3 Reporting:PresentationandCommunicationof DataandInformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.1 DefinitionofReportingObjectivesforDifferentTargetGroups . . . . . . . . . . . . . . 45
3.2 ContentofReporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.3 OrganisationalConditionsforReporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
transfercollecteddataintoacoherentandtransparentreportingsystem,
define reporting objectives for different target groups (e.g. (internal) accountability, strategic decision-
making,qualityassurance),
setupareportstep-by-step,consideringaspectssuchastargetgroups,afundamentalplan/actualdata
analysis,andanappropriatecompositionofvalidandrelevantinformation,
supportthedevelopmentofareportsystematyourinstitution.Youwillbeabletodetermineresponsibili-
tiesandfunctions,defineworkflows,deadlinesandreportingfrequencies,aswellasanappropriateformat
ofreporting.
On successful completion of this chapter, you should be able to…
Chapter 3
Reporting: Presentation and Communication of Data and Information
Chapter 3: Reporting: Presentation and Communication of Data and Information
45
3 Reporting: Presentation and Communication of Data and Information
3.1 Definition of Reporting Objectives for Different Target Groups
According toBlohm,a reportingsystem includesallunits, regulationsandactivitiesofahighereducation
institutionwhichsupportcollecting,analysingandcommunicatinginformationforinternalandexternaluse
(BlohminGrochla1980,316).Basedonthis,thedistributionandexchangeofinformationiscarriedoutby
reportswhich“includesummarisedinformationthatrefertoanoverridingaimandaninformationpurpose”
(translatedfromBlohm1974,15).
Therefore,reportsplayanimportantrolewithregardtoqualityassuranceandenhancementathigheredu-
cationinstitutions.Theyhelptodocumentevaluatedstatusquosandtodescribeopportunitiesandthreats
on theway to achieveexpectedperformance levels. Furthermore, they serve accountability purposeson
achievedoutputs-statusinthecorefieldsofteachingandlearning,researchorservicesbyprovidingafunda-
mentalbasisfordecision-makingprocesses.
Qualitymanagerscanbeassignedwithdevelopingsuchreportsorsupportingotherstaffmembersduringa
reportingprocess.Thisiswhytheyshouldhaveabasicunderstandingabouttheobjectivesofreportingto
thedifferenttargetgroups.Basedonthis,theyshouldbeabletodesignanadequatereportstepbystep(e.g.
coordinatingresponsibilities,workflows,deadlines,reportingfrequenciesorformats).
Inthefollowing,youwillgettoknowdifferenttypesofreportingthatcanbeusedfordifferentpurposesand
targetgroups.Basically,wecandifferentiatethreedifferenttypesofreporting:standardreports,reportson
demandanddeviationreports(Hórvath2011,535;Horváth2008,21etseq.;Küpperetal.2013,231etseq.;
Göpfert2007,3etseq).
Standardreportsarepublishedinregularlyfixedperiods.Theyarestandardisedinformandcontent,based
onadefinedsetofinformationneeds(e.g.standardisedteachingreports,reportforthetopmanagement/
ministry,evaluationreport).Generally,inthiscasetheaddresseehastoidentifyandselecttheinformation
thatisrelevanttoher/himfromthereportonher/hisown.Oneproblematicaspectofsuchstandardreports
isthequestionoftheirsignificancewithregardtoanoverarchingpurpose.Duetothestandardisationitcan
occur, thatcertain informationneedsofanaddresseearenotreportedcorrectly.Or,dependingonwhich
informationaddresseesselectfromthereport,theycaninterpretwrongorunclearcorrelations.
Consideringtheseproblems,reportsondemandaregainingrelevanceandcansubstitutestandardreports
withregardtocertainpurposes.Reportsondemandarenotbasedonstandardiseddata,butaredesigned
Distribution ofinformation basedon reporting
Chapter 3: Reporting: Presentation and Communication of Data and Information
46
forspecificinformationdemandsoftheaddressees.Theydonothaveapre-fixedrhythmofbeinggenerated.
Basedonadatabasethatincludesallrelevantdataforhighereducationmanagement,theaddresseescan
generatetheindividualinformationneededontheirownwithdirectaccess.Therefore,theaddresseestake
onamoreactiverole,onlyselectingsuchinformationthatisrelevanttothem(e.g.informationtobeconsid-
eredinaself-reportofaself-evaluationinteachingandlearning/research).Usingsuchreportsrequiresthe
addresseestoknowhowtousethedatabaseinordertobeabletogeneratesuchinformationrequests.
Thethirdtype,deviationreports,servestofocusonplan-actual-deviationsofmanagementissuesthatexceed
orfallbelowcertaindefinedtolerancevalues.Suchreportsareonlyusedwhennormalprocessesareinter-
ruptedbyconspicuousdeviationsordisturbancestoreachtheexpectedoutcomes(e.g.non-predictablefallin
students’enrolment).Thecontentandformatofthesereportsarenotstandardisednormally.Theaddressees
can,forexample,bedeansoffaculties,acontrollerorthetopmanagement.
3.2 Content of ReportingHowcanhighereducationinstitutionsdesignandusereportsadequatelywithregardtotheirpurposesand
withjustifiableworkload?
Figure 6 Criteria to design reports (translated illustration adapted from Tropp 2002, 70)
In the following, it is suggested that some fundamental conditions shouldbe consideredwhendesigning
reportsforinformationtransferpurposes.
Designing reports
Purpose ofReporting
Chapter 3: Reporting: Presentation and Communication of Data and Information
47
1. Why Reporting? (purpose)
Reportsareusedtofulfilpre-definedpurposesandthereforearenotanendinthemselves.Areport´spur-
poseisdeducedfromtheinformationneedsoftherespectivetargetgroup.Hence,areportcanbeusedfor
accountabilityanddocumentationreasons,bothfrominternalaswellasexternaladdressees(Küpperetal.
2013,230).Examplesincludeprotocols,listsofapproved/not-approvedexaminationsorself-reportsofinter-
nal/externalevaluation.Furthermore,reportsserveformanagementpurposesandwithitforpreparingand
controllingdecision-makingprocesses.Forexample,basedonareportoffinancialliquidity,theseniorman-
agementcandecideaboutthedistributionorcutbacksoffinancialresourcesindifferentfieldsoftheinstitu-
tion.Besidesservingformanagementandaccountabilityreasons,reportscanreleaseworkflows.Forexam-
ple,abudgetreportofacertainunitcanentailstartingarevisionoftheexpectedtargetsandresourcestobe
neededtoachievethesetargets.Concerningprojects,reportsareespeciallyusedtomonitortheprocessing
andtherespectivelevelsoftargetachievement.
2. What to Report?
Dependingonthepurpose,wehavetodecideonwhichinformationistobereported.Isitinformationtobe
usedforaccountingpurposesthatshouldreportonthecurrentstateofacertainarea?Willtheinformation
beusedforinternal/externalcomparability?Consideringthepurposeofareport,wehavetodecideabout
thescopeandthelevelofaccuracyandaggregationofinformation,sotheycanbeusedappropriately:Isthe
informationrelevantfortherespectivepurpose?Doestheinformationgiveadequateanswerstothedesired
informationneeds?Reportsshouldonlyincludesuchinformationthatisneeded,notmoreandnotless(“as
muchasnecessary,aslittleaspossible”).Thecollecteddatashouldsignificantlyhelptoanalyseandillustrate
thelevelofdefinedobjectives.Theillustrationofquantitativedataandindicatorsshouldbecompletedwith
qualitativedescriptionsandevaluationstobecomeasexactandunderstandableaspossible.Consideringindi-
vidualcontextsaswellasrelevantcorrelationsoroverlapswithotherobjectivesatahighereducationinsti-
tutionmayhelptodesignapictureofrealitythatisasexactandundistortedaspossible.Forexample,when
wecollectdataaboutteachingcapacities,itisnotenoughtocollectdatathatreferstotheteaching-workload
leveloflecturers.Eataaboutresearchworkloadoradministrativeobligationsshouldbeconsideredaswell.In
thefollowing,thisdatacollectionshouldbeanalysedbasedonaqualitativedescription.
Atthesametimewealwayshavetokeepinmindtobalancethenecessaryworkflows(includingstaffandtime
resourcesneeded)fortheexpectedinformationprovisionandtheexpectedresultsadequately,toavoiding
graveyards.
3. How to Report? (structure / format)
Aclearstructureaswellasthewayofpublishing(e.g.onlineorpaper-based)influencetheaddresseesand
how they are using the information. For example, by using visualisations and graphic illustrations special
issuesbecomeclearerora report iseasier toread.Furthermore, reportsshouldhaveastandardisedand
repeatingstructure,whichcanbeastandardisedheadingorthesameorderofindividualandaccumulated
information.Thismeans,thereportstructureshouldbechosenaccordingtotheneedsoftheaddresseesand
thusmakingsurethepresentedinformationisreadableandunderstandabletothem.
Chapter 3: Reporting: Presentation and Communication of Data and Information
48
4. Who Reports to Whom? (sender / addressee)
Beforewritingareport,theaddresseehastobedefinedclearlyinordertobeabletoillustratethecontent
ofthereportaccordingtotheneedsofthetargetgroup.Thatdoesnotmeanpreparinganindividualreport
foreveryaddressee.Instead,reportscanbecomposedbyusingamodulestructure.Individualmodulescan
includeadditionalinformationforspecifictargetgroupsaccordingtotheirneeds.
Furthermoretheaddresseeshouldalsoknowaboutthesenderwhoisresponsibleforthereportingprocess.
Thisisimportanttoimprovethereportedinformation´stransparency.Thesenderfinallydecideswhichinfor-
mationistransmittedhowandensuresthatthereportisunderstoodandacceptedtobeusedwithregardto
itspurposes.
5. When to Report? (reporting periods and dates)
Focussingonthetimeframe,ithastobeclarifiedwhenareporthastobefinishedandwhetheritisaone-
timeoraregularandrepeatingreporting.Thisincludesdefiningthereferenceperiodofthereport,e.g,isthe
reportbasedondatacollectedforeachsemesterorforeachstudyyear?
Basedonthis,thetimeframesforthedifferentworkflowsofdesigningthereporthavetobedetermined,as
wellasthescopeofdataandinformationthatistobecollected,analysedandtransmittedduringthisperiod.
3.3 Organisational Conditions for ReportingDuetoincreasinginternalandexternalinformationneedsathighereducationinstitutions,thecoordination
ofinformationsupplysystemsbecomesincreasinglycomplex:Composingandaggregatingdifferingdata-for-
mats,data-sourcesaswellaspaper-basedtemplatesismoredifficultandwithitalsoerror-prone.Disturbing
parametersresultinincreasinginformationgaps,andthustheyreduce(orevenprevent)expectedoutcomes
(Koch1994,71inGladen2003,240etseq.).
Todealwiththisproblem,highereducationinstitutionshavestartedtouseprofessionalIT-softwarethatinte-
gratethedifferentcorefieldsinacompletecampusmanagementsystem10 .
Nevertheless,asweallknow,evenautomaticIT-systemsdonotworkwithoutpeoplewhopushtheelectronic
buttonsandwholinktechniqueswithhumanworkflowsandcommunicationflows.
Thisisthemomentwhenqualitymanagerscanplayanimportantroleagainbyhelpingtohandletheafore-
mentionedobstaclesofcomplexinformationsystems.Theycanfindoutaboutexistinginformationor/and
communicationdeficits.Togetherwiththerespectiveinvolvedpartiestheycandiscusshowtosolvethese
deficits.Ifnecessary,theycanalsocommunicatethesepossibilitiesandtheiraccompanyingadvantagesand
disadvantagestotherespectiveauthoritiestotakedecisions.
10 The EuropeanUniversity Information Systems (EUNIS) organisation offers an online platform for institutions to develop their IT landscapebysharingexperiencesandworkingtogether.http://www.eunis.org/
Implementing reporting systems
Chapter 3: Reporting: Presentation and Communication of Data and Information
49
Thefollowinglistsummarisesfrequentshortcomingswhendesigningreports(basedonGleich,Horváth&
Michel2008,38)
Nosufficientorientationtowardtheaddressee(“Idon’thavetheinformationIneedtomanagethebusiness
effectively”(Axson2007,131inHorváth2008,36).
Reportsarebasedonavailabledataandinformation,butnotontheinformationneeded:Transferofnon-pur
posefulinformation.
Orientationonarigidandinflexibletimefrequency
One-sidedorientationonaccountingquantitativedata
Noclearlydefinedperiodofreference:Datacollectionreferstodifferingperiods/dates.
Competitionbetweenperiodoftime,levelofaggregationandscopeofdata:Ontheonehandtoomuch
unnecessaryandunclearinformationcanbetransferred.Ontheotherhandtoogeneralinformationcan
reducesignificanceaswell.
Misunderstandingsduetounclearlydefinedterminologiesandmissingqualitativeanalysesofthedata
material.Inthefollowing,theaddresseemightdevelopwronginterpretations.
Dataisoldandnotup-to-date.Thelessup-to-datethedata,themoredifficulttoguaranteeaccuracy.
Toreducesuchshortcomingswhenimplementingareportingsystem,someessentialcriteriashouldbecon-
sidered(Horváth2008):
1. Avoiding double data collections:Whencollectingandanalysingdataforreportingpurposesyoushould
makesurethattheyarecollectedonlyoncefromasinglesourceandmayinthefollowingbeusedfordif-
ferentpurposes.Forexample,veryoftenweneedthesamedataforinternalandexternalqualityassurance
purposesathighereducationinstitutions.Thatmeans,weusedata,collectedfromthesamedatasource
andonlyaggregateandcombineitaccordingtotherespectivepurposesandneeds.
2. Efficiency:Thecoordinationbetweennewdatademandsandalreadyexistingdatasourcesshouldbeeffi-
cient.Basicdataforspecificfieldscanbeprovidedtogiveanoverviewfor interestedstakeholders,e.g.
bypublishing themona (internal)website.Anothermoreelaboratedformofusingdataefficientlyare
so-called“data-warehouse-systems”,whichhavebecomeofincreasinginterestforhighereducationinsti-
tutions.Suchacentralonline-toolisabletointegratedifferentdatasetswithdifferentpossibilitiesofdata
retrieval,andwithitfacilitatesdiversesynergyeffectsathighereducationinstitutions.Usingsuchatool,
addresseesareabletogeneratemoreexactlytheinformationtheyneed.Nevertheless,itistobeconsid-
eredthatthereexistdataprotectionregulationsincludingspecificaccessrights–beitforinternalorexter-
naluseofspecificdataorinformation.
3. Comparability of data:Toachieveapurposefulandreasonableuseofdata,itisimportanttocomposethemin
astructured,clearandtransparentway.Thismeansthatyoushouldcoordinatestandardiseddefinitions,ter-
minologiesanddatacollectionprocedures.Thisisafundamentalrequirementforenablinginternal/external
comparisons(e.g.rankingorbenchmarkingofhighereducationinstitutions,(internal)facultiesorprogrammes).
Shortcomings inreporting
Chapter 3: Reporting: Presentation and Communication of Data and Information
50
4. Reliability, validity and consistency: Followingthecriteriaofcomparability,youshouldalsocoordinate
ratherstandardisedmethodologicalproceduresofdatacollectiontopreventidenticaldatasamplesfrom
differingreferenceperiodsleadingtodifferingresultsandcreatingirritationsorequivocations.Tocontrol
datavalidity,theyshouldnotbeillustratedinisolationbutincombinationwithothercomparablevalues,
e.g.byindicatingabsolute,relativeoraccumulateddata.
5. Timeliness of data:Aprompt reportingona certain issue increases the significanceofdataand infor-
mation.Still,youshoulddifferentiatebetween“intermediatedatainrealtime”or“outcomedata”tobe
usedforspecificdefinedindicatorsandperiods.Informationthatisneededregularlyshouldbecollected
andprovidedperiodically.Tobeabletodosoandtomeetpredeterminedreportingperiodsrequiresthat
(internal)providersofdatatobeontimeaswell(e.g.submissionofassessmentresultsintoasystem).
6. Transparency:Toensuretransparencyandresponsibilitiesithastobeclear,whohasworkedonwhichdata
fromwhichdatasource.
Recommendations on How to Proceed When Designing Reports
(basedonastudyoftheCHEofthereportingsysteminSaxony-Anhalt,afederalstateinGermany:
Yorck,H.,Güttner,A.,&Müller,U.(2010)).
1. Coordinationbetweentheaddresseeandthesenderwithregardtothereportingstructureandits
elements.
2. Theinformationsystemforinternalprocessesatahighereducationinstitutionsshouldbethebasis
andfacilitatedecisionsonexternalreporting.Thismeansexternalreportsshouldbelinkedtoand
basedoninternalhighereducationmanagerialaccounting.
3. The content analyses in reports should be oriented on the objectives and thus outcome-/out-
put-based.
4. Data, indicators or parameters should be defined according to fixed and comparable stand-
ards.
5. Thereportingformatofinformationshouldincludethecollectedquantitativedata,comprisingof
somedescriptivequalitativetextwithvisualisations,tablesorotherillustrations.
6. Developmentof adata-pool for internal/externalpurposes (data-warehouse) tobeable todeal
withtheincreasingcomplexityofdatasourcesanddataformats.
7. Adhoc reports should only be based on data and information from internal data-sources.
8. Reportsforexternaluseshouldrefertoconcreteaddressees,andshouldbedesignedbasedona
modularstructurethatcanbeadjustedaccordingtotherespectiveinformationneeds.
Chapter 3: Reporting: Presentation and Communication of Data and Information
51
9. Informationpyramid: Starting from internal reports fordecision-makingprocesses andmanage-
mentofthehighereducationinstitutions,theirlevelofdetaildecreaseswhenitcomestoamore
abstractpublicuse.
10.Theentirescopeofareportshouldbeshortandreadable(notmorethan20pages,ifpossible).
11.Coordinationofaregularreporting(e.g.forexternalpurposesannualreportingwithdatacollec-
tiononapredetermineddate).
12.Annualreportsrefertothepreviousyearandshouldincludeperspectivesonthefollowingyear.
13.Structuringareport:Clearseparationbetweenoverviewanddetaileddescriptions.Generalstruc-
ture according to compulsory requirements to be supplementedwith amore detailed outline
accordingtothespecificneedsofahighereducationinstitution(e.g.firstsummarisinginformation,
thandescriptionofparticularissuesanddetailedadditionalinformation).Anexample:
a. Executivesummary
b. Teaching,learningandfurthereducation
c. Researchandyoungscientists
d. Cooperationandknowledgetransfer
e. Qualityenhancementinteaching,researchandservices
f. Highereducationstrategicandfinancialplanning
14.Developingastandardisedsetofindicators(consideringregionalcompulsoryindicatorsystems)
a. Guaranteeingcomparability
b. Relevance-basedselection
c. Possibilityforindividualhighereducationinstitution-basedinterpretations
d. Describinghighereducationperformancesaccordingtotheirrespectivedimensions,e.g.
i. Research,teaching,services
ii. Monetaryvs.non-monetaryindicators
iii. Finances,processes,potentials,compatibility
Further Reading
The Commission on Institutions of Higher Education (CIHE) in New England has defined different
reportingguidelines:
CommisiononInstitutionsofHigherEducationNewEngland(CIHE).Reporting Guidelines.Retrieved
onJanuary25,2015,fromhttps://cihe.neasc.org/institutional-reports-resources/reporting-guide-
lines
Chapter 4: Elaborated Information Systems – Examples for Data Sharing
52
4 ElaboratedInformationSystems– ExamplesforDataSharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.1 CaseStudyoftheETHZurich:AnnualAcademicAchievementsReporting . . . . 53
4.2 CaseStudyoftheUniversityofVienna:CourseControlling . . . . . . . . . . . . . . . . 55
4.3 Unidata–FactsandFiguresatthePushofaButton–ACaseStudyfromAustria 56
differentiateapproachesofusingdataathighereducationinstitutionsappropriately,
deduceappropriateareasandmechanismstostartwithwhendevelopinginformationmanagement
systemsatyourownhighereducationinstitution.
On successful completion of this chapter, you should be able to…
Chapter 4
Elaborated Information Systems – Examples for Data Sharing
Chapter 4: Elaborated Information Systems – Examples for Data Sharing
53
4 Elaborated Information Systems – Examples for Data Sharing
4.1 Case Study of the ETH Zurich: Annual Academic Achievements Reporting
ThereportingsystemoftheETHZurich,theso-called“AnnualAcademicAchievements”(AAA)isanaca-
demicreportingofprofessors,facultiesandstudyprogrammes.Thereportsincludeinformationonthe
corefieldsofteaching,researchandservices.Theobjectivesrefertothreeessentialconcerns:
1. Collectionofmanagement informationbasedondecision-relevantdataandsignificantperformance
indicatorsforthefieldsofteaching,researchandservices,whichtheseniormanagementneedstofulfil
theirtasks.
2. Thereportsserveasacademicperformancecertificatesoftheprofessors.Theycompletetheregular
facultyevaluationsandsupportthedialoguewiththeseniormanagement.
3. Reportingtoexternalthird-parties(e.g.ETH-board,ministry).
Byusingthesamedatasetsforthesethreeconcerns,theETHtriestoreducedataandinformationasym-
metries.
TheAAAreportingsystemisdesignedasanelectroniconlineportal.Itcanbeunderstoodasabigpoolthat
importsandillustratesdatafromdifferentsystems,suchasthefollowing:
Teachingdatabase(lectures,assessments,completedBA/MAtheses,completeddoctoraltheses)
SAPR/3(stockforfinancialexpendituresandactivitiesoutsidetheuniversity)
Researchdatabase(researchprojects)
E-Citations(publications)
Hermes–database(patents,licences)
Databaseoftheorganisation(internalcommissions,functions)
Chapter 4: Elaborated Information Systems – Examples for Data Sharing
54
Figure 7 Import and illustration of data from different data base systems at the ETH Zurich (ETH Zürich 2013)
TheinformationgatheredfromthesedatabasesistransferredautomaticallyintotheAAAportal,i.e.,they
donothavetobedouble-entered.Furthermore,insomefields,datathatwasenteredinapreviousyearis
transferredtothefollowingyearaswell.Thismeansusersonlyhavetoaddchangesmanually,ifapplicable.
Besidescollectingquantitativedata,usershavethepossibilityofaddingadditionalqualitativereportsthat
describetheiractivitiesinmoredetail(e.g.selectedpresentations,organisationofaconferenceetc.).
TheAAAportal isonly accessible from the internal ETHnetwork.Onlyheadsofunits that are subject to
reportshaveaccess.They,inturn,havethepossibilitytodelegatetheiraccessrightstofurtherstaffbyselect-
ingtheindividualnecessaryaccessrightsfromtheportal-menu.
Chapter 4: Elaborated Information Systems – Examples for Data Sharing
55
4.2 Case Study of the University of Vienna: Course Controlling
AttheUniversityofViennateachingplanningisorganisedbyfocussingonthe(required)teachingload.The
teachingloadisdifferentiatedintothecategoriesinternalteaching,externalteachingandnon-paidteaching
andtutorials.Asquantitativenumericalvaluetheyexaminetheweightedteachingload.Dependingonthe
groupofpersonsandtherespectivepublicserviceslaw,theyusedifferentwages-codes.Theweightingfactors
helptoreduceresultingbias.
Thereportforteachingplanningisdesignedbythedepartmentoffinancesandmanagerialaccounting.They
providefourdifferenttypesofreportsfordifferentpurposes:
1. Overviewonactualteaching-performancewithcomparisonoftheprecedingyear(general/detailedsur-
vey),
2. Overviewonplan-actual-teaching-performanceattheendofthestudyyear,
3. Overviewonplan-actual-teaching-performanceduringastudyyear(general/detailedsurvey),
4. Overviewonactual-teaching-performanceonfacultylevel(includingteachingimport;general/detailed
overview).
Thefirstreportdeliversageneralsurveyofthedistributionofteachingload,differentiatedintodifferentstaff
categoriesduringastudyyear,includingacomparisontothepreviousstudyyear.
Thethirdreportdeliversanoverviewontheachievedteaching-performancelevelatadefineddateduringa
studyyear.Ithelpstoevaluatewhethertheteachingloadhasbeen/willbeaccomplished.
Thefourthreportisespeciallyusedforthesemesterplanningofafaculty,focussingonthedistributionand
theaccomplishmentofteachingload.
Thesecondreportfocusesontheplan-actual-comparisonofteaching-performanceattheendofastudyyear
andisusedasabasisfortarget-performance-agreementsinteachingbetweenseniormanagementandfac-
ulties.Therefore,theteachingload(inhours)isdefinedforthedifferentteachingcategories,asmentioned
intheillustrationbelow(internal/externalteaching,non-paidteaching,andtutorials).Inthefollowing,the
departmentoffinancesandmanagerialaccountingmatchtheagreedteachingloadresultsfortheteaching
categoriestotheteachingstaffavailable(professors,associateprofessors,academicassociates,tutorsetc.),
bygatheringthisinformationintheplan-actual-reports.Toachieveplanningvaluesfortheteachingloadthat
considerstheactualconditionsofcoordinatingthestudyprogrammes,theuniversityhasaninternalsetof
criteriawhichenablesrequiredshiftsbetweentheplannedvaluesofthedifferentteachingcategories(e.g.
whenlecturesofaprofessorhavetobecancelledduetoaresearchsemester).Suchshiftingproceduresare
usuallyalreadydiscussedbetweenthestudyprogrammecoordinatorandthedepartmentoffinancesand
managerialaccountingbeforehavingthetarget-performancetalksinordertochecktheshiftingpossibilities.
Chapter 4: Elaborated Information Systems – Examples for Data Sharing
56
.
Table 4 Report for teaching planning at the University of Vienna (adapted from the course controlling of the University of Vienna)
4.3 Unidata – Facts and Figures at the Push of a Button – A Case Study from Austria
Unidata isthestatisticalhighereducationinformationsystemoftheFederalMinistryofScience,Research
andEconomy(MSRE)inAustria.Themainpurposeofthisreportingsystemistoproviderecentdataandfacts
abouttheAustrianhighereducationsystem.Unidataisaninternetportalthataddressesstudents,research-
ers, education experts, employers, and especially higher educationmanagers and decision-makers of the
MSRE.11
Dependingontheaccessrights,theportalgivescontinuousaccesstostatisticalinformationinfieldssuchas
budget,students,graduates,staffandfacilitymanagement,aswellasindicatorsforteachingandresearchof
universitiesanduniversitiesforappliedsciences.Furthermore,Unidatacomprisesacentralcollectionofpub-
licationsoftheMSREandhighereducationreporting.Thestatisticaldatacanberetrievedasdynamicstand-
ardreports,includingthepossibilitytoreducethemondetailedparameters.
11 MoreinformationonUnidatacanbefoundontheirwebsite:https://oravm13.noc-science.at/apex/f?p=103:36:0::NO
Chapter 4: Elaborated Information Systems – Examples for Data Sharing
57
Thisdatapool includesaquantitativedocumentationofallhighereducationperformancesinthefieldsof
teaching,researchandservices.Therefore,ithelpstoprovidetransparentpossibilitiesofcomparinguniver-
sitiesordifferentdisciplinesinAustriatomonitorhighereducationtargetfields(e.g.gender,Bolognamon-
itoring).Furthermore,Unidata isavalidfundamentforevidence-baseddecision-makingprocessesandfor
deducingmanagementinformationandprogramme-initiativestoberealisedinthehighereducationarea.
Thepurposesofunidatarefertothefollowingkeyaspects(seeunidatawebsite)
FactsandfiguresabouttheAustrianhighereducationsector,
permanentaccessforrecentquantitativedataandqualitativeanalyses,
collectionofrelevantreportsandpublications,
freeinformationplatformforallinterestedusers,
decision-makinginstrumentfortarget-performance-agreements,aswellasmonitoringofquantitative
aspectsforperformance-agreementsandotherhighereducationtargetfields(BolognaProcess,gender
monitoringetc.),
implementationofaworkingplatformformutualdata-clearingbetweenhighereducationinstitutions
andtheministry.
Unidata,anditscentralisationoftheindividualinformationsystemsofAustrianhighereducationinstitutions
andtheministryhasinitiatedanddevelopedprocessesthatareessentialinhelpingincreasedataqualityin
highereducationstatistics.Forexample,datasourcesoftheministryandthehighereducationinstitutions
arenowsynchronisedviaanelectronicplatform.Before,thisprocessofdata-synchronisationwasregulated
bylaw.Thegainedstandardiseddatasetsshallcontributetoachievingmoreliabilityandreduceoutput-asym-
metriesbetweenhighereducationinstitutionsandtheministry.
Questions & Assignments
1.Pleasenameanddescribeaprocessinthefieldofteaching,researchorservicesatyourHEIthat
hasasystemisedinformationprocessing.Whichinformationiscollected,whatfor,bywhomand
inwhatperiod?Arethereanyinformationgapsthatyoucanobserveinthisinformationprocess?
Ifso,whatareyoudoingatyourHEItoclosesuchgaps?Areyou,asqualitymanager,involvedin
theseprocesses?
References
Alter, S. (1996). Information systems. A management perspective (2nd edition). Menlo Park: Benjamin
Cummings Pub. Co.
Axson, D.A. (2007). Best practices in planning and performance management (2nd edition). Hoboken, NJ: John
Wiley & Sons.
Balasubramanian, K. (2009). ICTs for higher education. Background paper from the commonwealth of learning.
Paris: UNESCO; World Conference on Higher Education; Commonwealth of Learning.
Babbie, E. R. (2004). The practice of social research (10th edition). Southbank: Wadsworth.
Barrie, S. & Ginns, P. (2007). The linking of national teaching performance indicators to improvements in
teaching and learning in classrooms. Quality in Higher Education, 13(3), 205-286.
Beitz, A., Dharmawardena, K. & Sam Searle, S. (2012). Monash university research data management strategy
and strategic plan 2012 – 2015. Melbourne: Monash University.
Blohm, H. (1974). Organisation, Verwaltung und Arbeitswissenschaft. Die Gestaltung des betrieblichen
Berichtswesens als Problem der Leitungsorganisation (2nd revised edition). Herne: Verlag Neue
Wirtschafts-Briefe.
Boston University (2015). What is “research data”? Retrieved from http://www.bu.edu/datamanagement/
background/whatisdata/
Bundesministerium für Wissenschaft, Forschung und Wirtschaft Österreich (Ed.) (2015). Datawarehouse
Hochschulbereich. Unidata - Zahlen und Fakten auf Knopfdruck. Retrieved from https://oravm13.
noc-science.at/apex/f?p=103:36:::NO:::
Chalmers, D. (2008). Teaching and learning quality indicators in australian universities. Outcomes of higher
education: Quality relevance and impact. Paris: Programme on Institutional Management in Higher
Education.
Commision on Institutions of Higher Education New England (CIHE) (2015). Reporting guidelines. Retrieved
from https://cihe.neasc.org/institutional-reports-resources/reporting-guidelines
Data Warehousing and Business Intelligence Organization (2014). Classifying data for successful modeling.
Retrieved from http://dwbi.org/data-modelling/dimensional-model/16-classifying-data-for-
successful-modeling
Demski, J. (1980). Information analysis (2nd edition). New York: Addison-Wesley.
Demski, J. (2008). Managerial uses of accounting information (2nd edition). New York: Springer-Verlag.
ETH Zürich (2013). ETHIS-Schulung Annual Academic Achievements (AAA). Akademische Berichterstattung
der Professuren, Departemente und Studiengänge. Retrieved from https://www1.ethz.ch/sap/
applications/aaa/Folien
European University Information Systems (EUNIS). Retrieved from http://www.eunis.org/
Frese, E. (Ed.) (1992). Handwörterbuch der Organisation (3rd revised edition). Band 2 der Enzyklopädie der
Betriebswirtschaftslehre. Stuttgart: Schäffer-Poeschel Verlag.
Gladen, W. (2003). Kennzahlen- und Berichtssysteme. Grundlagen zum Performance Measurement (2nd
revised edition). Wiesbaden: Gabler.
References
58
References
Gladen, W. (2011). Performance Measurement. Controlling mit Kennzahlen (5th revised edition). Wiesbaden:
Gabler Verlag / Springer Fachmedien Wiesbaden GmbH.
Gleich, R., Horváth, P. & Michel, U. (Ed.) (2008). Management Reporting: Grundlagen, Praxis und Perspektiven.
München: Rudolf Haufe Verlag.
Göpfert, I. (2007). Berichtswesen. In Küpper, H.-U. & Wagenhofer, A. (Ed.), EdBWL. Handwörterbuch
Unternehmensrechnung und Controlling. Stuttgart: Schäffer-Poeschel Verlag, 143–156.
Grochla, E. & Thom, N. (Ed.) (1980). Handwörterbuch der Organisation (2nd revised edition). Band 2 der
Enzyklopädie der Betriebswirtschaftslehre. Stuttgart: Schäffer-Poeschel Verlag.
Harvey, L. (2004-14). Analytic quality glossary. Resarch quality international. Retrieved from http://www.
qualityresearchinternational.com/glossary/performanceindicators.htm
Higher Education Funding Council for England (HEFCE) (2011). Performance indicators in higher education.
First report of the performance indicators steering group (PISG). London: HEFCE.
Horváth, P. (2008). Grundlagen des Management Reportings. In Gleich, R., Horváth, P. & Michel, U. (Ed.),
Management Reporting: Grundlagen, Praxis und Perspektiven. München: Rudolf Haufe Verlag, 15–
42.
Hórvath, P. (2011). Controlling (12th edition). München: Vahlen.
Horváth, P. & Michel, U. (Ed.) (2013). Controlling integriert und global. Erfolgreiche Steuerung von komplexen
Organisationen. Stuttgart: Schäffer-Poeschel Verlag.
Kaplan, R.S. (2011). Strategic performance measurement and management in Nonprofit Organizations.
Nonprofit Management & Leadership, 11(3), 353–370.
Kaplan, R.S. & Norton, D.P. (1993). Putting the balanced scorecard to work. Harvard Business Review, 71(5),
134-147.
Kaplan, R.S. & Norton, D.P. (1996). Using the balanced scorecard as a strategic management system. Harvard
Business Review, 74(1), 75-85.
Kimball, R. & Ross, M. (2002). The data warehouse toolkit. The complete guide to dimensional modeling (2nd
edition). New York: Wiley.
Koch, R. (1994). Betriebliches Berichtswesen als Informations- und Steuerungsinstrument. Frankfurt a.M.
[u.a.]: Verlag Peter Lang.
Koreimann, D.S. (1976): Methoden der Informationsbedarfsanalyse. Berlin, New York: De Gruyter.
Küpper, H.-U. (1997). Das Führungssystem als Ansatzpunkt für eine wettbewerbsorientierte Strukturreform
von Universitäten. Beiträge zur Hochschulforschung, 19(2), 123–149.
Küpper, H.-U. (2001). Rechnungslegung von Hochschulen. Betriebswirtschaftliche Forschung und Praxis, 53(6),
578–592.
Küpper, H.-U., Friedl, G., Hofmann, C., Hofmann, Y. & Pedell, B. (2013): Controlling. Konzeption, Aufgaben,
Instrumente (6th edition). Stuttgart: Schäffer-Poeschel Verlag.
Noé, M. (2010). Vom Qualitätsmanager zum internen Managementberater. Die neuen Anforderungen
souverän meistern. München: Hanser.
59
References
Nusselein, M.A. (2002). Empirische Erkenntnisse einer Informationsbedarfsanalyse an bayerischen
Hochschulen. Beiträge zur Hochschulforschung, 24(1), 100-114.
Nusselein, M.A. (2003). Inhaltliche Gestaltung eines Data Warehouse-Systems am Beispiel einer Hochschule.
München: Bayerisches Staatsinstitut für Hochschulforschung und Hochschulplanung.
Organisation for Economic Co-operation and Development (2003). Education policy analysis - 2003 edition.
Paris: OECD.
Röbken, H. (2003). Balanced Scorecard als Instrument der Hochschulentwicklung. Projektergebnisse an der
Reykjavik University. Beiträge zur Hochschulforschung, 25(1), 102-120.
Saupe, J.L. (1981). The functions of institutional research. Tallahassee: Association for Institutional Research.
Schenker-Wicki, A. (1999). Moderne Prüfverfahren für komplexe Probleme. Evaluation und Performance-
Audits im Vergleich. Wiesbaden: Deutscher Universitätsverlag/Springer Fachmedien Wiesbaden.
Scheytt, T. (2007). Strategieorientiertes Performance Management in Hochschulen: Das Konzept der Balanced
Scorecard. Hochschulmanagement, 2007(1), 15–21.
Taylor, J. (2014). Informing or distracting? Guiding or driving? The use of performance indicators in higher
education. In Menon, M., Terkla, D., Gibbs, P. (Ed.), Using data to improve higher education. Research,
policy and practice. Rotterdam: Sense Publishers.
Tropp, G. (2002). Kennzahlensysteme des Hochschul-Controlling. Fundierung, Systematisierung, Anwendung.
München: Bayerisches Staatsinst. für Hochschulforschung u. Hochschulplanung.
Varghese, N.V. (2004): Incentives and institutional changes in higher education. Higher Education Management
and Policy, 16(4), 27–39.
Volkwein, J.F. (1999). What is institutional research all about? A critical and comprehensive assessment of the
profession. San Francisco: Jossey-Bass ( J-B IR Single Issue Institutional Research, Book 41).
Weilenmann, G. & Scheitlin, V. (1972): Informationstechnik für Führungskräfte. Moderne Information u.
Kommunikation in d. Unternehmung. Stuttgart: Schäffer-Poeschel Verlag.
Wittmann, W. (1980). Information. In Grochla, E. & Thom, N. (Ed.), Handwörterbuch der Organisation (2nd
revised edition). Band 2 der Enzyklopädie der Betriebswirtschaftslehre. Stuttgart: Schäffer-Poeschel
Verlag.
Yorck, H., Güttner, A., & Müller, U. (2010). Berichterstattung für Politik und Staat von Hochschulen im Land
Sachsen-Anhalt. Wittenberg: Wissenschaftszentrum Sachsen-Anhalt.
60
61
List of Tables
Table 1 Information sources and requirements from different stakeholders (adapted from Nusselein 2002). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Table 2 Criteria of success for data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Table 3 Challenges of (performance) indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
Table 4 Report for teaching planning at the University of Vienna (adapted from the course controlling of the University of Vienna) . . . . . . . . . 56
List of Tables
62
List of Figures
Figure 1 Layer model for higher education institutions (Tropp 2002, 2) . . . . . . . . . . . 15
Figure 2 Gathering information based on needs, supply and demand (translated based on Picot & Frank 1988, 608 in Hórvath 2011, 311) . . . . . . . 18
Figure 3 Based on the project “Computer-based management tool for the institutions of higher education in Bavaria” (CEUS) (Nusselein 2002, 4) . . . . . . . . . . . . . . . 21
Figure 4 Types of interferences during the process of information distribution (adapted from Küpper et al. 2013, 241) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Figure 5 System of a Balanced Scorecard (adapted from Scheytt 2007) . . . . . . . . . . . . 38
Figure 6 Criteria to design reports (translated illustration adapted from Tropp 2002, 70) . . . . . . . . . . . . . . . . . . . 46
Figure 7 Import and illustration of data from different data base systems at the ETH Zurich (ETH Zürich 2013) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
List of Figures
63
With financial support from the Supported by