Experience of self evaluation as part of a long-term change process : the UA as a case study from...
-
Upload
margaret-pope -
Category
Documents
-
view
212 -
download
0
Transcript of Experience of self evaluation as part of a long-term change process : the UA as a case study from...
Experience of self evaluation as part of a long-term change process : the UA as a
case study from Portugal
J.A. Rafael and M. Assunção , 10-11 July 2008
Conference on self-evaluation of HE institutions, Belgrade, Serbia
Contents
• About the University of Aveiro
• profile
• organisation
• QA internal implementation
• Setting the scene at European and National level
• Legal changes
• Opportunity to re-think QA in UA
The U. Aveiro incorporates University Departments and
Polytechnic Schools (since 1997). Four years ago a
programme aimed at post-secondary education was started
and is now run by the Polytechnic Schools.
17 University departments (Aveiro)
35 graduation degrees
4 Polytechnic Schools (3 cities)
15 graduation degrees
16 CETs (11 cities)
The University of Aveiro as a Regional Network for Education and Training
AGUEDA
Ol. DE AZEMÉIS
AVEIRO
AGUEDA
Ol. DE AZEMÉIS
AVEIRO
engineering
› Electronics and Telecommunications
› Environment and Planning
› Ceramics and Glass Engineering
› Mechanical Engineering
› Civil Engineering
education
› Sciences of Education
› Didactics and Educational Technology
science› Chemistry› Biology› Physics› Mathematics› Earth Sciences
arts and social sciences
› Languages and Cultures
› Media and Artistic Studies
› Social and Political Sciences
› Economics, Management and Industrial Engineering
University Departments
health Sciences
Functional Organisation & Management
Rector Senate
ScientificCouncil
PedagogicCouncil
ResearchInstitute
Undergraduate UniversityEducation Institute
Postgraduate UniversityEducation Institute
PolytechnicEducation Institute
Departments (17)Polytechnic
School 1Polytechnic
School 4…
Study Programme 1 Study Programme 2 Study Programme n… …
[Degree Director] [Degree Director] [Degree Director]
●
●
●
●
●
●Dept. 1
Dept. 2
…
Dept. m
…
Resources
Study Programme Matrix Organisation
Research Unit1 Research Unit 2 Research Unit n… …
[Unit Coordinator] [Unit Coordinator] [Unit Coordinator]
●
●
●
●
●
●
Dept. 1
Dept. 2
…
Dept. m
…
Resources
Research Units Matrix Organisation
Promotion of a Quality Culture: the beginning (1)
• 1992, Portuguese Rectors Conference hosts a seminar on Quality Systems (GB, F, NL) for HE
• 1995 CRE institutional quality audit
• 1997, Vice-Rector for Quality
• 1999 Follow-up of CRE audit
• Initiatives to create awareness of the quality issue
– Atenção UA!
– Hosting a CRE/IMHE seminar - UA was presented as a case study
– Repensar os Curricula (“Bologna before Bologna”)
Promotion of a Quality Culture: the beginning (2)
• Decision to develop a single (integrated) information system
• Creation of specific bodies responsible for quality development
– GAGI (Information Management Office)
– GAQAP (Quality, Evaluation and Procedures Office)
• Redefinition of the university structure to include institutes for
coordination of function
Internal Processes (1)
• Transversal instruments– LEGUA (Legislation, Regulation and Procedures
Knowledge Base)• Quality Protocol
– Single Information System• Supporting and monitoring all the university activities in an
integrated way
Internal Processes (2)
• Teaching activities
– Managed through the Education Institutes• Centrally• Locally through the figure of the Study Programme Director
– Transparency of the global information system enabling that teaching and learning activities have their results available to all relevant actors
– Indicators referring access numbers, drop-out rates, time to graduation and student performance are produced on a semester basis; employment data and feedback from employers - initially on a five year cycle, now on an annual basis
– Since 1994, on a five-year cycle basis, the UA study programmes have been evaluated in accordance with a national framework
Study Programme Evaluation (at national level)
• Promoted by FUP (Portuguese University Foundation) and, afterwards, CNAVES
• Comprises– Self-evaluation (SE)
• SE Committee appointed by the Head of the Department responsible for the Study Programme
• Based on standard evaluation guidelines (include questionnaires to teachers and students)
– External evaluation• Evaluation team appointed by FUP, constituted by experts of other
Portuguese universities• Site visit
– To analyse and discuss the SE report with SE Committee, Teachers and Students
• Evaluation report with recommendations– Recommendations of the evaluation report were discussed and totally or
partially implemented
Internal Processes (3)
• Research activities
– Research Institute coordinates the activities of research units and encourages quality practices through support for high quality research
– At Research Unit/Associated Labs level there exist internal mechanisms which regulate production of research, promoting quality through internal funding rules
– Research activities are subject to an external quality processes in the sense that they are only funded if they meet the requirements of the funding institution (e.g. FCT, EU, …)
– The Research Units are also subject of an external evaluation programme by FCT, carried out by international panels, on a three-year basis
– Academic career itself foments quality research by valuing some types of publications over others and making promotion more dependent on research rather than teaching or other activities
Research Evaluation (at national level)
• Promoted by FCT (Science and Technology Foundation)
• Comprises– Activity report
• Written by the Coordinator of the Research Unit• Based on standard evaluation form issued by FCT
– External evaluation• Evaluation team designated by FCT, constituted by
international experts• Site visit
– To analyse and discuss the Activity Report with the members of the Research Unit
• Evaluation report and a classification of the Research Unit (from Poor to Excellent) with direct impact on Unit funding
• Administrative processes– Definition and management of these quality processes are the
responsibility of the appropriate Head of Service
– Mechanisms which monitor and regulate administrative processes are enforced by an internal network of quality-pivots coordinated centrally (by GAQAP)
– They are disseminated by internet through LEGUA, and/or incorporated into informatics applications that support actual activities
– Annually GAQAP gives a general questionnaire to all staff and a specific one to Heads of Units for identifying staff training needs
– Implementation of a system for evaluation of administrative and technical staff within the scope of the national SIADAP (Integrated System for Evaluation in Public Administration)
Internal Processes (4)
Setting the scene at European level (1)
• The initiative to introduce the European Higher Education Area was
launched in 1998, in Paris, and confirmed, one year later, in Bologna
where ministers from 29 countries signed a declaration to the purpose
of creating a borderless EHEA by 2010.
• The central idea of the Bologna process is to promote student and
staff cross-border mobility through:
– the adoption of a system of comparable two cycle degrees
(Bachelor and Master) and,
– the introduction of a harmonized credit system ECTS (European
Credits Transfer System).
Setting the scene at European level (2)
• Quality has always played a role in Higher Education but its importance
increased enormously during the last decade.
• Among the new challenges facing Higher Education Institutions are of
paramount importance the combined requirements of promoting the
Bologna process and participating in the European Higher Education
Area and of creating the European knowledge society.
• In both cases quality is seen as a key factor to achieve these
objectives.
• Why Quality?
– To guarantee the necessary trust between institutions, which is
essential to make sure they will accept students who have acquired
a certain number of credits in another institution, particularly in
another country, the quality of institutions and its audit, as well as
the recognition of degrees, have gained a central role in the whole
process
Setting the scene at European level (3)
• However
– The emergence of quality as a key factor to the successful
implementation of the EHEA is quite recent.
– In 2003 it became the first policy objective of the Berlin
Communiqué (September 2003). There, it is stated that quality of
higher education is at the heart of the setting up of a European
Higher Education Area and consequently it is essential the
development of quality assurance at institutional, national and
European level
Setting the scene at European level (4)
• Henceforth the need to develop mutually shared criteria and
methodologies on quality assurance.
• It is also stressed that the primary responsibility for quality assurance in
higher education lies with each institution itself.
• It also invited the QA community, together with the Higher Education
Institutions, to develop an agreed set of standards, procedures and
guidelines on quality assurance and to explore the possibility of a peer-
review of QA agencies
Setting the scene at European level (5)
• In 2005 the E4 group (ENQA, EUA, ESIB and EURASHE) proposed to
create an European Register for QA agencies.
• In April 2007, at the 4th EUA Convention, a recommendation was issued
emphasizing two points:
– quality should be all embracing and quality assessment must also
recognize diversity and encourage, rather than constrain, innovation
and creativity in teaching and learning.
– the European Register of QA agencies, expected to be approved at the
2007 Ministers meeting, has been developed by an innovative
partnership between stakeholders and is seen as an important block in
this area, and an essential component of the global attractiveness of
European Higher Education.
Setting the scene at European level (6)
• The Portuguese Higher Education System has profoundly changed in the last thirty five years: the Public University Sector was enlarged and a private higher education sector emerged and expanded very rapidly. An important feature is the existence of a binary divide between university-level and polytechnic-level programmes.
• The system now comprises:
– University Public Sector• 14 universities and one non-integrated university institute
• 4 Military University Institutions
– Polytechnic Public Sector• 15 Institutes
• 21 non-integrated colleges which include nursing, education, military and police
Setting the scene at National level (1)
• The private sector is much larger in terms of number of institutions, with more than 100, but accounting for less than 30% of the students, revealing the fragmentation of the system.
• It is also noteworthy that all the public universities are comprehensive in terms of fields of study and they are, by large, the main actors of research in Portugal, carried primarily by the academic staff.
Setting the scene at National level (2)
• In the current context of stabilization in the number of enrolments and severe shortage of public funds for higher education, the system and the institutions face considerable constraints.
• This setting presents great challenges for the regulation and steering of the system, requiring clear policies on the overall mission and goals to attain, supported by transparent and stable funding mechanisms and by a regular and consistent evaluation framework.
• Failing to do this will not only hinder institutions’ efforts to adopt a strategic approach, but will also fail to provide the adequate information for students, parents and other stakeholders, an issue of paramount importance for the enrolment decision process and for the relationships between institutions and the society.
Setting the scene at National level (3)
• Most public universities have sought international expertise to carry out a institutional evaluation.
This was the case, in particular, of the Institutional Evaluation Programme run by the European
University Association (EUA), formerly the Council of European Rectors (CRE).
• During the nineties most public universities, Aveiro, Porto, Minho, Technical University of Lisbon,
University of Lisbon, Universidade Nova de Lisboa, University of Algarve and Universidade
Católica, have invited EUA to carry out a evaluation of the institution. Some of these institutions
have also done one follow-up.
• These exercise was self-contained within the institution evaluated, so the benefits of the exercise
were not extended to the different type of institutions and a systemic perspective could not be
obtained.
Evaluation at National level: the institutions
Evaluation at National level: the system
• The Higher Education sector in Portugal is supporting the Glasgow declaration: Strong Universities for a strong Europe. Europe needs strong universities as motors in the knowledge triangle of education, research and innovation.
• At national level it is accepted that Portugal needs universities/Higher Education Institutions that are able to move out of the shadows of nation state and are able to do both: go for excellence in research as well as in teaching and provide broad, equitable access to basic higher education in Europe.
• In 2005 the government launched their programme which included as a priority the integration the Higher Education System in the European Area. In this context it was emphasised the need to carry out an all embracing evaluation of the system, according to the best practices in Europe and elsewhere.
• Background: Portuguese government promoted discussion on three
levels:– HE system level (OECD);
– setting-up of a national evaluation system (ENQA);
– Institutional level (EUA)
– Ministry (of CTES) supports programme for institutional evaluation
New developments and new legal framework (1)
• During 2006 ENQA reviewed the accreditation and quality assurance practices in the Portuguese higher education against the best European practices and following the Guidelines for Quality Assurance in the European Higher Education Area and issued recommendations that are now being implemented at national level.
• These recommendations emphasised the European trend of a distinction between accreditation at programme level and academic audit and evaluation at institutional level
New developments and new legal framework (2)
• The Portuguese Quality Assurance Agency will adopt the following
definition of accreditation:
– The characteristic of accreditation is that the process is based on
established external criteria and that the result of the process is a
decision as to whether these criteria have in fact been met by the
programme under accreditation.
New developments and new legal framework (3)
• The government decided to encourage ALL the Portuguese Higher Education
Institutions, universities and polytechnic public and private to apply for an Institutional
Evaluation according to the EUA International Evaluation Programme and set up the
following focus for such an evaluation:
– The main characteristics, positive and negative of the Portuguese Higher
Education institutions, their stage of development in the European context and
their efficiency.
– The level of institutional autonomy and the practices related to governance and
management. The report should make suggestions for improvement.
New developments and new legal framework (4)
– The institutional capacity for change and the way institutions respond to the
increasing challenges facing higher education.
– The access and the way students are recruited. Special attention should be paid to
LLL and the opening of the institution to the new publics and professional continuous
education.
– The relevance of the diplomas awarded.
– The diversification of funds and the way these are managed.
– Recommendations on how to promote the rationalization of he offer and
diversification of institutions.
New developments and new legal framework (5)
• 2007– RJIES
• Law that establishes new governance models for HEI. Institutions should have new statutes until June 2008
– RJAES• Law that establishes the legal environment for the evaluation
of HE (focus on evaluation of study programmes)
New developments and new legal framework (6)
• The UA decided (2006) to apply for EUA’s institutional evaluation programme
• The UA is finalising its statutes
– Changes in university governance bodies, both at central and Department/School levels,
may/will lead to a new approach and organisation for the quality assurance mechanisms
New developments and new legal framework (7)
Organic Unit 1…..
Organic Unit n
Management
Edu
catio
n
Res
earc
h
Adm
inis
trat
ive
sup
port
CentralManagement
Operational role
Edu
catio
n
Res
earc
h
Adm
inis
trat
ive
sup
port
LocalManagement
Re-Structuring the QA Organisation
Key areas of the Action Plan
• From self-evaluation (SWOT) and contribution of external evaluation team
– Performance indicators
– Bologna Process
– Partnerships
– Funding
– Organisation and governance