Evaluation - what is needed for the Universities

Lígia Maria RibeiroUniversity of Porto

lmr@reit.up.pt

Evaluation of Research using a CRISAnnual CRIS Seminars Series

13-14 September 2010 Brussels

• What is EUNIS?• EUNIS main activities• Using CRIS at U.PORTO• EUNIS Research Workshop• Universities requirements• Conclusions

Agenda

• European University Information Systems– www.eunis.org

What’s EUNIS?

• IS in European HE&R institutions– communication– collaboration – co-operation

• To liaise with major suppliers• To liaise with organizations setting the strategic agenda • From its beginning, EUNIS often acts as:

– a European complement to national organizations– an opening of minds beyond the national frontier

• Foster the development, the quality and innovation of IS for HE&R in Europe.

EUNIS Objectives

• Members– ≈150

• Over 130 HE&R institutions and about 10 national consortiums

– 29 countries • Linked Organizations

– TERENA– EDUCAUSE & ECAR– EAIE (European Association for

International Education)– EUA (Presidents)– CAUDIT (Australia)– ASAUDIT (South Africa)

EUNIS today

EUNIS annual conferences1995 Düsseldorf (Germany)

1996 Manchester (UK)

1997 Grenoble (France)

1998 Prague (Czech Rep.)

1999 Helsinki (Finland)

2000 Poznań (Poland)

2001 Berlin (Germany)

2002 Porto (Portugal)

2003 Amsterdam (Netherlands)

2004 Ljubljana (Slovenia)

2005 Manchester (UK)

2006 Tartu (Estonia)

2007 Grenoble (France)

2008 Aarhus (Denmark)

2009 Santiago de Compostela (Spain)

2010 Warsaw (Poland)

NEXT >

www.eunis.ie

• Workshops– ELFT (eLearning Task Force) – RS3G (Rome Student Systems and Standards Group)

• EAIE Digital Student Data Portability (DSDP) Task Force – MIS Task Force

• Rectors seminars • University Information Systems Strategy and Policy

– Paris, November 20-21, 2003• Impact of the Bologna Process on Information

Technology in European Universities – Paris, March 23rd-24th 2006

– Discuss strategic subjects in relation with the usage of ICT in HE&R

EUNIS other events

http://www.eunisresearch2010.it/

Future rector seminar ?

http://en.wikipedia.org/wiki/File:John_Henry_Newman_-_Project_Gutenberg_13103.jpg

Universities’ Mission

“ Knowledge is capable of being its own

end”

(J. H. Newman)

“Knowledge drives productivity growth. In advanced economies such as the EU, knowledge, meaning R&D, innovation and education, is a key driver of productivity growth. Knowledge is a critical factor with which Europe can ensure competitiveness in a global world where others compete with cheap labor or primary resources.”

http://eur-lex.europa.eu/smartapi/cgi/sga_doc?smartapi!celexplus!prod!DocNumber&lg=en&type_doc=COMfinal&an_doc=2005&nu_doc=24; retrieved 28 August 2010

A new start for the Lisbon Strategy (2005)

• Growing concern for:– Transparency– Accountability– Comparability– Performance indicators – Assessment

• University rankings– simple indicator-based systems are attractive

• reduced costs; effortless of reading

– more appropriate and sophisticate systems are needed• quantitative & qualitative indicators; multidimensional approach

R&D+I

• Several assessment exercises are in place– International assessments– National assessment

• International panels

– Self-assessment

– Institution – Study-programs– Research– Staff

Assessment Exercises

• Collect and organize data– Substantial financial and HR resources involved

• Need to reduce the overhead• IT helps!

– Information Systems• Institution-wide (integrated view)

– Teaching and Learning– R&D– Administration– Management

• Easily relate the information needed• Produce semi-automatic reports• Motivate the users to populate the IS

The role of IT

• State University created the 22nd March 1911• Largest in Portugal• 14 Faculties, 1 Business School and 69 R&D units• ≈31 000 Students / 2 300 Professors and Researchers / 1700

Staff

University of Portowww.up.pt

SIGARRA

Front-Office (CRIS +)

GA

Student Management

Primavera

Financial Management

Aleph

Library Management

GRH

Human Res. Management

Moodle

Learning Management

DSpace

Content Management

Radius

LDAP

Identity Management

Oracle DBMS WF engine App. Server

Unidirectional info flow

Bidirectional info flow

CORDIS

SIGARRA :: Full Text and access levels

keywords: CORDIS

SIGARRA :: Open Access Repository

SIGARRA

Full textOpen access

SIGARRA :: Thesis orientation

Open Access Policy :: Authors• Registration of all

U.PORTO scholarly research in SIGARRA.

• Include full text for each publication (PDF).

• Select open access whenever possible.

• Try “SPARC Author Adendum”

http://www.arl.org/sparc/author/addendum.shtml

Open Access Policy :: Academic Services• Registration in SIGARRA of

all Master and PhD Thesis (full text, PDF).

• Select open access whenever possible.

Open Access Policy :: Libraries

• Guarantee meta-data quality for each publication registered in SIGARRA.

U.Porto Internal assessment

• Self-assessment– Institution– Study-programs (+ external accreditation)

• Annual assessment of professors– implications on career progression

• Quantitative indicators– Automatic from SIGARRA

• Qualitative information – Filled in SIGARRA

SIGARRA

• Institution-wide integrated IS: not so many– connection of IS with IR is very weak

• CERIF compliance: weak (but increasing...)

EUNIS Research Workshop

IR CRIS

very weak

• Which are the main research data sharing needs that should be addressed at European level?– research products, citations, impact factors, research

groups, instruments, project related data – Skills, metrics

• Which are today the main barriers to the satisfaction of these needs?– Technological, cultural, political, organizational…– Cultural, organizational

Bologna Workshop Panel [1/2]

• Can we identify a roadmap to meet research data sharing and exploitation requirements?– Should the solution be centralized or distributed– Distributed, standards (CERIF-XML)

• What should be done to address the first step?– Survey, pilot, interoperability working group…– Survey, working group, pilot

Bologna Workshop Panel [2/2]

• To reduce the overhead for assessment exercises– Improve information management across the

institution– Integrated institutional-wide information systems– Support assessment exercises on IS/CRIS

• To provide valid information on HE across national borders– Set of world-wide comparable indicators– Ensure interoperability• IS/CRIS and IR• CERIF (EU recommendation)

Universities requirements

• Research evaluation processes are complex and sophisticated transparency tools are needed

• A distributed, open and interoperable European system is a goal to achieve– cooperation between organizations working with

HE & R institutions in Europe– awareness of stakeholders

Conclusions