Present Status of Evaluation of Medical Schools in Italian University ROME TOR VERGATA UNIVERSITY...

Present Status of Evaluation of Medical Schools Present Status of Evaluation of Medical Schools

in Italian Universityin Italian University

ROME TOR VERGATA UNIVERSITYROME TOR VERGATA UNIVERSITYITALYITALY

Prof. Stefano Elia, Department of Surgery, Italy Tempus Prof. Stefano Elia, Department of Surgery, Italy Tempus CoordinatorCoordinator

Prof. Claudio Cortese, Department of Internal MedicineProf. Claudio Cortese, Department of Internal Medicine

Prof. Antonio Volpi, Department of Internal MedicineProf. Antonio Volpi, Department of Internal Medicine

Project TEMPUS IV 159328 – 1 – 2009 – FR – TEMPUS - SMHESLife Long Learning Framework for Medical Teaching Staff

The main goals of Medical Faculty The main goals of Medical Faculty

are:are:

•To improve student’s learning To improve student’s learning

processprocess

•To ensure itTo ensure it

•To evaluate itTo evaluate it

• It is important to realize that true learning It is important to realize that true learning

process can only be managed by the student process can only be managed by the student

himselfhimself

• It occurs in his mind where teachers cannot It occurs in his mind where teachers cannot

enterenter

• Teachers can only create a stimulating Teachers can only create a stimulating

environment and support mental activities environment and support mental activities

necessary to facilitate the learning processnecessary to facilitate the learning process

• Therefore we have to switch from the Therefore we have to switch from the

historical « teacher-centered » education to historical « teacher-centered » education to

the « student-centered » educationthe « student-centered » education

Any reform of medical education has to take Any reform of medical education has to take

into consideration the simultaneous reform of:into consideration the simultaneous reform of:

•CurriculumCurriculum

•Teaching strategies for an effective learningTeaching strategies for an effective learning

•EvaluationEvaluation

CurriculumCurriculum

The Association of American Medical Colleges The Association of American Medical Colleges

(AAMC) has identified 5 main roles for the future (AAMC) has identified 5 main roles for the future

medical doctor:medical doctor:

a)a)Life-long learnerLife-long learner

b)b)ClinicianClinician

c)c)Teacher/communicatorTeacher/communicator

d)d)ResearcherResearcher

e)e)Manager Manager

Any type of reform of curriculum should take Any type of reform of curriculum should take

these roles into due consideration these roles into due consideration

Teaching strategies for an effective Teaching strategies for an effective

learninglearning

Teacher’s role is to stimulate in the student’s mind:Teacher’s role is to stimulate in the student’s mind:

•CreativityCreativity

•Problem solving abilityProblem solving ability

•Mind flexibilityMind flexibility

•Criticism skillnessCriticism skillness

•Opening of ideasOpening of ideas

•Analysys and synthesys of complex materialAnalysys and synthesys of complex material


learninglearningPresent worldwide medical teaching tends Present worldwide medical teaching tends

to emphasize scientific knowledge more to emphasize scientific knowledge more

than encouraging scientific comprehension, than encouraging scientific comprehension,

clinical reasoning, practical skills, and the clinical reasoning, practical skills, and the

development of character, empathy and development of character, empathy and

integrity integrity


learninglearning Medical teachers have to adapt themselves Medical teachers have to adapt themselves

to behavioural changes in the present to behavioural changes in the present

society focusing their action on:society focusing their action on:

• More effective practical trainingMore effective practical training

by systematically repeated clinical tasksby systematically repeated clinical tasks

• Development of new evaluation methods preferably Development of new evaluation methods preferably

focused on competencesfocused on competences

• Improvement of research standards Improvement of research standards

• Change the negative concept ofevaluation into an Change the negative concept ofevaluation into an

information step that helps students to improve information step that helps students to improve

their daily worktheir daily work

EvaluationEvaluation

• Medical teachers traditionally think that the Medical teachers traditionally think that the

evaluation procedure comes at the end of the evaluation procedure comes at the end of the

learning processlearning process

• This creates a gap between student’s learning and This creates a gap between student’s learning and

evaluation result evaluation result

• Evaluation is part of the teaching/learning processEvaluation is part of the teaching/learning process

• Teachers have to identify and communicate end-of-Teachers have to identify and communicate end-of-

course goals to the students in order for them to course goals to the students in order for them to

actively participate in their achievement actively participate in their achievement

• New evaluation models have to be used during New evaluation models have to be used during

learning process e.g. through progress testlearning process e.g. through progress test

Progress TestProgress Test

• Should be available along the whole Should be available along the whole

educational track and used to continuously educational track and used to continuously

monitor:monitor:

progression of knowledgeprogression of knowledge

implementation of knowledgeimplementation of knowledge

possible loss of previous knowledgepossible loss of previous knowledge


AdvantagesAdvantages

• It avoids the student’s approach to study It avoids the student’s approach to study

because « I need to take the exam » because « I need to take the exam »

• It gives the opportunity to identify It gives the opportunity to identify

student’s strengths and weaknesses both student’s strengths and weaknesses both

globally and in single topicsglobally and in single topics

• It evaluates long term knowledge and gives It evaluates long term knowledge and gives

confirmation of the achievement of goalsconfirmation of the achievement of goals

• It prevents wasting of time in repeating It prevents wasting of time in repeating

exams exams


GoalsGoals

• To make students identify the important To make students identify the important

conceptsconcepts

• To motivate students in studyingTo motivate students in studying

• To identify weak areas that need further To identify weak areas that need further

applicationapplication

• To identify weakness both in the course To identify weakness both in the course

and teacherand teacher

• To determine the final score ( give or deny To determine the final score ( give or deny

pass)pass)

Italian System of Education

High School

Bachelor Degree3 years

Master of Science2 years

PhD programs

One-cycle courses (5 or 6 years)Architecture/Law/Medicine

Italian University System

First Reform: 1999 Law n.509

AUTONOMY

Teaching

By-laws and rules

Financial

Organizational

TEACHING AUTONOMY

1. 1st cycle: Degree Courses(goal: to ensure appropriate empowerment of

general scientific methods and contents and acquisition of specific professional knowledge)

2. 2nd cycle: Postgraduate Courses and 1° level Master (goal: provide an advanced level of education to perform high quality activity in a specific environment)

3. 3rd cycle: a) Postgraduate Courses and 2°nd level Master

(goal: provide knowledge and capability for functions required by peculiar professional activities)

b) Ph.D. Courses (goal: acquisition of a correct research

methodology)

Medical Faculty is an exception since it is articulated in a unique cycle of 6 years

Study Classes (Law 270)

Courses of the same level that share goals and qualifying educational activities

University Education Credits (Crediti Formativi Universitari - CFU)

One credit = 25 working hours (comprehensive of lesson, individual study, lab, verification, etc.).

Teaching and research autonomy

<<art and sciences are free and free is their teaching »…., meaning that the teacher is

independent in exercising his profession as far as programmes and methods are

concerned>>

Art. 33 of Italian Constitution

According to present law it has to provide by 30th April each year the National Committee for Evaluation of University System (CNVSU) the following data :

•Educational offer•Students’ data •Personnel’s data •Financial data•Structures•Scientific research data

According to present law it has to provide by 30th April each year the National Committee for Evaluation of University System (CNVSU) the following data :

•Educational offer•Students’ data •Personnel’s data •Financial data•Structures•Scientific research data

Nucleo di Valutazione di Ateneo(University Evaluation Committee)Nucleo di Valutazione di Ateneo

(University Evaluation Committee)

From 2006 (law no.262)Agenzia Nazionale di Valutazione del Sistema Universitario e della Ricerca

(ANVUR)National Agency for Evaluation of University

System and Research

From 2006 (law no.262)Agenzia Nazionale di Valutazione del Sistema Universitario e della Ricerca

(ANVUR)National Agency for Evaluation of University

System and Research

Quality control system

Efficiency of training process is evaluated through: a) Mean teaching hours per year and average number of CFU ( Crediti Formativi Universitari) per studentb) Enrolled/attending studentsc) External yearly evaluation of quality system by CNVSU (Comitato Nazionale per la Valutazione del Sistema Universitario)d) Tracking of educational process by means of : withdrawal rate in the first two years, average yearly CFUs per student and yearly percentage of graduates within planned course duration. Efficacy of training process is evaluated through:a) Level of students’ satisfaction (anonymous questionnaire)b) Employment rate at one, two and three years from graduation.

Quality control system

Efficiency of training process is evaluated through: a) Mean teaching hours per year and average number of CFU ( Crediti Formativi Universitari) per studentb) Enrolled/attending studentsc) External yearly evaluation of quality system by CNVSU (Comitato Nazionale per la Valutazione del Sistema Universitario)d) Tracking of educational process by means of : withdrawal rate in the first two years, average yearly CFUs per student and yearly percentage of graduates within planned course duration. Efficacy of training process is evaluated through:a) Level of students’ satisfaction (anonymous questionnaire)b) Employment rate at one, two and three years from graduation.

BEME guidelines for medical education

Structure by problems;Central role of clinical education;Investigation of emerging problems, with reference and verification of scientific literature (aiming at searching for evidence proofs);Evaluation of evidence proofs (accessibility, reliability, relevance, pertinence) ; Analysis of transferability of evidence into an actual clinical case; Self-evaluation in terms of self-learning of the work being done.

Main reasons for resistance to change

Sluggishness that pushes to defend your own actionRecognition of clinical committments as prioritary compared to teaching tasksNon acceptance of education as a science with its principlesIgnoring fundamental elements of the educational processPoor interest and minor care for educational activitiesComplaining for lack of services and support for educational activities

Life-Long Learning Framework for Medical University Teaching Staff

Goals:1. Professional update >> all medical doctors2. Educational pedagogic update >> only for those

physicians who play an educational role

Transversal tools:a. Guidelines;b. Of scientific literature (and meta-analysis);c. Network of experts.

Ability to read and correctly and constantly interpret the reviews. It is a type of self-learning that has to be taught within the basic education of the medical doctor in order to build a mental dress oriented to constant education (and self-education) .

undergraduate

(to include into core curriculum)

postgraduate


Pedagogics and medicine

Epistemiological specificity:

The object of the study is a subject, the human being, that can never be reduced to an object;

It is not applied in a neutral context but in a world of values.

Clinical VS Experimental Method


Pedagogics and Education in Medical School 1

Pedagogics

Education

Last goals of education. It elaborates on the values included and pursued by educational action.

Educational goals

Means and strategies to be used to achieve educational objectives

Docimology: it assesses if , how and how much the established objectives have been achieved.


Therefore the educational process is made of three dimensions on which to work:

Planning >> pedagogics

Implementing >> education

Evaluating >> docimology

Core curriculum

Teaching function

Educational

management

Pedagogics and Education in Medical School 2


Core curriculum

Undergraduate

Implementation of clinical activity

Implementation of humanities: phylosophy, sociology, history of medicine, psychology

Postgraduate for medical doctors/teachers

Pedagogics (education – educational technology)

Docimology (experimental pedagogics)


Ethical aspects

Which are the reference values of the learning subject ?

Which criteria he has to use in decisional processes that involve ethical aspects ?

Which responsibilities?

How to harmonize ethics and deontology?


Epistemological aspects

What is the scientific culture of contemporary medicine ?

How research models and methodological presumptions are applied?

Which competences for continuous updating ?

Clinical

Biomedical


Educational Management

Academic institutions devoted to education

Administrative, legal, logistic ties

Plays the role of putting into relationship each other

The management acts besides institutions and teachers in macro-planning and cooperates in the evaluation of the entire course.

Some transversal work settings may be:

1) Study curriculum;

2) Courses and lessons such as: “history of medicine”;

3) Contextualize humanities that live within biomedical and clinical knowledge;

4) Recognize and implement research methodologies;

5) Reconstruct the formation of fudamental concepts for professional practice such as health/disease”

Qualification requirements for medical university teaching staff (Teaching skills)

Pedagogics

Normative NOT Describing Science

The following hypothesis is a proposal of competence NOT a

picture of the existing

Lesson

Planning Implementation Evaluation

Management CommunicationRelationship


Management competences

Frontal lesson (mimics, voice management, space occupation…)Cooperative didactics (organization of working groups for more complex goals than simple memorization such as decision making e problem solving). Management of groups (for the analysis fof conflicts and ability to negotiate) Simulation and role playing (active and participating didactic to elicit transformation of knowledge into competence and to create situations which are similar to professionale ones) Technique of case-study (decision making and individual problem solving).


Relational competences

Physical Setting (spaces, times, disposition of participants) Psychological Setting (space of relation: care is not given to

“removed” psychological aspects but to conditions that may be necessary to achieve different specific goals. E.g.: leadership type, definition of roles

Communication competences

Verbal/paraverbal (oral exposition: times, ways, organization, clearness)

Written (needed to transfer the experience) Tecnifor didactic communication (e.g. ppt, audiovisuals,

electronic blackboard, videorecorded simulations, up to e-learning and teleconference or online working/study groups)

Communication: content and exposition

Site of power exerting on a single individual concerning the management of knowledge

Evaluation competences

Times/ways of evaluation (ex ante - in itinere - ex post) Quantitative Methodologies (test creation) Qualitative Methodologies (creation and interpretation)

Metareflexive Competences

Observation Analysis Euristics

Planning competences


Lab and practising

Metareflexive Management Evaluation

Definition of methodoloigcal premises

Research methodology

Logic of discovery

Tutoring Scaffolding Planning of

significant working settings

Observation: Of practical and

tehnical procedures Of strong

communications signals

Of weak communication signals

Of communication abilities

Of relational ailities Of team working

abilities

Didactic evaluation:

1) Teachers’ performance

2) Students’ performance

3) Course performance

The evaluation may be quantitative (tests, questionnaires, multiple choice and filling answering, standard interview) and qualitative (non standard interview, participated observation, etnographic method)

Medical didactic goals have to be related to the need for care therefore the overall evaluation of a Medical School/Faculty has to include the analysis of the relationship between personnel education and long term changes produced as an answer to individual needs.

How to put this model into practice?

1. Need for an appropriate normative

2. Need for “education for educators”

undergraduate

postgraduate

3. Evaluation system


Present Status of Evaluation of Medical Schools in Italian University ROME TOR VERGATA UNIVERSITY...

Documents

Transcript of Present Status of Evaluation of Medical Schools in Italian University ROME TOR VERGATA UNIVERSITY...