Yecf[j[dY[ - CPSPelogbook.cpsp.edu.pk/elogbook/supervisors_workshop... · 2011-03-16 · Contents...

99KKKK==KKKKEE==FFLL99KKKK==KKKKEE==FFLLee\\ ee\\

YYeeccff[[jj[[ddYY[[YYeeccff[[jj[[ddYY[[

C o n t e n t sPPaaggee NNoo.. 11 -- 22

Introduction 3

Overview 5

Competence 6

Systems approach 9

Taxonomy of educational objectives 11

Guidelines for stating specific learning outcomes 15

Table of specifications 17

Principles of assessment 20

Characteristics of a good examination 21

Types of validity 23

Essay questions: characteristics 24

Essay questions 25

Short essay questions 26

Scoring strategies 29

Analytical vs holistic rubrics 33

Options for selecting rubrics 35

Analytical rubric: example 40

Blank rubric template for practice 43

1Assessment of Competence

DAY

ONE

ASSESSMENT OF COMPETENCE

S C H E D U L E

DAY ONE

Inaugural

Problems with assessment in Pakistan

What do we need to assess

Why do we need to assess

How can we improve assessment

Making a TOS

Principles of Assessment

Characteristics of a good exam

Validity and its types

Essay questions

Characteristics of short essays

How to write short essays

Critique

Scoring strategies

Analytical vs. Holistic rubrics

Making rating scale for short essays

Critique of rating scale


DAY

ONE

INTRODUCTIONThe College of Physicians & Surgeons Pakistan (CPSP) was established in1962 for the purpose of training local doctors in higher medical education.Its founder’s spelt out the following objectives for setting up thisinstitution:

By the end of the workshop, you will be able to:l to promote specialist practice in medicine, surgery, gynecology and

obstetrics and such other specialties;l to arrange postgraduate medical training;l to hold and conduct examinations for grant of diplomas and admission

to the Fellowship;l to provide for research.

Today, CPSP is recognised as one of South Asia’s major postgraduatemedical education centres with an international reputation in education,training and research. It has strong linkages with leading sister institutionsabroad.

Traditionally postgraduate training in Medicine in Pakistan used to followthe pattern as in the United Kingdom (UK). However, with the passage oftime the College has evolved its own system of training and evaluation toproduce a large pool of internationally recognised specialists who areinstrumental in raising the standard of health care in the country. TheCPSP has produced more than 10,000 specialists till date.

Being a premier institution, CPSP is acutely aware of the quality of itstraining and assessment. It has consistently been taking steps forimproving all aspects of the degree that it awards. One of the key steps inimproving the quality of its training is the training of its existing and to-be-supervisors in prime facets of imparting training. This includes planning atraining event, implementing it and then evaluating it.

The four newly designed workshops are focused around these threeaspects. Here is a brief overview of each of these:

Educational Planning and evaluation:This is geared towards how to plan an academic and structured trainingprogram. Devising a structured training program, or for that matter anyacademic event, a planned event and goes beyond making a time table orschedule. It requires being aware of the needs of the learner and thesociety, being aware of the steps in planning and the basic steps inassessing the competence of the learner.


DAY

ONE

This workshop is the first in series of workshops that a clinician has toattend. It is hoped that in this first exposure. The participants will be ableto devise and plan effective academic events.

Assessment of competence:Assessment is a tricky field to play on. It requires careful planning. If onefollows the procedures outlined in Educational Planning & Evaluation andcarries them out, the following steps for assessment of competence willbecome easy for the clinician and supervisor. One can not but agree thatin most cases, assessment does drive learning. Keeping this perception inmind, a supervisor has to be adept at planning and selecting tools forgauging competence. This workshop is directly linked to the previous oneand is intended to equip the participants with important skills of writingshort essay questions, one best and extended matching. It will also helppeople develop and run TOACS to assess psychomotor skills. Assessmentof Competence is important also since it endeavors to explain the qualityassurance procedures that any academic forum should follow in order toproduce reliable and valid decisions about competence or lack thereof.

Supervisory Skills:Having learnt about planning academic experiences it becomes imperativefor the supervisor to learn how to implement the whole plan. This workshopdiscusses practical measures that a clinician has to take when he / sheassumes the responsibility of being a trainer of fellow ship programs. Ithelps the participant gain experience of dealing with practical issues likeconflict management, training juniors in counseling, implementing atraining program.

Research Methodology and Medical Writing:An important aspect of supervision is organizing the conduct of researchof trainees for their dissertation and articles which are a pre- requisite forsitting in fellowship exams.In this workshop, supervisors get to learn the format and standards CPSPexpects from researchers.

All workshops lay great emphasis on hands on training and frequentinteraction between participants and facilitators. Participants of the workshops need to follow the sequence as:But there is a sequence that has to be followed in the remainingworkshops:1st Educational Planning and Evaluation, then Assessment of Competenceand finally Supervisory Skills. Research Methodology and Medical Writingmay be attended at any step.


DAY

ONE

OVERVIEWThe Department of Medical Education, College of Physicians & SurgeonsPakistan is interested in raising the standard of medical education.Assessment of student achievement is one of the most importantresponsibilities of teachers. Both written tests and per formanceassessments serve many functions, but at the same time poorly conceivedand executed tests may do more disservice to the learners. Therefore it isimportant that assessment instruments be well thought out and wellmade. It is also important for examiners to know what to do with theresults. It is not sufficient to simply paste results on the notice board withpercentages since it does not serve much purpose.

The present workshop is a continuation of the same effort and is beingundertaken to introduce to a larger number of faculty members of variousmedical colleges, issues related to two very important techniques ofassessment i.e. short essays and MCQs. Among MCQs, we lay stress onOne best and Extended Matching since internationally and nationally theyare proving to be better than true false. This is because the former two arecapable of gauging higher cognitive thinking much better than the latter.Also chances of guessing and cues are reduced (but not eliminated). Alsohands-on experience will be provided in developing and using tools forassessing psychomotor sheets and affects.

This workshop is based on mutual group activity and it is important for itssmooth running that everyone participate fully. It is for this reason that youare requested to attend all the sessions and avoid any other commitmentduring these hours.

Note: No telephone calls or messages will be delivered during thesessions. Please keep your mobiles and pagers switched off while thesessions are in progress.

OBJECTIVES

By the end of the workshop, participants will be able to:

By the end of the workshop, you will be able to:l Write objectives sutable for their training programl Develop a table of specifications l Develop an assessment plan based on characteristics of a good

examinationl Develop analytical rating scales for assessing cognition, procedural

skills and affectsl Write Short Essays questions, One best and Extended Matching MCQsl Organize and conduct examinations for the assessment of Clinical skillsl Interpret analysis of results based on reliability, dif ficulty and

discrimination indicesl Discuss the uses of an exam result


DAY

ONE

COMPETENCE

Is defined as an individual’s demonstrated knowledge, skills and abilities(KSAs) performed to a specific standard. Competencies are observable,behavioral acts that require a combination of KSA’s to execute. It is

necessary to have them documented for effective performance of a job ortask.

CORE COMPETENCIES:Certain competencies are essential to all members regardless of thediscipline in which they are. For example, history taking, patientexamination, communication with patient, requesting relevantinvestigations, interpretation of various lab data etc. these are factorswhich all disciplines will lay great stress on. These are the skills withoutwhich no person can be certified as a safe and ‘competent practitioner.

SPECIFIC COMPETENCIES:Are those skills, knowledge and abilities that are going to be usedspecifically in a particular discipline. For example the competenciesrequired in a histopathology department will be slightly different from thatin a neonatal care unit. Similarly, there will be a substantial difference inthe competencies required in a Psychiatry unit as compared to anOphthalmology unit.

SOFT-SKILL COMPETENCIES:Are competencies that are required for acquiring skill in auxiliary areas like l Computer skills,l Management skills, l Knowledge of research skills, l Time management l Conflict management l Negotiation skills etc.

Such competencies considered essential for the smooth running of thetraining unit many clinicians consider.


DAY

ONE

CLINICAL COMPETENCIESApplication of Knowledge= The application of clinical and basic science knowledge in order to

diagnose and manage a patient's medical problems.

History Taking= The technical skills involved in interviewing a patient. The skill of asking

questions relevant to the patient and building on to the informationbeing provided. A constant analysis and expertise a required by anadept clinician.

Physical Exam Technique (Data Collection)= Conducting a satisfactory physical examination. Includes knowing all

the physical examination maneuvers,

= the dexterity of conducting an examination efficiently with minimalduress to the patient is an important component of competence.

Clinical reasoning = Recognition of significant findings established through questions

(history), inspection, palpation, Auscultation, and percussion.

Interpretation of clinical data= The ability to integrate information about the patient and arrive at a

reasonable differential diagnosis.

Plan of investigation = Systematic choice of laboratory tests and other diagnostic

investigations.

Diagnostic procedures= Ability to perform diagnostic procedure.

Patient Problem Identification= Integrating the information in a most plausible problem.

Procedural skills= The ability to perform a technical procedure with causing none or

minimal discomfort to the patient

Patient Management= The ability to plan and carry out an appropriate approach to manage the

patient's health related problem.


DAY

ONE

Communication= The ability to communicate clearly and effectively with patients and

colleagues in both written and spoken forms.

Counseling= The ability to assist the patient arrive at a decision on his / her own

effectively regarding the nature of the medical problem; the plan fordealing with the problem and the patient's role in management.

Professional Behavior= The ability to interact and work effectively with patients and other health

professionals. This includes key personal traits as organization,dependability, and ethical conduct.


DAY

ONE

SYSTEMS APPROACH

Systems approach to education is a conceptual framework,

l which explains what the various components of an effective educational plan are,

l in what sequence they appear and l how they are inter-related.

It aids us in tackling problems that we experience in making our studentslearn better and hence, be more competent.

OBJECTIVES:

Objectives are statements of what the educational process hopes to

achieve. For medical education in Pakistan, they are primarily laid down by

the PMDC, which draws upon the services of health officials and experts

in different fields in designing them. The needs of the nation, community,

profession, students etc. determine the objectives.

Currently, the educational worlds is focusing on outcomes of an

educational program. Hence, the term ‘objective’ has been replaced by

‘outcome’. So, through out this text we will refer to objectives as

outcomes. Consequently, the first major step in the systems apporach is

the writing of learning outcomes. Since these have to be written in specific

and measureable terms, they are called Specific Learning Outcomes or

SLOs. SLOs inform the readers what the learner will have accomplished

after going through the course.

INSTRUCTIONAL STRATEGY:This is the outline or plan of how the objectives should be met. In practical

terms, this is the timetable and schedule which each institution and

department prepares in order to achieve the objectives. The strategy is

partly dependent upon the resources although it is possible to maximize

the available resources by fully using the principles of learning.

EVALUATION:Evaluation, is a general term, which incorporates a number of modalities.

For example, evaluation could be to find out the quality of a teaching

program, the teaching abilities of a faculty member, the strength of a

curriculum etc.


DAY

ONE

Assessment, on the other hand, is used specifically to mean evaluation ofstudents' knowledge, psychomotor competence and/or attitudes. Withinthe same context, assessment is the process of determining whether theobjectives have been achieved or not. A properly designed assessment isthus determined by including a sufficient number of SLOs in the test orexam; this is called sufficient sampling. Student assessment is better ifthe principles of evaluation are utilized in designing it.

The systems approach can be used in our lives also, i.e. first we make aplan (objectives) to do some work and then we perform it and then weevaluate what we did by thinking about how our work went. So usually we‘evaluate’ our work mentally by matching what we did with what we hadplanned to do. So the evaluation is always according to the initial plan. Ifthings do not go as per our plan (objectives) we review where we wentwrong. This review is not just of how we did our work (instructional strategy)but also of whether we evaluated our plan correctly or not and whether ourinitial plan was correct or not. So, review is ALWAYS of each and everyaspect of the systems approach.

REVIEW:It is imperative that once the whole circle has been followed, experts in thesubject hold a meeting in which they review the whole plan. They need to:

1. Go over the results of the test that they have administered and askthemselves: does the test have indicators of high quality? If not, thenthe items in the test have to be reviewed and

2. Matched with the objectives (do the SLOs need to be changed also?). 3. Some changes in the instructional strategy may then be necessary

and/ or4. Changes in the test that was administered.


DAY

ONE

TAXONOMY OF EDUCATIONALOBJECTIVES

HISTORICAL BACKGROUNDEducational development has been proceeding since the 19th century. The

beginning of the 20th century saw a rapid growth in work done in the field

of education. Educational psychologists evolved their ideas on the basis of

much research that kept pouring in from all over the world. Hence, a

number of educational philosophies evolved with time. So, we had the

‘Traditional philosophy’ of education (1897); then came the ‘Experiential

philosophy’ (1931); ‘Structure of the Disciplines’ (1953) became very

popular after world war II; ‘ Behavioral philosophy’ gained prominence

during the late 1950’s; during the late 1960’s the ‘Cognitive philosophy’

became very much in vogue since, by then, scientists had learned enough

about the functioning of the mind to be able to apply the information to

learning and education.

It was actually during the mid 50’s that it was thought of classifying

knowledge and information. Bloom, in 1956, was amongst the pioneers in

presenting this classification to the world. We have tried to simplify the

extensive taxonomy or classification presented by Bloom. Following is its

description.

TAXONOMY OF EDUCATIONAL OBJECTIVESThe taxonomy of educational objectives is intended to provide for

classification of the goals of our educational system. Although the specific

learning outcomes resulting from a course of study may run into hundreds,

most of them can be classified under a small number of headings.

Any such classification serves several useful purposes. It:

l indicates the type of learning outcomes that should be considered.

l provides a framework for classifying those outcomes.

l directs attention towards changes in student performance in a variety of

areas.

l forms the blueprint for forming your assessment tool.

l provides common grounds and makes comparison of various teaching

programs easier.


DAY

ONE

The system proposed by Bloom first divides objectives into the followingthree major areas:

Cognitive domain: Knowledge outcomes, intellectual abilities and thinkingskills.

Psychomotor domain: Perceptual and motor skills.

Affective domain: Attitudes, interests, appreciation, and modes of

adjustment.

Each of the three domains is subdivided into a number of sub-categories.

For the sake of convenience and based on literature from National Board

of Medical Education USA, the original categories and have been simplified

grouped into three sub catagries;

Following are the levels of Cognitive domain:

C1: involves recall of facts, principles, processes, patterns and methods

necessary for efficient performance of a professional task.

C2: involves deriving meaning from a single source of information.

Laboratory data, X-rays, ECGs or graphs are interpreted to determine

whether they are normal or abnormal in a given situation. Interpretation is

not possible without the knowledge of relevant facts and thus questions

aimed at testing this level automatically test C1 as well.

C3: start with gathering data from all the available sources. This data is

then interpreted, and highest synthesized and then analyzed to reach a

conclusion encompasses making diagnoses, management play, clinical

decision-making, clinical reasoning. This, at best, includes finding

solutions for a problem arising from new situations with very little or no

precedence to serve as a guide but by recalling facts and analyzing the

infomation at hand. Problem solving and clinical management. This is the

best and since it incorporates the previous two levels automatically.


DAY

ONE

Cognition 1 (C1): Recognition and recall: entails recall of facts and figures

Cognition 2 (C2): Interpretation: involves recall, synthesis and desisions about a single piece of information

Cognition 3 (C3): Problem Solving: consists of recall, synthesis, analysis and judgement

Following are the levels of psychomotor domain:

P1) IMITATION:

This is the first level of competence of performing any skill. During this

stage the person has to devote all his attention to the work at hand and is

unable to perform complex tasks unless they are systematically broken in

to smaller tasks. He needs constant supervision and instant feedback

about his actions and mistakes, which are frequent.

P2) CONTROL:

The performer is slightly more adept at the skill. He still needs to pay attention

but the level of attention required is not as great as before. He is able to

perform complex tasks in bigger chunks and relatively more speed. At this

stage he still needs supervision but to a much less degree. The number of

mistakes that he makes has reduced. He is able to judge when he makes a

mistake but has difficulty in making correction. He also has a lot of problems

in transferring his recently acquired skill to new and unique situations.

P3) AUTOMATION:

The performer is fully skilled at the work and needs no supervision. He is

able to focus his attention on things other than the task while performing.

The frequency of his mistakes has reduced remarkably. He is able to self

analyze the mistakes he makes and correct them himself. He is also able to

use his skill in new situations with relatively more ease. He is now able to

reflect on what he has done and how he can improve upon it.


DAY

ONE

Psychomotor 1 (P1): Imitation: Only small and simple tasks possible, one at a time, many mistakes, high tress level, maximumattention and heavy supervision required

Psychomotor 2 (P2): Control: Slightly complex tasks, fewer mistakes, moderate or low stress/and less supervision required, inability to deal with unique situations

Psychomotor 3 (P3): Automation: Complex multi-tasking, very few mistakes, self analysis and correction, command during unique situations, no supervision required

Following are the levels of Affective domain:

A1) RECEPTIVITY: This stage implies sensitivity to existence of a

certain phenomenon and includes willingness to receive. Example:

Noticing the anxiety of a patient awaiting the result of a laboratory test for

a disease that can have serious consequences.

A2) RESPONSE: This implies sufficient interest in the phenomenon

noticed to do something about it. Example: in the case described above in

the previous example, the response would be to say a few reassuring

words to that patient so that he does not feel alone.

A3) INTERNALIZATION: This level of affective domain implies that your

perception of a phenomenon has found a place in your scale of values and

has affected you long enough for you to adapt yourself to the value system

of the other person. This enables you to adapt your attitude to the other

person as if you were experiencing the same phenomenon yourself.

Example: On the death of a child, your attitude to the members of his

family will show them that you care about their grief and are ready to help

them get over it. This does not mean that you have internalized their grief

but that you have internalized the attitude that enables you to offer them

genuine, effective help.


DAY

ONE

Affective 1 (A1): Receptivity: Sensitivity to a phenomenon and willingness to receive

Affective 2 (A2): Response: Action in response to a phenomenon due to inherent interest

Affective 3 (A3): Internalization: Inclusion of the perception within the personal value system

GUIDELINES FOR WRITING SPECIFICLEARNING OUTCOMES

Objectives are statements of intentions they have to be thought of and

documented in the initial phase of any educational activity (as per the

systems approach).

The whole of ‘Behavioral philosophy’ of learning is based on stating

intentions of how or what the teacher expects the students to perform or

do as a result of instruction.

The technique of writing objectives has evolved over the decades. They

are now also called Specific Learning Outcomes (SLOs).

In the subsequent sections we will be dealing with the mechanics of

writing SLOs.

STEPS FOR STATING SPECIFIC LEARNING OUTCOMES(OR OBJECTIVES)

1. List a representative sample of topics.2. Write a blanket statement.3. Begin each objective with an action verb that specifies

observable performance (e.g. identifies, manages).4. Include enough objectives to describe adequately the

performance of students who have attained the objective.5. Keep the objectives sufficiently comprehensive and global

and not too restricting.


DAY

ONE

STATE SLOS AS EXPECTED STUDENT PERFORMANCEStart by writing a blanket statement, an example of which is as follows:“At the end of the session/ unit/ course, the learner will be able to:”l Name the cells involved in acute inflammationl Enlist the drugs that block inflammation and are anti-pyretics.l Differentiate between a normal and an abnormal ECG.l Outline a management plan for a patient with a septic wound and

uncontrolled diabetes mellitus.l Justify investigations required to diagnose a case of high-grade fever in

children under one year of age.

Don’t state them in terms of: l Teacher performance (e.g. teach inflammation)l Learning process (e.g. student learns the concept of immunization)l Course content (e.g. studies the management of enteric fever)l Two objectives (e.g. student knows and understands concepts in

peri-operative management)

USING OBSERVABLE VERBSl Each statement must begin with a verb. However, the verbs are specific

and indicate definite, observable responses; that is, responses that canbe seen and assessed by an observer.

l These verbs state learning objectives in terms of observable studentperformance, e.g. describes, identifies, differentiates, plans.

l Learning objectives should specify the types of reaction the students aresupposed to make to the content and not identify the content itself. So,do not go into too much detail. It is always beneficial to have objectivesthat do not restrict the learning activities. Try and make them global.

OBJECTIVES SHOULD NEITHER BE TOO RESTRICTING NOR TOO VAGUEThe following are too specific and restricting:Describes the phase of isovolumic contraction in a cardiac cycle. Describes the phase of rapid ejection in a cardiac cycle.The following is better:Describes how the heart functions in a normal adult human.

The following is too broad and vague:Deals with cases of endocrine disordesThe following is better:Manages cases of Phaechromocytoma effectively


DAY

ONE

TABLE OF SPECIFICATIONSA TABLE OF SPECIFICATIONS is a chart that contains information aboutthe topics of your speciality, the level you want each topic to be taught, i.e.whether you want a particular topic to be at C1, C2 or C3, and the total ofthese topics along with this total expressed in percentage. STEPS IN THE MAKING OF A TABLE OF SPECIFICATION:1. Make a table with the following headings (or columns)

2. First, fill the contents section. write all the topics that you and your

team want.

3. Decide how much ( i.e. relative) importance ( or weightage) you want toassign to each topic

4. In the extreme right column write the % of importance for a particulartopic. For example, in medicine, you may decide to assign 35% to

Hypertension, & 30% to Diabetes Mellitus.

5. Now, continuing with the above example, you have to decide that forHypertension, how much of this 35% you will assign to Cognition,Psychomotor and Affects. Within Cognition, then, you have to decidewhether you want to lay more emphasis on recall ( C1), interpretation(C2) or problem solving (C3). Examples are provided below:

6. Note: It is not possible to have three diffrent levels of Psychomotorsince a person can be at only one level of dexterity at one time. In thecolumn of ‘P’ write either 1, 2 or 3 to indicate the desired outcome.


DAY

ONE

Contents P A % 1 2 3C o g n i t i o n


Hypertension, 35

Diabetes

Mellitus 30

7. Note: For Affective domian use the same principle as that ofPsychomotor domain stated above in point 6.

DECIDING WHICH TOPIC/ UNIT TO GIVE MORE IMPORTANCE TO

This depends on the prevalence of disease in the country. For example, inPakistan if the most prevalent disorder is iron deficiency anemia orpulmonary tuberculosis, then probably most importance should be given toeither ‘Blood’ or ‘Respiratory system’ instead of to CNS or Locomotorsystem..

DECIDING WHETHER ONE SHOULD KEEP MORE OF C1, C2 OR C3

That depends on the level of students you are dealing with. If it is a juniorundergraduate, then there should be relatively more of C1 and C2. If it is asenior undergraduate then the number of C3s should be relatively more.If it is a post graduate student then C3s should be clearly more than theC1s and C2s.

COMMON MISTAKES TO BE AVOIDEDl It is not necessary for ALL the topics to be present in this table. Leave

out the relatively unimportant ones, even if they seem interesting to you.l It is not necessary for every topic to be divided into the two cognitive

domains. It is definitely possible for a topic to have no C2 (e.g. 1st yearMBBS student, topic: CNS) or no C1 (final year MBBS/ resident yearone, topic: CVS).

l Make sure that the total is always given is percentage also. This willhelp you if you want to change the number of items different from thatgiven in the total of the table.

HOW DO WE USE A TABLE OF SPECIFICATION

A table of specifications may be used to decide not just the anatomy of anexam but also the type of instructional strategy that is best suited for aparticular topic. A table of specifications can, thus, help us decide the:

A. Type of instruction that will be used for each sub-topic:l Number of items that should be present in a theory exam paper.l Relative number of topics that should be present in the theory exam.l Cognitive level for assessing each topic and sub-topic l Psychomotor importance that you assign to a unit or sub unit.l Affective importance that you want to assign to a unit or sub unit.


DAY

ONE

B. Type of assessment that is applicable for each topic and domain:It doesn’t make sense to assess how well a candidate can perform acertain skill by giving him a written or oral test about it. The best thing todo in such a case is to observe how he performs that skill/ task. So if yoursub unit has some psychomotor component to it, then a good exam wouldinclude not just a written part but also a practical. Similarly if you want toassess if your student has learnt how to introduce himself to the patientand how he deals with the patient then you should have a system ofevaluating his attitude regularly in the OPD or ward.SAMPLE OF COMPLETE TABLE OF SPECIFICATIONS

HOW TO WE USE IT

A table of specifications may be used to decide not just the anatomy of anexam but also the type of instructional strategy that is best suited for aparticular topic. A table of specifications can, thus, help us decide the:

A. Type of instruction that will be used for each sub-topic:l Number of items that should be present in a theory exam paper.l Relative number of topics that should be present in the theory exam.l Cognitive level for assessing each topic and sub-topic l Psychomotor importance that you assign to a unit or sub unit.l Affective importance that you want to assign to a unit or sub unit.

B. Type of assessment that is applicable for each topic and domain:It doesn’t make sense to assess how well a candidate can perform acertain skill by giving him a written or oral test about it. The best thing todo in such a case is to observe how he performs that skill/ task. So if yoursub unit has some psychomotor component to it, then a good exam wouldinclude not just a written part but also a practical. Similarly if you want toassess if your student has learnt how to introduce himself to the patientand how he deals with the patient then you should have a system ofevaluating his attitude regularly in the OPD or ward.


DAY

ONE


Hypertension, 16 12 2 30

Diabetes

Mellitus 10 12 8 30

Acute Renal 3 5 2 10

Failure

Tuberculosis 5 6 12 7 30

5 35 41 19 100

PRINCIPLES OF ASSESSMENT

ASSESSMENT: QUEST FOR IMPROVEMENT

A ssessment is a process of determining the extent to which studentshave achieved the instructional objectives.

The following principles form a general framework within which thisprocess may be viewed:

1. Clarify the purpose of assessment 2. Determine the content that is to be assessed.3. Ensure adequate content sampling by following a table of specifications.4. Select a suitable assessment tool (to fit the purpose).5. Try and use a combination of various assessment tools.6. Use the results to improve learning and training.

Assessment is a means to an end, not an end in itself: once an exam isover, the results should not be stored in a computer or a file but should beused to:

= review the instructions

= review the teaching program

= review teacher performance

= review student performance

= certify students

= upgrade / detain students.

TYPES OF ASSESSMENTFORMATIVE:l This is designed to inform students about the amount he still has to

learn. l It measures the progress made by the learner and enables learning

activities to be adjusted according to the progress made. l It must never be used for certifying purposes since its main aim is

diagnosis of the amount of learning done and needed further.

SUMMATIVE:

l This is designed to protect society from incompetent professionals. l It is traditionally used for placing students in order of merit and justifying

decisions as to whether they should move up to the next level or beawarded a diploma or not.

l It is carried out less frequently than formative assessment and isusually at the end of a unit or period of instruction.


DAY

ONE

CHARACTERISTICS OF A GOOD EXAMINATION

An exam is said to be ‘good’ if it has the following characteristics:1. Feasibility2. Objectivity3. Reliability4. Validity

FEASIBILITY:An exam should be cost beneficial and easy to score. It should not be aburden on financial and human resources of the organization.

OBJECTIVITY:An exam is ‘objective’ if the acceptable answer to the questions is definedprior to its administration and any competent person given a key can markit. Objective tests need not be multiple choice only; essays and oral examsmay be objective as well. However the objectivity of the latter two is lowerthan of MCQs.

RELIABILITY:It refers to the lack of error in measurement or consistency ofmeasurement. That is, how consistent test scores or other assessmentresults are from one measurement to another. Reliability is essentiallyconcerned with restricting measurement errors within acceptable limits.

Reliability refers to the results (obtained with an assessment instrument)and not to the instrument itself. So, when you comment about thereliability of a test you are actually talking about its result or score and notabout the test itself. It is important for a test to be reliable because onlythen can we confidently say that this instrument will give us the sameresult if we were to re-administer it to a similar set of students. Reliabilityis usually measured by a computer after the scores have been fed in. Thename of the method is Cronbach’s Alpha Reliability. The value ofCronbach’s alpha varies from 0 to 1. An internationally minimum acceptablevalue is 0.8.

FACTORS IMPROVING RELIABIITY:

Reliability improves with:

= increasing testing time,

= computer marking,


DAY

ONE

= good item design,

= item-banking and re-use of proven psychometric quality and

= examiner training.

VALIDITY:Validity is essentially concerned with:

= the content of the assessment ( are the right things being tested?)

= the effect of learning ( are the students learning what we want them tolearn?)

= the assessment methods ( are we using the most appropriate methods forthe purpose?)

In other words, it is the degree to which the test ‘truly’ measures what itis intended to measure. It also includes an evaluation of the adequacy andappropriateness of the uses that are made of the assessment results. So,when we talk of reliability, we are talking about minimizing measurementerrors; when we talk about validity we are talking about appropriateness ofinterpretations made from the results. Validity is a matter of degree; itdoes not exist on an all or none basis. So, there is no such thing as validor invalid. We always refer to a test result as having high validity or lowvalidity. There are many ways of accumulating evidence to support orchallenge the validity of an interpretation or use of an assessment result.Some of these categories are: face validity, content validity, constructvalidity, test-criterion relationship and consequence validity. Validitydoes not have a numeric value. It is a judgement based on the abovementioned evidence.

FACTORS IMPROVING VALIDITYValidity improves with:

= defining the curriculum/ syllabus

= mapping exam content against the curriculum or using a table ofspecifications to develop an assessment plan

= emphasis on essential and important curriculum content

= selecting methods according to objectives


DAY

ONE

TYPES OF VALIDITY

Face validity asks, "Does the test ‘appear’ to be suitable for

the level and discipline for which it is intended ?"

Content validity asks, “Is the sampling adequate?"

This is said to be high if the paper setters have followed the

table of specifications.

Construct validity asks, e.g. "Is this test actually assessing

what I want it to assess?"

Predictive validity asks, "How well does performance on

this test predict performance in the future?"

Consequence validity "What is the effect of this

assessment on learning ?


DAY

ONE

ESSAY QUESTIONS CHARACTERISTICS

1. FEASIBILITY: high (easy to make and administer, cost effective.)

2. OBJECTIVITY: low (unless a valid key is given to the rater, the marks

will differ widely among the examiners of the same question).

3. RELIABILITY: low. (by Cronbach’s alpha)

4. FACE VALIDITY: medium to high. (They ‘seem’ to measure basic

recall/ knowledge application among medical students of a particular

level.)

5. CONTENT VALIDITY: low. (Will an essay paper really be able to cover

ALL the aspects of a table of specifications given the time constraint?)

6. CONSTRUCT VALIDITY: medium to low. (Considering the type of essay

questions made currently, are we really judging reasoning ability,

knowledge and skills of a subject?)

7. PREDICTIVE VALIDITY: low. (Can an essay paper really predict how

well a student will do in the future in the field of medicine?)

8. CONSEQUENCE VALIDITY: low. (They usually promote rote

memorization of seemingly higher order medical processes such as

pathogenesis, management and diagnoses, so higher order thinking is

not fostered.)


DAY

ONE

ESSAY QUESTIONSAn Essay question is a test item in which an examinee composes a

response. It is usually in the form of one or more sentences of a nature

that no single response or pattern of response can be listed as correct and

the accuracy and quality of which can be judged subjectively only by one

skilled or informed in the subject.

Essay questions are one of the oldest formats to assess the knowledge of

a student about the subject. Despite extensive development of new

assessment procedures and criticism on older methods essay type

examinations are still widely used and continue to flourish in many parts

of the world including in United States.

There are two types of essay questions:

== Short essay questions

== Long essay questions

This division depends upon the amount of freedom offered to the

examinee to express his ideas and restriction over the scope of topic.

Essay type question are an appropriate tool for evaluating complex

performance e.g. synthesis of a group of complex concept, summary of a

document, cause analysis.


DAY

ONE

SHORT ESSAY QUESTIONS (SEQs)RULES FOR WRITING & ADMINISTERING BETTER SEQs:

SEQs are a good way of measuring an examinee’s ability of higher order

intellectual achievement e.g. synthesis of a group of complex concept,

summary of a document, cause analysis.

They should not be used to test just factual recall

= Define objective clearly

= Formulate well-focused question

= Relate the question to the objective

= Ask question with appropriate terms like summarize, compare, define,

don’t use broader terms like discuss or write etc

= The phrasing of questions should be simple and unambiguous e.g.

terms like ‘describe briefly’ & ‘explain concisely’ should be avoided

= Indicate time limit approximately for each question.

= Indicate marks distribution for each question, especially if it varies

from question to question

= Use of optional question should be avoided because it is difficult to

construct questions of equal difficulty. Furthermore, a better student

may opt for a difficult question without having any added benefit.

= Prepare an expected answer of a question before administering it.

= Pre test the question if possible in field condition. If the question has

already been administered get feed back analysis for record of question

bank.

ADVANTAGES OF SEQs:l SEQs are an important tool of assessing higher cognitive learning

particularly at C3 (problem solving) level.

l Examinees are free to select, relate and present the idea in their own

words.

l Many topics can be examined in a short time (improved sample size)

l SEQs are more reliable & valid than long essays.

l SEQs are precise and clear because they are made according to

well-defined objectives.

l The form of response that is expected is clearly indicated in questions

by using appropriate action words e.g. enumerate, enlist, compare etc.

l Improved precision of objective allows better rating & lessens subjective

bias, although it does not eliminate it completely.


DAY

ONE

l The ability of examinees to organize and summarize thoughts is

effectively tested. This leads towards better communication skills.

l Examinees have to think more and write less, thereby relying less on

writing speed than organization of thoughts.

l SEQs give no cues since examinees have to supply responses instead

of selecting them.

l Use of unfair means is not very probable since the time allowed for each

response is very limited.

LIMITATIONS OF SEQs:

l Compared to long essays the reliability of SEQs is high, but generally

their reliability is low if we compare it with that of MCQs. This is because

of difficulty in grading the free response type and inters rater

differences.

l SEQs can be remembered by students and communicated to the next

batch of examinees easily so they are not fit for frequent reuse.

l As the number of SEQs remains limited in any test so few areas are

measured intensely while others are left unattended

l Scoring cost is higher, and time consuming because they can be

checked only by qualified assessors and not by computers or clerks.

l Some people are disadvantaged because of their poor, illegible or slow

writing speed.


DAY

ONE

EXAMPLES OF SHORT ESSAY QUESTIONS

A) Question no.1

You are attending patients in the emergency of a tertiary care hospital.

A patient has been brought to you from a village of interior Sind with H/O:

High grade fever –one week.

O/E

Drowsiness present

Dehydration ++

Moderate splenomegaly present.

Name 5 lab investigations and justify how these will help you to reach the

most appropriate diagnosis and subsequent management.

(Your answer should not exceed one page of response sheet)

B) Question no.2

Write the management of a 4-year-old child bitten by a stray cat resulting

in 1/2cm deep wound on face .The cat was hidden in the corner of the

house and attacked the child when he went near it.

(Note: the parents of the child are quite affording)

C) Question no.3

A man, while crossing a road, was hit on the right knee joint by an

automobile. List the probable positive clinical findings you expect if

you examine him two hours after accident in your clinic.

D) Question no.4

A poor woman, working as a domestic servant, is pregnant and severely

anemic. She comes to consult you in a govt. dispensary. You fear that you

will not see this patient again. Discuss the management of this case by

stating not more than five steps.


DAY

ONE

SCORING STRATEGIESI. GUIDELINES FOR PREPARING A KEY FOR ESSAY

QUESTIONSA key is a set of instructions for scoring written answers. It defines andsets the minimum factual knowledge/comprehension or ability to solve aproblem, which a candidate must have in order to secure passing marks.

STEPS IN THE PREPARATION OF A SCORING KEY.For knowledge and comprehension questions: Prepare an outline of the expected answer.Assign a score or weightage to each item to be included in the answer.

Example:The following is the key of question A. The pass mark is 60%.

Key:Point m 20 Point n 10 Point o 30 Point p 20 Point q 10 Point r 10

100

The candidate could earn 60% marks by writing

points o + m + n or o + n + p or m + n + p + q or m+n+p+r or any other combination totaling 60%.

If you wish that to pass, the candidate must write one or two facts/ points,which you consider essential, then you should specify these in the key.There are different ways in which this can be done.

Give so much weightage/marks for those particular points that thecandidate cannot get the pass marks without it. In the above examplesuppose you want that point o is mandatory then you can put 50 as theMarks for o and if the candidate misses it or does it very poorly he doesnot get the passing marks. Taking the same example quoted above,suppose 25 marks are deducted for omitting point o. Irrespective of howmany marks the candidate gets in the rest of the question, he/ she cannotpass after losing 30 marks for not having mentioned point o and a furtherdeduction of 25 marks for this act of omission. Thus the total markscandidate losses for missing point o are 55.


DAY

ONE

If you want the candidate to mention all the above points then distributethe marks in such a way that they score total 60% or the pass marks forthat question. The rest of the marks could then be for additional points andetter organization.

Note: In certain types of examinations, the candidates are only declaredpass/fall. In such cases the question setter is required to identify theelements that must be present in the answer. These are called the"Essential Points". A candidate must mention all the essential points inorder to pass the question. The additional elements in the answer arecalled

"Optional Points" and are considered only when all the essential pointshave been mentioned.


DAY

ONE

II. ANALYTICAL VS HOLISTIC RUBRICS

Combining scores from analytic rubric scaleIf you decide to use an analytic rubric, you may wish to add or averagethe scores from each of the scales to get a total score.

Scoring rubrics may be specific to a particular assignment or they maybe general enough to apply to many different assignments. Usually themore general rubrics prove to be most useful, since they eliminate theneed for constant adaptation to particular assignments and because theyprovide an enduring vision of quality work that can guide both studentsand teachers.

Although it is possible to construct your own rubrics from scratch, manypeople find it simpler to adopt or modify existing rubrics. Examples ofscoring rubrics may be found in education journals and other professionalpublications. These rubrics may be adopted, adapted, or they canprovide ideas and serve as models.

There is no single best rubric for all purposes and many different rubricscould be applied to the same task. The following criteria for evaluatingscoring rubrics are adapted from Herman, Aschbacher and Winters(1992), Arter (1990) and ISBE (1994).

Does the rubric relate to the outcome(s) being measured?The rubric should address all aspects of the outcome(s) being measuredand it should not address anything extraneous. For example, spelling andgrammar might be considered extraneous on a science assessment,unless it is measuring an outcome that deals specifically withcommunication. If the assessment is to address critical thinking skills,does the rubric reflect that?

Does it cover important dimensions of student performance? Do the criteria reflect current conceptions of excellence in the field? Does the rubric reflect what you emphasize in your teaching? Does the highest scale point represent a truly exemplary performance or product? When you evaluate rubrics, you need not be concerned about having acertain number of students score at every point on the scale. It may bethat no student will attain the highest scale point. Nonetheless, it stillmay be worthwhile to have that point on the scale as a standard ofexcellence for which students should strive.


DAY

ONE

Are the dimensions or scales well-defined? Is it clear to everyone whateach scale measures?Is there a clear basis for assigning scores at each scale point? Is it clearexactly what a student needs to do to get a score at each scale point?Can you easily differentiate between scale points? An easily understood rubric with clear definitions of each score point is theideal. Conversely, it is usually best to avoid scales that are labeled only atthe highest and lowest points.

Can the rubric be applied consistently by different scorers?Inter-rater reliability depends on how well the scales and scale points aredefined and the extent to which you and your colleagues can arrive atconsensus about how performance should be measured and whatconstitutes good performance.


DAY

ONE

ANALYTICAL VS HOLISTIC RUBRICSA rubric with two or more separate scales is called an analytical rubric.This contrasts with a scoring rubric that uses only a single scale that yieldsa global or holistic rating.

Holistic scoring is often more efficient, but analytical scoring systemsgenerally provide more detailed information that may be useful in planningand improving instruction and communicating with students.

Combining scores from analytic rubric scaleIf you decide to use an analytic rubric, you may wish to add or average thescores from each of the scales to get a total score.

Scoring rubrics may be specific to a particular assignment or they may begeneral enough to apply to many different assignments. Usually the moregeneral rubrics prove to be most useful, since they eliminate the need forconstant adaptation to particular assignments and because they providean enduring vision of quality work that can guide both students andteachers.

Although it is possible to construct your own rubrics from scratch, manypeople find it simpler to adopt or modify existing rubrics. Examples ofscoring rubrics may be found in education journals and other professionalpublications. These rubrics may be adopted, adapted, or they can provideideas and serve as models.

There is no single best rubric for all purposes and many different rubricscould be applied to the same task. The following criteria for evaluatingscoring rubrics are adapted from Herman, Aschbacher and Winters (1992),Arter (1990) and ISBE (1994).

Does the rubric relate to the outcome(s) being measured? The rubric should address all aspects of the outcome(s) being measuredand it should not address anything extraneous. If the assessment is toaddress critical thinking skills, does the rubric reflect that?


DAY

ONE

Does it cover important dimensions of student performance? Do thecriteria reflect current conceptions of excellence in the field? Does therubric reflect what you emphasize in your teaching? Does the highestscale point represent a truly exemplary performance or product?

When you evaluate rubrics, you need not be concerned about having acertain number of students score at every point on the scale. It may bethat no student will attain the highest scale point. Nonetheless, it still maybe worthwhile to have that point on the scale as a standard of excellencefor which students should strive.

Are the dimensions or scales well-defined? Is it clear to everyone whateach scale measures? Is there a clear basis for assigning scores at eachscale point? Is it clear exactly what a student needs to do to get a scoreat each scale point? Can you easily differentiate between scale points? An easily understood rubric with clear definitions of each score point is theideal. Conversely, it is usually best to avoid scales that are labeled only atthe highest and lowest points.

Can the rubric be applied consistently by different scorers?Inter-rater reliability depends on how well the scales and scale points aredefined and the extent to which you and your colleagues can arrive atconsensus about how performance should be measured and whatconstitutes good performance.


DAY

ONE

OPTIONS FOR SELECTING RUBRICS1. Adopt

q Use an existing rubric "as is."

2. Adapt q Modify or combine existing rubrics. q Re-word parts of the rubric. q Drop or change one or more scales of an analytical rubric. q Omit criteria that are not relevant to the outcome you are

measuring. q "Mix and match" scales from different rubrics. q Change the rubric for use at a different grade. q Add a "no-response" category at the bottom of the scale. q Divide a holistic rubric into several scales.

3. Do it yourself q Build your own rubric from scratch about

QUESTIONS TO CONSIDER WHILE MAKING RUBRICS

1. Can the rubric be understood by students and parents? Can it beexplained without technical jargon and in terms that even youngstudents can understand?

2. Is the rubric developmentally appropriate? Does the rubric specifysome score points appropriate for the age of the student?

3. Can the rubric be applied to a variety of questions?The most useful rubrics can be applied to more than one task.

4. Is the rubric fair and free from bias? Does it reflect teachable skillsor does it address variables over which students and educatorshave no control, such as the student's culture, gender or homeresources? Does the rubric reward or penalize students based onskills unrelated to the outcome being measured? Have all studentshad an equal opportunity to learn the content and skills addressedin the rubric? Is the rubric appropriate for the conditions underwhich the task was completed?

5. Is the rubric useful, feasible, manageable and practical? Does therubric make sense to you? Will it provide the kind of informationyou need and can use effectively? Does the rubric have areasonable number of scales and score points?


DAY

ONE

It is difficult to rate student work on many scales at once and it may behard for students to assimilate data from many score scales. Using longscales is often more time-consuming than using shorter ones.

It's impossible to know for sure how well a rubric will work until you andyour colleagues actually try it out on some actual examples of studentwork. You may want or need to pilot test several rubrics on the samestudent products or performances (it's easier if you can tape theperformances) to see which one works best for you. It's also important tosee whether you and others can learn to apply the rubric consistentlyenough so that you can generally agree on what score to assign to a pieceof student work. This consistency or reliability is crucial if the assessmentis to yield valid, meaningful data.

One method for assessing reliability involves having each raterindependently score several examples of work produced by high-achieving,average, and low-achieving students, and note the extent to which theraters assign the same score to a piece of work. If there is no consensus,having the raters explain and discuss their scores can be instrumental inclarifying the rubric and fostering uniform scoring.

SUMMARY:

Things to consider when selecting a rubric

l Does the rubric relate to the outcome(s) being measured? Does it

address anything extraneous?

l Does the rubric cover important dimensions of student performance?

l Do the criteria reflect current conceptions of "excellence" in the field?

l Are the categories or scales well-defined?

l Is there a clear basis for assigning scores at each scale point?

l Can the rubric be applied consistently by different scorers?

l Can the rubric be understood by students and parents?

l Is the rubric developmentally appropriate?

l Can the rubric be applied to a variety of tasks?

l Is the rubric fair and free from bias?

l Is the rubric useful, feasible, manageable and practical?


DAY

ONE

QUALITY ASSURANCE IN RUBRICS:

In everyday life we usually try to get the facts before we make a decision.The greater the amount of accurate information we can gather, the moreconfident we can be that we are making the right decision. The same thingholds true in the classroom when we assess students to get informationthat's used to decide what they know and can do, to diagnose their needs,to decide if we need to teach something in a different way, to planinstruction, to assign grades, to communicate with parents, and to guideus in the development of school improvement plans. The consequencesof making a poor decision can be very serious, so it's important to makesure that we're basing our decisions on an adequate amount of soundinformation. Or to put it another way we need assessment results that areboth reliable and valid for our particular purpose.

An assessment is reliable if it yields results that are accurate and stable.In order for a performance assessment to be reliable, it should beadministered and scored in a consistent way for all the students who takethe assessment.

An assessment is valid for a particular purpose if it in fact measures whatit was intended to measure. An assessment of a learning outcome is validto the extent that scores truly measure that outcome and are not affectedby anything irrelevant to the outcome.

Some important aspects of validity are content coverage, generalizabilityand fairness. The assessments for a given outcome should be alignedwith the both the outcome and instruction and, when taken together,should cover all important aspects of the outcome. The assessmentsshould address the higher-order thinking skills specified in the outcome.The tasks used should have answers or solutions that can't be memorized,but which, instead, call on the student to apply knowledge and skills to anew situation.

Assessment results are generalizable [to the extent that] if there isevidence that scores on one assessment can predict how well studentsperform on another assessment of the same outcome.

Once you decide on a rubric, the best way to promote reliable scoring is tohave well-trained scorers who thoroughly understand the rubric and whoperiodically score the same samples of student work to ensure that theyare maintaining uniform scoring.

Another way to increase reliability is to try hard to stick to the rubric as youscore student work. Not only will this increase reliability and validity, but


DAY

ONE

it's only fair that the agreed-upon rubric that's shared with students andparents should be what is actually used to rate student work.Nonetheless, human beings making subjective judgments mayunintentionally rate students based on things that aren't in the rubric at all.The conscientious scorer will frequently monitor his or her thinking toprevent extraneous factors from creeping into the assessment process.The box below contains a list of some extraneous factors to watch out for.

After you decide upon a rubric, you might find it helpful to reformat it in away that makes it more easily usable by you and your students. Forexample, an analytical scale may be easier to use or more meaningful if itis typed or summarized in tabular form.


DAY

ONE

PROBLEMS AND PITFALLS ENCOUNTERED BY SCORERS

Positive-Negative Leniency Error: The scorer tends to be too hard or tooeasy on everyon.Trait Error: The scorer tends to be too hard or too easy on a given trait,criterion or scale.Appearance: The scorer thinks more about how the paper or project looksthan about the quality.Length: Is longer better? Not necessarily.Fatigue: Everybody gets tired.Repetition Factor: This paper is just like the last 50. Order Effects: If you've just read 10 bad papers, an average one may startto look like Shakespeare by comparison.Personality Clash: It's tougher if you don't like the topic or the student'spoint of view.Skimming: Doesn't the first paragraph pretty well tell the story?(Hint: No.)Error of Central Tendency: Using an odd-numbered scoring scale? Bewarethe dreaded "mid-point dumping ground."Self-Scoring: Are you a perceptive reader? Be sure what you're scoring isthe writer's work--not your own skill.Discomfort in Making Judgments: Remember that you are rating thepaper, product or performance, not the student. This is just oneperformance assessment--not overall ability.The Sympathy Score: "The student was really trying..." "...seems to besuch a nice kid," "...chose a hard topic," "...had a tough day," etc.--Adapted from Culham and Spandel (1993)

REFERENCES:l Arter, Judith. Per formance Rubric Evaluation Form (Metarubric).

Portland, OR: Northwest Regional Educational Laboratory, September1990.

l Culham, Ruth and Spandel, Vicki. Problems and Pitfalls Encountered byRaters. Developed at the Northwest Regional Educational Laboratoryfor the Oregon Department of Education, 1993.

l Herman, Joan, Aschbacher, Pamela, and Winters, Lynn. A PracticalGuide to Alternative Assessment. Alexandria, VA: Association forSupervision and Curriculum Development, 1992


DAY

ONE


Exc

elle

nt

(10)

The

writ

er p

rovi

des

high

ly lo

gica

lar

gum

ents

. B

oth

(pos

itive

and

neg

ativ

e)as

pect

s ar

e pr

esen

ted

very

nic

ely.

The

writ

erha

s ve

ry c

lear

lypr

esen

ted

his/

her

ow

nst

ance

on

the

topi

c an

dha

s de

fend

ed it

suita

bly.

Pro

per

refe

renc

es a

re p

rovi

ded

for

the

argu

men

ts.

The

info

rmat

ion

pres

ente

d m

atch

esex

actly

with

the

requ

irem

ents

. It

isev

iden

t th

at

refe

renc

es a

re f

rom

varie

d so

urce

s an

din

clud

e a

good

num

ber

of n

ew o

nes

Typi

cally

cle

arbe

ginn

ings

, m

iddl

es,

and

ends

.N

o di

gres

sion

s ar

epr

esen

t.Tr

ansi

tions

are

sm

ooth

and

logi

cal.

Mec

hani

cal e

rror

s ar

eab

sent

.Id

eas

are

wel

lor

gani

zed

and

com

mun

icat

ed c

lear

lyan

d co

ncis

ely.

The

lear

ner

has

deve

lope

d a

high

lylo

gica

l pie

ce o

f w

ork

base

d on

cle

arly

defin

ed p

rinci

ples

/th

eory

. The

re s

eem

s to

be n

o co

nfus

ion

in t

hele

arne

r’s m

ind

as t

oho

w t

o us

e th

e th

eory

/pr

inci

ples

tha

t w

ere

prov

ided

. The

wor

kde

velo

ped

is h

ighl

yus

eabl

e.

Go

od

(08)

The

writ

er p

rovi

des

logi

cal a

rgum

ents

. B

oth

(pos

itive

and

neg

ativ

e)as

pect

s pr

esen

ted

are

bala

nced

. The

writ

erha

s tir

ed t

o pr

esen

t hi

s/he

r ow

n st

ance

on

the

topi

c an

d ha

s de

fend

edit

at m

ost

plac

es.

Arg

umen

ts a

resu

ppor

ted

by p

rope

rre

fere

nces

.

The

info

rmat

ion

pres

ente

d m

atch

es t

here

quire

men

ts.

It is

evid

ent

that

re

fere

nces

are

fro

mva

ried

sour

ces

and

incl

ude

a go

od n

umbe

rof

new

one

s

Gen

eral

ly w

ell

orga

nize

d ac

cord

ing

tode

finite

pla

ns.

Typi

cally

cle

arbe

ginn

ings

and

end

s.M

ost

tran

sitio

ns a

resm

ooth

and

logi

cal.

Mec

hani

cal e

rror

s do

not

conf

use

read

er,

but

in e

ach

com

posi

tion

ther

e m

ay b

e se

vera

lm

inor

err

ors

or o

ne o

rtw

o m

ajor

err

ors.

The

lear

ner

has

deve

lope

d a

cred

ible

piec

e of

wor

k ba

sed

oncl

ear

cut

prin

cipl

es /

theo

ry. T

here

see

ms

tobe

alm

ost

no c

onfu

sion

in t

he le

arne

r’s m

ind

asto

how

to

use

the

theo

ry/

prin

cipl

es t

hat

wer

e pr

ovid

ed. T

hew

ork

deve

lope

d is

high

ly u

seab

le.

Lev

el o

f ac

hie

vem

ent

Arg

um

enta

tio

n/

Just

ific

atio

ns

Co

nte

nt

Exp

ress

ion

(sty

le o

f w

riti

ng

)P

ract

ical

Ap

plic

atio

n

AN

ALY

TIC

AL

RU

BR

IC E

XA

MP

LE


Ad

equ

ate

(06)

The

writ

er p

rovi

des

logi

cal a

rgum

ents

.A

dequ

ate

bala

nce

ism

issi

ng in

onl

y a

few

case

s. T

he w

riter

has

trie

d to

pre

sent

his

/ he

row

n st

ance

on

the

topi

can

d ha

s de

fend

ed it

at

mos

t pl

aces

. S

uita

ble

refe

renc

es a

re p

rovi

ded

for

the

argu

men

ts in

mos

t ca

ses.

The

info

rmat

ion

pres

ente

d is

mos

tlyac

cord

ing

to w

hat

has

been

ask

ed. T

henu

mbe

r fo

r re

fere

nces

incl

uded

is b

arel

ym

inim

al t

o su

ppor

t th

ear

gum

ents

. La

test

refe

renc

es a

re in

suffi

cien

t nu

mbe

rs.

End

ings

may

som

etim

es b

e aw

kwar

dor

abr

upt.

Tran

sitio

ns a

re t

ypic

ally

logi

cal b

ut m

ay o

noc

casi

on la

ck d

epth

and/

or d

irect

rel

evan

ce.

Dig

ress

ions

may

be

foun

d he

re a

nd t

here

but

they

do

not

ham

per

the

flow

of

text

.

The

writ

er is

unc

lear

at

som

e pl

aces

reg

ardi

ngw

hat

prin

cipl

es/

theo

ryto

app

ly. T

here

is s

ome

conf

usio

n in

the

lear

ner’s

min

d as

to

how

to

use

the

theo

ry/

prin

cipl

es t

hat

wer

epr

ovid

ed. T

he w

ork

deve

lope

d is

vag

ue a

tso

me

plac

es o

nly.

At

som

e pl

aces

lack

s flo

wof

pro

cess

/ th

ough

t.M

ajor

ity o

f th

e id

eas/

prop

ositi

ons

are

usea

ble.

Inad

equ

ate

(04)

The

writ

er p

rovi

des

logi

cal a

rgum

ents

.B

alan

ce is

mis

sing

inso

me

case

s. T

he w

riter

pres

ents

his

/ he

r ow

nst

ance

on

the

topi

c bu

tfa

ils t

o de

fend

itad

equa

tely

. R

efer

ence

sar

e no

t pr

ovid

ed f

or t

hear

gum

ents

in m

ost

case

s.

The

info

rmat

ion

pres

ente

d is

at

plac

esac

cord

ing

to w

hat

has

been

ask

ed. T

henu

mbe

r fo

r re

fere

nces

incl

uded

is in

adeq

uate

to s

uppo

rt t

hear

gum

ents

. La

test

refe

renc

es a

rein

suffi

cien

t in

num

ber.

Pos

sibl

e di

gres

sion

s or

elab

orat

ions

are

conf

usin

g to

rea

der.

Som

e tr

ansi

tions

are

logi

cal.

Beg

inni

ngs

and

endi

ngs

may

be

awkw

ard

or a

brup

t.K

ey e

lem

ents

may

be

unev

enly

dev

elop

ed o

rom

itted

.M

echa

nica

l err

ors,

som

e m

inor

, so

me

maj

or,

whi

ch m

ay o

noc

casi

on c

onfu

sere

ader

.

The

writ

er is

unc

lear

at

mos

t pl

aces

reg

ardi

ngw

hat

prin

cipl

es/

theo

ryto

app

ly. T

here

isco

nfus

ion

in t

hele

arne

r’s m

ind

as t

oho

w t

o us

e th

e th

eory

/pr

inci

ples

tha

t w

ere

prov

ided

. The

wor

kde

velo

ped

is v

ague

at

mos

t pl

aces

. At

mos

tpl

aces

lack

s flo

w o

fpr

oces

s/ t

houg

ht.

Maj

ority

of

the

idea

s/pr

opos

ition

s ar

eus

eabl

e.

Lev

el o

f ac

hie

vem

ent

Arg

um

enta

tio

n/

Just

ific

atio

ns

Co

nte

nt

Exp

ress

ion

(sty

le o

f w

riti

ng

)P

ract

ical

Ap

plic

atio

n


Po

or

(02)

The

writ

er f

ails

to

prov

ide

any

argu

men

tat

all.

The

writ

er d

oes

not

pres

ent

his/

her

own

stan

ce o

n th

e to

pic

and

fails

to

defe

nd it

.R

efer

ence

s ar

e no

tpr

ovid

ed f

or t

hear

gum

ents

.

Lack

of

nece

ssar

yin

form

atio

n.To

pic

may

be

clea

r bu

tno

ove

rall

orga

niza

tiona

l pla

nsex

ist.

The

re a

re m

any

digr

essi

ons

or o

ver-

elab

orat

ions

or

little

deve

lopm

ent

alto

geth

er.

The

re is

littl

e se

nse

ofbe

ginn

ings

or

endi

ngs.

Man

y m

echa

nica

ler

rors

inte

rfer

e w

ithun

ders

tand

ing.

Inco

mpl

ete

sent

ence

sar

e pr

esen

t.

Inco

mpl

ete

sent

ence

sar

e pr

esen

t.T

here

see

ms

to b

e to

tal

lack

of

clar

ity t

o th

ere

ader

as

to w

hat

prin

cipl

es/

theo

ry h

asbe

en a

pplie

d. T

here

seem

s to

be

grea

tco

nfus

ion

in t

hele

arne

r’s m

ind

as t

oho

w t

o us

e th

e th

eory

/pr

inci

ples

tha

t w

ere

prov

ided

. The

wor

kde

velo

ped

is v

ague

,la

cks

flow

of

proc

ess/

thou

ght

and

does

not

seem

to

be u

seab

le.


Exc

elle

nt

(10)

Go

od

(08)

Lev

el o

f ac

hie

vem

ent

BLA

NK

RU

BR

IC T

EM

PLA

TE F

OR

PR

ACTI

CE


Ad

equ

ate

(06)

nad

equ

ate

(04)

Lev

el o

f ac

hie

vem

ent


Po

or

(02)

Yecf[j[dY[ - CPSPelogbook.cpsp.edu.pk/elogbook/supervisors_workshop... · 2011-03-16 · Contents...

Documents

Transcript of Yecf[j[dY[ - CPSPelogbook.cpsp.edu.pk/elogbook/supervisors_workshop... · 2011-03-16 · Contents...