Bloom's Taxonomysites.psu.edu/.../2010/04/Blooms-and-Question-Types.docx · Web viewAnalysis...

40
E-learning and Assessment Version 4. Reference http://wiscinfo.doit.wisc.edu/teaching-academy/Assistance/course/q uestions.htm The education strategy (March 2004) has highlighted that Bristol University:- 1. Is a “research intensive” university where the student learning experience is informed by research and is delivered by research active staff. 2. Wishes to retain its high quality teaching. 3. Needs to widen access and to look at diverse progression routes into HE. 4. Needs to expand postgraduate numbers – particularly on taught masters programmes 5. To retain their highly rated international research output and to deliver high quality teaching, requires that teaching is cost effective. Whilst E-learning has a relevance to all of these points, this paper will focus on point 5 i.e. the requirement for teaching to be cost effective. The contribution of E-learning to assessment is a major area where substantial gains in efficiency can be achieved. Currently staff spend a significant amount of time 1. setting examinations 2. ensuring that they match course learning objectives, 3. liasing with the external examiner 4. marking 5. exam related student administration Much of this activity is repeated at least twice per year and sadly the outcome is often only summative rather than formative. Several staff within the university have already adopted CAA (computer aided assessment) as a means of addressing this problem and have found pedagogic as well as efficiency gains. However this approach has not yet become embedded. Reasons for this limited uptake include:- 1

Transcript of Bloom's Taxonomysites.psu.edu/.../2010/04/Blooms-and-Question-Types.docx · Web viewAnalysis...

E-learning and AssessmentVersion 4.

Reference http://wiscinfo.doit.wisc.edu/teaching-academy/Assistance/course/questions.htm

The education strategy (March 2004) has highlighted that Bristol University:-

1. Is a “research intensive” university where the student learning experience is informed by research and is delivered by research active staff.

2. Wishes to retain its high quality teaching.3. Needs to widen access and to look at diverse progression routes into HE.4. Needs to expand postgraduate numbers – particularly on taught masters

programmes5. To retain their highly rated international research output and to deliver high quality

teaching, requires that teaching is cost effective.

Whilst E-learning has a relevance to all of these points, this paper will focus on point 5 i.e. the requirement for teaching to be cost effective. The contribution of E-learning to assessment is a major area where substantial gains in efficiency can be achieved.

Currently staff spend a significant amount of time 1. setting examinations 2. ensuring that they match course learning objectives, 3. liasing with the external examiner4. marking5. exam related student administration

Much of this activity is repeated at least twice per year and sadly the outcome is often only summative rather than formative. Several staff within the university have already adopted CAA (computer aided assessment) as a means of addressing this problem and have found pedagogic as well as efficiency gains. However this approach has not yet become embedded. Reasons for this limited uptake include:-

1. Lack of staff time2. Lack of priority by university managers3. Lack of reward4. Techno phobia5. Lack of knowledge/understanding of how e-learning can help. Specifically many

staff still believe that CAA is only capable of “simple multi choice questions” which test recognition.

6. A multitude of software packages which are evolving and for which support is varied. Academic staff whose interest is primarily the pedagogy of their own subject, find the technological developments both confusing and frustrating.

7. Academic staff perception that the technology is driving the pedagogy, rather than the student learning experience being informed by research and delivered by research active staff.

It is therefore essential that any development of CAA within the University of Bristol must address these issues. Specifically the technology must:-

1

654321

1. Match the needs of academic staff 2. Be relatively simple to use, be well supported and not evolve rapidly in the

short term. It is therefore important to get the specification right first time and not be continually updating it. The current design should therefore match not just current (short term) but also our future (medium term) needs.

3. Be capable of assessing at all levels of Blooms Taxonomy (see later)4. Fit within our portal (MLE)5. Support the latest technical interoperability standards to facilitate easy

transfer of content to another CAA application if required at a later date6. Be secure, both meet Data Protection/privacy requirements and have robust

mechanisms in place to deal with machine or network failures7. Be scalable - while any CAA might initially be implemented in one or two

areas it should have the potential to be used across the University

Before examining question and assessment types, it's beneficial to establish a common language for discussing cognitive processes. To assess specific levels of student mastery of course content, needs a framework for describing those levels. Bloom's taxonomy is a system created to improve testing precision by categorising cognitive functioning into distinct levels. Appropriate questions can then be developed to assess the desired level. Psychologist Benjamin Bloom developed this system at the University of Chicago in the late 1940s. His goal in formulating this classification was to increase precision in the discussion of educational goals among teachers, administrators, and researchers.

Bloom's levels are:

1. Knowledge 2. Comprehension 3. Application 4. Analysis 5. Synthesis 6. Evaluation

"Taxonomy" is simply a system of categorizing and organizing. In this case, the taxonomy is hierarchical; each level is subsumed by the higher levels. In other words, a student functioning at the "application" level has also mastered the material at the "knowledge" and "comprehension" levels. Before taking a closer look at the levels, it should be noted that Bloom's taxonomy is not the only system of classification for cognitive domains. However, it is a widely accepted system, and is useful in illustrating how certain question types are better choices for assessing different levels of student mastery. What's important is the idea that student knowledge is not all equal; there are levels of mastery ranging from simple recitation of facts, to formulating informed opinions regarding complex issues. An awareness of these levels can help one determine how well do students really know course content.

2

Table 1 : Blooms Taxonomy of Learning

Level Learner Action Question CuesKnowledge Recall content in the exact form that it was presented.

Memorisation of definitions, formulas, or procedures are examples of knowledge-level functioning

List, define, label, identify, name

Comprehension Restate material in their own words, or can recognise previously unseen examples of a concept.

Describe, associate, categorise, summarise

Application Apply rules to a problem, without being given the rule or formula for solving the problem.

Apply, calculate, illustrate, solve

Analysis Break complex concepts or situations down into their component parts, and analyse how the parts are related to one another.

Analyse, compare, separate, order, explain

Synthesis Rearrange component parts to form a new whole. Combine, modify, rearrange, "what-if"

Evaluation Evaluate or make judgments on the worth of a concept, object, etc. for a purpose

Assess, decide, grade, recommend, explain, judge

3

Introduction to Exam Question Types

1. True/False 2. Matching 3. Multiple Choice4. Short Answer5. Calculation6. Translation 7. Essay8. “Problem based”9. Comprehension - 'Gobbets'10. "Simulation testing"11. "Performance"

True/false, matching and multiple choice are examples of closed-ended questions. That is, they do not require the students to construct an answer themselves. Rather, the answer is among the possible options. Closed-ended questions test recognition; that is, they evaluate a student’s ability to recognize the answer. However questions that are built around a scenario, some data, a graph or a table of numbers can require students to ‘deal’ with the information in order to determine the correct response.

Open-ended questions include short answer and essay types. These questions require the creation of a written response. Open-ended questions test student recall; the student is asked to come up with the answer on their own, as opposed to picking it out from a list of potential options.

"Problem based", "simulation testing" and "performance" all assess the student's ability to undertake a process and therefore need to assess skills as well as knowledge. Traditional written examination formats clearly are not an appropriate form of assessment. E-learning provides an opportunity to develop assessment tools which are matched to these learning objectives.

1. True/False Questions

a. Advantages:True/false questions are among the easiest to write, and can be scored electronically.

b. Disadvantages:True/false questions are limited in what kinds of student mastery they can assess. They have a relatively high probability of student guessing the correct answer (50%). True/false also assesses recognition of information, as opposed to recall. However properly crafted True/False questions require students to recall relevant knowledge, reason or make some deduction about the mechanism and so arrive at their most likely correct response which hopefully is one of the choices.

c. Most Appropriate For:Factual information and naturally dichotomous information (information with only two plausible possibilities). Dichotomous information is "either/or" in nature. Examples include male/female, analog/digital, and internal/external.

4

2. Matching Questions

a. Advantages:Matching items can assess a large amount of information in a confined space on the exam page, relative to multiple-choice questions; if developed carefully, the probability of guessing is low. To decrease that probability further, avoid equal-sized lists by including a few “distractor” items in the second (answer) column.

b. Disadvantages:Matching assesses recognition rather than recall of information.

c. Most Appropriate For:Assessing student understanding of related information. Examples of related items include Symptoms and diagnosis, intervention and risks, terms and definitions, tools and uses, and events and dates.

5

TABLE 2 Question typesQuestion type Description Blooms levelsTrue/false Present a statement, and prompt the student to choose

whether the statement is truthful Knowledge Comprehension

Matching questions

Involve paired lists that require students to correctly identify, or “match,” the relationship between the items

KnowledgeComprehension

Multiple-choice

Ask students to choose from a list of possible answers. Most multiple-choice questions feature one correct answer, and two to four "distractor" choices that are not correct.

KnowledgeComprehensionApplicationAnalysis

Short-answer Open-ended questions that require students to create an answer. Short-answer items typically require responses of one word to a few sentences. “Fill in the blank” and “completion” questions are examples of short-answer question types.

KnowledgeComprehensionApplicationAnalysis

Calculation Present a statement which includes variables from a formula that is being assessed. The particular instance has variables replaced by specific numbers so the calculation can be performed. Can also test understanding of dimensions appropriate to the calculation. Can also handle inexact responses - so making students aware of the precision of answers.

Knowledge Comprehension ApplicationAnalysis

Translation Students are required to translate a passage from one language to another specified language observing the grammatical structure, idiomatic expression and literary tone of the piece.

Knowledge, Comprehension, Application, Synthesis

Essay Like short-answer, are constructed-response questions. However, essay answers are typically much longer than those of short-answer, ranging from a few paragraphs to several pages.

Knowledge Comprehension Application Analysis SynthesisEvaluation

Comprehension'Gobbets'

Students must respond to a passage or image, often without the prompt of a specific question. They will have to interpret (and sometimes identify) the passage/image and should be able to relate it to its wider context. These questions specifically test the student's ability to create answers and to make sensible links between specific items and wider academic debates.

Knowledge, Comprehension, Application, Analysis, Synthesis, Evaluation

“Problem based”

Students are asked to interpret information and make an informed decision as to what further information is required so that judgements, decisions and course of action can be decided upon.

Knowledge Comprehension Application Analysis SynthesisEvaluation

6

"Simulation testing"

Use of a simulation capable of testing (in a controlled way) a student's ability to carry out a process.

Knowledge Comprehension Application Analysis SynthesisEvaluation

"Performance" Examination by performance raises particular difficulties of assessment and documentation. Performance-based assessments test the student's interpretation and presentation of material and observation of the conventions of genre (dramatic genre, scholarly presentation etc). Students are also assessed on their ability to work as a group and to communicate to an audience.

Knowledge Comprehension Application Analysis SynthesisEvaluation

3. Multiple Choice Questions

a. Advantages:Multiple-choice questions are the most versatile of the closed-ended question types. This versatility stems from the fact that the questions can contain more elaborate scenarios that require careful consideration on the part of the student. The probability of student guessing is also relatively low.

b. Disadvantages:When compared to true/false and matching, multiple-choice items can be more challenging to write. They also require the creation of plausible “distractors,” or incorrect answer options. As with other closed-ended questions, multiple-choice assesses recognition over recall.

c. Most Appropriate For:Factual, conceptual, or procedural information. Unlike true/false and matching, multiple-choice is not limited to a particular type of information for which it is best suited. Because of the versatility of multiple-choice questions, the examples in this section will focus on demonstrating how an effective multiple-choice question can test for each of the first four levels of Bloom’s Taxonomy.

Knowledge LevelThe value that occurs most frequently in a given data set is the:a. Meanb. Standard deviationc. Moded. Median

The answer is “c.” This is a knowledge-level question because it requires only that the student remember the definition of mode.

Comprehension LevelWhich of the following measures involves ordinal data?

7

a. The score of a football gameb. Football player’s jersey numberc. A team’s rank in league standingsd. Number of fans in attendance

The answer is “c.” This is an example of a comprehension-level question, in that it requires the student to recognize a previously unseen example of ordinal data.

Application LevelThe standard deviation of the sample data 6, 9, 10, 12, 13 is closest to:

a. 3b. 5c. 8d. 10

The answer is “a.” This question assesses application-level mastery, or the ability to apply a rule or formula in a new context.

Analysis LevelSusan, a student in Mr. Smith’s statistics class, asks Mr. Smith what her average score is for the three exams he has given the class. He replies that her average is +1.7. Which of the following assumptions about the students’ scores on these tests is most plausible?

a. The standard deviations of scores on all three tests were similar.b. None of the tests produced extremely skewed distributions.c. All of the students did poorly on at least one of the tests.d. The correlations between the three sets of test scores were low.

Answer is “d.” This item requires the test-taker to recognize unstated assumptions and relationships. The test-taker has to recognize the steps Mr. Smith has taken (converted scores into Z-scores) and why he would do so (the tests probably have very unlike distributions or lack of variation in one or more tests).

4. Short-Answer Questions

a. Advantages:Short-answer questions assess unassisted recall of information, rather than recognition. Compared to essay questions, they are relatively easy to write.

b. Disadvantages:Short-answer items are only suitable for questions that can be answered with short responses. Additionally, because students are free to answer any way they choose, short-answer questions can lead to difficulties in scoring if

8

the question is not worded carefully. It’s important when writing short-answer questions that the desired student response is clear.

c. Most Appropriate For:assessing the correct spelling of items, or in cases when it is desirable to ensure that the students have committed the information to memory (medical students, for example, will require recall of information more than recognition by the nature of their jobs).

4. Calculation Questions

a. Advantages:For simple formula this is a very easy and obvious question type. It can generate a large set of potential questions automatically so each test can be different. Handling dimensions and marking that aspect separately is another useful feature. Method also allows for inexact responses so making student aware of imprecision.

b. Disadvantages:For more complex formula the capabilities might be too limited. Only suitable for formula based questions.

c. Most Appropriate For:Scientific material based on formulae that have a single solution.

5. Essay Questions

a. Advantages:Essay questions can effectively assess all six levels of Bloom’s Taxonomy. They allow students to express their thoughts and opinions in writing, granting a clearer picture of the level of student understanding. Finally, as open-ended questions, they assess recall over recognition.

b. Disadvantages:There are two main disadvantages to essay questions: time requirements and grading consistency. Essays are time-consuming for students to complete, and require careful instructions on the part of the test writer. Scoring can be difficult because of the variety of answers, as well as the “halo effect” (students rewarded for strong writing skills as opposed to demonstrated mastery of the content). Students may also plagiarise published work and present that as their essay. Currently available software can be used to screen student essays to deter plagiarism.

c. Most appropriate for:Assessment that cannot be accomplished with other question types. Because essays are the only question types that can effectively assess the highest levels of student mastery, they are the only option if the goal of testing is the assessment of synthesis and evaluation levels.Essay questions can be written for all of Bloom's taxonomy, but their true advantage is the ability to assess the highest levels. Because essay questions can be time-consuming for student to complete and difficult for instructors to grade, this question type should be chosen carefully. If the desired knowledge can be assessed with a different question type, it should be considered. However, if higher-level processing is a goal of a course (as it likely will be), well-developed essay questions are the key to accurate

9

assessment.

6. “Problem based”

a. Advantages :Similar to an essay question can effectively assess all six levels of Bloom’s Taxonomy. It is particularly suited to assessing how effectively students can apply knowledge gained in a novel situation. The student’s ability to make effective decisions is assessed and thus it is appropriate to Blooms level 5 & 6 (synthesis and evaluation). For example you can ask a question and the next question depends upon the answer given to the former. The result is an assessment based as much on the path taken through the questions as it is on the final question answer. It can identify if a student can choose a good strategy or not. You can get the student to enter into a dialogue with the computer, and record the responses. Time stress can be applied which has relevance in some areas.

b. Disadvantages :Difficult questions to write, requiring detailed data/information to be presented to the student in a structured way. The question is likely to need to integrate information from several sources and have several “pathways” open for the student to choose. (see figure 1)

c. Most appropriate for :Suited to differential diagnosis of clinical cases. Presenting problem can be presenting clinical symptoms, which requires provisional diagnosis (possibly more than one). This needs to be refined by clinical examination and laboratory tests (pathology, microbiology, biochemistry, histology, ECG, ultrasound, X ray etc.) The diagnosis once refined may need further tests before a conclusive diagnosis can be made. This assessment process is highly relevant to the principles of differential diagnosis and evidence based medicine.Once a diagnosis has been made then a suitable treatment needs to be decided upon and a prognosis made. Each of these steps can be assessed and ranked in order of importance. Negative marking will be required if an option is taken which would actually harm the patient.

d.

10

Presenting informationwhich defines the

problem

Additional informationrequested by the student

[To be self selected from several sources, but which is ranked in terms of significance. If inappropriate or excessive information is requested then student looses marks]

Figure 1 Outline of a problem based assessment

Provisional hypothesis

Modified hypothesis

Decision (several options)

Course of action choice

11

7. "Simulation testing"

a. AdvantagesMany things we teach actually are high level processes that must be learnt - interpreting, translating, writing, manipulating, etc. Simulation of the actual process can be a powerful tool for facilitating learning, and can be used for both summative and formative assessment. However the assessment of these processes can often be difficult and complex. The development of virtual reality tools and E-learning provides an exciting opportunity to overcome many of these problems and to be cost effective at the same time.

b. DisadvantagesSimulations can be costly to develop both in staff time and the cost of appropriate equipment. Currently the aerospace and defence world uses simulations (flight simulators) as part of both learning and assessment. But in this very expensive example, it is worth building a simulation because the cost of making a mistake on a real one is huge. This is not always (often) the case. There are cheaper ways of simulating however, a good example being the CD test currently used to test driving students hazard awareness skills. Another example being developed at Glasgow University is an interactive model used to test a students ability to pregnancy diagnose a cow.

c. Most appropriate for :Testing students skills, which currently is often not done because of the financial cost, demands upon staff time and safety implications.

8. "Performance"

a. Advantages Almost by definition the performing arts require both formative and summative assessment tools capable of making judgements about student learning outcomes based upon a performance or a piece of creative work. E-learning tools such as video or image capture can play an important role in storing the evidence of student outcomes (on which the assessment is based) and in supporting formative feedback.

b. DisadvantagesThere are few shortcuts for marking performance and elearning tools cannot automatically mark work as they can for multiple choice type questions for example. Performance marking can be seen as subjective and the marking criteria needs to be properly thought out. Performances are ephemeral events, so unless the performance is captured in some way, (for example with video recording equipment) the performance is lost. If it is a group performance, it can be difficult to mark individual students for what is essentially a group effort.

c. Most appropriate for : Performance tests practical application of theory and methodology, gauging creative interpretation of source material. Performance can also test communication skills, for example in medicine a students ability to communicate with patients (often actors) can be assessed. There are

12

parallels with science practicals, but in arts subjects such as music, it the performance itself that is assessed, rather than the results of the practical.

The University of Bristol/ILRT has been proactive both in developing itself and in buying in technologies capable of delivering on-line assessment (see attached appendix 2).

Whatever the decision as to which tools will be chosen for the next step, it will be important that they can be used to assess all 6 levels of Blooms taxonomy.

13

Appendix 1: Examples of CAA which evaluate differing levels of Blooms taxonomy

LEVEL 1: knowledge/recognition

Question: Newton's three famous laws of motion include these concepts (select three): 

1. Speed2. Velocity3. Mass4. Vector5. Energy6. Gravity

Question: For each of the following dates, match the event or person associated with it.   Question Items Answer Items

- 1. 1066 A. Norman Conquest of Britain- 2. 1100s B. Angkor Wat is built- 3. 1215 C. Magna Carta signed- 4. 1347 D. Bubonic Plague spreads in Europe- 5. 1455 E. Gutenberg's movable-type printing press produces the Bible- 6. 1517 F. Martin Luther initiates the Reformation- 7. 1664 G. Newton's theory of universal gravitation- 8. 1764 H. Mozart (aged 8) writes first symphony- 9. 1867 I. Japan ends 675-year shogun rule- 10. 1947 J. Gandhi's civil disobedience movement leads to an

independent India

14

LEVEL 2: Comprehension

Question: Indicate what class is this drug?

1. Alkyl phenol2. Steroidal anaesthetic3. Barbiturate4. Non-steroidal analgesic5. Muscle relaxant

Question: Which of the following statements is correct? 

1. stereotypies are repetitive behaviours that develop into redirected behaviours2. stereotypies sometimes develop from redirected behaviours and approach /

avoidance behaviour3. apathetic behaviour, redirected behaviour and aggressive behaviour are all

examples of stereotypies4. stereotypies only develop in animals that have already shown vacuum behaviours

15

LEVEL 3 : Application

Question: Indicate what is the effect of this suture pattern on the wound edges

i. Apposes edgesii. Inverts edgesiii. Relieves tension at edges

Question: Pet cats show hunting behaviour. According to control theory models of motivation involving feedback mechanisms, if the functional consequence of this hunting behaviour (food. is given to the cat, the behaviour should stop.  

True or False

Level 3/4 Application/Analysis

The following data were obtained from an experiment in which four diets were fed to growing rats. Three diets were formulated to contain protein (CP) entirely as casein, fishmeal or kidney bean meal. The fourth diet was protein-free. Food intake, faecal and urinary excretion were measured over 5 days. The following results were obtained.

Which diet has the highest true protein digestibilty

1. casein2. fishmeal3. kidney bean

Incorrect Answer Feedback Example of casein

where EFCP = endogenous faecal crude protein

16

Thus casein has the highest true protein digestibility

Question: The following data were obtained from a digestibility trial with growing cattle weighing 300kg.

Nutritive value of foodDry matter (DM) concentration 0.25Gross energy (GE) 18.8 MJ/kgDMNeutral detergent fibre 550g/kgDM

Animal dataFood intake 13.5/kg fresh weightFaecal output 1.22kgDM/day (GE = 15.2MJ/kgDM)Urinary output 150gDM/day (GE= 16.1MJ/kgDM)Methane output 160 litres (STP)/day (GE = 39.5kj/lSTP)

First calculate the ME value in MJ/kgDM of the feed and use it to calculate the ME intake. Use this value to determine the

Indicate ME intake : ME requirement ratio.

1.51.31.0 

 Incorrect Answer Feedback

Diet Metabolisable Energy concentration (MJ/kg/DM) is calculated from

which equals

thus ME concentration = 10.72

ME intake = food intake (fresh) x DM xME concentration

= 13.5 x 0.25 x 10.72 = 36.18

17

ME requirement = 0.5 x W0.75 = 0.5 x 3000.75 = 36.04

Therefore ME/Mem ration = 36.18/36.04 = 1.0

18

LEVEL 4 : Analysis

Question A sow is introduced to another sow on one occasion and shows no oestrous response but, a few minutes later, shows a strong oestrous response when introduced to a boar, even though the two are not allowed to mate. The same procedure is followed again a week later and this time the sow shows no oestrous response to the boar. These observations indicate that:  

1. oestrous behaviour is controlled by external causal factors only2. oestrous behaviour is controlled by internal causal factors only3. both external and internal causal factors control oestrous behaviour4. the identity of the boar strongly influences whether oestrous behaviour is shown

Question The graph below shows the relationship between levels of stereotypic circling behaviour and baseline circulating corticosterone observed in 20 randomly selected singly housed mice.

These results indicate that:  1. Circling behaviour helps the animals to cope with the environment by reducing

stress2. There is no discernible relationship between circling and corticosterone3. Corticosterone levels play a causal role in the development of circling behaviour4. Animals that show high levels of circling may be more stressed5. High activity could be linked to elevations in corticosterone levels

19

LEVEL 4/5 Analysis/synthesis

Question The following results were obtained from routine haematology / biochemistry (normal ranges appear after each value):

Total protein 60 g/l (57 - 78) Albumin 28 g/l (32 - 38)Potassium 2.5 mmol/l (3.5 -

4.5)White blood cells 4 x10^9/l (6 - 18)Neutrophils 1.5 x10^9/l (3 - 12)Haematocrit 0.58 (0.35 - 0.55)

Which of the following differentials is most likely in this case?

1. Hepatic failure2. Chronic renal failure3. Auto immune haemolytic anaemia4. Parvovirus infection5. Small intestinal foreign body  

  LEVEL 5/6 Synthesis/Evaluation

Question On day 1 of a study, 6 groups of captive otters (treatment A) are prevented for swimming for 7 days while 6 other groups are not (treatment B). On day 8, otters in treatment A are allowed to swim once more. The graph below shows how much time (mean and standard deviation. otters from the two treatments spent swimming on day 8. These results indicate that:

 

1. otters are highly motivated to swim and this motivation accumulates since they last swam

2. otters are motivated to swim20

3. deprivation of swimming is stressful for otters4. the control of otter swimming behaviour is unlikely to be as predicted by Lorenz's

psychohydraulic model

 

21

Question In an experiment on learning, 10 rats were injected with lithium chloride 2h after consuming their daily ration of familiar rat pellets. A further 10 rats were handled and given a sham injection. The rats were then replaced in their individual home cages with ad libitum familiar rat pellets and water. After 24 h they were given a choice between familiar rat pellets, and pellets flavoured with a novel but palatable substance. What happens? 

1. The lithium chloride stimulates the rats' appetite and the experimental rats eat more of both types of pellets than control animals.

2. The lithium chloride induces a temporary sickness and a specific food aversion for the familiar rat pellets is formed. In the test, only novel pellets are consumed.

3. The lithium chloride induces a temporary sickness but no specific food aversion learning takes place because the pellets were familiar. Rats eat the familiar pellets in the test.

4. No food aversion learning takes place, because the injection was given 2h after food consumption. For learning to occur the injection would have had to be given immediately after food consumption. Rats eat the familiar pellets in the test.

5. The lithium chloride induces a feeling of sickness that persists for 24h. In the test, nothing is consumed.

22

LEVEL 6: Evaluation

This example is produced using CALnet and currently can only be used for instruction since it does not include the facility for assessment. It is planned to provide this as an add-on. Basically it consists of a problem solving exercise using clinical material – in this case a horse presenting with symptoms of Colic.

Stepwise the student needs to :

1. make a provisional diagnosis based upon the presenting symptoms

2. carry out a clinical examination – by asking pertinent questions of the software. In the assessed version (to be developed) students will be assessed on the amount of irrelevant information they request.

Material that can be provided to the student can be quite diverse in its format including, text, numeric data, audio, pictures and video.

3. make a diagnosis

4. Decide upon treatment

5. Assess likely outcome of the case.

The following screen shots give a flavour of how the case looks for a student.

23

24

25

The following are the answers provided in pop-up boxes when individual treatments have been selected : Note some are appropriate, others would be bad news for the case!

26

AnalgesiaAnalgesia is always indicated in colic to control pain and reduce self-inflicted injury and exhaustion. The pain of impaction colic tends to be intermittent, so that analgesics are usually given intermittently to effect. ProkineticsDrugs which promote peristalsis are contraindicated because in the presence of an obstruction they tend to increase pain and, at worst, they may cause gut rupture. Liquid ParaffinLiquid paraffin is a mild laxative which softens the mass and lubricates the gut. It is delivered by nasogastric tube in large volumes, twice daily if necessary, until the mass starts to break down and liquid paraffin appears in the faeces. Because of its viscosity it is easier to deliver a suspension of the oil in warm water or 0.9% saline (50:50 volumes). This may be done under gravity using a funnel or by the cautious use of a pump. Volume guide: pony 2-3L (total); horse 4-5L; large horse 6-7L. Intravenous FluidsIntravenous polyionic fluids may be useful in the first day or two of treatment to ensure proper hydration of gut tissues. They are positively indicated in dehydrated patients.AnthelminticsAn optional treatment but it is unlikely that parasites are implicated here. Nevertheless, the presence of strongyle larvae and tapeworms may interfere with normal gut motility. The mare's history recounts use of a proprietary benzimidazole every six weeks to control alimentary parasites, but at normal therapeutic doses BMZs are not effective against migrating strongyle larvae or tapeworms. If considering anthelmintics as part of colic therapy, use products which are effective against strongyle larvae and tapeworms. AntibioticsIf the impaction were associated with peritonitis, then antibiotics would be indicated. However, they are not indicated in the clinical work up of this case.Teeth raspingThe teeth should be examined as part of the clinical work up. Sharp points on the cheek teeth can be associated with inadequate chewing of hard feed, which can predispose or exacerbate impactions. Teeth rasping should be considered. Pathological changes, particularly vascular compromise of the gut, is reflected in the colour changes seen in peritoneal fluid. In this case: normal clear yellow fluid

27

3. Progress and conclusion of the caseThis mare was hospitalised on a bedding of shavings and treated with low doses (one-third doses) of flunixin as required. Five litres of liquid paraffin in warm 0.9% saline were given twice daily by nasogastric tube. No food was offered for 36 hours, after which wet bran mashes were given twice daily. After 48 hours scant hard faeces were produced, followed by a larger volume which was seen to contain liquid paraffin. At this point the administration of liquid paraffin was stopped and grass was introduced into the diet as a soft feed. No signs of colic recurred. Two days later the faeces showed an absence of liquid paraffin and a rectal examination confirmed disappearance of the impaction. Hay was gradually introduced into the diet over a period of three days, during which time the mashes were continued to regulate the faecal consistency. Thereafter the usual diet was introduced and the clients were warned not to use straw bedding again. No further problems were reported at a six month follow up.

28

Appendix 2 : Comparison of various CAA tools

One of the key elements in any eLearning strategy will be the tools to deliver eLearning programmes or tutorials. A variety has been used within the University to date (and the following list is not exhaustive) partly because of ad-hoc experimentation but more importantly due to cost and because one single product has not been found to meet all needs.

Blackboard - has the all important element of central University support but its eLearning capabilities consist of the ability to store documents or the means to deliver quizzes and questionnaires which are limited in their format and content and which only allow results for the most recently taken quiz to be stored. However, Blackboard and other popular VLEs do not specifically support eLearning content creation - merely its delivery (i.e. VLEs, to a large extent, assume a priori existence of eLearning content).

TAL - is a Bristol-produced computer-based assessment package capable of producing online and offline tests. The online tests can be authored and delivered in both Blackboard directly or via CALnet. TAL focuses on summative assessment and does not allow inclusion of standalone questions inside formative eLearning content.

CALnet - enables non-technical staff to create interactive eLearning content in the form of online tutorials that support a much broader pedagogy than the minimal built-in features of VLEs. However, CALnet currently lacks any form of student tracking or feedback to the lecturer. CALnet was developed at ILRT and is available free of charge to University of Bristol staff.

CalScribe - is an offline tutorial authoring tool on top of the commercial off-the-shelf ToolBook multimedia authoring tool and as such brings a high degree of technical and pedagogical flexbility but at the unrealistic cost of requiring academics to become novice programmers. Every user requires a ToolBook licence in order to use CalScribe. LTSS has helped CalScribe users to switch to CALnet but at the loss of some authoring features.

WinEcon - eLearning tools built on top of CALnet for the WinEcon II project, feature richer interaction modes and question types than either CALnet, CalScribe or Blackboard but currently in a form that is specific to the WinEcon II project; the WinEcon tools have never been ported back to CALnet for wider use by the university.

The ideal eLearning tool would include at least the following characteristics: Free at the point of use to staff and students (at least the perception of being free

for the individuals involved) Delivered via the Web and usable across all commonly used browsers Enable creation and deployment of online programmes of work that lead students

through a mixture of text, images (still and moving), links to online resources, questions and feedback

Provide the option to have questions and quizzes scored and have these results stored over time

Tutorials/Assessments which are repeatable… o Students can take a tutorial more than once and can compare their scoreso Tutorials can be copied by staff to form the basis of a new tutorial

Allows lecturers to review their students' tutorials/assessment

29

Conform to the emerging standards for interoperability of content (IMS, SCORM) so that no one is tied to a particularly piece of software as tutorials can be moved between software packages

Conforms to guidelines and legal requirements for accessibility and usability Allows local customisation of tutorials to the extent that the look and feel can be

altered to reflect departmental styles but not to the extent that accessibility and usability are compromised

Adequately supported by online documentation (downloadable documents and/or online help) for both tutorial authors and students to the extent that direct training is not required

Can be hosted centrally so that both tutorials and results are held on a secure server whose data is regularly backed-up

Can be technically supported by the University with no, or very minimal, increase in IT staff or hardware

Can potentially be linked to University databases (e.g. Datahub for student records)

Ideally within the technical control of University staff (open source / in-house developed) to better facilitate enhancement and extension of features

30