Charlton perez pbl_met

Introducing Problem Based Learning Approaches in Meteorology

Andrew Charlton-PerezDept. of Meteorology, Univ. of Reading

Submitted in partial fulfillment of the Postgraduate Certificate in Academic Practice

January 2011

Abstract

Problem based learning, despite recent controversies about its effectiveness, is used extensively as a teaching method throughout higher education in the UK and at the University of Reading. In Meteorology, there has been little attempt to incorporate PBL techniques into the curriculum. Motivated by a desire to enhance the reflective engagement of students within a current field course module, this project describes the implementation of two test PBL activities. By the end of a two-year program of design, implementation, testing and reflection/reevaluation two robust, engaging activities have been developed which provide an enhanced and diverse learning environment on the field course. The results suggest that PBL techniques would be a useful addition to the Meteorology curriculum and suggestions for courses and activities which may benefit from this approach are included in the conclusions.

Contents

1. Introduction to the problem and existing course design1.1. The problem - passive engagement of students1.2. Test module - Atmospheric Science field course1.3. Assessment of current course design1.4. Potential approaches to the problem considered

2. Problem based learning2.1.Broad advantages and disadvantages2.2.Implementation in higher education

3. Proposed changes to module3.1. Introducing two problem based learning elements3.2. Ozonesonde launch3.3. Climate monitoring station design3.4. Revised course design

4. Methodology of implementation and assessment4.1. Timetable of implementation and practical considerations4.2. Assessment methods

5. Results from implementation in 20095.1. Raw survey results5.2. Reflection on student experience5.3. Informal consultation with colleagues5.4. Consistency of evaluation using all three evaluation methods5.5. Proposed changes to activities

6. Results from implementation in 20106.1. Raw survey results6.2. Reflection on student experience6.3. Consistency of evaluation using all three evaluation methods6.4. Informal consultation with colleagues

7. Summary and discussion of impacts7.1. Module specific success of PBL activities7.2. Wider implications for teaching of Meteorology7.3. Potential areas in which small PBL projects would be beneficial7.4. A proposal for an improved undergraduate final year project/dissertation7.5. Reflections on course design in general

1. Introducing the problem and existing course designThis project will assess both the feasibility and usefulness of problem based learning approaches in Meteorology teaching. Two new problem based learning activities will be introduced to an existing fieldwork based Meteorology module. The activities are designed in line with best practice guidelines for problem based learning but are designed to be sufficiently different that conclusions about the overall suitability of problem based learning for Meteorological teaching can be drawn. The success of the new activities will be assessed using a combination of student feedback, peer observation, analysis of resulting student outputs and personal reflection. Given the small number of students involved in the activity designing a range of evaluation techniques is crucial to measuring itʼs success. A control group of students will take part in both activities so that they can be compared.

1.1 The problem - passive engagement of studentsMeteorology as a subject has a strong practical, experimental component. Teaching students how to make effective measurements and how to use the data collected appropriately is a key part of our undergraduate curriculum which also provides a strong transferable skill. Although a large element of practical work is included in the University of Readingʼs Meteorology and Climate BSc and MMet programs, in its current form much of this teaching follows a relatively traditional model of several self-contained experiments with well defined expected outcomes known prior to students conducting the experiments. While this approach has definite value to students, it fails to allow students to address key components of the most widely held view of experiential learning, the Kolb learning cycle (Kolb, 1984). This model of experiential learning is reproduced below:

Figure 1: Kolb learning cycle

1.2 Test module - Atmospheric Science field courseThe module chosen to test the implementation of PBL approaches in Meteorology is MT37H/MT4XH Atmospheric Science field course. This module is jointly taught with colleagues from the University of Leeds. The course is residential and takes place over 8 days based at a field centre on the Isle of Arran. Typically there are around 35 students on the course, split 50:50 between students from Reading and Leeds. The course is offered at both level 6 and level 7. The background of students on the course is diverse, with a wide range of mathematical skill in particular a major challenge. Activities on the course are primarily field based and include an all day hike to the top of Goat Fell (~850m) taking measurements on the way. The course is worth 10 credits and for students on the level 6 version all analysis and commentary on the results is completed in the field.

The traditional approach to practical experimental learning incorporates only the active experimentation and concrete experience stages of the Kolb learning cycle. A continuing challenge is to incorporate the aspects of more reflective observation of what has been

1. Introducing the problem Problem based learning approaches in Meteorology

Andrew Charlton-Perez 1

Concrete experience

Active experimentation Reflective observation

Abstract conceptualization

learned in the experiment and to then use that observation to form more general abstract conclusions about how the atmosphere works. To try to tackle this problem, in 2006 the atmospheric science field course was introduced to the BSc program. I have been a member of staff on this course since its inception and the co-convener of the module since 2008. On this field course, students have the opportunity to participate in several different experiments at once, allowing them the opportunity to try to piece abstract concepts about the atmosphere together. However, a remaining problem on the course is that all the experiments have been designed by the staff participating to have simple outcomes, known at the outset by both staff and students. Therefore, the reflective observation link in the Kolb learning cycle chain is often opaque or broken, making it difficult for the students to move to higher-level abstract conceptualization. In addition, from 2008 the course has been offered at both the BSc and MMet level, without adequate provision for students at the M-level to further enhance their learning through more detailed abstract conceptualization. The problem for the course, and by extension the BSc and MMet programs in general can therefore be summarized as:• There is not adequate opportunity for students to reflect upon the experimental choices made in practical meteorology courses. This leads, through truncation of the Kolb learning cycle, to little or no fundamental abstract learning about appropriate generic experimental design, a key transferable skill.• Additionally, there is currently no existing pathway for gifted and talented students, generally registered on the MMet program, to progress their learning along these lines. The lack of this pathway prevents the MMet from providing adequate training at the M-level suitable for students who may progress to higher academic scholarship.

1.3 Assessment of current course designTo fully examine the current structure of the course and the way that itʼs current structure maps to the Kolb learning cycle a course map (Conole, 2010) was completed. Mapping the course in this way provides a concise summary of its current state and highlights the issues discussed in the previous section. Since the test module is made up of a series of discrete activities, it has also been possible to map these activities to the Kolb learning cycle. As described above this identifies the lack of activities which both follow the learning cycle model and those which miss crucial steps, particularly the opportunity for reflection.

A video diary describing the initial mapping of the course and the problem at hand can be found at: http://cloudworks.ac.uk/cloud/view/3813. By mapping the course in this way, additional issues associated with the course are highlighted or emphasized:• The lack of opportunity for reflection in the course is clear, only the weather forecasting activity (blue dot) provides ways for students to examine their own work or put it in the context of others work. As a consequence many of the activities ʻshort-circuitʼ the Kolb learning cycle.• Along with this lack of reflective elements, no opportunity is provided to the students for formative feedback on their work. While the high staff-student ratio on the course does allow staff to informally have a dialogue with students to improve their understanding, there is no way for students to gain feedback on their written work, which is in some ways a more concrete demonstration of their understanding.• The Weather forecasting activity is clearly quite separate and distinct from the other activities on the course and provides a good way of introducing a variety of teaching styles. However, since many of the other activities are quite similar, an additional activity which filled some of the gaps identified by this analysis would be beneficial in further broadening the teaching of the module.



http://cloudworks.ac.uk/cloud/view/3813


Figure 2: Course map for Atmospheric Science Field Course as of December 2008 when the project began. Coloured dots in each panel correspond to the activities

identified in the top right of the figure.

Figure 3: Mapping of individual course elements onto Kolb learning cycle. Colours are the same as those used in the course map.



1.4 Potential approaches to the problem consideredTo address the problems identified in the previous sections and highlighted in the Atmospheric Science Field Course by the course mapping exercise several possible changes to the module were considered.

Modification Advantages Disadvantages

Pre-trip exercise: Ask students to complete an exercise on existing exercises before arriving on site (Carnduff and Reid, 2003). Ask students to reflect on their answers once on site and completing exercises.

• Engage students in practical before start of exercises.• Little additional resource cost.• Some chance for reflection and formative feedback.• Opportunity to add exercises at both level 6 and 7.

• Difficult to implement practically since course is taught during vacation time and across two institutions.• Ensures participation in exercise, but may not particularly inspire students. • Existing experiments are not well suited to major modification by students, hence limiting the chance for reflection.

Peer assessment: Ask students to assess work from other group members midway through the course.

• Little additional resource cost.• Some chance for reflection and formative feedback from peers.

• Little additional benefit for student engagement.• Hard to implement at different levels for Masters students.• Impractical, all students have not participated in every exercise by the middle of the week and would add additional time pressure which may be unhelpful.• Hard to match prior expectations of students from two different Universities.



Modification Advantages Disadvantages

Full problem based learning approach: Radically shift the focus of the course by first setting students a problem to solve during their field work and then providing resources/staff time for them to design experiments to solve the problems.

• Allows students great flexibility to explore atmospheric measurement and time for reflection.• Encourages student motivation.• Ability to tailor activity to ʻreal-worldʼ examples - improved employability for students.

• Impractical, a much higher staff to student ratio would be required to ensure equipment could be used both safely and effectively. Large additional cost.• Unlikely, given the timescales involved, that students would achieve meaningful results during their time on Arran.• Hard to determine the level of the course and to offer the activities at different levels for Masters students.• May not deliver enough core content for students.

Hybrid problem based learning approach: Maintain current course content, but introduce a small number of more flexible problem based learning elements.

• Allows students some flexibility to explore atmospheric measurement and time for reflection.• Moderate additional resource cost.•Encourages student motivation.• Ability to tailor activity to ʻreal-worldʼ examples - improved employability for students.

• Maintains some activities which do not fully complete the Kolb learning cycle.• Only limited opportunities for reflection and formative feedback.

Of the options considered, it was clear that the hybrid problem based learning approach represented a compromise between a desire to add more flexible, open-ended content to the module which allowed appropriate opportunity for reflection and formative feedback and practical considerations of how to deliver a broad and successful field course in a remote location with students and staff from two different institutions. It is also the case that a hybrid option has a comparably low risk profile compared to some of the other options, since in the instance of complete failure of the exercise, the PBL element can simply be removed from consideration when assessing students. By adopting a cautious approach to the implementation of PBL elements, it will be possible to rigorously test their success in comparison to the more traditional elements of the course.



2. Problem based learning

Problem based learning (PBL) is an approach to teaching and learning which forms part of a broader spectrum of techniques known as enquiry based learning. Enquiry based learning can be broadly defined to have the following characteristics (Kahn and OʼRouke,2004)• Engagement with a complex situation or scenario that is sufficiently open ended to allow a variety of responses or solutions• Students direct the lines of enquiry and the methods employed• The enquiry requires students to draw on existing knowledge and to identify their required learning needs• Tasks stimulate curiosity in the students, encouraging them to actively explore and seek out new evidence• Responsibility falls to the student for analysing and presenting that evidence in appropriate ways and in support of their own response to the problem.

PBL in particular involves students addressing a problem in a small group and defining the further knowledge and investigation that they require to solve the problem. In many ways PBL is as much about identifying the key unknowns in a problem and appropriate ways to tackle these problems as it is about solving the problem at hand. The PBL approach to learning does not require students to have mastered a body of knowledge before the completion of a project (as in a typical undergraduate or masters dissertation) but allows the understanding of the student and their ability to solve the problem to evolve together.

2.1 Broad advantages and disadvantagesKahn and OʼRouke (2004) list a large number of potential advantages of PBL as a teaching style particularly associated with student motivation and engagement and employability. As they identify “...the modern “knowledge economy” places a premium on the ability to create relevant knowledge that helps to solve specific problems...”

PBL provides a way of encouraging students to participate in constructive, experiential learning, as in the Kolb learning cycle (Fig. 1). This happens by encouraging students to engage in active experimentation to test their ideas and then using their experience of the outcomes of their experimentation to reflect on their grasp of the knowledge at hand. This reflective element is particularly important and can be enhanced in the PBL model by the chance for students to contrast their own performance and knowledge with that of their peers.

Despite these widely accepted benefits of PBL in the educational literature, there is current controversy over the effectiveness of minimally guided techniques in general. This controversy links to the paper of Kirschner, Sweller and Clark (2006, KSC06) who make the case that minimally directed techniques are incompatible with our knowledge of human cognitive architecture (in particular the Atkinson and Shiffrin (1968) sensory memory–working memory–long-term memory model). KSC06 argue that since the capacity of working memory is limited, placing heavy demands on it by requiring problem-based searching should be avoided. KSC06 argue that numerous studies have suggested that a more directed learning approach, particularly incorporating numerous ʻworked-examplesʼ is a more efficient use of novice and intermediate learners cognitive resources. Several responses to KSC06 exist in the literature (Schmidt et al. (2007), Hmelo-Silver et al. (2007), Kuhn (2007)) along with a commentary on these responses by the original authors of KSC06 (Sweller et al. (2007)). Common to the discussion between most of the authors is the idea that PBL techniques without any guidance are inferior to those with some

2. Problem based learning Problem based learning approaches in Meteorology


strong scaffolding provided by the course leader. They also agree that much more careful research with properly controlled experiments is required to fully assess the advantages and disadvantages of different educational techniques.

In practical terms, much of the discussion of the advantages and disadvantages of minimally guided techniques is focussed on rather fundamentalist positions of fully guided or fully unguided teaching. In reality, any implementation of PBL in Meteorology and in the Arran field course in particular is likely to exist somewhere between these extremes with some guidance provided by course tutors. It should also be recognised, however, that PBL techniques may be more appropriate for intermediate and advanced learners and hence for course at levels III and IV rather than levels I and II.

Despite the controversy about PBL techniques in the literature it seems appropriate to investigate their usefulness in the Meteorological context, provided that this is within a course with a range of different instructional techniques including directed learning. In this way PBL techniques can be evaluated but at low potential detriment to students involved in the course if they prove to be of limited value.

2.2 Implementation in higher education and in MeteorologyVarious reviews of the implementation of problem based learning approaches in higher education exist in the literature (e.g. Boud and Feletti, 1997, Savin-Baden 2000). Even a cursory glance at these texts reveals three things about the implementation of PBL in higher education: • PBL has been used to refer to a broad range of educational activities from the design of an individual element of a problem class to the design of a full three-year curriculum.• The implementation of PBL varies greatly between different subjects in UK higher education. Those with a strong element of practical problem solving (e.g. Medicine and Law) have been by far the most enthusiastic adopters of PBL.• A barrier to the implementation of PBL more widely is the lack of understanding amongst academic staff on their role within a PBL exercise.

There has been little implementation of PBL techniques in Meteorology or in related Earth and Environmental science fields. Some literature on the implementation of PBL in GEES subjects is available in a special edition of Planet (http://www.gees.ac.uk/planet/index.htm#PSE2). Of the articles in this issue, the most relevant is that which describes the implementation of PBL on a field course module by Perkins et al. A particularly interesting aspect of this article is the adoption of the ʻSeven-Jumpʼ Maastricht model for PBL tutorials (Gijselaers, 1995). This provides a framework model for tutorial structure for PBL activities which is adopted in the two new activities introduced in section 3 (with some modification for activities which take place entirely on Arran). This model characterises PBL learning as a series of seven ʻjumpsʼ:



http://www.gees.ac.uk/planet/index.htm#




Jump Activity Timing

1 Clarify terms and concepts not readily comprehensible Meeting 1

2 Define the problem

3 Analyse the problem and offer tentative explanations

4 Draw up an inventory of explanations

5 Formulate learning objectives

6 Collect further information through private study Between Meetings

7 Synthesise new information and test it against original problem. Reflect and consolidate learning

Meeting 2

Perkins et al. report that PBL was had a generally positive impact on the field activities and was equally at home in ʻhard-scienceʼ subjects (although as above, clear tutor guidance was a key factor in its success). One major difference between our own field course and that of Perkins et al. is the length of preparatory time which is long (16 hours) in the case of Perkins et al. and relatively short in our case (1 hour).

2.3 PBL and work-related learningAlong with itʼs benefits for student engagement, motivation and reflection PBL also has the advantage that it fits well within a work-related learning context. As identified by Moreland (2005), work-related learning is rapidly becoming an important part of higher education, particularly with the recent focus on ʻemployabilityʼ (Yorke, 2004). Yorke defines employability as:“a set of achievements – skills, understandings and personal attributes – that makes graduates more likely to gain employment and be successful in their chosen occupations, which benefits themselves, the workforce, the community and the economy.”Improving the employability of graduates from higher education is key to ensuring both that our courses offer students value for their substantial investment in their future and in maintaining the competitive position of the UK economy in the face of increased international competition and change. Moreland identifies key areas of work-related learning which can help contribute to graduate employability:• Learning and practising skills and personal attributes of value in the world of work (Skillful Practices);• Experiencing the world of work (or facsimiles thereof) to provide insights and learning into the world of work predominantly associated with the subjects of oneʼs higher education studies (Understandings); and• Experiencing and learning how to learn and manage oneself in a range of situations,

including (of course) those to be found at work (Metacognition).The prime example of this work-related learning already present in the Atmospheric Science Field Course is the Weather Forecasting activity which is designed to be a facsimile of the real-world forecasting problem. PBL activities are an additional way to introduce some of these elements into the Meteorology curriculum by providing a facsimile of a real-world research or operational problem for students to address. In designing new PBL elements for the course, it is important to remember the fit to these work-related learning goals.



3. Proposed changes to module3.1 Introducing two problem based learning elementsWith the key messages of the proceeding literature in mind, two similar but different PBL approaches were introduced into the atmospheric science field course module. The first of these PBL activities involves students on both the BSc and MMet programs and students from our partner the University of Leeds. It focuses on trying to address issues of missing stages in the Kolb learning cycle outlined above. The second activity involves only those students on the MMet program and is completed over a longer period upon return to Reading. The aim of this activity is to provide a second M-level route to obtaining appropriate professional skills in environmental monitoring.

3.2 Ozonesonde launchThis activity involves the design of an experiment to launch an ozonesonde, a piece of equipment attached to a weather balloon, designed to measure ozone concentrations throughout the atmosphere. The experiment is a key integrative part of the field course module since it combines many of the concepts of atmospheric chemistry and physics dealt with in other parts of the course. Students are already be part of mixed University of Reading/University of Leeds teams for other activities. The students are told that we only have the resources to launch a single ozonesonde and that they should design an experiment to maximize the benefit of observations from a single launch.The activity proceeds as follows:• The activity is introduced in a short lecture and through course documents. Some information about ozone in the atmosphere is given along with some technical details about the equipment available for use.• Students break into teams to discuss how and when to launch the ozonesonde. They have access both to staff (who act as facilitators) and forecast information about future weather conditions which determines when an interesting time to launch would be. (This is the initial abstract conceptualization phase).• Students are asked to write a short ʻgrant proposalʼ for the launch. The proposals are then be presented to a steering committee of staff who assess which of the proposals to take forward. (This is the first part of the active experimentation phase).• The ozonesonde is launched according to the instructions of the successful bid and data provided to all of the groups to analyze. (This is the second part of the active experimentation phase).• Following the launch students analyze both the data produced by the experiment but also the differences between the winning bid and their own. They comment on the differences between their bid and the winning bid and identify any deficiency based on the results of the experiment. This part is crucial since it requires the students to enter the reflective phase, based on the experimental design and to build this reflection back into their original abstract conceptualization. A detailed description of the activity is attached to this document as appendix I.

3.3 Climate monitoring station designThis activity takes place following the return of students on the MMet program from Arran. This is a relatively small group of students, but all members are common with activity 1 giving an opportunity to compare and contrast results. The problem given to the students is to design a new climate monitoring station for Arran based both on their experience of the field course location and meteorology and further original research from existing literature. The activity is facilitated by the module convener and two members of Meteorology research staff in three one-hour discussion sessions.

3. Proposed changes to module Problem based learning approaches in Meteorology


Students are asked to produce a 15 page design specification for the climate monitoring station detailing equipment used, fit to national and international monitoring priorities and operating procedure. This activity is designed to be much more open-ended than activity I. The first task of the students will be to decide on the priorities for the climate monitoring based on their own analysis of the literature and discussion in group forum. In this sense the activity specifically targets the reflective observation and abstract conceptualization elements of the Kolb learning cycle, whilst using the observational experience gained on Arran as the active experimentation and concrete experience phases. The final assessment of the design specification emphasizes these aspects.

A detailed description of the activity is attached to this document as appendix II.

3.4 Revised course designFigure 4 shows the revised course design incorporating the new PBL elements. Notice that many of the gaps in the previous design, particularly the lack of reflective elements and formative assessment have been filled by incorporating the new PBL elements. A key part of this design is making the assessment of the initial research proposals formative rather than summative since this allows for reflection on the part of the students in their workbooks based on our assessment of their work. By making the reflective element part of the summative assessment we ensure that students value and take part in the reflection. A similar approach has been adopted for the observing system activity by requiring students to actively criticise each others work in the tutorial sessions which support the activity (although this reflective element is not included as part of the assessment). PBL activities, by their nature, map well onto the Kolb learning cycle (Figure 5). The more open-ended questions that they pose mean that students have to refine their understanding through testing, particularly while refining the approaches to tackle the problem at hand.



Figure 4: Course map for Atmospheric Science Field Course as of December 2009 following the implementation of new activities described in section 3. Coloured diamonds

show new activities. Blue text shows additional contribution to course elements.

Figure 5: Mapping of 2009 course to Kolb learning cycle. New elements are included as cyan and brown arrows. Notice that these elements map a complete Kolb cycle.



4. Method of implementation and assessment4.1 Timetable of implementation and practical considerationsDesign of the new PBL methods took place during academic year 2008/9 and was introduced into the course in Autumn 2009. The experiment was then repeated with some modification in autumn 2010. One practical difficulty in the implementation in the first year was that I was on paternity leave at the time of implementation of the ozonesonde activity and did not take part in this exercise. All equipment required for the ozonesonde activity was tested before deployment and produced good results. One difficulty encountered in designing the ozonesonde activity was the very broad background knowledge of the students. Hence, it was thought likely that the background material provided in the course handbook would need to be refined substantially after first implementation and that course staff might be required to give more guidance than for a normal PBL activity in this first instance.

4.2 Evaluation methodsWith any new teaching and learning activity a crucial part of its successful introduction is a robust evaluation (Fry, Ketteridge and Marshall, 2008). Project evaluation was conducted using a range of techniques including student feedback, peer observation, analysis of resulting student outputs and personal reflection.

Student feedback was obtained through a carefully designed diagnostic questionnaire (Gibbs, Habeshaw and Habeshaw, 1988) which specifically explores the distinctions between the PBL approach and more traditional approaches used for the majority of the field course. A similar diagnostic questionnaire was applied to both activities and some questions were added to the questionnaire for activity II to specifically explore the differences between the two activities. In particular these questions explored if a more open ended and longer activity was preferable for the control group of gifted and talented students.

Peer observation from other staff was also easy to implement since both activities take place within a staff intensive environment. Feedback was obtained broadly through a separate diagnostic questionnaire and through an unstructured interview with colleagues. Again the emphasis was on which aspects of the PBL approach work well within a meteorological context.

The third stream of evaluation was through examination of student outputs for each activity and personal reflection from this perspective. Since a set of clear goals for the improvement of the teaching and learning experience has been drawn it should be possible to qualitatively measure student outputs against them.

4.2.1 Student and staff questionnairesFor each of the activities, feedback was sought from both staff and students with a carefully designed questionnaire. Example, blank questionnaires are shown in Appendix III. The motivation behind each question is described in Appendix IV.

4.2.2 Unstructured interviews with colleaguesTo gain further insight into staff impressions of the PBL activities I conducted short (30min) interviews with experienced colleagues who participated in the two activities. The topics to be covered were dependent upon answers to the staff and student questionnaires. The interviews were used to check that answers to the questionnaires were truly diagnostic, providing an independent check of the methodology. Since there was a strong correlation between the answers given on the questionnaires and results from the interviews, I am

4. Implementation and assessment Problem based learning approaches in Meteorology


confident that this is the case and do not provide a detailed breakdown of results from the interviews.

4.2.3 Assessment of students workFollowing submission of final work assessment was also made of students work on the project. In general results were mixed, but generally consistent with performance in other activities in the course and showed that students had actively engaged with the PBL activities. I am comfortable that the reflective element of the activities was well incorporated since all students provided some reflection on their own and others work. Since results of this analysis are somewhat more qualitative than the diagnostic questionnaire and broadly agree with its results they are mentioned only briefly below.

4. Implementation and assessment Problem based learning approaches in Meteorology


5. Results from implementation in 2009The two activities were first implemented as part of the course during academic year 2009/10. The course took place between 4th and 11th September on the Isle of Arran. I did not attend the course since I was on paternity leave. 32 students took part in the course, 16 from Reading and 16 from Leeds. Of those students, 3 from Reading took the course at the M-level and also participated in the observing system design activity during the autumn term 2009/10. The average mark for the course overall was 63% with a standard deviation of 5%. The ozonesonde activity had an average mark of 64% with a standard deviation of 10%. The observing system design activity had an average mark of 62% (no standard deviation is recorded since only three students participated).

5.1 Raw survey results

2009 Assessment of PBL Meteorology activities

CRITERIA O3 STUDENTS

S.D.O3

STAFFS.D.

OBS. SYS. STUDENTS

OBS. SYS STAFF

How well did students understand the task?

How easily did groups quickly focus on the key questions required?

Was the activity a good simulation of a ‘real-world’ case

Did you anticipate the activity would improve your specific subject understanding?

How well did students engage with specific reflective activity

Was all the information required provided to you in the project text?

How much were staff used to give subject specific information

How much were staff used to give generic skills information

Did comparison with other groups/students help students to reflect on their work?

3.2 1.2 3.5 2.1 6.0 3.5

3.5 1.6 2.3 1.9 4.3 2.5

4.4 1.6 6.3 1.0 4.7 3.5

3.7 1.7 5.0 2.9 4.7 3.5

2.9 1.3 2.7 2.0 2.0 1.5

3.9 2.2 5.3 2.1 7.7 1.5

2.8 1.9 5.3 2.1 1.7 4.5

4.9 2.7 6.8 2.1 1.7 4.5

3.0 1.7 6.3 2.8 N/A 1.0

5. Results 2009 Problem based learning approaches in Meteorology


CRITERIA O3 STUDENTS

S.D.O3

STAFFS.D.

OBS. SYS. STUDENTS

OBS. SYS STAFF

Did reflection help students improve their understanding?

Did students agree with the staff assessment?

Did the activity improve students generic skills?

Did the activity improve students subject specific skills?

Did you prefer the time constraint in the O3 activity to the open-ended Obs. Sys. activity?

Did you prefer working on your own in the Obs. Sys. activity rather than in a team in the O3 activity?

The Obs. Sys. Activity improved my subject specific knowledge more than the O3 activity?

The Obs. Sys. Activity was at a higher educational level than the O3 activity?

5.3 3.0 4.7 1.5 N/A 6.0

2.8 2.3 N/A N/A N/A

N/A 2.7 1.5 N/A 3.0

3.7 1.5 3.3 1.5 N/A 2.0

N/A N/A 3.0 N/A

N/A N/A 5.0 N/A

N/A N/A 4.0 N/A

N/A N/A 2.0 N/A

Table 1: Results of student survey of PBL activities following implementation in year 1 (2009). Marks are awarded by participants on a scale of 1-10 with 1 being the highest mark. N/A means a question was not asked to gain this information. Statistics are based on 18 student surveys and 4 staff surveys for the ozonesonde activity and 3 student surveys and 2 staff surveys for the observing system activity.

5.2 Reflection on student experienceIn general both activities were well received by the students who assessed generally high grades in most categories. The questions can be useful divided up into four broad categories on which to assess the success of the PBL implementation. The first set of questions assess how well the activity was structured and communicated to students. Clearly the small group of students who took part in the observing system activity did not fully understand their task and this may have reduced their motivation in taking part (Action: Improving documentation and introduction of task for 2010). There is an interesting discrepancy between the perception of the ozonesonde activity as a good simulation of a real world task between the students (who generally thought it was) and the staff (whoʼs reaction was more mixed). In many ways this is a positive outcome since it suggests that the task was simpler than a complex real-world grant proposal but that this did not detract from itʼs appeal to the students. In all activities both staff and students judged the students to engage well with the reflective part of the activity which is a key part of the Kolb cycle and crucial to this new activity which is positive. Interestingly, the extent



to which the students and staff believed that the reflection helped the students improve their understanding was more mixed (Action: Reconsider the reflective part of the activity to ensure it boosts student understanding).

The second set of questions deals with how students gained required information for the task. This shows the expected split between the two activities with the ozonesonde activity providing most of the required information in written form and the observing system activity requiring students to do their own research and engage with staff. When assessing how staff were used, students were generally more pessimistic about their own input and claimed staff influenced both their subject specific and generic skills more than the staff perceive. This is perhaps to be expected, but it is important for the success of the activity that the students believe that it is their input and decisions which influenced the direction of both projects. It is important that staff act as facilitators of the discussion since part of the PBL learning process is shaping and refining the problem at hand (Action: Reiterate to staff that their role should be advisory only and that they should allow groups to make their own project decisions - even if this leads to a weaker project overall).

The third set of questions deals with the assessment of the activity post-facto by both groups. As mentioned above, both staff and students were somewhat mixed in their assessment of the utility of the reflective elements of the activities. Interestingly, students believed that the comparison with other groups was a very helpful part of the ozonesonde activity, whereas staff were more circumspect. In general the projects scored well amongst all groups in their ability to improve both generic and specific skills.

Finally, the group of students who participated in both the ozonesonde and observing system activities were asked to compare them. Interestingly for broader applications of PBL in Meteorology there was a clear preference for the time-limited ozonesonde activity and the focus that this brought to discussion. However in general the students believed the observing system activity to be at a higher educational level, which again fits well with the course design.

Participants were also asked to make specific and general comments on the activities. Few comments were received, but some of the most interesting are shown below:

Student“I didnʼt have much of an idea of what I was supposed to be doing or how to get a good mark in this.”“Good but should only be done sometimes.”“Encourages time keeping.”“Makes you think more for yourself which encourages learning.”“I prefer more lecture based teaching, not a fan of large research projects stuff. It is important it is more real-world, but 40% is still too heavy a weighting.”“Initial knowledge of the area needs to be taught first to better be able to do these activities, but it challenges you to think about stuff in a more realistic context which is good.”“It encourages you to think for yourself more. Although I didnʼt like it to begin with it has taught me a lot.”

Staff“Encourages vibrant interaction between staff/student so that ideas are created and developed quickly. Allowed for quickly working through problems and assimilation of scientific knowledge.”



“Good activity, although students found assessment of the speaking part a bit vague.”“You cover a lot less content but it may be more effective and the student learns a lot more from it by making mistakes and learning/developing things by himself. Combined with traditional approaches to teach the basics I think it is highly useful.”

5.3 Informal consultation with colleaguesInformal consultation with colleagues revealed that both activities had been well received in the first instance and had enabled students to be more actively engaged in their learning and to explore different facets of both problems than they might otherwise have done. The major discussion point for the ozonesonde activity was the lack of training of staff both for the PBL process and in the specifics of the activity itself. There was particular concern about the role that the reflective activity should play. The major discussion point for the observing system activity was the lack of engagement between students and staff members outside contact hours. Both staff members felt that the students were disinclined to ask for help and expertise even though this was explicitly offered.

5.4 Consistency of evaluation using all three evaluation methodsA coherent picture of the successes and failures of the activities in their first implementation arises from consideration of all three methods of evaluation. In general, staff and students found the activity to be worthwhile and both in the questionnaire evaluation and the informal interviews thought that the PBL approach promoted active engagement amongst the students. Evaluation of student work, informal staff interviews and the questionnaire responses highlighted the problems in the introduction of the reflective elements, particularly in relation to the way in which staff participated in the activity. There are however, some elements in which the different evaluation techniques give different pictures of the activities. Although the survey results suggested students didnʼt fully understand the purpose of the observing system activity the student outputs (both in terms of a qualitative or quantitative evaluation) did not suggest that they performed any better or worse than in the ozonesonde activity or in the course in general. Staff interviews seemed to confirm the view that part of the dissatisfaction with the observing system activity was dominated by the views of one student who set a very confrontational tone from the beginning of the tutorials.

5.5 Proposed changes to activitiesIdentified actions to improve the activity for 2010 are:• Improving documentation and introduction of observing system task for 2010. The documentation of the activity was revised to the current version shown in Appendix II. In addition, research staff who assisted in teaching the module were asked to give two short presentations in the first tutorial to provide key information on observing techniques to the students. These presentations were shared with the students via Blackboard.• Re-consider the reflective part of the ozonesonde activity to ensure it boosts student understanding. • Re-iterate to staff that their role should be advisory only and that they should allow groups to make their own project decisions - even if this leads to a weaker project overall.These two changes were brought about by enhanced staff training prior to the exercise and by communication of the staff assessment of the proposals to the students. All staff were informally briefed on what was expected from the students and the purposes of the reflective activity. Staff then ensured that the purpose of the activity was explained to students when they asked questions about it. Staff were also encouraged to act as facilitators of each groups thinking. Once the staff ʻsteering-committeeʼ had assessed the proposals, in addition to communicating the result to students, 10 minutes was spent explaining the reasons for this choice to them.



• Add informal contact periods (ʻoffice hoursʼ) to the observing system activity to encourage informal contact between staff and students.This was done, although it did not seem to encourage informal contact.

These actions were undertaken during academic year 2010 and modified activities were introduced into the course in September 2010. The assessment of the activities is repeated and described in the next chapter.



6. Results from implementation in 2010The second implementation of the two activities occurred as part of the course during academic year 2010/11. The course took place between 5th and 12th September on the Isle of Arran. 35 students took part in the course, 12 from Reading and 17 from Leeds. Of those students, 5 from Reading took the course at the M-level and also participated in the observing system design activity during the Autumn term 2010/11. The average mark for the course overall was 61% with a standard deviation of 4%. The ozonesonde activity had an average mark of 56% with a standard deviation of 4%. It should be noted that a different academic colleague at Leeds was responsible for marking the ozonesonde activity in each year of the course. While every effort is made to standardize marking, experience in previous years shows that the lower mark in the 2010 implementation is partly related to this change in marker. The observing system design activity had an average mark of 65% with a standard deviation of 7%.

6.1 Raw survey results

2010 Assessment of PBL Meteorology activities

CRITERIAO3

STUDENTSS.D.

O3 STAFF

S.D.OBS. SYS. STUDENTS

OBS. SYS STAFF

How well did students understand the task?

How easily did groups quickly focus on the key questions required?

Was the activity a good simulation of a ‘real-world’ case

Did you anticipate the activity would improve your specific subject understanding?

How well did students engage with specific reflective activity

Was all the information required provided to you in the project text?

How much were staff used to give subject specific information

How much were staff used to give generic skills information

3.2 2.1 4.0 n/a 3.2 3.0

4.2 1.9 3.0 2.7 2.6 3.0

4.2 2.3 4.7 3.8 2.4 2.5

3.7 2.3 2.3 1.5 1.6 2.0

3.4 2.2 3.7 3.8 1.2 2.0

3.5 1.7 3.5 3.5 2.8 2.5

2.2 1.1 6.7 2.3 1.4 5.5

3.8 2.7 4.7 3.1 3.4 4.5



CRITERIAO3

STUDENTSS.D.

O3 STAFF

S.D.OBS. SYS. STUDENTS

OBS. SYS STAFF

Did comparison with other groups/students help students to reflect on their work?

Did reflection help students improve their understanding?

Did students agree with the staff assessment?

Did the activity improve students generic skills?

Did the activity improve students subject specific skills?

Did you prefer the time constraint in the O3 activity to the open-ended Obs. Sys. activity?

Did you prefer working on your own in the Obs. Sys. activity rather than in a team in the O3 activity?

The Obs. Sys. Activity improved my subject specific knowledge more than the O3 activity?

The Obs. Sys. Activity was at a higher educational level than the O3 activity?

2.3 1.1 2.5 0.7 N/A 2.5

2.8 2.6 3.5 0.7 N/A 4.5

3.6 2.6 N/A N/A N/A

N/A 3.0 1.7 N/A 3.5

2.6 1.7 3.3 2.5 N/A 3.5

N/A N/A 6.8 N/A

N/A N/A 3.6 N/A

N/A N/A 3.4 N/A

N/A N/A 3.6 N/A

Table 2: Results of student survey of PBL activities following implementation in year 1 (2010). Marks are awarded by participants on a scale of 1-10 with 1 being the highest mark. N/A means a question was not asked to gain this information. Statistics are based on 21 student surveys and 3 staff surveys for the ozonesonde activity and 5 student surveys and 2 staff surveys for the observing system activity.

6.2 Reflection on improvement to PBL activities in second year of implementationIn summary, results from the evaluation of the PBL activity in the second year of implementation are extremely positive. In most cases where the evaluation of the 2009 module revealed that the activity had been successful this positive result was maintained. In the areas where the 2009 evaluation identified improvements could be made the changes made to the PBL procedure have generally improved both student and staff evaluation, specifically:

• The improved documentation and introductory lectures incorporated into the observing system activity significantly improved scores in the first part of the survey, particularly for



students showing that they understood the task better, were able to quickly focus on the task at hand, that they felt that the task was a reasonable simulation of a real-world activity and that they engaged strongly with the reflective activity.• The improved oral description and staff training for the reflective part of the ozonesonde activity significantly improved the scores of both staff and students in this part of the survey. Particularly pleasing is the gain in the mark for subject specific skills for both staff and students.

Another interesting result of the second evaluation, perhaps related to the small sample size and variation between student groups is the lack of preference for the time constrained, ozonesonde activity in the 2010 cohort. While there was a strong preference for this activity in the 2009 cohort, the 2010 was enthusiastic about the observing system activity, but expressed no clear preference for this PBL style as opposed to the more limited, focussed ozonesonde activity.

The 2010 control cohort which participated in both PBL activities also produced a number of interesting comments and suggestions on PBL in general:

“...applying what you learn to a 'real-lifeʼ situation focuses oneʼs mind and gives the learning/research , etc., a full purpose...”“I thought it was a very good way to go, in that we got the benefit of people which much more expertise. Also it was done in a relaxed way which was good.”

They also had some interesting thoughts on how PBL might be applied more generally in their degree program:“In Meteorology, it would be good to have more of this form of teaching...”“...to do it justice, it should come at a time where other deadlines are not imminent.”“Maybe with the final project a little more.”

Staff comments highlighted that this approach was only really successful with outgoing and able students (a comparison between the two cohorts participating in the observing system activity was quite revealing). The 2009 cohort which was socially dominated by one student of quite low background ability did not engage fully with the exercise and as a result the cohort had a worse impression overall of the exercise and performed more poorly. The second cohort, which was generally of higher background ability engaged fully with the exercise and were more content with itʼs learning objectives and had overall better performance.

6.3 Consistency of evaluation using all three evaluation methodsAs in the assessment of 2009 results, there is broad consistency between the three methods of evaluation of the activity (questionnaire, staff interviews, student outputs). In particular the improvements to the reflective part of the activity were echoed through all methods of evaluation. This lends confidence to the overall evaluation of the PBL elements described above. Although the student output mark for the ozonesonde activity is lower than in the 2009 implementation and low compared to the overall course mark, more detailed qualitative comparison of student outputs suggests that there was not a decline in quality between the two cohorts. The change in mark is likely related to the change in the marker as noted above.

6.4 Continue implementation of PBL in field course?The two test implementations of PBL to the field course have generally been a success as shown by the generally positive evaluation of the activity by students and staff alike. The



improvements to the activities implemented for the 2010 version appear to have produced a robust methodology for both approaches which can and should be continued. For further development, as suggested by a colleague from Leeds in informal interview, it would be beneficial to modify the ozonesonde activity so that more outcomes of the experiment are possible. This would increase the diversity of student proposals and lead to a more fruitful reflective portion of the activity. One suggestion would be to replace the ozonesonde in the proposal with a standard additional radiosonde. For technical reasons, there are more chances that this would lead to interesting and provocative observations, giving greater scope for student creativity in the activity.



7. Conclusions and discussion7.1 Overall reflection on the use of PBL in this moduleIn conclusion the experiments with a PBL approach in Meteorology have proved to be very successful and have provided useful new content for an existing course in an innovative style unfamiliar to students. In general students enjoyed the freedom given to them by this approach and felt that it was a reasonably faithful simulation of a real-world activity thereby improving their motivation for the task in question.

We plan to continue the experiment in future years and to seek to refine the methodology used to improve its implementation. One idea for the ozonesonde activity would be to switch the science experiment in question to one with more potential outcomes and experimental strategies to improve the diversity of student responses and observed features. Nonetheless, clearly the PBL methodology has an important part to play in the module, coupled with other teaching approaches.

7.2 PBL in Meteorology more generallyMore generally, I certainly see a role for PBL teaching within Meteorology as a complement to our existing teaching styles. Teaching in our department is already highly diverse and well regarded and PBL should be considered as simply ʻanother tool in the boxʼ to help refresh and revise our teaching. I would not however advocate moving our whole curriculum to a PBL or EBL style as is done in some disciplines and institutions (particularly in the medical sciences). Since Meteorology represents somewhat of a departure for some students from their previous background knowledge and general approach to learning, a full PBL curriculum would not be able to provide the required breadth and depth of material that students require, particularly in their first two years of higher education.

If I were to discuss with a colleague the ways in which PBL might be useful for them I would emphasise the real-world simulation aspect and its affect on student motivation. To successfully implement a PBL activity they would need to think carefully about the kind of activity they could introduce, if the students had significant training and maturity to deal with this kind of learning and carefully design resources which provided adequate but not too comprehensive background material for the students. As was evident from staff responses, there is also a clear need to educate staff involved in the activity about the limits and purpose of their role in the activity and the module convener should consider how best to do this in conjunction with designing the activity.

7.3 Potential areas in which small PBL projects would be beneficialAs mentioned above, there are some clear benefits to a limited amount of PBL teaching in module which could be incorporated into other parts of our Meteorology curriculum. Having considered our program in general, there are a few obvious candidates for small tests of PBL to see if the lessons learnt in this project transfer to other modules. All of these modules are ʻadvancedʼ modules i.e. they would be taken in the final year of the students three or four year degree program

Practical classesMeteorology, as a large department, benefits significantly from a large number of well qualified laboratory support staff who construct and run experiments either in our fluid dynamics laboratory or on our field site. Since the equipment used is complex, the typical model of a class is for the convener to design the experiment, for the support staff to execute its construction and for the student to vary the parameters of the apparatus or to process the output. This model is adopted for convenience but tends to leave the student

7. Conclusions Problem based learning approaches in Meteorology


with little feel for how to design an experiment or the challenges of executing it. A PBL approach whereby the student is set a problem and seeks to design and execute it with the assistance of laboratory support staff could be an interesting variation on standard practice.

Climate ChangeAt both BSc and MSc level students are taught the science of climate change in a standard lecture based format. A module on similar material but at lower mathematical level is also offered more generally to Reading students. In all these contexts, an application of PBL could be tested. This subject is particularly suitable for a PBL approach since a large number of high quality resources, particularly the reports by the intergovernmental panel on climate change, exist which provide scientists new to the area with a comprehensive but easy to access introduction to current scientific understanding. A potential theme for a PBL activity would be for students to be asked to produce a synopsis of the science for a government or a large corporation, a task that many of them may be required to do if employed as climate change consultants or by government post-degree.

7.4 A proposal for an improved undergraduate final year project/dissertationAs mentioned above, I think in UK taught degree programs which are typically highly compressed in time and content the scope for broad implementation of PBL is limited. Certainly, in Meteorology if a pure PBL approach to all learning was adopted then too little core content could be delivered. That said, there is certainly scope for a broader application of the tools and techniques of PBL to current programs to encourage students to engage more fully with them and reflect on their own and other peoples work. In Meteorology, along with the two examples above, the best example of this is the current final year project which forms a significant percentage of the final mark. The current structure of the project is:

• Student selects a project during summer term offered by research staff• Student works on project during autumn term• Short presentation on project to other staff (staff assessment and feedback)• Student works on project during spring term, the majority of time being spent on preparing a long (~30 page) document describing the project• Project supervisor reads and comments on first draft• Student submits project document (staff assessment)• Student prepares and presents a poster (peer and staff assessment)

Although the current structure has some scope for reflection through staff comments on work there is little else to offer feedback to the student or to allow them to reflect on their own and other students work till the end of the project - by which time they are unable to modify their own performance. To enhance the project, I propose to use some of the ideas from PBL to make the activity more like a real-world case and allow students much greater freedom to explore an area of science and provide feedback on each others work. A proposed alternative structure for the project is:

• Week 5, summer term, student selects a staff member to work with based on their research interests and list of staff research interests.• Before vacation, meets with staff member to discuss some research ideas.• Week 1, autumn term, student submits one-page research proposal to research committee consisting of one staff member and two student members.



• Committee reads and assess proposal (10% of final mark) and provides feedback to student by start Week 2.• Student works on project with supervisor through autumn term.• Week 1, spring term, student submits six page ʻresearch-paperʼ to student research journal. Continues to work on project.• Research paper is reviewed by two anonymous student reviewers. Staff member acts as editor and communicates essential and nonessential corrections to student by Week 5. The quality of the student review is assessed by the staff member (20% of final mark).• Student submits revised six-page research paper to journal by end of week 8. Staff member assess research work (50% of final mark).• Student prepares poster for final poster conference. Assessed by both peer and staff assessment (20% of final mark).• All final research papers are bound into a final volume which is archived in departmental library.

This system would move the project closer toward the modern, real-world practice of meteorological science. Students would still be expected to perform an in-depth research project but they would have ownership of the process at all stages. They would also learn to participate in the process of constructive peer review and understand the complexities of a peer-review system in which opinions on work are often different and sometimes wrong. Finally, by producing a short, final document they would learn to communicate key scientific messages and move away from simply adding all their results to a single document. At the end of the project, an attractive and accessible record of research performed by them and their peers would be available.

No doubt, implementing such a major change to current practice would be challenging and would require staff members to reconsider the aims and objectives of the project. I feel however, that it would also much better prepare our students for the world of work or further study.

7.5 Reflections on course design in generalAs mentioned in my first video diary (http://cloudworks.ac.uk/cloud/view/3813) the process of formally redesigning the Atmospheric Science Field course has been very informative for the course design process in general. It has been interesting to be exposed to more of the formal course design procedures adopted by other disciplines/universities. It is my experience thus far in my academic career that this formal course design process is very rarely adopted in my department or subject area. The typical process of course design in Meteorology begins with course content and produces a course design which fits around this content. There is little consideration at the outset of how the overall course might be structured or how different teaching and learning techniques might be used to improve the overall quality of provision. Most course design is similar to the project described here, in that the course convener will adapt a previous module and make small changes to improve or enhance that module. The course mapping tool is particularly useful in this context, since it allows a ʻbirds-eye viewʼ of the course to be developed before making changes. It is also useful to force oneself to consider the course from this perspective. I hope to continue to develop courses in this fashion in future.





References

Atkinson, R., & Shiffrin, R., 1968, Human memory: A proposed system and its control processes. In K. Spence & J. Spence (Eds.), The psychology of learning and motivation (Vol. 2, pp. 89–195). New York:Academic.

Carnduff J. & Reid N., 2003, Enhancing Undergraduate Chemistry Laboratories, Royal Society of Chemistry, London.

Conole G., 2010, http://cloudworks.ac.uk/cloud/view/2971

Boud D. and Feletti G. (eds.), 1997, The Challenge of Problem-Based learning, London: Kogan Page

Fry H., Ketteridge S. & Marshall S. (eds.), 2008, A Handbook for Teaching and Learning in Higher Education, New York: Routledge

Gibbs G., Habeshaw S. & Habeshaw T., 1988, 53 Interesting Ways to Appraise your Teaching, Bristol: Technical and Education Services

Gijselaers, W, 1995, Perspectives on problem-based learning; pp 39-52 in Gijselaers, W, Tempelaar, D, Keizer, P, Blommaert, J, Bernard, E & Kapser, H (eds) Educational Innovation in Economics and Business Administration: The Case of Problem-Based Learning. Dordrecht: Kluwer.

Hmelo-Silver C., Golan-Duncan R. and Chinn C. A., 2007, Scaffolding and Achievement in Problem-Based and Inquiry Learning: A Response to Kirschner, Sweller and Clark, Educational Psychologist, 42, 99-107

Kahn P. and K. Oʼ Rouke, 2004, Guide to Curriculum Design: Enquiry Based Learning, Higher Education Academy

Kirschner, P.A., Sweller, J. & Clark, R.E., 2006, Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry based teaching. Educational Psychologist, 41, 75-86.

Kolb, D.A., 1984, Experiential Learning, Prentice-Hall, New Jersey

Kuhn, D., 2007, Is direct instruction the answer to the right question? Educational Psychologist, 42, 109–113.

Moreland N., 2005, Work-related learning in Higher Education, Higher Education Academy Learning and Employability series.

Perkins C., Evans M., Gavin H., Johns J. & Moore J., Fieldwork and Problem based Learning, Special edition of PLANET, http://www.gees.ac.uk/planet/index.htm#PSE2

Savin-Baden, 2000, Problem-Based Learning in Higher Education: Untold Stores, Open University Press

8. References Problem based learning approaches in Meteorology


Schmidt H. G., Loyens S. M. M., van Gog T. and Paas F., 2007, Problem-Based Learning is Compatible with Human Cognitive Architecture: Commentary on Kirschner, Sweller and Clark (2006), Educational Psychologist, 42, 91-97

Sweller J., Kirschner P. A. and Clark R. E., 2007, Why Minimally Guided Teaching Techniques Do Not Work: A Reply to Commentaries, Educational Psychologist, 42, 115-121

Yorke M., 2004, Employability in Higher Education: What it is and what it is not, HIgher Education Academy Learning and Employability Series.

8. References Problem based learning approaches in Meteorology


Appendix IDocumentation of Ozonesonde Activity

8. Appendices Problem based learning approaches in Meteorology


Appendix IIDocumentation for Observing System ActivityIntroductionDuring your time in Arran you had a chance to observe many different measurement techniques and to see how they were used in an experimental setting. In this extended essay you will now have the opportunity to use the skills you learnt in practice to produce a design for a new climate monitoring station on Arran. You will spend this term thinking about the design and reading around the subject before submitting a design brief on Monday week 8.

Aims and design briefThe UK government wants to establish a new climate monitoring station at Loch Ranza. You have been asked to submit a design specification for the station. Your design specification should include:

1. A section outlining important parameters of the climate system that should be monitored and why.2. A section listing the equipment that would be required to measure these parameters. You should include a rough estimate of the costs of purchasing and maintaining the equipment you require.3. A section describing the particular challenges of monitoring the climate on Arran.4. A section describing how your monitoring station would contribute to overall monitoring of UK and global climate, considering other land and satellite based instrumentation.5. An outline of the operational procedure at the station and how data should be analysed once collected.

While on Arran you should think about the particular challenges involved in monitoring the climate and the equipment and techniques used to solve these challenges. You should also take the opportunity to talk to the lecturers from Reading and Leeds who are involved in the course about their particular expertise in Atmospheric monitoring. If you find some of the analysis you complete on Arran particularly relevant to this problem you should take a copy of the analysis, figures or data with you for later processing in Reading.

Your design specification should be no longer than 15 pages and can include diagrams and tables where necessary. References should be included in standard format at the end of the document. You should make use of the range of resources (including other academic and research staff) available to you in Reading.

Helpful contentSome ideas for helpful content are as follows.Textbooks in the Meteorology libraryAn excellent basic resource is:Measuring the atmosphere, Strangeways, 551.63Other additional background Mountain meteorology, Whiteman, 551.69143



The Encyclopedia of Atmospheric ScienceThere is also a large section on remote sensing under catalogue number 551.653

Web-based resourcesThe webpage of the global climate observing system organisation (GCOS) which contains many useful reports and technical documents:http://www.wmo.int/pages/prog/gcos/Met Office climate monitoring pagehttp://www.metoffice.gov.uk/climatechange/science/monitoring/General information and introduction to space-based monitoring from NASAhttp://climate.jpl.nasa.gov/General information and introduction to space-based monitoring from ESAhttp://earth.esa.int/applications/data_util/CLIMATE/

Journals with interesting contentMore cutting edge information on methods for atmospheric measurement is presented in the journals The Review of Scientific Instruments and Journal of Atmospheric and Oceanic Technology. Articles are available free of charge from campus machines.

It is worth also searching other journals for more specific areas of interested, but be careful to avoid a too broad search.

TutorialsIn order to help you with the project, during the Autumn term we will hold three, one-hour tutorials. Please check your timetable for the time and location of these sessions. At each tutorial you should bring along the work you have already completed and share this with the rest of the group. We will also ask other research staff to attend and participate in the tutorials to give you some additional ideas about what you might want to include in your design specification. The project is deliberately open-ended and we hope will allow you to be creative in the way that you work and in the resources you use to plan your monitoring station.

DeadlineThe deadline for submission of the report is Monday November 29th (week 8 of autumn term) and should be submitted in pdf format to the Blackboard drop box. You should expect results by Monday 17th January 2011 (week 2 of spring term).



http://www.wmo.int/pages/prog/gcos/

http://www.wmo.int/pages/prog/gcos/

http://www.metoffice.gov.uk/climatechange/science/monitoring/

http://www.metoffice.gov.uk/climatechange/science/monitoring/

http://climate.jpl.nasa.gov/

http://climate.jpl.nasa.gov/

http://earth.esa.int/applications/data_util/CLIMATE/

http://earth.esa.int/applications/data_util/CLIMATE/

Appendix IIIBlank questionnaires



Appendix IVDesign of questionnaireActivity 1: Ozonesonde launches on ArranStudent questionnaireThe student questionnaire is made up of 14 questions, 12 with numerical answers (and space for additional comments) and 2 more open ended questions. The motivation for each question is as follows.1. Did the students have a clear sense of the problem before beginning the task. Some

styles of PBL allow students to define ʻthe taskʼ themselves. In this activity this is not the intention so if instructions are not clear this could lead to valuable lost time and should be improved in future.

2. Did the group format work well. An important component of PBL is the additional benefits gained from this teaching style. One of these is enhanced group working. A key part of this is the ability of the group to work efficiently on the task in hand. If this is not the case then additional help should be given to focus group efforts early on in the task.

3. Did the students feel that the activity was realistic. Another important benefit of PBL is that it should simulate practitioner activities as closely as possible. It is important that the students feel the task is a realistic one in order for this benefit to be realised.

4. Did the activity also fulfill a core educational purpose. Besides the generic skills gained in the activity did the students also gain subject specific knowledge. If this is not the case is PBL the best approach here?

5. An important part of the motivation is encouraging students to participate in reflective observation. The aim of the oral presentations is to do this by comparing their own and other students work. Did the activity achieve this?

6. & 7. These questions explore how students formed their knowledge base to produce the proposal. Our aim is that not all the information is contained within the text and that the students will use other resources including staff to complete the task.

8. Staff should act as facilitators for each group rather than driving the discussion. Did the students feel this was the case?

9. This question also assess the ability of the activity to encourage reflective observation in the students. Did they feel that the comparison with other students work, required for the assessment was worthwhile?

10. Having done some reflective observation can the students now link their thoughts back to the beginning of the Kolb learning cycle and have a different abstract conceptualization of how to approach the task?

11. Did the activity overall improve the students knowledge of atmospheric chemistry or not?

12. Since there will always be multiple approaches to measuring the atmosphere, if the students are genuinely linking their reflective observation back to more abstract conceptualization I would expect them to disagree with this statement. If a majority agree, then the activity would require additional elements to encourage this link.

13. Are the students aware of the approach to teaching they experience? This may influence how they react to the teaching method. It may be useful to make the approach clear at the start of the activity.

14. This question encourages the students to give more general comments about the PBL approach which may identify issues not consider in the rest of the questionnaire.

Staff questionnaireAlmost all the questions in the staff questionnaire replicate questions in the student questionnaire. The aim of the staff questionnaire is to assess if the impressions of students matched those of the staff. The only difference is in questions 11 & 12. Here in the staff



questionnaire the aim is to determine which sets of skills staff believe the activity promoted, subject specific, general scientific or both.

Activity 2: A design specification for a new climate observing stationStudent questionnaire1.-8. The first part of the questionnaire repeats the questions from activity one. The aim is to be able to compare students impressions of the two activities.9. Did students feel that the longer more considered approach adopted in activity 2

benefited their learning or were the time-constraints imposed in activity 1 beneficial?10.Did the team working aspect in activity 1 enhance the PBL approach or not?11. As in activity 1, did the PBL approach also contribute to enhanced subject specific

understanding.12. For this activity in particular, did the students feel that it extended their knowledge and

was at the higher, masters-level in comparison to the activities in the rest of the field course. It is useful to assess if the PBL aspect here is important.

13. & 14. The students who will take both courses will be able to give a broad impression of their feel for PBL within Meteorology and suggest how widely it might be used.

Staff questionnaireThis questionnaire is designed to be very similar to the staff questionnaire for activity 1. The aim of the evaluation here is to compare and contrast the impressions of staff assisting with each activity to determine the positive and negative aspects of the two methods of PBL introduced into the course.



Charlton perez pbl_met

Education

Transcript of Charlton perez pbl_met