External Evaluation Proposal for the ADDER and … · Web viewThe Open, Transferrable, and...

1St Interim External Evaluation Report:

The OTTER Project

FINAL

21 July 09Andrew K Comrie

0

Contents

1. Introduction.............................................................................................................2

2. External Evaluation Focus and Methodology.........................................................5

3. Evaluation of Work Package 1................................................................................7

4. Evaluation of Work Package 2................................................................................9

5. Evaluation of Work Package 3..............................................................................15



8. Planning for Work Package 6...............................................................................20

9. Summary of Recommended Actions....................................................................21

Appendix 1.....................................................................................................................24

Appendix 2.......................................................................................................................27

1

1. Introduction

1.1 The Open, Transferrable, and Technology-enabled Educational Resources) Project (OTTER) is jointly funded by JISC and the Higher Education Academy (HEA) and is one of seven institutional pilot projects funded under the JISC/HEA OER Programme1

Aims and Objectives

1.2 The primary aim of OTTER is to pilot, analyse and model effective processes for the successful release of highly usable, adaptable, technology-enhanced Open educational resources (OERS) at the University of Leicester.

1.3 The objectives are to:

Contribute through effective models for OER release, towards the marketing and positioning of the University of Leicester and the UK HE Sector among prospective students globally.

Modernise, update, tag, categorise, and release, at least 360 credits worth of digital materials from seven academic departments, from staff development and from student support services for open use and re-purposing.

Integrate lessons learnt from previous OER experiences to identify the key challenges associated with the clearing of rights, licensing, and release of existing resources for free open access and repurposing.

Support individuals, teams and departments to release their digital content for free use and repurposing

Promote the sharing and reuse of high quality OERs within the University of Leicester and across the sector.

Populate, test the affordances of and inform future versions of JORUMopen and the University of Leicester’s open repository

Build capacity and provide evidence in usable formats, to influence future institutional and cross-sector policy in respect of OERs at the University of Leicester and elsewhere.

Widely disseminate OTTER outcomes locally, nationally and globally, well beyond the duration of the project through the University of Leicester’s high-profile presence at international conferences, communities of practice, publications via Plone, Leicester’s Institutional OER repository.

Outcomes

1 http://www.jisc.ac.uk/whatwedo/programmes/oer.aspx

2

http://www.jisc.ac.uk/whatwedo/programmes/oer.aspx

1.4 The outcomes from the project are:

A piloted model process and criteria that promote and enable extensive adoption of the concept and use of OERS in the future at the University of Leicester. Leicester’s established research to practice, innovation to mainstream, pilot to scale capability has shown that a substantial and well executed pilot is likely to lead to true transformation in the university.

Wider, improved institutional understanding of and commitment to the value of free OERS for promotion and positioning purposes in a global HE market, across all levels and categories of staff including senior managers, supporting its extensive distance and work-based learning marketing efforts.

Departmental and institutional awareness of OERS and associated process and benefits at Leicester.

Departmental and institutional workflows for managing content and resources.

Understanding of the limitations and benefits of different file formats for OERs by technologists at University of Leicester.

Understanding of the advantages and pitfalls of different platforms for OER sharing to inform future institutional choices.

Articulation of mechanisms for search engine optimisation and resource discovery such as tagging.

Enhanced capacity across all subjects involved in the generation and release of high quality OERs.

Strong evidence-based influence on institutional and sector policy on the future of OERs.

Improved staff commitment and motivation to engage in the collaborative development of OERs.

Increased satisfaction of future, current and past students through access to OERs.

A significant contribution to the learning and teaching, innovation, e-learning and widening participating policies and strategies at University of Leicester.

3

1.5 The project has specified that it aims to collect 360 credits of Open Educational Resources. This is detailed in Section 4 of this report.

Alignment with Institutional Strategy and National Developments

1.6 The University of Leicester’s Strategy for Learning Innovation (2009) was published in July 2009 as an addendum to the Learning and Teaching Strategy. The strategy is based on the principles of :

evidence for and evaluation of benefits of enabling innovation across the institution for students’ learning;

collaboration across the institution within strategic frameworks and through funded research and development projects;

raising the capability of all members of the University – students and staff- to exploit and benefit from the learning technologies of the 21st Century;

exploring ‘beyond the obvious’ to prepare for the future in unseen, unknown and unchartered territory for learning and teaching

1.7 The OTTER Project has real potential to evidence the new Learning Innovation strategy in action.

1.8 The OTTER project is well placed to inform OER development policy and strategy nationally through the contribution Professor Gilly Salmon is making through her membership of the Strategic Advisory Committee for OERs, the and the HE Academy Board and Council.

4

2. External Evaluation Focus and Methodology

2.1 This is the first of three interim evaluation reports of the OTTER project. Each interim report is structured around three core questions as defined in the JISC six-step evaluation model2:

What have we done/built/achieved, to what quality, and how efficiently?

What has been learned or confirmed through development activities?

How has the learning been acted on or fed back in?

2.2 Each interim evaluation is scheduled to take place at key points during the life of the project to enable the external evaluator to focus and report on different project work packages, outcomes and outputs. Full details are defined in the external evaluation framework which takes into account activities set out in the project’s evaluation plan.

2.3 Each interim report will:

confirm the extent to which work packages have been completed satisfactorily;

scrutinise evidence and confirm the extent to which the project’s intended outcomes are being realised and confirm the impact that this is having;

summarise key learning points from the project and where relevant, recommend actions for sharing learning with other stakeholders;

provide the project team with formative feedback in the form of recommended actions that are intended to help the team maximise outputs and/or, enhance evidence in support of outcomes;

provide formative feedback in the form of recommended actions that are intended to inform future project planning and prioritisation;

provide evaluative commentary on outputs from the project and recommend actions to enhance the output, as appropriate.

2.4 The first external evaluation has focussed on work packages 1,2 3, 4 and 5 and their associated outputs/outcomes. At this stage in the project, only work packages 1 and 2 are scheduled to have been completed. Work packages 3-5 are in progress and are not scheduled to be completed until the end of February 2010.

2 http://www.jisc.ac.uk/media/documents/programmes/reppres/evaluationhandbook.pdf

5

http://www.jisc.ac.uk/media/documents/programmes/reppres/evaluationhandbook.pdf

2.5 The methodology used to externally evaluate the OTTER project at this stage has included:

a. Focus Group Discussions with the OTTER Project Team, Library and Information Librarians

b. Literature Review. The literature review took into account:

Project Plan Steering Group Minutes Project BLOG Beyond Distance Research Alliance BLOG University of Leicester Strategy for Learning Innovation (2009)

6

3. Evaluation of Work Package 1

Overview

3.1 This work package was designed to set the project up and to establish the project’s governance arrangements. The Work package was scheduled to run between April and May 2009.

3.2 Evaluation has focussed on:

the extent to which the activities in the work package have been completed satisfactorily;

provide evaluative commentary on outputs from the project; key learning points from this work.

What have we done/built/achieved, to what quality and how efficiently?

3.3 The activities listed in Work Package 1 have been completed satisfactorily and have resulted in:

Formation of the project team. The project team has now been formed. This has taken longer than originally anticipated, however. The main contributing factor was the delay in freeing up the Copyright Administrator appointed to the project from the Library on a seconded basis from her current duties in the university. Communication between project team members is effective with team members meeting informally and formally (monthly) to review progress and agree priorities and actions.

Establishment of the Project Steering Group. The Steering Group has been established with a clearly defined remit and has met once since the start of the project. Minutes of the meeting are informative, clear and actions clearly assigned. A schedule of future meetings have been agreed and scheduled.

Participation in JISC programme level activities. Members of the project team are actively participating in the monthly online forum that JISC has established using ILLUMINATE. Team members are finding the online forum very valuable and are using it effectively to liaise with other OER Projects, to share ideas and approaches, discuss common issues/challenges with the other OER projects, agree common approaches and standards relating to the creation of OERs and learn more about tools that are currently in development such as JORUMOpen and GLO (OER Authoring Tool).

7

3.4 The outputs planned at this stage of the project have been created and include:

Project Plan. The draft and final versions of the project plan were submitted to JISC by the deadline set and has subsequently been approved and signed off.

The Project Website (BLOG) has been launched. Members of the project team are contributing to the project BLOG. However, the Beyond Distance Research Alliance also has a BLOG and some members are BLOGGING about OTTER here.

JISC Programme Launch. The team was represented and actively participated in the Programme Launch event on the 9 June 09.


3.5 Internal Secondments. A project proposal is strengthened by being able to list the names and expertise of staff that will participate in the project. However, the process for finalising the project plan must make sure it makes realistic adjustment toactivity timescales so that they more accurately reflect the time being taken to recruit ‘back-fill’ to release staff from current commitments to take up seconded posts.

Recommended Actions

The project team should ensure that BLOG entries about OTTER are recorded in one place (preferably the OTTER BLOG) and are linked from the Beyond Distance Learning Research Alliance BLOG and other relevant BLOGS.

The time taken to release seconded staff should be taken into account and more accurately reflected when preparing detailed project plans for future projects.

8


Overview

4.1 This work package was designed to collect existing digital materials and assess the volume and complexity of creating Open Educational Resources (OERs). Work package was scheduled to run between May and end of July 2009.


the extent to which the activities in the work package have been completed satisfactorily;

provide evaluative commentary on outputs from the project; identifying key learning points from this work. providing the project team with formative feedback in the form of

recommended actions that are intended to help the team maximise outputs and/or, enhance evidence in support of outcomes;

providing formative feedback in the form of recommended actions that are intended to inform future project planning and prioritisation;


4.3 The collection of digital materials and initial assessment of them is behind schedule. As a result, this work package is not complete and it is estimated that it will now not be complete until September 09. The following table summarises progress with materials collection to date:

9

Department Source of OERs Level Indicative Credit Weighting

Collected to Date

Processed to Date

Beyond Distance Research Alliance

Learning Futures,Learning Design,e-moderation, etivities, and the 5-stage modelSecond Life artefacts and resources

PG 12 12 3

Archaeology and Ancient History

Archaeological Theory (second life artefacts)

UG 3 3 3

Education Into to applied linguistics and TESOL,Phonetics and Phonology

PG 25 0 0

Law Company LawFinancial Services Crime

UG + PG

35 35 0

Politics and International Relations

The art of negotiationAscent to Pre-eminence: The USAVisions of the modern American PresidencyAmerican and the World: US Foreign PolicyDissertation Writing

PG 180 60 0

Psychology Selection & AssessmentThe psychology of organising, The individual at work, Ergonomics

PG 80 0 0

Staff Development Centre

Introduction for Lecturers new to Teaching in Higher Education

PG 5 5 5

10


Collected to Date

Processed to Date

Student Support & Development Service

Applying for JobsCareers Service ResourcesJob HuntingOptions after GraduationCareer Action PlansStudy, presentation, Writing Research, Numeracy and IT Skills

UG & PG

20 20

4.4 135 credits (38%) of the expected 360 credits worth of materials have been collected to date. There are three institutional partners that are contributing significantly to this deficit currently:

EducationPsychologyPolitics and International Relations

4.5 11 credits (3%) of the expected 360 credits worth of materials collected to date have been through initial assessment and IPR/ copyright clearance checks.

4.6 A number of additional departments have indicated that they are willing to contribute materials for inclusion in the project. They are as follows:


Collected to Date

Processed to Date

Genetics Still to be confirmed To be confirmed

30 0 0

Museum Studies Still to be confirmed Still to be confirmed

0 0

Mathematics Still to be confirmed Still to be confirmed

0 0

Physics Still to be confirmed Still to be confirmed

0 0

Actuarial Science Still to be confirmed Still to be confirmed

0 0

4.7 The project team do not envisage that this delay will impact on the overall project timescales. At present, evaluation confirms this view, but this needs to be kept under review at monthly project team meetings and as part of the evaluation process.

11

4.8 The process for conducting initial checks of digital materials is proving to be time consuming. Whilst the project team are not reviewing the accuracy of the content of materials being submitted, initial checks are required to ascertain:

the completeness of the materials being submitted; the best way of dis-aggregating (chunking) materials to maximise potential

for re-use and/or re-purpose; If there are any diversity issues.

4.9 The outputs planned at this stage of the project have been created and include:

An OER Quality Assessment and Work-flow Framework (CORE). The framework enables project team members to track the status of each OER at each stage of the development process. It also enables the team to track the amount of time spent on the OER at each stage of the process. After evaluation of the framework, further refinements have been suggested (see recommended actions). Furthermore, evaluation of the framework has suggested the need to develop additional tools and guidance for central teams and academic staff involved in the process of developing OERs with the aim of reducing the amount of time taken and ensuring that teams involved in the OER development and validation process are consistent when conducting initial assessments and validating the potential of the OER for re-use and/or re-purposing.


4.10 The complexity and time needed to conduct an initial assessment of source materials. From the activities in this work package, the team have learnt that:

The initial assessment of digital materials is time consuming and complex. There is potential for the team to be overwhelmed unless the initial assessment process can be more streamlined and the time taken to conduct initial assessment is reduced.

The project is already demonstrating the need for a central support team to remove the initial assessment burden from academic staff. Without it, it would be unlikely that academic staff would prioritise time to OER development. However, further guidance and OER literacy CPD is needed to ensure that academic staff have the knowledge and skills to carry out some initial checks before submitting source materials to the OER development process. This would reduce the OER development time and would ensure that any future OER development services were efficient and sustainable.

There is a need to record the work flow of OERs in development. This enables the development team to report on the status of individual OERs at any one time and monitor the time taken at each stage of development.

12

4.11 It is unclear how project partners from different academic departments and support areas are assigning a credit rating to source materials. This needs further exploration.


4.12 The team recognise the need to work together to further refine the CORE framework. Work is underway to review other OER development models to see if they provide insights to the way in which the CORE Framework could be further refined.

4.13 The CORE framework should be considered as an early deliverable from the project and the team should consider using the monthly JISC online Forum to share it with the other OER projects. This would enable the team to receive peer feedback which could inform further refinement of the framework.

13

Recommended Actions

Progress with Work Package 2 is kept under review at monthly project team meetings. The next interim evaluation visit scheduled for September should re-visit this work package to evaluate completion and ascertain if it can be signed off and confirm conclusively that the delays in collecting source materials are not going to impact on the achievement and/or timescales of the project overall.

The project team should work collaboratively to make and agree further refinements to the CORE framework that has been developed by the OER Evaluator.

All members of the project team should establish the discipline of using the CORE framework to record personal time spent and to record any other costs (e.g. copyright clearance costs) associated with the development of OERs at each stage of the development process. This data will be essential to enable the business case for OERs to be developed.

The project team should consider prioritising the development of guidelines and checklists for academic staff that are submitting source materials to the OER development process. Initial checks carried out by academics prior to the submission of source materials has the potential to reduce the amount of time needed overall, to carry out the initial assessment of source materials. These checklists could be incorporated into an OER Literacy framework ultimately. (See Section 7 for more details about the need for OER Literacy).

The team should use the JISC monthly online Forum to promote the CORE framework and encourage others to adopt it and provide feedback to inform further refinement.

At least one member of the project team should consider under-going diversity assessment training to ensure that OERs developed accord with current thinking and practice in diversity assessment.

The project team need to develop a workable and consistent model for assigning credit weighting to OERS. Possible sources that could assist this process is the Scottish Credit and Qualifications Framework (www.scqf.org.uk) and the QAA advice for levelling and credit rating under-graduate and post-graduate provision (www.qaa.ac.uk/)

14

http://www.qaa.ac.uk/

http://www.scqf.org.uk/


Overview

5.1 This work package has been designed to develop processes for conducting IPR and copyright clearance checks on source materials to ensure that any third party content in OERs have formal approval for use. The Work package is scheduled to run between May 09 and the end of February 2010.


key learning points from progress to date. providing the project team with formative feedback in the form of




5.3 Despite only being in post for one week, the Copyright Adviser is already making good progress assessing the source materials collected so far for any copyright/IPR issues.

5.4 IPR/Copyright assessment of source materials has not identified any major issues at this stage. Furthermore any copyright clearance has been obtained without incurring any costs to the project.

5.5 The outputs associated with this work package have not been realised as

yet. This will be evaluated further in February 2010.

Recommended Actions

The project team should use monthly meetings to monitor the volume of copyright clearance activity and associated costs.

The Copyright Adviser should use the CORE framework to record time spent and costs associated with obtaining copyright clearance.

15


Overview

6.1 This work package has been designed to develop processes for ensuring that OERs are technically ready for release. The Work package is scheduled to run between June 09 and the end of February 2010.






6.3 The learning technologist assigned to the project has already made good progress in establishing the formats for presenting OERS in different digital format . The JISC monthly online FORUM is proving an invaluable vehicle for discussing standardisation and formatting issues with the other OER projects and JISC experts.

6.4 The JISC Online FORUM has been used to get agreement on standards for meta-data/tagging and there is now reaching consensus that Dublin Core will be the standard used.

6.5 The JISC online FORUM is enabling Learning Technologist to keep abreast of updates to OER tools that are in development such as JORUMOpen and the OER authoring tool GLO.

6.6 The outputs associated with this work package have not been realised as

yet. They will be evaluated in February 2010.


6.7 JISC Monthly Online FORUM is proving invaluable in enabling the project team to engage with other the JISC Programme Team and other JISC/HEA funded OER projects to discuss and agree standards and common formats. This is helping the project to ensure that decisions that are being taken relating to standards and OER formats is consistent with other OER work.

16


6.8 Regular engagement with the JISC OER Programme team and the other OER projects is helping to ensure that OER projects are able to effectively review and provide feedback on BETA versions of JORUMOpen and other OER tools that are in development.

Recommended Actions

The learning technologist should ensure that the process for assigning meta-data/tags to OERS is streamlined as much as possible so that a balance is struck between providing enough meta-data to allow meaningful searching for OERS in open access repositories and ensuring that costs attributed to assigning meta-data is affordable and not prohibitive to ensuring sustainable OER development.

The project team should capitalise on the JISC Online FORUM and use is as a vehicle for disseminating outputs in draft form in order to encourage: peer review; efficiencies in development and minimal duplication of effort resulting from effective collaboration and workload sharing.

17


Overview

7.1 This work package has been designed to develop processes for engaging partners in the validation of OERs prior to release. The Work package is scheduled to run between July 09 and the end of February 2010.



recommended actions that are intended to help the team maximise outputs and/or enhance evidence in support of outcomes;

providing formative feedback in the form of recommended actions that are intended to inform future project planning and prioritisation.


7.3 The OER development process has been designed deliberately to minimise the contribution needed from academic staff.

7.4 THE CORE Framework has been developed to include the partner validation process.

7.5 The outputs associated with this work package have not been realised as yet. They will be evaluated in February 2010.


7.6 There is evidence to show that Academic staff’s lack of knowledge of OERs is resulting in misconceptions about making course materials and resources freely available to other institutions. There is a potential danger that unless staff awareness and knowledge of contributing and using OERs in course design and delivery is evolved, the OERs developed in this project and in others will not be used effectively in course delivery.

7.7 OER Literacy. The team have recognised that there is a need to develop academic and library staff’s awareness and knowledge of the OER development process and to develop curriculum design skills to ensure that OERs are embedded into course designs and learning activities.


7.8 Currently, the project team are reflecting on the need to prioritise the development of an OER Literacy framework and suite of OER CPD materials that will ensure staff have the knowledge and skills to use and/or re-purpose OERs in their courses.

18

Recommended Actions

Identify and exemplify (using case studies or other illustrative tools) the drivers for the development and integration of OERs into course design/delivery models and learning activities.

Collect evidence that can be used to illustrate the benefits and value of using OERs in course delivery models. These illustrations should exemplify efficiencies gained, time saved for academics charged with designing and developing course materials.

Collect evidence that can be used to challenge staff misconceptions about sharing content openly with other educational institutions.

Devise communication and internal dissemination events and activities that interest and engage academic teams in OER production and use.

Encourage academics with influence to undertake advocacy roles highlighting to peers the benefits of using OERs in course delivery.

Develop an OER Literacy framework aimed at academic and library/information staff that is used to further develop knowledge and understanding of for example:

the process for producing OERs; how to effectively access and search for OERs in open repositories; how to quality assess and validate OERs effectively; how to re-use and re-purpose OERs and integrate them into their own course

delivery models and learning activities; how to use OER Authoring tools (developed as part of the JISC Programme).

Explore ways of working collaboratively with other Institutions involved in OER development to produce and openly share CPD materials that support OER development and use.

19

8. Planning for Work Package 6

Overview

8.1 This work package has been designed to develop processes for engaging students in the validation of OERs prior to release. The Work package is scheduled to run between November 09 and the end of February 2010.


providing formative feedback in the form of recommended actions that are intended to inform future project planning and prioritisation.

8.3 In planning for work package 6, the project team need to give some thought to how best to identify students and provide suitable induction/briefing to ensure that they are effective in reviewing and providing meaningful feedback about the OERs developed.

Recommended Actions

The project team should ensure that student representatives on the project steering group are properly briefed on OER and the team should explore the potential for student representatives to assist with the student validation process.

The project team should consider lessons learnt from other projects in the Beyond Distance Learning Alliance Research Media Zoo especially in relation to capturing the student voice (e.g. CALF) to see if there are lessons learnt that could be adopted by OTTER.

20

9. Summary of Recommended Actions

The project team should ensure that BLOG entries about OTTER are recorded in one place (preferably the OTTER BLOG) and are linked from the Beyond Distance Learning Research Alliance BLOG and other relevant BLOGS.

The time taken to release seconded staff should be taken into account and more accurately reflected when preparing detailed project plans for future projects.

Progress with Work Package 2 is kept under review at monthly project team meetings. The next interim evaluation visit scheduled for September should re-visit this work package to evaluate completion and ascertain if it can be signed off and confirm conclusively that the delays in collecting source materials are not going to impact on the achievement and/or timescales of the project overall.

The project team should work collaboratively to make and agree further refinements to the CORE framework that has been developed by the OER Evaluator.

All members of the project team should establish the discipline of using the CORE framework to record personal time spent and to record any other costs (e.g. copyright clearance costs) associated with the development of OERs at each stage of the development process. This data will be essential to enable the business case for OERs to be developed.

The project team should consider prioritising the development of guidelines and checklists for academic staff that are submitting source materials to the OER development process. Initial checks carried out by academics prior to the submission of source materials has the potential to reduce the amount of time needed overall, to carry out the initial assessment of source materials. These checklists could be incorporated into an OER Literacy framework ultimately. (See Section 7 for more details about the need for OER Literacy).

The team should use the JISC monthly online Forum to promote the CORE framework and encourage others to adopt it and provide feedback to inform further refinement.

At least one member of the project team should consider under-going diversity assessment training to ensure that OERs developed accord with current thinking and practice in diversity assessment.

21

The project team need to develop a workable and consistent model for assigning credit weighting to OERS. Possible sources that could assist this process is the Scottish Credit and Qualifications Framework (http://www.scqf.org.uk/AbouttheFramework/Overview-of-Framework.aspx) and the QAA advice for levelling and credit rating under-graduate and post-graduate provision (http://www.qaa.ac.uk/academicinfrastructure/FHEQ/EWNI08/FHEQ08.pdf)

The project team should use monthly meetings to monitor the volume of copyright clearance activity .

The Copyright Adviser should use the CORE framework to record time spent and costs associated with obtaining copyright clearance.

The learning technologist should ensure that the process for assigning meta-data/tags to OERS is streamlined as much as possible so that a balance is struck between providing enough meta-data to allow meaningful searching for OERS in open access repositories and ensuring that costs attributed to assigning meta-data is affordable and not prohibitive to ensuring sustainable OER development.

The project team should capitalise on the JISC Online FORUM and use is as a vehicle for disseminating outputs in draft form in order to encourage: peer review; efficiencies in development and minimal duplication of effort resulting from effective collaboration and workload sharing.

Identify and exemplify (using case studies or other illustrative tools) the drivers for the development and integration of OERs into course design/delivery models and learning activities.

Collect evidence that can be used to illustrate the benefits and value of using OERs in course delivery models. These illustrations should exemplify efficiencies gained, time saved for academics charged with designing and developing course materials.

Collect evidence that can be used to challenge staff misconceptions about sharing content openly with other educational institutions.

Devise communication and internal dissemination events and activities that interest and engage academic teams in OER production and use.

Encourage academics with influence to undertake advocacy roles highlighting to peers the benefits of using OERs in course delivery.

22

Develop an OER Literacy framework aimed at academic and library/information staff that is used to further develop knowledge and understanding of for example:

the process for producing OERs; how to effectively access and search for OERs in open repositories; how to quality assess and validate OERs effectively; how to re-use and re-purpose OERs and integrate them into their own

course delivery models and learning activities; how to use OER Authoring tools (developed as part of the JISC

Programme).

Explore ways of working collaboratively with other Institutions involved in OER development to produce and openly share CPD materials that support OER development and use.

The project team should ensure that student representatives on the project steering group are properly briefed on OER and the team should explore the potential for student representatives to assist with the student validation process.

The project team should consider lessons learnt from other projects in the Beyond Distance Learning Alliance Research Media Zoo especially in relation to capturing the student voice (e.g. CALF) to see if there are lessons learnt that could be adopted by

OTTER.

23

Appendix 1OTTER Project External Evaluation

Structured Questions

WORK PACKAGE 1

1. To what extent has activities in work packages 1 been completed?

Project Staff in place Course Team Members identified from e-learning, Dept of Archaeology/Ancient

History, Linguistics/TESOL, Law, Psychology, Staff Development, Student Services

Steering Group setup. How many times has it met Participation in JISC activities – including Programme meetings,

WORK PACKAGE 2 – COLLECTION OF EXISTING DIGITAL MATERIALS FROM ALL PARTNERS


2. To what extent have OERS been identified and collected from the academic departments involved in the project, staff development and student services?

3. To what extent has the OERs collected matched the split between departments and the credit weightings anticipated in Table 1 of the OTTER Project Plan?

4. To what extent has the contribution from the Genetics department been finalised?

5. What quality criteria has been developed for evaluating OERs? How is this being applied to the OERs being collected?

6. How successful has the project been to date in collection OERs that satisfy the project quality criteria?


7. What have you learnt from the collecting process?

8. What has been the attitude of the partners involved in the project?

9. Have there been any unexpected outcomes or discoveries?

24


10. What has happened as a result of implementing your process for collecting and quality assessing OERS. How is this informing future activities?

11. How do you plan to continue to communicate and keep your stakeholders engaged with the project?

WORK PACKAGE 3 – CLEARANCE OF RIGHTS


12. Can you describe the process used to check OERs for IPR issues?

13. What are the time and cost implications?

14. To what extent has the project developed its IPR clearance process itself or worked collaboratively with sister projects/JISC?


15. What have you learnt from reviewing the OERS for IPR issues?

16. What cost/time implications are there for obtaining IPR clearance for OERS? Could cost/time be a barrier to sustainability of OER repositories? What needs to be done to minimise this? Is this something that the project can lead on/influence?


17. What actions are needed?

WORK PACKAGE 4 – FORMATTING AND STANDARDISATION


18. Can you describe the process used to format, add meta-data and tag OERs? To what extent does the process you are using reflect that of other sister projects?


25


20. What have you learnt from formatting, adding meta-data and tagging OERS so far?

21. Could cost/time be a barrier to sustainability of OER repositories? What needs to be done to minimise this? Is this something that the project can lead on/influence?



WORK PACAKGE 5 – VALIDATION WITH PEERS


23. Can you describe the process developed to peer validate OERs?



25. What have you learnt from peer validation of OERs so far?



26

Appendix 2

The following people participated in the interviews:

Individual Position

Gabi Witthaus Project Co-ordinator

Samuel Nikoi OER Evaluator

Simon Kear Learning Technologist

Tania Rowlett Copyright Administrator

Professor Gilly Salmon Principal Investigator/Project Director

27

External Evaluation Proposal for the ADDER and … · Web viewThe Open, Transferrable, and...

Documents

Transcript of External Evaluation Proposal for the ADDER and … · Web viewThe Open, Transferrable, and...