Post on 06-May-2015
description
Developing Digital Literacy:Baselining and evaluationHelen Beetham, Synthesis Consultant
Jay Dempster, Evaluation Consultant
4th October 2011
Maple House, Birmingham
Welcome – who we areDigital Literacy
Helen BeethamSynthesis consultant
Jay DempsterEvaluation consultant
Session outlineDigital Literacy
Why baseline?The programme-level view
The project-level view (1):How will practice/provision/literacy be developed?
The project-level view (2):Collecting and making sense of baseline data
Programme tools and resources for baselining
Why baseline?Digital Literacy
At programme level:Provide snapshot of the sector at start-upDevelop shared understandingReview what we already know and have achievedContextualise programme aims and rationale
At project level:Provide starting-point for evaluationProvide clear structure for collecting evidenceRefine objectives and success indicators
What do we know?
2008 Learners' experiences of e-learning programmeStudents' success depends on strategies for integrating ICT into academic practice; strategies and preferences differ widely
2009 Learning Literacies for a Digital Age studyDigital literacy needs to be integrated across the curriculum: learners develop through authentic tasks in meaningful situations
2010 Supporting Learners in a Digital AgeInstitutions are: addressing skills and expectations at transition; creating digitally-enabled learning spaces; listening to learners; embedding digital activities into the curriculum
2011 Digital literacy workshop seriesLessons and recommendations, tools for organisational and curriculum development; sharing best practice
2011-13 Developing Digital Literacies programme12 institutions, taking different views across the DL landscape
Digital Literacy
Why digital literacy?Digital Literacy
Impacts of digital technologyon practice
(research, teaching, learning, professionalism, knowledge transfer,
development)
New demands on the sector (graduate attributes and outcomes,
the learning experience, lifelong learning)
Fair access and opportunity(digital technology reduces some
barriers/inequalities, can introduce others)
Addressing digital literacy...Digital Literacy
inhigher/further education (curriculum development,
learning skills/academic practice)
through higher/further education (employability, graduate attributes,
the co-curriculum)
into higher/further education(transition, entitlement, access, student markets, participation)
Whose digital literacy?Digital Literacy
How do we conceive of digital literacy?Digital Literacy
It will be a big challenge to identify how digital literacy makes a difference to the individual in terms of how they think and what they do.Joe Nicholls, Digidol
We need to know more about the messy, day-to-day lived practices of students and staff.Lesley Gourlay, Institute of Education
How do we conceive of digital literacy?Digital Literacy
Presentation, Production and Performanceplay, performance, simulation, appropriation
Collaboration and work skillsmultitasking, distributed cognition, collective intelligence
Participating in information practices transmedia navigation, networking, negotiation
Ryberg and Georgsen (2010), drawing on Buckingham (2005) and Jenkins (2006)
At what level?Digital Literacy
access
skills
practices
identities 'I am...'
'I do...'
'I can...'
'I have...'
(Beetham and Sharpe 2009)
Discipline/subject
specialist focus
Generic/entitlement
focus
With what institutional impact?Digital Literacy
With what focus?Digital Literacy
(Sharpe et al. 2011)
Working with which professionals?Digital Literacy
Digital Literacy
'If we all do it properly, and climb and stay out of our bunkers, this will be a cross-community project on a
grand and productive scale'David Baume, SEDA
Activity in projectsDigital Literacy
Review some of the different ways of 'developing digital literacies'
Which have resonance for your project?
What part(s) of the landscape will your project focus on?
Write down three ways in which your project will bring about development
what will be developed (people, places, practices, processes, policies, curricula, opportunities...) and how?
Activity in projectsDigital Literacy
Review some of the different ways of 'developing digital literacies'
Which have resonance for your project?
What part of the landscape is your project focusing on?
Write down three ways in which your project will bring about development, i.e. what will be developed (people, places, practices, processes, policies, curricula, opportunities...) and how.
Share your statements with another project, working in project pairs. Review, compare, critique.
Baselining
From the original circular:
“Review current processes and practices to establish a baseline description of current digital literacy provision to aid in the evaluation of subsequent innovations.”
This aims to provide the evidence and ideas to support your project in planning your interventions.
Baseline report by end Dec/Jan
Many projects are looking to do a more detailed review by end of year 1.
Baselining as...
stakeholder review for shared/distinct meanings
contextualising ‘big picture’ evaluation questions
provide clear structure for collecting evidence
checking back against the ‘why and how’ of project
achievements
‘Baselining for success’
Combine your 30,000 ft view activity with 'what success looks like‘ ...
produce a ‘rich picture’ of your project’s intended outcomes
identify some clear indicators of success for your different stakeholder audiences
clarify starting points for sources & types of data for your evidence base
hard metrics e.g. statistics in audit documents, quantitative data collected from student feedback surveys
vs. narrative descriptions e.g. example in CDD of a project (T-Sparc) doing an entire baseline as a series of video clips .... on the Design Studio website at: http://blogs.test.bcu.ac.uk/tsparc/t-sparc-baseline-review/
Baselining data and indicators of success
Baselining Digital Literacies
Share your initial ideas/issues in pairs & discuss ...
for instance, you might touch on:
1. Is there a shared sense of what digital literacies mean/are?
2. Based on what/whose evidence?
3. How are you triangulating/cross-checking with stakeholders?
4. What are the assumptions made about the project and its context?
5. Are there existing measures available from prior audits/surveys/initiatives?
6. What descriptors/metrics/rubrics are emerging?
(towards a common focus and common language for how we define and measure
effectiveness & progress in digital literacies)
1. What are the most useful/feasible baseline data/ indicators of success for YOUR project?
Tools & Resources
Tools & Resources
http://bit.ly/pHxQnS
Design Studio pages for JISC DLL:http://jiscdesignstudio.pbworks.com/w/page/46421608/Developing%20digital%20literacies
Support Wiki pages for Baselining:https://programmesupport.pbworks.com/w/page/46353821/Supporting%20tools%20and%20frameworks
http://bit.ly/nfF8Mg
Support for baseliningDigital Literacy
Evaluation support- a starting point for conversations- DigLit programme wiki - follow up Elluminate session in November - project support Synthesis support- mapping project objectives and success criteria to programme view- develop resources on Design Studio- work with professional bodies
Jay Dempster
Helen Beetham