1

Learning Application Readiness (LAR)

A Paradigm for NSDL

Katy Ginger2011

2

LAR Definition Refers to how educational resources,

collections, and their related metadata are aligned to educational goals, curriculum, or professional development needs of users

AND

how readily said resources and collections can be embedded in tools and services that educators and students use.

3

Learning Application Definition

Tools and services educators use to accomplish learning tasks

AND

that have the ability to embed active & curated content from internal or external sources to support desired learning goalsThe goal is to support contextualized learning experiences, allowing users to find and use or deliver just the right digital content

4

Curriculum Customization Service (CCS)

External publisher content

Internal school district content

5

Why Learning Application Readiness

K-12 and Higher Ed reports emphasize critical missing pieces for educational & digital resources particularly: Lack of coherent integration Lack of platform

interoperability

NSF has asked NSDL to demonstrate an educational impact

6

LAR Guiding Principles: Resources

21st century contexts: advance critical thinking, problem solving, collaboration & recognize the interdisciplinary nature of knowledge

Relevant and reliable for STEM education: authored, meet pedagogical needs of educators and interests of learners

Accessible: rights, licenses, permissions, & technical requirements clearly stated

NSDL policy: Meet Resource Quality Guidelines

7

LAR Guiding Principles: Metadata

Complete: title, description, URL, educational level, resource type, audience, language, rights, access rights, creation date, contributors/creators, language, mime type and, if appropriate, educational standards

Concise: free of self-promotion; describes purpose and content of resource for comprehension

Accurate: the metadata is correct content for the field

8

LAR & NSDL Scope/Collections

NSDL has collections that are library appropriate but LAR concepts may not be applicable reference collections dataset collections

Collection sustainability/maintenance is important in maintaining LAR

Not always easy to determine LAR

9

Using LAR Resources Done programmatically by learning applications Learning apps using NSDL resources are filtering by:

education level resource type subject audience educational standards

For NSDL currently, human-crafted metadata (resource, annotations & usage data) exceeds machine-created metadata in terms of accuracy and quality.

10

Curriculum Customization Service (CCS)

Teacher selected digital library resourceTeacher added comment and rating

11

Collections Assessment Process for LAR

Examined individual metadata records to get: Collection size, growth & metadata format Collection use of educational metadata fields Readiness to embed collection in learning app

Analyzed educational metadata (fields in next slide)

Combined existing NSDL vocabularies & actual metadata values to create benchmark term set for assessment/categorization.

12

142,600 records; 131,342 resources; 121 collections

Metadata InfoMaximum # of unique terms

analyzed

% of records with any value

% of records categorized &

notAccess Rights 36 9.4% 100%Audience 157 41.3% 99.1% (562 not)Education Level 82,951 55.7% 99.2% (2,798

not)Educational Stds 1,078 3.78% 94.5% (304 not)Language 60 75.4% 99.9% (34 not)Mime Type 1,345 48.7% 94.0% (4,410

not)Resource Type 565 78.3% 99.7% (397 not)Subject 82,722 81.0% 91.9% (10,133

not) Total 168,915

13

Sample of Education Level Terms

14

Benchmark Term Set: Audience

Administrator Educator General Public Learner Parent/Guardian Professional/ Practitioner Researcher

15

Used to make resource criteria decisions programmatically

Field Actual NSDL Record Learning App Ready Record

Title Rainwater Harvesting Service Learning Project Rainwater Harvesting Service Learning Project

Descrpt In this service learning project, students, teachers and community members will work together to design and construct a rainwater harvesting system for their school campus. Research RWH design basics and local conditions Explore how RWH could be used on your campus and develop a basic design. Present findings and action plan to community partners, school administration and student body. Enact the action plan to construct a RWH system on your campus and raise community awareness for water conservation

Use the same description or here is another: Students will gain an understanding of the history, benefits, and components of a rainwater harvesting system and partner with community members to design and build a rainwater harvesting system for their school. Students will learn about rainfall patterns, the relationship between catchment area and rainwater volume and water use.

Res Type

Project, Service Learning Instructional Material: Project

Audience

None listed Educators, Students

Ed Level None listed but the actual resource has it on the 1st page. Middle School, High School, Higher Ed, Informal

Access Rts

None listed Free access

Language

None listed en-US

Metadata is Paramount

16

TeachEngineering Math Common Core Collection NSDL Science Refreshers Harvard Smithsonian Digital LibraryAlso: TeachingWithData Teachers' Domain Compadre SMILE MSP2

Most LAR Collections of NSDL

17

Examining NSDL Collections for LAR

What do the resources and metadata look like for individual collections?

18

Problem Resources & Metadata: GSDL

RESOURCES CONSIST OF: Standards documents from

the early 2000’s that are out of date or broken and not reachable

Reports that have been superseded by more recent reports on education issues

Good sounding career information but broken URLs

Not really learning resources but career info is valuable if links are updated

METADATA IS MISSING: Education level Resource type Access rights Rights Mime type Creation date

19

Good Resources - Metadata Needs Help: ChemCases

RESOURCES CONSIST OF: Web-based curriculum

supplements for teaching the 2nd semester of general chemistry

Freely available Textual tutorials with

explanatory images/diagrams

METADATA IS MISSING: Has education level but it

is in audience Has resource type (Text).

Should include tutorial, instructional & reference material

Audience Access rights Creation date

20

Good Metadata – Resources Need Help: Bridge Sea GrantRESOURCES CONSIST OF:

Has good resources also cataloged in other collections

These include: instructional materials, lesson plans

But: Overwhelmed by the

number of broken URLs Too many departmental

websites and landing pages in research journals

METADATA HAS: Education level Language Creator/publisher Mime type Resource type Rights Title/URL Description/keywords Creation date (sometimes) Missing access rights (has

cost instead) Missing audience

21

Good Resources & Metadata: OPL

RESOURCES CONSIST OF: Interactive resources for

teaching psychology Peer-reviewed materials Studies, data sets,

experiments, demonstrations

Teaching aid and instructor support

Some materials require class registration.

METADATA HAS: HAS: Education level Language Creator/publisher Mime type Resource type Rights Title/URL Description and

keywords Missing audience Missing creation date Missing access rights

22

LAR or Not?

RESOURCE QUESTIONS:

What are the resources?

What would be the perceived use of the materials?

Are the resources learning materials?

Are the materials only useful in an educational setting?

METADATA QUESTIONS: What information is

provided by the metadata? Are the metadata detailed

and appropriate for educational use of the resource.

Are metadata fields used correctly?

Can the metadata be used effectively & programmatically by learning applications?

23

Metadata Assessment of all Collections All 8 metadata

variables for each collections (Sept 2010): http://www.dls.ucar.edu/people/kginger/assessment/reports/

Audience, Ed Level and Resource type instantaneous for all collections: http://www.nsdl.org/browse/?subject=All

24

LAR Workshop Goals The NSDL Resource Quality Guidelines exist for

resources. What needs to be added if anything?

Develop NSDL Metadata Quality Guidelines to flesh out & formalize LAR ideas into criteria developed by and supporting the NSDL community

Develop a plan on how to apply the above docs/criteria across the heterogeneity of the NSDL?

How to promote LAR as a way of sustaining NSDL’s brand & reputation?