Website Comprehensibility Research Design

8/14/2019 Website Comprehensibility Research Design

1/14

Learner

Perception ProcessingInputsManifested

Behavior

Clustering

ObservedPatterns

LearningOpportunity

Advisory Engine

Classification

PedagogicalModel

Discrepancy Analysis FILL

LEARNER

SATISFACTION

Tests,

Questionnaires,

Implicit profiling

Pathways, an adaptive platform for learning and research


2/14

Website Comprehensibility

Determining an index for

information and instructional value


3/14

Comprehensibility Project

UASC and MIS collaboration

Determine whether an automated process

can be used to source and evaluate website pages for usage as learning material

Web presents a challenges for the self-

directed learner


4/14

Method

Used four professional librarians to source

and evaluate websites

Focus on STEM topics Use golden standard for human judgment

to determine a computational model to

simulate human judgment


5/14

Defining Comprehensibility

Extraction of instructional/learning value of

the information content in a web site

including the related links

The degree to which a web page provide

direct access to the substance of the

information of hypertext space without

distractions


6/14

Defining General Public Audience

Average reading level 8th grade

Assumed to have internet access

Internet and Information Literacy is notassumed

English reading proficiency

The population that is served by Librarians


7/14

STEM Topics

Science, Technology, Engineering, and

Math

Used 4 taxonomies

Flandrau Science Center

National Academies Press

Library of Congress

UNESCO


8/14

Website Selection

800 STEM sites selected by librarians

Varied quality levels

Not all topics are well represented on theweb

Used many searching strategies to find

content Eliminated many non educational websites


9/14

Site Evaluation

Skim or Read the site content with 10

minutes, click on related links as needed

to understand the site content

Rate site within 5 minutes

If prior knowledge of topic or familiar with

the specific website it was eliminated from

evaluation


10/14

Site Evaluation.

Concept recall to classify the website

Overall information and instruction rating

Detailed Rating

Information Value (text content)

Information Credibility (credibility)

Media Instructional Value (visual design clarity)

Affective Attention (overall appeal) Organization (structure, navigation)

Usability (functionality, interactivity)


11/14

Research Data

800 STEM website selected and

evaluated by 4 professional librarians

160 web page characteristics for each

website entry point being extracted and

analyzed

Info space model to define probability of

traversal


12/14

Model Implementation

Information Value Heuristic

Information Credibility Pattern matching

Media Instructional Value Heuristic + Pattern

Affective Attention Pattern

Organization -- Heuristic

Usability Heuristic


13/14

Challenges

What is comprehensibility of web content?

How do we define the information space?

How do we define the information andinstructional value of a website in amanner that may be computed?

Computational problem of determining

cognitive information processing andaesthetic reaction to content within thecontext of learning objective


14/14

Research Extension

After data analysis we will create an initial model, apply it

to the content then retest the index

Gather Learner demographic information

Connect learners to STEM content to gather feedbackdata and compare with Librarians evaluation

Expand content sampling through TPPL participation

Cluster learner and content to define research questions

more specifically Include task based analysis of scaffolding features

Website Comprehensibility Research Design

Documents

Transcript of Website Comprehensibility Research Design