Simple Sequencing in Training Delivery and Management

Neil Cramer, Manager Instructional Systems Northwest Airlinesneil.cramer@nwa.com

Simple Sequencing

A method for representing the intended behavior of an authored learning experience

Consistent sequencing of learning activitiesBased on rules and conditions tied to

activitiesControls the launching of activities and access

to learning resourcesThe manifestation of an instructional strategy

SCORM 1.3 (IMS) Simple Sequencing

What it isn't: Simple AI based sequencing Schedule based sequencing Sequencing requiring data from closed systems

and services (e.g. embedded simulations) Collaborative learning Customized learning

Terminology

Learning Activity – a meaningful unit of instruction (potentially with sub-activities or access to learning resources in a defined hierarchy)Have a discreet start and endHave well defined completion and mastery conditionsSub-activities can be nested to any depthMay or may not have associated SCOs or SCAsMay be associated with Parent or Child activities a la

SCORM Content Aggregation Model

Terminology

Tracking Activity – association of tracking status data with a Learning Activity impacted by learner interactions and impacting learning sequencesNo specification on data persistence beyond

current login sessionActive/Passive Content – active content sets

values of elements in data tracking model, passive content assumes default values

Terminology

Activity Tree – represents the organization of all elements, hierarchies, and relationships between components relative to Content AggregationDerived from the instructional design

parametersImplemented and controlled by the LMS

Terminology

Cluster – an activity that consists of child activities. Includes the parent activity and its first level children but not the descendants of the child activities.Combined to form the Activity TreeThe building blocks of content aggregation and

sequencing

Activity Tree and Clusters

Course

Lesson 3Lesson 2Lesson 1

Module 1

Module 2

Module 1

Module 2

Module 3

Module 1

Module 2

Topic 1

Topic 2

Cluster

Cluster

Cluster Cluster

Cluster

Terminology

Activity Duration Tracking – the temporal model for the learning experience Attempt – effort to complete a learning activity. Applies across all

components of the parent/child hierarchy up through the root. Status parameters are changed to determine subsequent attempts versus resumption.

Suspension/Resumption – learner remains within previous attempt rather than tracking new attempt. More than one activity can be in suspension at one time.

Sequencing Session – period from start of sequencing process to completion for given activity.

SCORM Sequencing Definition Model

Elements that may be used to define intended sequencing behavior:

Sequencing Control Modes Sequencing Rules Limit Conditions Auxiliary Resource Rollup Rules Objectives Objective Map Rollup Controls Selection Controls Randomization Controls Delivery Controls

Sequencing Conditions and Actions/Behaviors

Conditions

Satisfied Activity Progress Known

Objective Status Known

Attempted

Objective Measure Known

Attempt Limit Exceeded

Objective Measure Greater

Than

Time Limit Exceeded

Objective Measure Less

Than

Outside Available Time

Range

Completed

Precondition Actions

Skip

Disabled

Hidden From Choice

Stop Forward Traversal

Postcondition Actions

Exit Parent

Exit All

Retry

Retry All

Continue

Previous

Exit Actions

Exit

ThenIF

(not)

Sequencing in IBM/Lotus LMS 1.0

Two controls provide for management and launching of activities:Choice – free movement through enabled

activities identified as clickable linksFlow – controls the way in which a student

proceeds (or recedes) through learning activities

Typical Sequencing Strategies

Choice – AllAll learning activities can be launched at any

timeStudent controls what they launch and in what

orderGood for overview and general informationDoes not include prerequisites or assessments

(at least ones that drive sequence)

Typical Sequencing Strategies

Flow – AllActivities can not be launched from the course

listActivities have "Continue" and/or "Previous"

buttons on screenFlow is controlled by course hierarchyGood for highly structured course controlled by

design

Typical Sequencing Strategies

Choice and Flow – AllAllows for free movement but encourages a

sequence Student will have clickable link in course list

but also see "Continue" and "Previous" buttons on screen

Good for course that has both new and returning students

Typical Sequencing Strategies

Choice, then FlowTop level activities are marked as choice, child

activities are marked as flowStudent can choose any activity but must work

through sub-activities in prescribed orderTypically high level activities are self contained

with no prerequisitesActivity always starts at the beginning even on

subsequent retries

Objectives

Objectives link performance with sequencingObjective Status – the objective's defined

goal has been achieved through launch or completion

Score (Normalized or Objective Measure) – value from 0 to 100 (or 0.00000 to 1.00000) as a percentage of correct answers on an assessment

Information Passing

Local ObjectiveAssociated with a single course itemScore and status affect that one itemOne local objective is tagged as primary and contains

roll up information for LMS Global Objective

Passes information between local objectivesThe only type that can be accessed outside the scope of

a single course itemMust have unique ID throughout entire LMS

Basic Architecture

Course item Primary Objective

Sequencing Rule

Topic ParametersChoice = False

Flow = True

Objective Satisfied Objective not Satisfied

Course item

"Stop Forward Traversal"

Objective Satisfied by MeasureMinimum Normalized Measure = .80

. . .

Multiple Objectives Example

Pre-test assess two types of knowledge –basic and advanced

Must repeat assessment until basic knowledge is mastered

If basic knowledge is mastered but advanced knowledge is not student receives remediation

When both basic and advanced knowledge are mastered student moves to next topic

Multiple Objectives Example

Course

Topic 1

Topic 2

Content

Assessment

Basic Question(s)

Advanced Question(s)

Quiz Score

Choice = FalseFlow = True

Choice = FalseFlow = True

"Basic"

"Advanced"

By ObjectiveIs Master Exam = TrueReport Objectives = True

Objectives#1 Basic – Objective satisfied by measure = .80#2 Advanced – Objective satisfied by measure = .80

Sequencing Rules#1 If objective "basic" is not satisfied then retry#2 If objective "advanced" is not satisfied then

previous

Global Objective Example

Pre-test determines pathIf pre-test is passed, student proceeds to

advanced contentIf pre-test is failed, student proceeds to

introductory contentGlobal objective used to pass data from pre-

test to the introductory content

Global Objective Example

Course

Topic 1

Advanced

Pre-test

Question(s)

Quiz Score

Choice = FalseFlow = True

Global Objective – data container

Primary Local Objective - Objective satisfied by measure = .80

By ObjectiveIs Master Exam = True

Sequencing Rule(Pre-condition Rule)

Skip if Mastery > .80

Intro

Primary Local Objective - Objective satisfied by measure = .80Export Map

Import Map

Sample SCORM Sequencing Models

No SequencingStudent can choose learning activities in any

orderWhen all components of a cluster are complete

the cluster is completeIf all questions return "passed" the whole

course is passedChoice = ALL

Sample SCORM Sequencing Models

LinearGuided flowStudent progresses through course in predetermined

orderMust complete one component before moving to the

nextCluster is complete when all components are completeNo intermediary assessments, only final assessmentFlow = ALL

Sample SCORM Sequencing Models

Learner ControlledMust complete introduction, then presented with menu

of modulesEach module lists its component lessonsStudent can select any module, but not a lessonEach module has assessmentStudent may not attempt module more than onceChoice, then Flow

Sample SCORM Sequencing Models

Knowledge PacedStudent must complete introduction, then proceed to

module 1 assessment If assessment is passed, move to module 2 assessment If assessment is not passed, move to module, then

assessment again Etc. for all modulesFlow = ALL w/ sequencing instruction to repeat on

failure

Sample SCORM Sequencing Models

RemediationStudent completes introduction and follows linear

sequence through modulesStudent takes comprehensive assessmentOn pass course is completeOn fail of any given section of assessment (by

objective), student is moved to associated content module

On completion of remediated module student retakes module assessment

Flow = ALL w/ sequencing rules to repeat on specific objective(s) failure

Sequencing Controls

Meaning Options

Choice When set to True, indicates that students can choose which course items to complete.

True, False

Choice Exit When set to True, indicates that students can exit an activity that is part of a choice sequence.

True, False

Flow When set to True, indicates that the student has the ability to use Continue or Previous navigation.

True, False

Forward Only When set to True, students may not go back to previous course items higher in the course hierarchy.

True, False

Rollup Controls  Meaning Options

Rollup Objective Satisfied

When set to True, the status of the objective associated with the course item is included in the rollup calculations.

True, False

Rollup Objective Measure Weight

When greater than one, the objective for rolled-up items are weighted more heavily. (A weight of two would weigh these items twice as heavily as others, for example.)

A number of one or greater.

Rollup Progress Completion

When set to True, the complete and incomplete status is included in the rollup calculations.

True, False

Rollup Progress Duration

When set to True, the amount of time spent launching and working on the course items are included in the rollup calculations.

True, False

Limit Controls  Meaning Options

Attempt Control When set to True, students are bound by the Attempt Limit. True, False

Attempt Limit The number of attempts that are permitted for the course item. A number

Attempt Wallclock Duration Control

When set to True, students are bound by the Attempt Wallclock Duration Limit.

True, False

Attempt Wallclock Duration Limit

The amount of time students are allowed for one attempt on the course item.

A time limit in days, hours, and minutes.

Attempt Interaction Duration Control

When set to True, students are bound by the Attempt Interaction Duration Limit.

True, False

Attempt Interaction Duration Limit

The amount of time students can spend actively working on a course item in a single session. It does not include time when the course item is open, but not being worked on.

A time limit in days, hours, and minutes.

Wallclock Duration Control

When set to True, students are bound by the Wallclock Duration Limit. True, False

Limit ControlsMeaning Options

Wallclock Duration Limit The total amount of time students are allowed to have a course item open, across multiple sessions.

A time limit in days, hours, and minutes.

Interaction Duration Control When set to True, students are bound by the Interaction Duration Limit.

True, False

Interaction Duration Limit The total amount of time students can spend actively working on course items for all attempts they make.

A time limit in days, hours, and minutes.

Time Limit Control When set to True, students are bound by Begin Time Limit and End Time Limit.

True, False

Begin Time Limit Specifies on which day, hour, and minute of the course timeline the course item becomes available.

A time limit in days, hours, and minutes.

End Time Limit Specifies on which day, hour, and minute of the course timeline the course item becomes unavailable.

A time limit in days, hours, and minutes.

Delivery Controls Meaning Options

Tracked When set to True, the objective and progress data is recorded and the data contributes to the rollup for the parent course item.

True, False

Completion Set by Content

When set to True, indicates that the content sets the completion status. When set to False, progress data is sent by default tracking rules, rather than waiting for data sent from the content.

True, False

Objective Set by Content

When set to True, indicates that the content sets the status for the primary objective. When set to False, objective status is set by default tracking rules.

True, False

Rollup Rules Meaning Options

Child Set Determines how many child items (those contained as children of the course item in the course hierarchy) must meet the Rollup Condition. At Least Count is a minimum number of child items; At Least Percent is a minimum percentage of the child items.

All, Any, None, At Least Count, At Least Percent

Number Field The number that applies to At Least Count or At Least Percent Child Sets.

For "At Least Count," this is any number from 1 to 100.

For "At Least Percent," this is a number from 0 to 100 and you can specify up to five digits to the right of the decimal place; for example, 57.48529.

Rule Conditions Evaluate to Determines if all conditions for the rule must be met or at least one, in order for this rule to take effect.

All True, Any True

Specify Condition (operator) The selected condition is either stated as is with {none} or negated with Not.

{none}, Not

Specify Condition (condition) The course item status for which this rule applies.

Satisfied, Objective Status Known, Completed, Activity Progress Known, Attempted, Attempt Limit Exceeded, Time Limit Exceeded, Outside Available Time Range, and Always.

Perform Action The progress that is marked for the parent item when the condition is met.

Complete, Incomplete, Not Satisfied, and Satisfied

Sequence Rules Meaning Options

Rule Conditions Evaluate to Determines if all conditions for the rule must be met or at least one, in order for this rule to take effect.

All True, Any True

Specify Condition (operator) The selected condition is either stated as is with {none} or negated with Not.

{none}, Not

Specify Condition (condition) The course item status for which this rule applies.

Satisfied, Objective Status Known, Objective Measure Greater Than, Objective Measure Less Than, Completed, Activity Progress Known, Attempted, Attempted Limit Exceeded, Time Limit Exceeded, Outside Available Time Range, and Always.

Action Group Determines when this rule becomes active. Pre Condition Sequencing Rule, Exit Condition Sequencing Rule, Post Condition Sequencing Rule

Perform Action States what navigation occurs when the rule is applied.

Disabled, Hidden From Choice, Skip, Stop Forward Traversal. Choices depend on the Action Group selected.