August 2017

SIF AU Specification DevelopmentAssessment

Agenda – Tuesday 12th

1. Setting the Landscape (10-11)a. Introductions

a. Who is who? b. What do you hope to achieve over the next 2 days?

b. Where did we get to last year?c. Where are we up to now?

2. Confirm aim for the group.

This time last year…

Very basic support of formative assessment GradingAssignment, GradingAssignmentScore Drawing on work already done in SIF-US

Preliminary work on summative gradebook assessment (StudentGrade) Done with CEO Melb

NAPLAN-specific objects well underway Objects very much specific to NAPLAN structure Though basic structure is generalisable

At Previous Workshop: Key Findings

NAPLAN SIF objects will remain idiosyncratic It should be possible to use the same set of objects

for formative and summative assessment, at a first cut

Objects specific to particular assessment activities or use cases may be added as needed

At Previous Workshop: Immediate Changes

Existing objects fit for purpose. A few more links (Score to Grade; Assignment to

Sub-Assignment; Assignment to Learning Standard; Assignment to expected Score)

A few more attributes to Assignment, to contextualise it educationally

A few more attributes to Learning Standard, to genericise it

Immediate Changes: Assessment Instrument School authorities are rarely interested in gathering the

test instrument (assessment form, assignment); they are more interested in the assessment results. However the test instrument may be consulted to make sense of the results.

Assessment instrument represented in GradingAssessment as link to resource or binary attached resource (“AssessmentForm” in diagram)

Instrument is accompanied by a descriptor. At later stage, particularly in summative assessment,

AssessmentItem object may be provided. This is already done for NAPLAN.

Immediate Changes: Assessment Purpose The purpose of assessment differentiates assessments, as context

That includes where the assessment is intended for Purposes of assessment include:

Teachers scoring student work Teachers reporting on students’ progress vertically Teachers reporting on students’ progress to parents Diagnostic formative Paedagogical formative (which often ends up being diagnostic at a

different time scale)

The assessment purpose field in GradingAssignment will need to be enriched significantly, to provide proper context to assessment objects

Immediate Changes: Syllabus

Assessment needs to be contextualised against syllabus or curriculum being taught The syllabus is heterogeneous, and Australian Curriculum does not displace other syllabuses. An assessment may need to align to multiple curricula (e.g. International Baccalaureate) The syllabus can include local ad hoc entries, and entries specific to a student (learning plan). Cannot always mandate a syllabus link

Schools use syllabus to determine how much of curriculum covered through assessment

Both GradingAssignment and StudentGrade link to zero or more learning standards Learning standards are encoded as an option of: LearningStandardItem RefID; Australian

Curriculum URL; or Curriculum/Statement pair Not presupposed to be in the Australian Curriculum, or mapped to the Australian Curriculum Need not be presented within a hierarchy, or even be in a hierarchy Can belong to an individual learning plan, or an ad hoc learning objective Not all learning standard objects need belong to the same curriculum

Immediate Changes: Expected Score

Score received for assessment often contrasted against the expected or predicted score

Expected score can substitute for received score, where the received score is unavailable The student has not done the assessment, and will not be penalised for it

GradingAssignment has two links to GradingAssignmentScore. Current mandatory link remains, but becomes optional, and indicates the

received score. Additional optional link is added, indicates expected or predicted score.

Immediate Changes: Compositionality

Assessments can be aggregated into other assessments Assessments can be grouped by semester; by practical or written status; by

other categories Report of student’s progress over marking term is based on assessments

undergone during the term

A GradingAssignment can consist of other GradingAssignments, recursively

GradingAssignment/Category can describe a grouping of assessments. The StudentGrade object may link to the GradingAssignmentScore

objects that it aggregates over

Subsequent Changes: Shared Knowledge

Adding objects/attributes to represent context around assessment: Rubrics, Mark Info, Learning Standard Authorities Pre-Exchange handshaking will determine whether these contextual

objects will need to be exchanged, as mutual knowledge. The agreed knowledge between sender and receiver can change

for any two parties, and even for any given exchange. Within an enterprise, these contextual objects typically will not be

exchanged: their value will be understood. There may be a need for an external register of learning

standards (further discussion)

Subsequent Changes: Scoring Iterations

The scoring process may need to be captured Successive submissions of drafts can be tracked The student artefact (script) may be retained Successive attempts at an assessment may be scored differently

A Grading Assignment Submission Draft object will be added The object may include the student artefact The object is timestamped The object will indicate which iteration of submission the draft reflects Each Grading Assignment Submission Draft may be scored separately.

This is reflected in an optional link to a separate instance of GradingAssignmentScore

Agenda – Tuesday 12th

1. Gathering Business Drivers – Part 1 (11:15-12:45)a. NZ – Department of Education

SIF Assessment Workshop

NZ Education Sector Perspective

12, 13 September 2017

education.govt.nz

NZ Sector Projects

19

Prime driver is iED programme / SISI project (MoE - Ministry of Education)

Other agency initiatives are looking to mature information management and incorporate common data standards - NZQA (NZ Qualifications Authority) eg Digital Assessment - TEC (Tertiary Education Council)- Education Council

education.govt.nz20

education.govt.nz

SISI Data Hub

21

education.govt.nz22

sd Enrol Student - Retrieve Assessments & Achievements

:School Administrator

:SchoolManagement

System

«Service»:Student Personal

:National StudentRegister

«Service»:Student School

Enrollment

:SISI«Service»:Assessment Result

«Service»:Achievement

This scenario - excludes interactions for retrieval of student data other than assessment results and achievements- ignores interactions with ENROL - assumes that no interactions with NZQA required- assumes SMS will store local copy of student's historic assessment results and achievements.

Retrieve assessmentsand achievements()

Enter basic student information()

Store achievements()

Store assessment results()

inquire()

Find()

create()

inquire assessment results()

Search()

inquire()

Find matching students()

Select matching student()

create new enrolment()

inquire achievements()

Enrol student()

education.govt.nz

NZ Primary School Assessments

23

education.govt.nz

PAT

24

education.govt.nz

PAT

25

education.govt.nz

STAR

26

education.govt.nz

STAR

27

education.govt.nz

IKAN

28

education.govt.nz

IKAN

29

education.govt.nz

National Standards

30

education.govt.nz

National Standards

31

education.govt.nz

NZ Secondary School Assessments & Achievements

32

education.govt.nz

CourseOffered

33

education.govt.nz

NCEAResults

34

education.govt.nz

Cambridge

35

education.govt.nz

Achievement ResultsSecondary School

36

education.govt.nz

NZQA Record of Achievement

37

education.govt.nz

Conceptual Modelling

38

Concepts

• Curriculum == Programme + Component + Learning Standards + Competencies

• Tertiary: Programme Completion Qualification• Secondary: Credits (from LearningStandards) Qualification• Primary: Merely assessment to LearningStandards – ie: no

Qualifications.

9/13/2017 39

class Curriculum & Programme

Organisation

Name: String

Learning Programme

Education Level: StringTitle: StringVersion: String

Learning Standard

Component

Education Level: StringHours Per Year: DecimalIdentifier: StringLearning Area: StringSubject: StringTitle: StringVersion: String

NZQF Qualification

Qualification

Classification: StringCredit Value: IntegerEducation Level: StringIdentifier: StringTitle: StringVersion: String

Achievement Standard

Credit Value: IntegerIdentifier: StringTitle: StringVersion: Integer

Unit Standard

Credit Value: IntegerIdentifier: StringTitle: StringVersion: Integer

«enumerati...NZQF Level

Level 1Level 2Level 3Level 4Level 5Level 6Level 7Level 8Level 9Level 10

«enumeration»NZQF Qualification Type

Doctoral DegreeMaster's DegreePostgraduate DiplomaPostgraduate CertificateBachelor Honours DegreeBachelor's DegreeGraduate DiplomaGraduate CertificateDiplomaCertificate

Competency

CompetencySet

«enumeration»Programme Framework

New Zealand National StandardsCambridge International ExaminationInternational BaccalaureateNew Zealand Qualifications Framework

National Standard

Learning Area: stringYear Level: Year Level

«enumerati...Year Level

Year 1Year 2Year 3Year 4Year 5Year 6Year 7Year 8

Component Standard

Providers create their own Component definitions - child Components can be Learning Standards.

Provider

Provider Id: String

Provider Programme

Name: Curriculum & ProgrammeAuthor: Tony & StuartVersion: 1.0Created: 22/08/2017 12:00:00 a.m.Updated: 4/09/2017 11:16:39 a.m.

University Entrance, Vocational Pathways, and Literacy & Numeracy 'qualifications' are awarded by NZQA, based around accumulating appropriate numbers of specified credits.

Framework

is composed of

Courseis composed of

awards

is composed of

is composed of

is intendedresult of

defines

is composed of

Course

is composed of

NCEALevel

NZQF Level

NCEALevel

assesses

TeachingProvider

includes

may becomposed of

NZQF QualificationType

9/13/2017 40

Primary Assessment

• School performs multiple assessments• Against “National Standards” flavour of LearningStandard• Each Asessment has multiple ResultScores (eg. Mathematics,

has 9 ResultScores, 1 for each of 8 ‘Aspects’ (Competency) + 1 ‘Overall Teacher Judgment (OTJ)

9/13/2017 41

class Primary Sector - Assessment Results

Assessment Administration

Assessment Design

Assessment Scoring

Student

First NZ Schooling DateNational Student Number

Person

Date Of Birth Assessment Result

CommentPerformance LevelResult TypeWhen Assessed

Student Assessment

When Assessed

Assessment Session

When Scheduled

Result Score

MetricValue

Learning Standard

Organisation

NameAssessment Form

IdentifierTitle

Provider

Provider Id

Assessment Family

Default Score MetricShort Name

Assessment

Assessment Sub Test

IdentifierTitleWeightingNational

Standard

Learning AreaYear Level

Competency

CompetencySet

Name: Primary Sector - Assessment ResultsAuthor: Tony & StuartVersion: 1.0Created: 21/08/2017 12:00:00 a.m.Updated: 4/09/2017 11:08:50 a.m.

is result for

may becomposed of

is delivered as

comprises

is result for is composed of

consists of

is assessedagainst

isassessed

is conducted at

is composed of

may be scored by

AssessingProvider

schedules is composed of

is composed of

9/13/2017 42

Secondary Assesment• Schools & NZQA performs multiple assessments (Internal /

External)• Against “Unit Standard” & “Achievement Standard” flavours of

LearningStandard• AchievementStandards tend to have 1 ResultScore: on NAME scale• UnitStandards have multiple ResultScores on NA scale –

Competencies?• Don’t forget Cambridge (IGCSE, etc) & Internationale

Baccalaureate.

9/13/2017 43

class Secondary Sector - Assessment Results

Component

+ Education Level+ Hours Per Year+ Identifier+ Learning Area+ Subject+ Title+ Version

Learning Standard

«enumeration»Assessment Mode

External Internal

Programme Entry

PersonStudent

+ First NZ Schooling Date+ National Student Number

Learning Programme

+ Education Level+ Title+ Version

Component Entry

+ Entry Date

Assessment Result

+ Comment+ Performance Level+ Result Type+ When Assessed

Learning Standard Entry

+ Exam Answer Language+ Exam Paper Language

Organisation

Operational Entities

Knowledge Entities

«enumer...Result Status

Provisional Interim Final Result

Score

+ Metric+ Value

Provider Programme

Enrolment

+ Entry Date+ Reference Id+ School Enrolment Type

Instructional Period Enrolment

+ Academic Year+ Current Year Level+ Funding Year Level

CompetencySet

CompetencyComponent

Standard

Name: Secondary Sector - Assessment ResultsAuthor: Tony & StuartVersion: 1.0Created: 21/08/2017 12:00:00 a.m.Updated: 8/09/2017 10:33:01 a.m.

includes

is assessed via

is composed of

assesses

includes

+Courseis composed of

includes

comprises

may becomposed of

is composed of

is composed of

includes

is party to

is assessed via

may bescored by

records entry in

is composed of

is assessed via

9/13/2017 44

Secondary Achievement• Achievement is separate from Assessment.• Qualification Achievement is based upon rules about numbers of

Credits accumulated from LearningStandard passes.• Course Endorsement (Merit, Excellence)• NCEA Levels 1&2 – Endorsements (Merit/Excellence)

• NZQA doesn’t have concept of Qualification Entry - Qualifications are forensically awarded based on student’s collections of passed Standards.

9/13/2017 45

class Secondary Sector - Achievement Results

Component

Education LevelHours Per YearIdentifierLearning AreaSubjectTitleVersion

Learning Standard«enumeration»

Assessment Mode

ExternalInternal

Programme Entry

PersonStudent

First NZ Schooling DateNational Student Number

Learning Programme

Education LevelTitleVersion

Component Entry

Entry Date

Assessment Result

CommentPerformance LevelResult TypeWhen Assessed

Learning Standard Entry

Exam Answer LanguageExam Paper Language

Organisation

Operational Entities

Knowledge Entities

«enumer...Result Status

ProvisionalInterimFinal

Result Score

MetricValue

Provider Programme

Enrolment

Entry DateReference IdSchool Enrolment Type

Instructional Period Enrolment

Academic YearCurrent Year LevelFunding Year Level

CompetencySet

Competency

Component Standard

Name: Secondary Sector - Achievement ResultsAuthor: Tony & StuartVersion: 1.0Created: 21/08/2017 12:00:00 a.m.Updated: 8/09/2017 10:47:25 a.m.

Achievement Result

MetricResult DateValue

Qualification

ClassificationCredit ValueEducation LevelIdentifierTitleVersion

Achievement Justification

assesses

Course

is composed of

is assessed via

records entry in

awards

records entry inincludes

is composed of

justifies

comprises

is assessed via

is party to

includes

may becomposed of

recordsawarding of

includes

is justified by

is composed of

is assessed via

is composed of

may bescored by

includes

is composed of

is intendedresult of

9/13/2017 46

education.govt.nz

Options for Data Objects

47

9/13/2017 48

class Logical Data Objects - NZ

(Student)StudentPersonal

(Contact)StudentContactPersonal

StudentContactRelationship

PersonInfoType

(Staff)StaffPersonal

(Enrolment)StudentSchoolEnrollment

(Provider)SchoolInfo

TeachingGroup

(Component)SchoolCourseInfo

StudentAssessment

Assessment

Learning Standard

StudentAchievement

0..1

0..*0..10..*

1

0..1

1

0..*0..*1

0..1

0..1

0..*

10..*

0..1

0..*

0..1 0..*

1

0..*

0..*

1

0..* 1

0..*

0..*

0..1

0..*0..* 1

1

0..*

0..*

0..*

9/13/2017 49

class Assessment and Learning Standard (NZ)

Assessment

AssessmentFamily: AssessmentFamilyType [0..1]ContributionType: String [0..1]LearningArea: LearningAreaType [0..1]SessionDate: dateTime [0..1]

normalizedString

LocalIdType

GUIDType

RefIdTypeLearningStandard

Code: String [0..1]EducationLevel: EducationLevelType [0..1]Title: String [0..1](CreditValue) Weighting: String [0..1]

ProvidingAuthorityRefId

0..1

AssessmentProviderRefId

0..1

LocalId

0..1

RefId

1

LearningStandardRefId

0..1

RefId

1

9/13/2017 50

class Student Assessment and Achievement (NZ)

normalizedString

LocalIdType

GUIDType

RefIdType

StudentAssessment

AssessmentDate: dateTime [0..1]Component: ComponentType [0..1]EntryDate: dateTime [0..1]EntryStatus: AssessmentEntryStatusType [0..1]LanguageOfAnswer: LanguageType [0..1]LanguageOfExam: LanguageType [0..1]PerformanceLevel: PerformanceLevelType [0..1]ResultComment: String [0..1]ResultDate: dateTime [0..1]ResultStatus: AssessmentResultStatusType [0..1]

ResultScore

Competency: CompetencyType [0..1]Metric: MetricType [0..1]Value: normalizedString [0..1]

StudentAchievement

Award: AchievementTypeAwardedDate: dateTime [0..1]EducationLevel: EducationLevelType [0..1]Endorsement: String [0..1]LearningArea: LearningAreaType [0..1]

AchievementJustificationList

LocalId

0..1

StudentAssessmentRefId

0..*

RefId

1

JustificationList 0..1

RefId

1StudentRefId

1

LocalId

0..1

ProvidingAuthority

0..1

StaffPersonalRefId

0..1

AssessmentRefId

1

ResultScore 0..*

StudentRefId

1

education.govt.nz

National Standards

51

education.govt.nz

NCEA Results

52

education.govt.nz

Cambridge

53

education.govt.nz

Achievement Results

54

Agenda – Tuesday 12th

Lunch 12:45

Agenda – Tuesday 12th

Gathering Business Drivers – Part 2 (1:15-2:15) Other jurisdictions/vendors? What types of Assessment information need to

be exchanged? Further Assessment Requirements?

Impacts of NAPLAN online (2:15-2:45) Afternoon Tea (2:45)

Agenda – Tuesday 12th

Shared Terminology (3:00-4:00) Types of Assessment – agree on what to

support. Formative, summative, diagnostic, dynamic……?

Use Cases – Priorities Entities – Confirm Broadly Events – what are the triggers? Commands - what should be supported?