Mathematics Assessment by Group 2

Overview of PresentationOverview of PresentationOverview of PresentationOverview of Presentation1. Deciding on a developmental continuum

1.1 The Context, the task & Purpose of task 1.2 Framework of Mathematics in National

Curriculum1.3 Components in National Curriculum framework1.4 Learning Framework: the Dreyfus model

2. Developing a standards referenced assessment framework2.1 Building a pedagogical framework2.2 Identifying performance indicators2.3 Breaking down the domain2.4 Quality criteria & initial rubrics

3. Drafting the assessment items 3.1 Questions on test4. Panelling

4.1 Procedure4.2 Issues, concerns and themes

4.3 Comments from panellists5. Revision of Assessment framework and test

items5.1 Quality criteria & amended rubrics5.2 Further amendments & revised rubrics5.3 Final performance matrix

5.4 Revised test items

6. Implementation of assessment 6.1 Test conditions 6.2 Instructions for administrator of test7. Analysis of results

7.1 Zone of proximal development7.2 Guttman chart7.3 Analysis: Commonalities7.4 Analysis: Anomalies

8. Reporting 8.1 Scope of assessment and audience

8.2 Reporting for students & parents8.3 Reporting for teachers & school

9. Discussion 9.1 Intervention 9.2 Reliability & Validity 9.3 Scaling up10. Group Reflection 10.1 Reflection on the assessment task 10.2 Reflection on how the

interpretation was done 10.3 Reflection: last words

1 Deciding on a 1 Deciding on a Developmental ContinuumDevelopmental Continuum

1 Deciding on a 1 Deciding on a Developmental ContinuumDevelopmental Continuum

1.1 The Context, the Task & Purpose of task 1.2 Framework of Mathematics in National Curriculum1.3 Components in National Curriculum framework1.4 Learning Framework: the Dreyfus model

1.1 The Context, the Task & Purpose of task 1.2 Framework of Mathematics in National Curriculum1.3 Components in National Curriculum framework1.4 Learning Framework: the Dreyfus model

1.1 The Context1.1 The Context1.1 The Context1.1 The Context

General profile

•Autonomous, all-girls’ school•Upper Secondary •15 year olds, Express stream•Science subject combination

Class profile

•Generally weak in Math•Girls are very ‘Arts-driven’ (Drama) and not keen on the subject – preference for the Humanities•Students are self-motivated and are highly disciplined

1.1 The task1.1 The task1.1 The task1.1 The taskMathematics Syllabus D

Secondary 3 Express stream

30 minutes individual written testIndices & Algebraic Manipulations

1.1 Purpose of task1.1 Purpose of task1.1 Purpose of task1.1 Purpose of task

Formative assessmentThis test is conducted to allow students and teacher to check their knowledge and

understanding of Indices and Algebraic manipulation

From the assessment, students and teacher can strategize on how to improve

in the areas they are weak in.

1.2 Framework for Mathematics in 1.2 Framework for Mathematics in National CurriculumNational Curriculum

1.2 Framework for Mathematics in 1.2 Framework for Mathematics in National CurriculumNational Curriculum

1.3 Components in Mathematics 1.3 Components in Mathematics National Curriculum frameworkNational Curriculum framework1.3 Components in Mathematics 1.3 Components in Mathematics National Curriculum frameworkNational Curriculum framework

Skills ◦ Expansion and factorisation of algebraic expressions

(both)◦ Recognising and applying special products

Concepts Processes

◦ Application◦ Reasoning, communication and connections◦ Thinking skills and heuristics

Attitudes Metacognition

These three components

will be explored in

this pilot study

1.4 Learning framework: 1.4 Learning framework: the Dreyfus Modelthe Dreyfus Model

1.4 Learning framework: 1.4 Learning framework: the Dreyfus Modelthe Dreyfus Model

NoviceRigid adherence to taught rules or plans; Little situational perception; No discretionary judgment Having knowledge and remembering (recalling) of appropriate, previously learned information such as terminology, specific facts or ways and means of dealing with specific conventions and sequences. In this case, having knowledge of the mathematical nomenclatures in standard forms and numbers, and performing simple sequences in manipulation of numbers.

Advanced BeginnerGuidelines for action are based on attributes or aspects (aspects are global characteristics of situations that can be recognised only after some prior experience); Unable to see the entirety of a new situation (may miss some critical details)

CompetentPerceives actions at least partially in terms of longer-term goals; Conscious deliberate planning; Standardised and routinised procedures. Uses previously learned information in new and concrete situations to solve problems that have single or best answers.

ProficientSees situations holistically rather than in terms of aspects; See what is most important in a situation; Perceives deviations from the normal pattern and possesses a web or perspectives on a situation. For example when the students are able to identify strategies, differentiate and apply multiple laws of indices. Decision making is less laboured; Uses maxims for guidance, whose meaning varies according to the situation. A maxim is a brief expression of a simple truth, a code of conduct or cryptic instructions that make sense only if there is already a deep understanding of the situation.

ExpertNo longer relies on rules, guidelines or maxims; Operates from a deep understanding of the total situation; Analytical approaches are only used in novel situations or when a problem occurs. The student is able to manipulate complex algebraic expressions combined with laws of indices, and has a vision of what is possible.

2 Developing a standards 2 Developing a standards referenced assessment referenced assessment

frameworkframework

2 Developing a standards 2 Developing a standards referenced assessment referenced assessment

frameworkframework

2.1 Building a Pedagogical Framework2.1 Building a Pedagogical Framework2.1 Building a Pedagogical Framework2.1 Building a Pedagogical Framework

Pedagogy

DOMAIN: These are sets of skills, knowledge, behaviours and dispositions that enable us to sample and define phases, the strands or constructs

STRANDS & CAPABILITY: The big ideas that are learnable, teachable. They too are the set of skills, knowledge and expectations or learning outcomes

1

INDICATOR: Identified behavioural indicators. These indicative behaviours were things that a student could do, say, make or write, and from which we infer their capability in an area

CRITERIA: Finally, observational statements that detailed ‘how well’ each behaviour could be performed were created. These criteria underpin the profiling of learning pathways. The criteria are represented by ordered levels of increasing difficulty, sophistication, elegance, etc…

23n

2.2 Performance Indicators2.2 Performance Indicators2.2 Performance Indicators2.2 Performance IndicatorsManipulation of numbers in standard form notations

◦ Knowledge of mathematical nomenclature in standard form and numbers

◦ Comparison of numbers

Manipulation of algebraic expressions Application of laws of indices and conceptsMathematical reasoning and communication

*Relying on professional knowledge and experience of certain members of the group

2.3 Breaking 2.3 Breaking down the domaindown the domain2.3 Breaking 2.3 Breaking down the domaindown the domain

Manipulation of standard form

Manipulation of algebraic expressions

Application of Concepts Mathematical reasoning, communication and connections

Expert Compare and contrast alternative methods in the use of different laws

Explain errors / misconceptions in the: – algebraic manipulation, - use of the laws of indices

Proficient Perform the four operations in standard forms expressions

Manipulate algebraic fractions and polynomials (with positive, negative, zero and fractional indices), showing consistent and clear working

Apply multiple laws of indices concurrently

Competent Identify large and small numbers such as giga, micro, pico

Manipulate algebraic expressions with positive, negative, zero or fractional indices

Differentiate the laws of indices and apply laws appropriately

Identify errors in the four operations on indices, surds and use of laws of indices

Novice & Advanced Beginner

Express numbers in standard form

Perform four operations on simple algebraic fractions

2.4 Quality Criteria & Initial Rubrics2.4 Quality Criteria & Initial Rubrics

Manipulation of standard form


Application of Concepts Mathematical reasoning, communication and connections

Expert Compare and contrast alternative methods in the use of different laws

Explain errors / misconceptions in the: – algebraic manipulation, - use of the laws of indices

Proficient Perform the four operations in standard forms expressions

Manipulate algebraic fractions and polynomials (with positive, negative, zero and fractional indices), showing consistent and clear working

Apply multiple laws of indices concurrently

Competent Identify large and small numbers such as giga, micro, pico

Manipulate algebraic expressions with positive, negative, zero or fractional indices

Differentiate the laws of indices and apply laws appropriately



Express numbers in standard form

Perform four operations on simple algebraic fractions

2.4 Quality Criteria & Initial Rubrics2.4 Quality Criteria & Initial Rubrics

Students are introduced to the nomenclature, and learn to express familiar numbers in simple standard forms. At competent level, the manipulation of very small and very large numbers would be done. Moving to the proficient level, the students would be able to perform the operations, namely the addition, subtraction, multiplication and division of standard form expressions.

At the competent level, students would be able to manipulate algebraic expressions and fractions with indices (or powers). The nature of the math curriculum is spiral, such that students would be taught algebraic manipulation with increasing complexity. At this secondary three level, they would be taught how to manipulate algebraic fractions with indices.

The laws of indices is a major concept at upper secondary, so it was not considered as a novice or advanced beginner level.At the competent level, students are expected to be able to differentiate which laws to use, and to apply the laws appropriately. They would only be required to apply the isolated laws. As mentioned in the learning framework, there would be conscious deliberate planning. Students would show standardized and routinized procedures in applying each law.

The ability to articulate reasons, and communicate strategies and connections was beyond a novice or advanced beginner level, as students at lower secondary level need only solve questions in routinized manner, and rarely requiring them to state the reasons or laws used.At competent level, the students are able to identify and explain errors in the use of indices. It would be inferred that they have understood and concept, and could communicate their understanding using mathematical reasoning.

3 Drafting the Assessment 3 Drafting the Assessment ItemsItems

3 Drafting the Assessment 3 Drafting the Assessment ItemsItems

3.1 Questions on test3.1 Questions on testQuestion 1:

Manipulation of numbers in standard form notations - Comparison of numbers

Question 2: Manipulation of numbers in standard form notations - Knowledge of mathematical

nomenclature in standard form and numbers

Question 3: Manipulation of numbers in standard form notations

Question 4 & 5:◦ Manipulation of algebraic expressions ◦ Application of laws of indices and concepts

Question 6 & 7: ◦ Manipulation of algebraic expressions ◦ Application of laws of indices and concepts◦ Mathematical reasoning and communication

4 Panelling4 Panelling4 Panelling4 Panelling

4.1 Procedure4.1 Procedure4.1 Procedure4.1 Procedure

Panelling refers to: checking of the test items and initial rubric using a group (panel) of specialists chaired by one member of the group appointed as a leader

Five teachers checked the test items and rubrics.

Reviewing / panelling of draft test items:• Making specific ideas for change• Checking the learning area and year level• Identifying content range and gaps

Each panel member reviews their item and makes notes about faults and recommendations to fix the fault

Step 2: Panelling4.2 Issues, concerns and themes4.2 Issues, concerns and themes4.2 Issues, concerns and themes4.2 Issues, concerns and themes

• Refine questions to target students’ level and syllabus

• Rephrase questions to avoid ambiguity and biases

• Adjust criteria and levels

Step 2: Panelling4.3 Comments from panellists4.3 Comments from panellists4.3 Comments from panellists4.3 Comments from panellists

Refine questions to target students’ level and syllabus“I personally feel that for half hour we should keep to max 6 questions”“Are the questions too difficult for the students? Are they able to explain errors? Does the syllabus require them to do it?”

Rephrase questions to avoid ambiguity or biases“For question 1, should it be 'rank the following expressions numbers 1 to 4... with 1 representing highest value and 4 lowest " instead of 1 to 4?”

“Not sure which skill / concept question 1 is testing on the rubric - but I'm wondering if there is a chance of a student getting it right by guesswork? Or if they did get it wrong, can we tell the misconception from their answers? i.e. whether they had 1,3,2,4, or 3,2,1,4 etc. If so, should this be reflected in the rubrics?”

Step 2: Panelling4.3 Comments from panellists (con’d)4.3 Comments from panellists (con’d)4.3 Comments from panellists (con’d)4.3 Comments from panellists (con’d)

Adjust criteria and levels, ensure clarity and common understanding in criteria“I'm wondering for the last column if we can bring up the identifying error from competent to proficient…Also for standard form, there‘re no questions testing this criteria on identifying error…”

“the misconceptions in question 7 don't only specify laws of indices so I'm wondering if this will mean that they first need to have the ability to identify that the error is in the use of the laws?”

“what happens if students explain misconception in question 7, but did not show correct steps? … we will have to determine if the incorrect steps were due to careless mistakes, if it is conceptual, then the student does not achieve proficient level.”

“if student could identify errors in 4 operations (Competent), wouldn’t it mean that he could perform 4 operations in std form (Proficient)?”

“For 'Application of Concepts' column, is it possible for a student to be able to compare and contrast alternative methods (Expert) but not able to apply multiple laws of indices concurrently (Proficient)?”

“For 'Manipulation of Expressions' column, does Competent need to be able to show consistent and clear working as well (compared to Proficient)”

5 Revision of Assessment 5 Revision of Assessment Framework and Test ItemsFramework and Test Items5 Revision of Assessment 5 Revision of Assessment Framework and Test ItemsFramework and Test Items

5.1 Quality criteria & amended rubrics5.1 Quality criteria & amended rubrics5.1 Quality criteria & amended rubrics5.1 Quality criteria & amended rubrics Indicators >

Levels

Manipulation of numbers in standard form notations


Application of Concepts

Mathematical reasoning, communication and connections

Expert Compares and contrasts alternative methods in the use of different laws, and multiple laws

Explains errors / misconceptions in the:– algebraic manipulation, - use of the laws of indices

Proficient Performs operations in standard forms expressions

Manipulates algebraic fractions and polynomials (with positive, negative, zero and fractional indices), showing consistent and clear working

Applies multiple laws of indices concurrently


Competent Identifies large and small numbers such as giga, micro, pico

Manipulates algebraic expressions with positive, negative, zero or fractional indices, showing clear working.

Differentiates the laws of indices and apply laws


Expresses numbers in standard form

Performs operations on simple algebraic fractions

Amended rubric with Katie’s comments on 4 Feb 2010

5.2 Further amendments and revised rubrics5.2 Further amendments and revised rubrics

Seven individual rubrics were then set up for each question, to allow more clarity in the assessment of the students’ level.

At the same time, the questions, as well as the criteria and indicators required further refinement to the indicators and criteria - to reflect the skills and observable behaviours more accurately.

From the developmental perspective, we became more mindful of the considerations to be made with regards to the:

Content: Can the content (test items) define students’ learning levels?

Developmental levels: Can the order of levels reflect the developmental sequence?

5.3 Final performance matrix5.3 Final performance matrix5.3 Final performance matrix5.3 Final performance matrix

Manipulation of numbers in standard form notations

Knowledge of mathematical nomenclature in standard form and numbers

Comparison of numbers


Application of laws of indices and concepts

Mathematical reasoning and communication in algebraic manipulation

Mathematical reasoning and communication in indices

Cut off points

Expert Manipulates algebraic fractions and polynomials (with positive, negative, zero indices), showing consistent and clear working

Explains strategies used in the algebraic manipulation

13 to 14

Proficient Applies multiple laws of indices concurrently

States the types of algebraic manipulation

States the operations on indices, surds or laws of indices used in the working

10 to 12

Competent Performs operations involving standard form expressions

Manipulates algebraic expressions with positive and negative indices, showing clear working.

Differentiates the laws of indices and applies the law(s)

7 to 9

Advanced beginner

Manipulate numbers with negative, positive powers and standard forms by moving decimal places

Compares small and large numbers by using strategies e.g. converting to common powers or common forms

Performs operations on single algebraic expressions

4 to 6

Novice Manipulates numbers with either negative or positive powers by moving decimal place in a single direction

Expresses small and large numbers using standard form, giga, micro, pico etc.

Compares numbers without showing clear working.

1 to 3

See slide notes for explanation

Question 1: Manipulation of numbers in standard form notations Comparison of numbers

Question 2: Manipulation of numbers in standard form notations Knowledge of mathematical nomenclature in standard form and numbers

Question 3: Manipulation of numbers in standard form notations

Question 4: ◦ Manipulation of algebraic expressions ◦ Application of laws of indices and concepts◦ Mathematical reasoning and communication – in (i) algebra and (ii) indices

5.4 Revised test items5.4 Revised test items5.4 Revised test items5.4 Revised test items

5.4 Revised test items5.4 Revised test items5.4 Revised test items5.4 Revised test itemsQ1. Rank the following expressions using numbers (1), (2), (3) and (4) in

ascending order, with (1) representing the smallest value and (4) representing the largest value.

Ranking: ( ) ( ) ( ) ( )

340 × 10−4 , 0.000034 × 104 , 0.034 × 102 , 3·4 × 10−3

Show Working:

Q2. Convert the following lengths to metre, leaving your answers in standard form240 gigametres =

1370 picometres =

Q3. Given that a = 80 × 106 , and b =1.9 × 104 , express a + b in standard form.

Indicator 1: Manipulation of numbers in standard form notations will be shown through students’ working.

Indicator 3: Comparison of numbers. At advanced beginner level, students will show strategies via their manipulation.

Indicator 1: Manipulation of numbers in standard form notations Indicator 2: Knowledge of mathematical nomenclature in standard form and numbers

Indicator 1: Manipulation of numbers in standard form notations. Students at competent level would be able to perform the operation in standard form.

5.4 Revised test items5.4 Revised test items5.4 Revised test items5.4 Revised test itemsQ4. Simplify the following algebraic expression, leaving your answers in

a single fraction and positive indices. State mathematical reasons and strategies for every step that you take. Write these reasons and strategies on the same line as your working, stating the laws of indices where applicable. Working Reasons/ strategies

25142

827

22

xyyxyx

xyyx

Indicator 4: Manipulation of algebraic expressions

Indicators 6 and 7:Mathematical reasoning and communication – in (i) algebraic manipulation (indicator 6) and (ii) indices (indicator 7)

Indicator 5: Application of laws of indices and concepts

6 Implementation of 6 Implementation of AssessmentAssessment

6 Implementation of 6 Implementation of AssessmentAssessment

6.1 Test conditions6.1 Test conditions6.1 Test conditions6.1 Test conditions

•Conducted by a trainee teacher•During timetabled Mathematics period•Students did not have any remedial or re-cap of the topics prior to test as the topic of Indices was newly covered earlier in the year. •Algebraic manipulations topic was covered in 2009.

6.2 Instructions for administrator 6.2 Instructions for administrator of testof test

6.2 Instructions for administrator 6.2 Instructions for administrator of testof test

1. All bags & notes are placed in front of the class.2. Write time duration, start and end on the board.3. Remind students to do their work individually without

reference to any notes.4. Students should only have their stationery on the table. No

calculators are allowed.5. Distribute the test face-down and instruct students not to

turn over until told to do so.6. Cue for start following class clock, 10 minutes before end and

end time.

• NOTE: the school has a protocol for written tests and exams and all staff will be able to conduct the test in the manner expected.

7 Analysis of Results 7 Analysis of Results 7 Analysis of Results 7 Analysis of Results

7.1 Zone of Proximal Development (ZPD)7.1 Zone of Proximal Development (ZPD)

Vygotsky“...a state of readiness in which a student will be able to make certain kinds of conceptual connections, but not others; anything too simple for the student will quickly become boring; anything too difficult will quickly become demoralising”.

So, when is a student ready to learn?

A zone in which an individual can learn more with assistance than he or she can manage alone.

Name 1a 3a 3b 1b 1c 4a 2a 4b 5a 6a 7a 6b5b

4c Score Level

Norazah 1 1 1 1 1 1 1 1 1 1 1 1 1 1 14 5

Ke Tian 1 1 1 1 1 1 1 1 1 1 0 1 1 0 12 4

Marissa 1 1 1 1 1 1 1 1 1 1 1 1 0 0 12 4

Stacy 1 1 1 1 1 1 0 1 0 1 1 1 0 0 10 4

Isabelle 1 1 1 1 1 1 0 1 1 0 1 0 1 0 10 4

Suzanne 1 1 1 1 1 1 0 1 1 0 0 0 1 0 9 3

Kelly 1 1 1 1 1 1 1 1 1 0 0 0 0 0 9 3

Dianne 1 1 1 1 1 1 1 1 0 0 0 0 0 0 8 3

Cherlyn 1 1 1 1 1 1 0 1 1 0 0 0 0 0 8 3

Nasirah 1 1 1 1 0 1 0 1 1 0 0 0 0 0 7 3

Nur Zahwah 1 1 1 1 1 0 0 0 1 0 0 0 0 0 6 2

Eunice 1 1 1 1 1 0 1 0 0 0 0 0 0 0 6 2

Namrata 1 1 1 1 1 0 1 0 0 0 0 0 0 0 6 2

Amanda 1 1 1 1 1 0 0 0 0 1 0 0 0 0 6 2

Celestine 1 1 1 1 0 1 0 0 0 0 0 0 0 0 5 2

Sharifah 1 1 1 1 0 0 1 0 0 0 0 0 0 0 5 2

Peo Shan 1 1 1 1 1 0 0 0 0 0 0 0 0 0 5 2

Cheryl 1 1 1 1 1 0 0 0 0 0 0 0 0 0 5 2

Nicole 1 1 1 1 0 0 0 0 0 0 0 0 0 0 4 2

B. Rohini 1 1 1 1 0 0 0 0 0 0 0 0 0 0 4 2

Gitanjali 1 1 1 1 0 0 0 0 0 0 0 0 0 0 4 2

Annetta 1 1 1 0 0 0 0 0 0 0 0 0 0 0 3 1

41 41 40 37 24 23 19 18 15 13 10 10 8 6

7.2

Gu

ttm

an

Ch

art

7.2

Gu

ttm

an

Ch

art

7.2

Gu

ttm

an

Ch

art

7.2

Gu

ttm

an

Ch

art

At this level, the recommended next intervention is to strengthen manipulation of

algebra

At this level, the recommended next intervention is to strengthen manipulation of

numbers in standard form

At this level, the recommended next intervention is to strengthen reasoning and

application of multiple laws for indices

At this level, the recommended next intervention is to strengthen reasoning for

algebra and manipulation of fractional algebra


4c Score Level

Norazah 1 1 1 1 1 1 1 1 1 1 1 1 1 1 14 5

Ke Tian 1 1 1 1 1 1 1 1 1 1 0 1 1 0 12 4

Marissa 1 1 1 1 1 1 1 1 1 1 1 1 0 0 12 4

Stacy 1 1 1 1 1 1 0 1 0 1 1 1 0 0 10 4

Isabelle 1 1 1 1 1 1 0 1 1 0 1 0 1 0 10 4

Suzanne 1 1 1 1 1 1 0 1 1 0 0 0 1 0 9 3

Kelly 1 1 1 1 1 1 1 1 1 0 0 0 0 0 9 3

Dianne 1 1 1 1 1 1 1 1 0 0 0 0 0 0 8 3

Cherlyn 1 1 1 1 1 1 0 1 1 0 0 0 0 0 8 3

Nasirah 1 1 1 1 0 1 0 1 1 0 0 0 0 0 7 3

Nur Zahwah 1 1 1 1 1 0 0 0 1 0 0 0 0 0 6 2

Eunice 1 1 1 1 1 0 1 0 0 0 0 0 0 0 6 2

Namrata 1 1 1 1 1 0 1 0 0 0 0 0 0 0 6 2

Amanda 1 1 1 1 1 0 0 0 0 1 0 0 0 0 6 2

Celestine 1 1 1 1 0 1 0 0 0 0 0 0 0 0 5 2

Sharifah 1 1 1 1 0 0 1 0 0 0 0 0 0 0 5 2

Peo Shan 1 1 1 1 1 0 0 0 0 0 0 0 0 0 5 2

Cheryl 1 1 1 1 1 0 0 0 0 0 0 0 0 0 5 2

Nicole 1 1 1 1 0 0 0 0 0 0 0 0 0 0 4 2

B. Rohini 1 1 1 1 0 0 0 0 0 0 0 0 0 0 4 2

Gitanjali 1 1 1 1 0 0 0 0 0 0 0 0 0 0 4 2

Annetta 1 1 1 0 0 0 0 0 0 0 0 0 0 0 3 1

41 41 40 37 24 23 19 18 15 13 10 10 8 6

7.2

Gu

ttm

an

Ch

art

7.2

Gu

ttm

an

Ch

art

7.2

Gu

ttm

an

Ch

art

7.2

Gu

ttm

an

Ch

art


4c Score Level

Norazah 1 1 1 1 1 1 1 1 1 1 1 1 1 1 14 5

Ke Tian 1 1 1 1 1 1 1 1 1 1 0 1 1 0 12 4

Marissa 1 1 1 1 1 1 1 1 1 1 1 1 0 0 12 4

Stacy 1 1 1 1 1 1 0 1 0 1 1 1 0 0 10 4

Isabelle 1 1 1 1 1 1 0 1 1 0 1 0 1 0 10 4

Suzanne 1 1 1 1 1 1 0 1 1 0 0 0 1 0 9 3

Kelly 1 1 1 1 1 1 1 1 1 0 0 0 0 0 9 3

Dianne 1 1 1 1 1 1 1 1 0 0 0 0 0 0 8 3

Cherlyn 1 1 1 1 1 1 0 1 1 0 0 0 0 0 8 3

Nasirah 1 1 1 1 0 1 0 1 1 0 0 0 0 0 7 3

Nur Zahwah 1 1 1 1 1 0 0 0 1 0 0 0 0 0 6 2

Eunice 1 1 1 1 1 0 1 0 0 0 0 0 0 0 6 2

Namrata 1 1 1 1 1 0 1 0 0 0 0 0 0 0 6 2

Amanda 1 1 1 1 1 0 0 0 0 1 0 0 0 0 6 2

Celestine 1 1 1 1 0 1 0 0 0 0 0 0 0 0 5 2

Sharifah 1 1 1 1 0 0 1 0 0 0 0 0 0 0 5 2

Peo Shan 1 1 1 1 1 0 0 0 0 0 0 0 0 0 5 2

Cheryl 1 1 1 1 1 0 0 0 0 0 0 0 0 0 5 2

Nicole 1 1 1 1 0 0 0 0 0 0 0 0 0 0 4 2

B. Rohini 1 1 1 1 0 0 0 0 0 0 0 0 0 0 4 2

Gitanjali 1 1 1 1 0 0 0 0 0 0 0 0 0 0 4 2

Annetta 1 1 1 0 0 0 0 0 0 0 0 0 0 0 3 1

41 41 40 37 24 23 19 18 15 13 10 10 8 6

Gu

ttm

an

Ch

art

Gu

ttm

an

Ch

art

Gu

ttm

an

Ch

art

Gu

ttm

an

Ch

art

7.3 Analysis: commonalities7.3 Analysis: commonalitiesPupils Learning difficulties Possible

interventions

This applies to all students except Norazah. For example, both Isabella & Suzanne could apply multiple laws of indices concurrently, but seemed to have difficulty in reasoning for algebra in this assessment.

General observation of greater difficulty in algebra compared to indices.

Revision of concepts and understanding

14 out of 22 students These students seemed to have difficulty with the language for Math i.e. Identifying large and small numbers such as giga, pico.

Emphasis of the importance to remember the language

7.4 Analysis: anomalies7.4 Analysis: anomaliesPupils Learning difficulties Possible interventions

Ke Tian She seemed to have problems stating the laws of indices in this assessment, though she was able to apply multiple laws.

Cause – Problem with languageIntervention – Enhancing the use of math language

Stacy She seemed to have problems differentiating the laws of indices in this assessment, though she was able to apply multiple laws.

Cause – Learning the steps by rote learningIntervention – Starting from the laws of indices, students learn how questions can be derived from individual laws.

Amanda She seemed to exhibit reasoning for strategies she used. She had problems with manipulation and application of indices and algebra in this assessment.

Cause – Understanding of the concepts were not in depthIntervention – Enhancing conceptual understanding

Nur Zahwah

She seemed to be able to differentiate the laws of indices. She had problems with manipulation and application of indices and algebra in this assessment.

Cause – Understanding of the concepts were not in depthIntervention – Enhancing conceptual understanding

8 Reporting8 Reporting8 Reporting8 Reporting

8.1 Reporting for students and parents8.1 Reporting for students and parents8.2 Reporting for Math teacher and school 8.2 Reporting for Math teacher and school

8.1 Reporting for students and parents8.1 Reporting for students and parents8.2 Reporting for Math teacher and school 8.2 Reporting for Math teacher and school

ReportingReportingReportingReporting

Since this assessment is formative in nature, the results of the assessment would be helpful to the individual students to help them develop their competencies.

There are two parts to the individual report:◦Rocket report ◦The individual report on the performance on

each indicatorFor the teacher another report with the

performance of students grouped by competencies would be generated too.

8.1 Scope of Assessment & Audience8.1 Scope of Assessment & AudienceScope of Assessment

Term Tests &Semester Examinations Class Tests

Students & Parents

Teachers

School

Reporting audience

Section Method of reporting Purpose / Implications

Subject teacher

8b

•Student performance by indicators•Student overall performance descriptors

1. Pick out anomalies and commonalities2. Categorise whole class intervention,

remediation, buddy or individual3. Report to other teachers and/or HOD

HOD Maths / Buzz team (school)

•Student performance by indicators•Student overall performance descriptors

1. Adjust SOW to include past year revision which serves as foundation for current topics

2. Review teachers’ teaching and assessment abilities for testing the concepts, skills, processes aligned to national framework.*

Parents

8a

•Student rocket report which includes intervention suggestion

1. Monitor that student does the work at home and support the intervention

Student •Student rocket report•Student individual performance

1. Student will be able to strategise on the area they need to work on.

2. Check their own progress.

8.1 Reporting8.1 Reportingfor students and parentsfor students and parents

8.1 Reporting8.1 Reportingfor students and parentsfor students and parents

Performance Indicators

Student individual progress report: Norazah (28)Student individual progress report: Norazah (28)Student individual progress report: Norazah (28)Student individual progress report: Norazah (28)

A

B

Class performance middle 50%


Norazah Level 5Norazah Level 5

B. Student can perform simple sequences in the manipulation of numbers (i.e. with either positive or negative powers).


D. Student can manipulate numbers in standard form and algebraic expressions. Student is able to routinely apply the law(s) of indices.


F. Student can perform and explain the manipulation of complex algebraic expressions combined with laws of indices. Student is proficient in demonstrating clear strategies and stating their analytical approaches. that deep understanding of application of the laws of indices.


C. Student can perform manipulation of numbers and single algebraic expressions. Student is able to compare magnitude of numbers using clear strategy.


E. Student can manipulate numbers in standard form and algebraic expressions. Student is proficient in the application of multiple laws of indices concurrently.


A. There is no evidence of student’s ability to manipulate numbers and algebraic expressions.


C

D

E

F

1. Manipulation of numbers in standard form notations

2. Knowledge of mathematical nomenclature in standard form and numbers

3. Comparison of numbers4. Manipulation of algebraic expressions 5. Application of laws of indices and

concepts6. Mathematical reasoning and

communication in algebraic manipulation 7. Mathematical reasoning and

communication in indices

Suggested intervention: Well done! Do work on your Mathematical metacognitive abilities by verbalising strategies with your partner.Parent’s signature:Date: ___________


Student individual progress report: Celestine (6) Student individual progress report: Celestine (6) Student individual progress report: Celestine (6) Student individual progress report: Celestine (6)

A

B



Celestine Level 2

Celestine Level 2













C

D

E

F

1. Manipulation of numbers in standard form notations

2. Knowledge of mathematical nomenclature in standard form and numbers

3. Comparison of numbers4. Manipulation of algebraic expressions 5. Application of laws of indices and

concepts6. Mathematical reasoning and

communication in algebraic manipulation 7. Mathematical reasoning and

communication in indices

Suggested intervention: Work with your partner to work on the performance indicators checklist.Parent’s signature:Date: _______________

Summary statements of learning at Summary statements of learning at each leveleach level

Expert13-14

Proficient10-12

Competent7-9

Advanced Beginner4-6

Novice1-3

Indicators 1. Manipulation of numbers in standard form notations

2. Language of common terms in standard form and number

3. Comparison 4. Manipulation of algebraic expressions

5. Application of concepts

6. Mathematical reasoning, communication for algebra

7. Mathematical reasoning, communication for indices

Level 4: At this level, the student is learning to apply multiple laws of indices concurrently. She/he is also learning to state the types of algebraic manipulation and the operations on indices, surds or laws of indices used in the working.

Level 1: At this level the student is learning to manipulate numbers with either negative or positive powers by moving decimal place in a single direction. She/he is learning to use the knowledge of common terms like standard form, small and large numbers such as giga, micro, pico. She/he is also learning to compare numbers without showing clear working.

Level 5: At this level, the student is learning to manipulate algebraic fractions and polynomials (with positive, negative, zero indices), showing consistent and clear working. She/he is also learning to explain strategies used in the algebraic manipulation.

Level 3: At this level, the student is learning to manipulate algebraic expressions with positive, negative, zero or fractional indices, showing clear working. She/he is also learning to differentiate the laws of indices and applying laws.. Level 2: At this level the student is learning to manipulate numbers with negative and positive powers by moving decimal places. She/he is also learning to compare numbers by using strategies e.g. converting to common powers or common forms. She/he is also learning to perform operations on simple algebraic fractions.

Performance level descriptorsPerformance level descriptorsPerformance level descriptorsPerformance level descriptors

A

B

C

D

E

F

50% of the class can be located within this range

The student is estimated to be at the location

Inter-quartile range

Inter-quartile range

Student Achievement

Level

Student Achievement

Level

Level descriptions

Level descriptions

LevelsLevels













Student rocket reportStudent rocket reportStudent rocket reportStudent rocket report

28. NORAZAH


6. CELESTINE TAN


8.2 Reporting8.2 Reportingfor teachers and schoolfor teachers and school

8.2 Reporting8.2 Reportingfor teachers and schoolfor teachers and school

Student performance by indicators : Student performance by indicators : teachers can sieve out how to help students progress in specific teachers can sieve out how to help students progress in specific

areasareas

Student performance by indicators : Student performance by indicators : teachers can sieve out how to help students progress in specific teachers can sieve out how to help students progress in specific

areasareas

Student performance by indicators Student performance by indicators

9 Discussion9 Discussion9 Discussion9 Discussion

9.1 Intervention9.1 Intervention9.2 Reliability& Validity9.2 Reliability& Validity

9.3 Scaling up 9.3 Scaling up

9.1 Intervention9.1 Intervention9.2 Reliability& Validity9.2 Reliability& Validity

9.3 Scaling up 9.3 Scaling up

9.1 Intervention9.1 Intervention9.1 Intervention9.1 Intervention

When we talk about When we talk about general interventions, general interventions, what do we need to what do we need to consider?consider?

1. The students’ individual general developmental levels

2. The students’ group developmental levels

3. The assessment history of the students.

Student Results

At this level the recommended intervention for the group is...

At this level, the recommended intervention for this student is…

Developmental level = x.

Group Targets

Individual targets

Target students

Intervention strategies

Area owner / description Resources

Whole class then one-to-one

Review resultsTeacher- student individual conferencing

Teacher to review results and compare to previous results as well as prior knowledge about students to sieve out ‘abnormal’ results.Call students individually to ask what they were going through when they did the paper. Establish if the error is based on mathematical misconception.

Teacher training to analyse based on individual profiles.

Whole class Include revision before teaching new proper

HOD to review input of previous year revision prior to new topic in SOW (teacher to recommend). Review inclusion of metacognition and building mathematical language during lessons. Consider including certain types of questions in formative assessment.

Teacher training in building mathematical language teaching and assessment. Sharing during buzz sessions.Teachers need to have a database of questions they can tap on to be used for other intervention strategies. From the maths sharing portal, pick out questions that test different skills, concepts, processes etc and categorise these questions.

Whole classn>20

Review of topic

Teacher to take one period to go through the important skills or go through lesson to build mathematical language.

Affected students10<n<20

Remediation based on indicators

Teacher to go through similar questions and common misconceptions

Affected students1<n<10

Short term buddy system based on indicators

Individual students would be paired up with students who have been identified to be able to do these sub-sections well to verbalise strategies and try similar questions together at own time. Student need to verbalise strategies to buddy.

Checklist / rubrics for buddy and partner to work on so they can monitor their own progress and report to teacher during stipulated times.

Level 1 students

One-to-one attention with teacher: face to face & online

Students to meet up with teacher to go through the questions they have problems in. Have online questions to do individually. Teacher can monitor students’ progress online.

Online learning portal* All students have access to computer and internet.

9.2 Reliability and Validity9.2 Reliability and Validity9.2 Reliability and Validity9.2 Reliability and Validity

Type of validity Suggestions for improvement

Content validity - This assessment was showed that the content of the assessment tasks were closely related to the school syllabus and subject matter (national math syllabus). This was also contributed by the rich teaching experiences of three math teachers, including the assessor.

Use of multiple tasks and multiple sources of evidence as the basis for judgment.

ValidityValidity

9

Validity: alignment to national syllabusValidity: alignment to national syllabusValidity: alignment to national syllabusValidity: alignment to national syllabus

Validity: alignment to school syllabusValidity: alignment to school syllabusValidity: alignment to school syllabusValidity: alignment to school syllabusSuggested Activities

(Including Teaching Processes)

RESOURCES [PD]STRATEGIC FOCUS (Aesthetic,

NE, HOM, ICT)

Week Learning Outcomes Exercises Exploration Thinking Writing IT/HOM NE/Aesthetic

1-4(6

periods)

Chapter 1: IndicesUnderstand and use concept of indices and laws of indicesUnderstand and use concept of indices and laws of indicesManipulate zero and negative indicesManipulate fractional indicesRewrite and express numbers in standard formExpress very large or small numbersSolve problems involving indices

Ex 1.1: 4, 5, 6, 7, 8Ex 1.2: 1, 2, 3, 4, 5, 7, 9Ex 1.3: 1, 2, 5, 7, 8Ex 1.4: 3, 4, 5, 6Ex 1.5: 2, 3, 4, 5Prefixes and Powers of 10 (pg 24)Ex 1.6: 1Chapter Review: 6, 11, 12, 13

Inclass Activity pg 8, 11Inclass Activity pg 19Inclass Activity pg 22Exploration Task pg 29 [Curriculum Differentiation]

Thinking Time pg 6Thinking Time Pg 11Thinking Time pg 15, 16Thinking Time pg 23

NE [1]

Use standard form to represent big numbers and tie in with size of Singapore population – impact on space, economy

Ref: Pg 23 Example 3, pg 25 #5, #6

4-6(6

periods)

Chapter 2: Solving Quadratic EquationsSolving quadratic equations by factorization (revision of Sec 2) Solve quadratic equations by completing the squareUnderstand the quadratic formula and use it to solve quadratic equations Solve fractional equations that can be reduced to quadratic equationsSolve problems involving quadratic equations

Ex 2.1: 3, 4, 7Ex 2.2: 3, 4, 5Ex 2.3: 1, 3Ex 2.3: 2Ex 2.4: 6, 8, 10Chapter Review: 10,11

Exploration Task pg 49[Cooperative Learning]

Looking Back pg 32Thinking Time pg 42

ICT[1]- Graphmatica

2010 Secondary 3 Specific Instructional ObjectivesNew Express Mathematics

Type of Reliability Suggestions for improvement

The overlap between 0s and 1s is not fairly wide (Guttman chart ) which shows fair amount of consistency.

A fairly reliable assessment allows a more clearly defined ZPD.

Standard administration – The team established and documented clear assessment procedures/instructions for collecting, analysing and recording outcomes. The team used multiple tasks of evidence as the basis for judgment.

Inter-rater reliability through paneling – There was a consistency of judgement and moderation of the judgements across different team members using the same assessment task and procedure.

Involvement of expertise - The team members and assessor are experienced teachers, three of them are specialised in teaching math and one curriculum officer who have demonstrated competence in the field.

Elimination of noise - ‘Noise’ due to individual bias is eliminated when the team reviewed and moderated the competencies in the assessment framework. The team members reflected on their judgement error in competency based assessment and biases.

Reliability could be improved with the help of others. As we believe that professional development is social in nature, team effort is useful in helping to improve the reliability. We could use assessors with expertise in competency based assessment.

Maintain representative sample of assessment tasks to compare from context to context/year to year and use a panel of independent assessors to evaluate this sample.Use multiple sources of evidence as the basis for judgment.

ReliabilityReliability

9

9.3 Scaling Up9.3 Scaling Up9.3 Scaling Up9.3 Scaling Up

Scaling upScaling upScaling upScaling up

9

There are issues to consider when scaling up from class test to school formal assessments

or from one department to many or from written tests to projects and presentations. For example, will the leadership support teacher training or create more platforms for teachers to discuss in a professional learning team etc.

These complexities will be discussed in details in the following slides.

Scaling up in standards based assessment- At school level : Use of quality criteria and

Guttman charts for future term tests across levels and across department.

Implications on:1. School policy and leadership2. Teaching practices3. Teacher PD and competencies4. Student learning and engagement5. Parent support

Scaling UpScaling UpScaling UpScaling Up

9

1. School Policy, Structure & Culture1. School Policy, Structure & CultureSchool leadership has influence on school policy, which determine the structure and culture that would support the standards based assessment.

Structures◦ Provide time for teachers to collaborate, analyze data and

student work within Professional Learning Communities / Teams

◦ The construction within a school organization such as role and job descriptions as well as decisions on time and space, clarity of the goals and means forms the fundamental essence to the success of this educational initiative (Evans, 1996). The rationale and procedures associated with initiating the change should be constantly conveyed clearly throughout the implementation process (Evans, 1996).

School Cultures ◦ When schools promote this initiative, they inevitably bring

about a significant change in the school culture.

The program will not succeed if the Principal is:DisinterestedAwareInterestedSupportive

The program will succeed if the Principal is:InvolvedEngagedCommittedOwnership Support and understanding from school leadership to create a conducive environment for cultivating trust and collegiality

9

1. School Leadership1. School Leadership

• As assessment carries a lot of stake in Singapore, it is important that the school leaders are willing to support such scaling up.

• In addition, with the support of the school leaders, resources such as time would be given for the teachers to work on the assessment.

• Furthermore, we could then look at possibly scaling up to other subjects.

1. Support of School Leaders1. Support of School Leaders

2. Teaching Practices2. Teaching PracticesTeaching practices needs to be evidence based.

Teachers need to focus on strategies, as well as intervention strategies, for differentiated teaching. Teachers need to model the skills that they wish for students to exhibit, e.g. teachers demonstrating reasoning of process.

Teachers need to learn to work collaboratively, sharing resources, ideas and keeping abreast of new changes and research.

Other teacher factors include: Teacher beliefs and attitudes on

◦ Class organisation◦ Differentiated teaching◦ Targets for all students

Teachers’ pedagogical skills in◦ Individualised learning◦ Flexible use of resources

Teachers’ knowledge of◦ Discipline expertise◦ Developmental learning◦ Assessment and reporting◦ Use of data

Teaching or Pedagogy Practices

Teaching or Pedagogy Practices

Evidence Based

Evidence Based

Intervention StrategyIntervention Strategy

Sharing of resourcesSharing of resources

3. Teacher PD 3. Teacher PD Equip teachers PD in these 5 areas:

◦ Leadership training in leading PLT, facilitating data dialogues to help teachers move beyond what the data mean to actions that will close the gap.

◦ Use of data to examine data, understand students’ strengths and weaknesses, and identify interventions.

◦ Equip teachers in assessment competency, provide guide and coaches (e.g. master teachers, subject specialists)

◦ PLT strategies, to create a non-threatening, supportive environment that encourages and allows teachers to be open and honest in planning, assessment, data analysis and reporting.

◦ Knowledge of the developmental theory, which provides a common language and common theoretical framework.

Note on PD: Follow-up support to teachers needs to be continuous and on-going

Teacher PD in 5 areas

Teacher PD in 5 areas

PLT Strategies

PLT Strategies

Assessment and

Reporting

Assessment and

Reporting

Data UseData Use

Leadership

Leadership

Developmental

Theory

Developmental

Theory

3. Teacher Competencies3. Teacher Competencies

• As mentioned in PLTs, we could get the teachers to go through the whole process of assessment from designing of rubrics to designing of questions to implementation and analyzing the data to reporting and intervention.

• The key is about learning and so the number of questions could be kept small.

3. Teacher Competencies3. Teacher Competencies

When the teachers gained the greater competencies and confidence, we could further scale up by having more rubrics and more questions in the tests and finally incorporate the questions into semester examinations.

4. Student learning & engagement4. Student learning & engagement

This requires a cultural shift towards formative practices of assessments in four areas: questioning, feedback through marking, peer-and self-assessment (Black & William, 1998b)

Going beyond processes, skills and concepts to includes attitudes and metacognition (refer National Curriculum framework; interventions by peer coaching as suggested by the interventions)

4. Scaling up process at school level 4. Scaling up process at school level (an example)(an example)

4. Scaling up process at school level 4. Scaling up process at school level (an example)(an example)

9

5. Support of Parents5. Support of Parents

We also need to engage the parents to allow them to understand and appreciate the way assessment is done.

With the support of the parents, intervention would be more effective and this would further enhance the confidence of the parents in the assessment process.

5. Support of Parents5. Support of Parents

For example, parents could provide the following support:

Setting up interest groups or Parent Support Group (PSG) Local support group

Close communication with the teachers and school

Monitor and encourage children at home

10 Group Reflection10 Group Reflection10 Group Reflection10 Group Reflection

Reflection point Thoughts/ImplicationsClarity of Questions • The way the questions was being asked could confuse the

questions and mask the competencies of the students.•The way the questions was being asked could also guide the students to work aspects such as meta-cognition.

Balance between Space for Alternatives and Specific Skills to be Tested.

Questions are to be asked to test the ability of the students in that competency. In this case, the assessors will need to be prepared to accept all other plausible acceptable methods that arrive at correct answer, which would not reflect the students competency in standard form.

9

10.1 Reflection on the 10.1 Reflection on the assessment task assessment task

Reflection point Thoughts/ImplicationsImportance of Teacher Judgment There were many situations where it was not clear cut that the

students demonstrated or did not demonstrate the competency. Teacher’s expertise and experience are important to make accurate judgments. This implied that the teacher needs to be an expert and experienced one. If not, having more than one marker would help in both getting the more accurate judgments and developing the competencies of the teacher.

Understanding the continuity of development We should not just look at this assessment as an isolated event. Instead, knowing the students and their development in Mathematics as a whole would help the teacher develop better intervention for the students.

10.2 Reflection on how the 10.2 Reflection on how the interpretation was doneinterpretation was done

9

9

Seeing the trees and the forest◦Bearing in mind the linkages between the

objectives, assessment rubrics, reporting, and intervention has helped us gain a perspective of a holistic assessment process.

◦This new perspective not only enables us to see assessment as an integral of teaching but also how it could be done.

Reflection: Last wordsReflection: Last words

ReferencesReferences Dreyfus, H. L. & S. E. Dreyfus. (2004). From Socrates to

Expert Systems: The Limits and Dangers of Calculative Rationality: Regents of the University of California. http://socrates.berkeley.edu/~hdreyfus/html/paper_socrates.html

Gillis, S. & Bateman, A. (1998). Assessing in VET: Issues of validity & reliability. Review of Research.

Griffin, P. (2006). Strategies for developing measures of the skill levels 1. University of Melbourne. Assessment Research Centre.

Black, P., Harrison, C., Lee, C., Marshall, B., & William, D. (2003). Assessment for learning. England: Open University Press.

Mathematics Assessment by Group 2

Education

Transcript of Mathematics Assessment by Group 2