CBA Autumn 2015 Principal Information Seminar 19 th, 22 nd and 23 rd June 2015 Anne Cromie and...
-
Upload
baldric-bryant -
Category
Documents
-
view
214 -
download
0
Transcript of CBA Autumn 2015 Principal Information Seminar 19 th, 22 nd and 23 rd June 2015 Anne Cromie and...
CBA Autumn 2015
Principal Information Seminar
19th, 22nd and 23rd June 2015
Anne Cromie and Dorothee Wagner
1
Learning Intentions
At the end of the session you will be aware of the:
•context for CBA in Northern Ireland for autumn 2015;•findings of independent reviews of computer-based assessments (CBA) including the OECD Review;•developments made to CBA in response to feedback from schools; •reports and outcomes produced and how they can be used to inform planning; and •guidance and support available from CCEA and C2k.
2
Overview
9.00 or 13.00 Registration
9.30 or 13.30 Welcome
Context for CBA Autumn 2015
10.30 or 2.30 Tea/Coffee
10.45 or 14.45 Developments to CBA
CBA Reports and Outcomes
Next Steps
11.45 or 15.45 Questions, Evaluation & Close
3
Focus on Raising Standards
“The introduction of the CBAs is consistent with the national strategy to improve pupil performance within literacy and numeracy.” p61
“…fit well with an official assessment strategy that focuses on assessment for learning and a wider national strategy to reduce inequity in student outcomes.” p61
4
OECD Review of Evaluation and Assessment in Northern Ireland Education 2013
Step Change
5
Step Change
“Literacy and numeracy are bedrocks of a successful education
system. Getting this right significantly enhances a young person’s
employability and lays the foundations for further success
in all areas of learning and life.”
Step Change, October 2014, www.cbi.org.uk
6
Step Change
• At primary school, literacy, numeracy, communication and ICT skills have been made a central focus – and computer-based assessment is increasingly used as a tool for embedding ICT skills into broader curriculum study.
• Helping young people to be confident and familiar with ICT is important.
• However, greater improvement is still needed. On leaving primary school, 18% of students still do not have the expected level of communication and mathematics skills for their age.
• The policy focus on literacy, numeracy, communication and ICT is strongly supported by industry
7
Northern Ireland employers’ top priority areas for action (%)
8
Top priority areas for action (%) in primary schools
• Clearly defined goals on literacy and numeracy 86.54%
• Development of communication skills 53.85%
• Development of character / behaviours 46.15%
• Clearly defined goals on science 30.77%
• A broader definition of achievement at school
covering exam results and other factors essential 17.31%
to success in life and work
9
Recent Evaluation and Review
10
Evaluation and Review of CBA since 2012
Standard Evaluation
Commissioned Review Additional
Annual and ongoing evaluation by CCEA
1.CCEA review the operation of the new assessments.2.ETI review to determine how effectively schools make use of assessment information.
3.Independent review of the technical issues - why they occurred and what lessons can be learnt for the future.
4.Independent review of policy and practice.
5.Department review the policy of statutory CBA.
OECD Review
11
Evaluation and Review of CBA since 2012
• Covered all aspects from procurement to operation – Phased approach in autumn 2013– 2014 mini trial– Phased approach in autumn 2014 and 2015
• Overall findings– Consistent– Reinforce key messages
• Since 2012:– 184 schools chose to use CBA in 2013 – 194 in 2014– 276 in 2015
12
Moving Forward
Procurement Cycle
•Five year procurement cycle
Technical Support/ Transformed Primary Network
•Continued partnership – CCEA/C2k/Capita/suppliers•Enhance CCEA/C2k helpdesk•Monitoring network stability•Ensure ICT equipment set up for CBA•Revised Pre-flight check for IE11
– Support CBA on transformed network
13
OECD Review
14
OECD:Particular Strengths of CBA
OECD has identified the following as particular strengths:
•Standardised to NI school population•Aligned to revised curriculum•Adaptive (difficulty of the questions adapts to the level of the pupil)
– reduces the risk of demotivation– provides nuanced, fine-grained profile of the pupil’s performance.
•Consistent with the national strategy to:– improve pupil performance in literacy and numeracy– focus on assessment for learning
15
OECD:Main Benefits of CBA
OECD has highlighted CBA as a reliable basis for…
•assessing progress and learning needs •planning teaching to meet learner needs•supporting self-assessment and individualised target setting•reducing the workload of teachers as no marking is involved•providing teachers with clear and reliable information for parents on their children’s strengths and weaknesses
16
OECD:Learning for CBA
OECD identified the following regarding CBA:
•Technical challenges– procurement process– screen freezes/sound– physical dexterity (keyboard and mouse)
•Feedback of results– adaptivity– delay in standardised scores– parent reports
•Lack of continuity– procurement process– InCAS– longitudinal data– two providers
•Timeframe
17
Ministerial Statement
• OECD Review positive
‘The practice of having this sort of universal, formative assessment in primary school, mapped to our own curriculum and delivered at the start of the academic year, is noted with approval by the OECD’
John O’Dowd, Oral Statement (11 March, 2014)
18
Legislative Context
In an email to schools on March 26, 2015, the Assessment and Qualifications Team (DENI) noted that:
•Since the introduction of NILA and NINA assessments in 2012, CCEA has worked with both assessment providers to improve both CBAs to best meet school needs. •Participant schools reported increasing confidence in the educational value of the assessments, with over 90% stating that they plan to use them again, if available. •All primary schools are now invited to register their interest in participating in a further voluntary CBA programme in autumn 2015 and the Minister would like to encourage as many schools as possible to consider taking part.
19
NILA and NINA
20
What do they have in common?
•May be completed only by Year 4 to Year 7 pupils•Web-based and pupil responses to questions are saved automatically online•Adaptive and not time-bound•Longer a pupil takes to answer the longer the assessment will take•Can be paused and re-started from that point•Available in English & Irish•Special Entry Points available•Pupils cannot repeat assessments•Similar tables and reports•Reports and outcomes available within 48 hours of completion
NILA and NINA
21
22
Focus on Skills
Aspect of Literacy Skills Focus Pupils’ ability to …
Listening Information Retrieval
Listen to and understand information and ideas.
Reading Sequence Understand information and ideas
Information Find and use information
Features of Texts
Understand different types of texts
Opinions Respond to different layers of meaning
Writing Punctuation Punctuate text
Spelling Recognise accurate spellings
Grammar Use grammar accurately23
The Literacy Assessment
• Single assessment • No time limit• Most pupils complete 46 questions• Average time of 40 minutes• Four difficulties of questions
– age appropriate for the pupil (=), for a younger pupil (<), an older pupil (>) or an even older pupils (>+)
• SEN Entry Point starts with (<)• Question number is displayed at the bottom of the screen• 8 different answer techniques
– cloze, drag and match, drag into order, drag punctuation, drag to hot spot, pick multiple, pick one and select word
• Practice assessment
24
Adaptive Routing in NILA
25
Let’s take a few minutes to look through the CBA Objectives Report
26
Focus on Skills
Skills Focus Pupils ability to …
Strand 1 Number Counting & Understanding Number;
Facts Facts and Mental Arithmetic
Operations Operations with Whole Numbers, Integers and Decimals
Fractions Fractions, Decimals, Percentages, Ratio & Proportion
Problems Problems – including Money, Function Machines & Algebra
Strand 2 Shape and Space
Shape and Space
Measures Measures
Handling Data Handling Data
Skills Focus Pupils’ ability to …
Strand 1 Number Counting & Understanding Number;
Facts Facts and Mental Arithmetic
Operations Operations with Whole Numbers, Integers and Decimals
Fractions Fractions, Decimals, Percentages, Ratio & Proportion
Problems Problems – including Money, Function Machines & Algebra
Strand 2 Shape and Space
Shape and Space
Measures Measures
Handling Data Handling Data27
The Numeracy Assessment
• 2 strands:
– Strand 1 Understanding & Using Number – Strand 2: Shape & Space, Measures and Handling Data
• Neither has a fixed time for completion• Each strand assessed using an adaptive approach• Pupils take an average of 20 minutes per strand• Only mental arithmetic questions are timed• Irish Medium version only available to IM schools or those with IM
units• Pencil and paper should be available to pupils• Practice area• Voiceover available for every question
28
Adaptive Assessment
• Start in their target year - one year below their current school designated year e.g. Year 3 questions for a Year 4 pupil
• SEN Entry Point is 2 years below target year• 3 difficulties of questions
– Easy, medium and difficult
• First question is a ‘medium’ one• First set of 6-8 questions assesses general numeracy• Move up, down or stay in the year• Move up or down can be reversed• Process repeated until assessment determines they are answering
questions appropriate to their ability
29
Adaptive Assessment
30
Let’s take a few minutes to look through the CBA Objectives Report
Promotions and Demotions by Year
31
32
Coffee Time10.30 / 14.30
CBA Developments
33
Developments since 2012
34
SEN Entry Point Both
HI Entry Point NILA
Skills Focus Report Both
Year Group Report NINA
Improved Parent Report Both
SS and AROs available within 48 hours Both
Improved filtering of reports NILA
Improved IM selection Both
Modified pupil checklist Both
Developments since 2012
NILA and NINA administrator’s access/permissions aligned Both
Improved assessment monitoring NINA
Item Analysis Both
Question number added to the bottom of the screen NILA
Reduction in question techniques NINA
Improved practice area NINA
Software adapted to operate on the C2k transformed network Both
Pre Flight Check Both
35
Special Entry Points
SEN (Special Educational Needs) Entry Point:
•for pupils experiencing difficulties with literacy/numeracy– quicker access to easier questions – more positive and potentially shorter assessment experience– offers same access to the full range of questions as other pupils
HI (Hearing Impaired) Entry Point:
•for pupils who are receiving support for a hearing impairment – removes the Listening aspect of the assessment– allows pupils to be assessed in Reading and Writing
Use of the Special Entry Points will not be obvious to pupils.
36
Qualifying for Special Entry Points
The teacher and the Senior Management Team must:
For the SEN Entry Point•reasonably expect that the pupil is performing two or more years below their chronological age; and•be in agreement that an SEN Entry Point is appropriate for the pupil.
For the HI Entry Point•have evidence that the pupil is receiving support for a Hearing Impairment; and•be in agreement that an HI Entry Point is appropriate for the pupil.
Only the Administrator can set a Special Entry Point
37
Setting an SEN Entry Point for a Pupil (NILA)
38
Setting an SEN Entry Point for a Pupil (NINA)
39
NINA SEN Entry Point
Pupil’s Curriculum Year
First Non-SEN Entry Point questions will be from:
First SEN Entry Point questions will be from:
Year 4 Year 3 Year 1
Year 5 Year 4 Year 2
Year 6 Year 5 Year 3
Year 7 Year 6 Year 4
40
General SEN improvements
• Specific guidance included in the NILA / NINA Guidance Documents;
• Improved font size, style and colour;• Language modification;• Setting an SEN entry point for individual pupils;• Setting an HI entry point (removes Listening questions from NILA);• Improved ‘drag punctuation’ questions in NILA;• Voice in NILA and NINA Demos slowed down;• Improved pupil demos (for the practice areas and the real
assessment);• Streamlined NINA Practice area; and• Simplified pupil checklists.
41
Monitor Assessments
NILA EM completed NINA completed
NILA EM started NINA started
NILA IM completed NINA not started
NILA IM started
NILA – to help monitor progress during an assessment the question number is displayed at the bottom of the screen (most pupils answer 46 questions).
NINA – each strand takes approximately 20 minutes.
42
NILA: Monitoring Assessment Progress for the School
Shows that the pupil has started NILA in English
Shows that the pupil has completed NILA in English
Shows that the pupil has started NILA in Irish
Shows that the pupil has completed NILA in Irish
43
NINA: Monitoring Assessment Progress for the School
Shows that the pupil has started NINA
Shows that the pupil not started NINA
Shows that the pupil has completed NINA
44
Item Analysis
NILA•Additional questions added to NILA at the lower and upper end.•Drag punctuation questions:
– Improved ‘grab’ of the box; and– More accurate placement of the punctuation mark.
•Improved graphics and screen display.•Improvements for Hearing Impaired (HI) pupils.
NINA•Reviewed small number of questions with high proportion of incorrect answers.
45
Reduction of Question Techniques
46
Pupils are allowed to use pencil and paper
Cross Out
47
Calculation
48
Multi-drag
49
50
Computers
• Use pupil workstations.• New teacher laptops can be used, if required.• Connect to the C2k network at least 30 minutes
before CBA use.• Computers not been connected to the C2k network
recently may take longer to update.• Pupils must first log into the C2k network with their
individual C2k username and password. • Multiple pupils logged on with the same username
and password will not be able to access NILA and NINA.
51
Check the Sound
NINA•Go to the practice area •Select any practice question and •Click the sound icon•You should be able to hear the sound play
NILA•Go to the practice assessment•Click the sound icon•You should be able to hear the sound play
52
Accessing CBA
53
Teacher ‘MY-SCHOOL’ homepage
• ‘View All’• ‘Curriculum Links’• ‘NILA’ or ‘NINA’
Accessing CBA
54
Pupil ‘MY-SCHOOL’ homepage
• ‘My Links’• ‘NILA’ or ‘NINA’
Pre Flight Check (PFC)
Key Points
•Simple and user-friendly resource for schools•Automatically runs when pupils click on NILA or NINA link via their ‘MY-SCHOOL’ homepage•Completed prior to commencing assessments•Checking correct versions of software are installed•In case of incorrect set-up - provide guidance and advice to schools•When successful takes you straight to the assessment homepage
NINA NILA
http://preflightcheck.ccea.org.uk/nina.html http://preflightcheck.ccea.org.uk/nila.aspx
55
Pre Flight Check (PFC)
56
Something’s Wrong
57
Pre Flight Check (PFC)
• CCEA captures all PFC information• CCEA is notified if a PFC has a problem• Monitoring provides frequency of information and helps identify
patterns • Completing the pop up gives details of:
– school, – workstation – teacher’s comment
• Need to include both DENI number and workstation number – little sticker on computer
58
Outcomes and reports
59
What NILA / NINA Tables and
Reports are available?
• NILA / NINA Results Tables provide schools with Age-Related Outcome (ARO) Ranges and Standardised Scores (SS). These are available for individual pupils, classes, year groups and at whole school level. These build over time to show additional data as it becomes available for new academic years.
• NILA / NINA Parent Reports contain the results table for an individual pupil, showing only the Age-Related Outcome (ARO) Range.
• NILA / NINA Objectives Reports provide specific information about the questions asked and whether the pupil answered correctly or incorrectly.
60
Results Table (School, Year Group and Class)
61
Results Table (Individual Pupil)
62
Parent Report
63
Skills Focus Reports
64
Using Outcomes
CBA “… are intended for diagnostic purposes and the assessment data is primarily used for:
•Assessing pupil progress and identifying their learning needs
•Supporting self-evaluation and target-setting
•Helping teachers shape their teaching to address the learning needs of their pupils
•Providing information for parents on their children’s strengths and weaknesses within literacy and numeracy.”
OECD Review of Evaluation and Assessment in Education (2013) p.54
65
Next Steps
66
Roles and Responsibilities
Role Responsibility
PrincipalOverview of school CBA
• arranging the SIMS upload to WebXchange• accessing the school username and password • allocating the CBA administrator role
AdministratorManages at school level
• teacher passwords• SEN/HI Entry Points
TeacherManages at classroom level
• pupil usernames and passwords for class• preparing classroom• practising with pupils• administering assessments
Classroom assistantSupports pupils
• continuation of general classroom good practice
67
SIMS Upload to WebXchange
Role to be assigned by Principal
•Quick to complete•Complete well in advance of using assessments•Need to have:
– Instructions sent to principals (approx. 17 August)
•Does not affect use of practice assessment/area by pupils•Last date to complete is 25 September 2015
If you experience any difficulty with SIMS/Webxchange please contact the C2k Service Desk on 0870 6011666.
68
Moving Forward with the CBA 2015
69
Admin passwords available to schools 12 August
Preparation instructions provided to schools 17 August
Manuals and posters delivered to schools 4 September
Phase 1 assessments completed 7 - 11 September
Phase 2 assessments completed 14 - 18 September
Final date for SIMS upload 26 September
Phase 3 assessments completed 28 September - 11 December
Evaluations live 7 September - 11 December
CBA Passwords
The School Administrator Username is the school DENI number.
Principals can access the new Administrator Password from ‘Principal Documents’ within the ‘MY-SCHOOL’ area of the C2k Exchange.
See CBA guidance documents page 10.
To access teacher usernames and passwords please refer to
page 11.
70
Assessment Preparation
Teachers should:
•use the online course CBA section of www.ccea.org.uk; •refer to the NILA / NINA Information for Schools Guidance Documents;•be familiar with the NILA / NINA pupil demos;•explore the NILA practice assessment and the NINA practice area;•check pupil usernames and passwords;•ensure that pupils can have a break during the assessment/s;•ensure that pupils know that NINA allows them to have the question/s read aloud if required;•consider resources required; and•check the equipment.
71
Assessment Preparation
Teachers should ensure that pupils:
• view the pupil demos as part of a class discussion;
• know how to pause or exit an assessment by clicking on the red x at the top right hand side of the screen and closing the window;
• know how to resume an assessment by clicking “Start Assessment button”. Pupil should log in again with the same username and password and see “Continue (first) assessment”; and
• understand the assessments they will be completing and have had sufficient opportunity to use the NILA Practice Assessment / NINA Practice Area.
NB - Pupils should be provided with pencils and paper for the NINA assessment.
72
Pupil Demos
Pupil Demos
•Accessible via www.ccea.org.uk
•Optional sub-titles available
•Scripts are available
NILA Practice Assessment Demo / NINA Practice Area Demo:
Introduces pupils to NILA / NINA and the techniques they need to be aware of when completing their assessment.
NILA / NINA Real Assessment Demo:
Reinforces this knowledge and prepares the pupils for logging in to complete their actual assessment.
73
The Practice Assessment / Practice Area
Be sure to:
• take your pupils through the various question/response types;
• draw attention to the instructions for each question type;
• demonstrate how to answer using a data projector or smart board;
• give your pupils plenty of opportunity to use the practice assessment (NILA) / practice area (NINA);
• show pupils how to undo/change an answer;
• evaluate how pupils cope with question types and advise; and
• show that an answer can be deleted or changed any number of times before the ‘Next’ (NILA) / ‘Submit’ (NINA) button is clicked.
NB - Pupils can access the Practice Assessment / Practice Area without usernames/password. No log-ins required.
74
Trouble-shooting
Question not fully loaded (images/buttons are misplaced/missing)
Step 1: If this issue continues for more than 10 seconds click on ‘F5’ once.
Step 2: If the problem persists for this user, close Internet explorer (IE) and get the pupil to log back into their assessment.
Step 3: If the problem is still not resolved, please remove this device from the assessments and use another workstation if available.
Step 4: If the problem remains, contact the C2k Service Desk on 0870 6011666.
75
Phoning C2k
Please make a note of the following information before calling the C2k Helpdesk:
•date and time of issue •assessment being attempted (NINA or NILA)•description of issue•workstation ID •pupil ‘C2k / NINA / NILA’ account user name•if possible please take a screenshot.
76
CCEA Support and Resources for Autumn 2015
• Face-to-face training (17 August – 28 September)
• Online Training Course (from 17 August)
• Videos available on CCEA website
• FAQs available on CCEA website
• Preparation instructions – SIMS upload, school username/password
• NILA / NINA Guidance Documents, including guidance for pupils with SEN
• ‘How to change an answer before you submit it in NINA’ poster
• Pupils demos and scripts
• NINA practice area
• NILA practice assessment
77
Websites and Helpdesks
78
Resources and Information for Schools•Latest information about CBA and online resources for schools on www.ccea.org.uk
Computer-Based Assessment Helpdesk
Hardware/ IT Difficulties or help with computers/laptops•C2k helpdesk 0870 6011666•C2K Exchange (desktop icon)
Monday – Friday (except for public holidays)
Email [email protected]
Questions, Evaluation and Close
79