Data Management Procedures
description
Transcript of Data Management Procedures
1 3rd NRC Meeting, 9-12 June 2008, Windsor
Data Management Procedures
Michael Jung, IEA Data Processing and Research Center
2 3rd NRC Meeting, 9-12 June 2008, Windsor
Contents
Receipt of FT Data and Documentation
FT Data Processing at the IEA DPC
Evaluation of FT Data Quality
WinW3S - Within-School Sampling Software
WinDEM – Data Entry Manager
SurveySystem – On-line Data Collection Software
Material Submission after Test Administration
MS Data Processing at the IEA DPC
3 3rd NRC Meeting, 9-12 June 2008, Windsor
Receipt of FT Data and Documentation
4 3rd NRC Meeting, 9-12 June 2008, Windsor
Data from 32 countries altogether– 16 data sets arrived on time
– 16 data sets arrived after January, 25th
Receipt of FT Data and Documentation
5 3rd NRC Meeting, 9-12 June 2008, Windsor
25. Nov.
5. Dez.
15. Dez.
25. Dez.
4. Jan.
14. Jan.
24. Jan.
3. Feb.
13. Feb.
23. Feb.
4. Mrz.
14. Mrz.
24. Mrz.
3. Apr.
13. Apr.
23. Apr.
3. Mai.
13. Mai.
23. Mai.
0 5 10 15 20 25 30
Receipt of FT Data
25-Jan
6 3rd NRC Meeting, 9-12 June 2008, Windsor
Data from 32 countries altogether
– 16 data sets arrived on time
– 16 data sets arrived after January, 25th
Documentation from 32 countries altogether
– 10 countries sent on time and complete
– 22 countries sent after January, 25th and complete
Receipt of FT Data and Documentation
7 3rd NRC Meeting, 9-12 June 2008, Windsor
18. Dez
28. Dez
07. Jan
17. Jan
27. Jan
06. Feb
16. Feb
26. Feb
07. Mrz
17. Mrz
27. Mrz
06. Apr
16. Apr
26. Apr
06. Mai
16. Mai
0 5 10 15 20 25 30
25-Jan
Receipt of FT Documentation
8 3rd NRC Meeting, 9-12 June 2008, Windsor
FT Data Processing at the IEA DPC
9 3rd NRC Meeting, 9-12 June 2008, Windsor
Data Processing at the IEA DPCData Processing at the IEA DPC
Data & Codebook
sInstrument
s & Forms
INP
UT
IEA DPC
Country
1. Import to SAS and adapting the
file structure
10 3rd NRC Meeting, 9-12 June 2008, Windsor
Adapting the File StructureAdapting the File Structure
WinW3S database and WinDEM data files have been merged, renamed and converted to SAS data files
Unique system numbers (CSYSTEM, TSYSTEM, SSYSTEM) have been added
National Scores and derived variables (e.g. final participation indicator) have been added
National variables have been removed (will be sent to countries in a separate data file on demand)
11 3rd NRC Meeting, 9-12 June 2008, Windsor
Structure of the Merged Data Structure of the Merged Data SetSet
Variable ValueIDPUNCH 1
...,ITPART1..2,
IDSCHOOL,
IDCLASS,...
3...
8001
800101
IDSTUD 80010101
IDCHECK 373
TOKEN01 3
CS2GSEX 9
SSYSTEM,... 000001
Information:
TrackingForms Information:
StudentQuestionnair
e
Addedduring
cleaningprocess
Example: Student Questionnaire Data File
WinDEM(ISGxxxF2.DBF)
12 3rd NRC Meeting, 9-12 June 2008, Windsor
Data Processing at the IEA Data Processing at the IEA DPCDPC
Country Databas
eNew Data
Structure
Reports, Statistic
s & Docu.
INP
UT
OU
TP
UTStructur
e Check
Background
Cleaning
IEA DPC
Country
ID Cleaning
Linkage Cleanin
g
Data & Codebook
sInstrument
s & Forms2. Cleaning
13 3rd NRC Meeting, 9-12 June 2008, Windsor
Evaluation of FT Data Quality
14 3rd NRC Meeting, 9-12 June 2008, Windsor
Evaluation of FT Data QualityEvaluation of FT Data Quality
General data quality in almost all countries was VERY GOOD
Most common problems– Not all changes in the instruments reflected in the
codebooks
– Not administered variables deleted in codebooks instead of hidden
– WinDEM quality control checks not performed or not all problems have been solved
– Inconsistencies between Student Questionnaire files and Occupation data files
– Information in WinW3S not completed and thus database not exported
15 3rd NRC Meeting, 9-12 June 2008, Windsor
WinW3S - Within-School Sampling Software
16 3rd NRC Meeting, 9-12 June 2008, Windsor
WinW3S
Management tool for all ICCS within-school sampling operations
Identifies the schools, classes, teachers and students that will participate in ICCS
Creates a hierarchical ID system + passwords for ODC
Samples classes + teachers
Assigns booklets to the students
Produces and prints all forms and labels
17 3rd NRC Meeting, 9-12 June 2008, Windsor
WinW3S – What's NewNew option for participation in Asian Module added
Import of students and teachers from MS Excel files
New participation code for students added: „participated, but excluded”
New participation code for teachers added: „teacher on temporary leave”
Assignment of reliability booklets changed: random assignment of ~100 R booklets per booklet type
Assignment of different class IDs for ICCS Target Grade and Additional Grade students
Listing of teachers birth information and gender optional on Teacher Listing Forms
Possible to list more than 99 teachers per school
Teacher and school principal questionnaire mode can be changed on individual level– ODC optional ONLY for countries that have trialed ODC in FT
Participation status for on-line participants can be imported from SurveySystem Monitor
18 3rd NRC Meeting, 9-12 June 2008, Windsor
WinDEM – Data Entry Manager
19 3rd NRC Meeting, 9-12 June 2008, Windsor
WinDEM
Provides mask for data entry from booklets and paper questionnaires
Based on codebooks, provided by the IEA DPC
Allows adaptation of codebooks
Validates data during data entry
Provides various data verification checks after data entry
Allows for comparison of double-entered data
20 3rd NRC Meeting, 9-12 June 2008, Windsor
WinDEM – What's New
New coding module for coding of occupations integrated in WinDEM– Entry of occupation data directly from WinDEM– Occupation data will be stored in an MS Access database– Text answers and occupation codes also available in
table mode
21 3rd NRC Meeting, 9-12 June 2008, Windsor
IEA SurveySystem
On-line Data Collection Software
22 3rd NRC Meeting, 9-12 June 2008, Windsor
ODC – What's New
Export tool in SurveySystem Monitor application for participation status of teachers and school principals– data can be imported into WinW3S later on
23 3rd NRC Meeting, 9-12 June 2008, Windsor
Material Submission after Test Administration
24 3rd NRC Meeting, 9-12 June 2008, Windsor
Material to be submitted
WinW3S database exported for IEA DPC Sampling
(MICS<country>1.dbf)(MICS<country>2.dbf)
Send Material to IEA DPC Sampling
National Adaptation Forms (IV) (*.xls)
Set of final National Instruments (*.pdf)
Send Material to ISC at ACER
Send Material toIEA DPC
25 3rd NRC Meeting, 9-12 June 2008, Windsor
Material to be submitted
Send Material to IEA DPC Sampling
Send Material to ISC at ACER
Send Material toIEA DPC
National Adaptation Forms (IV) (*.xls)
Set of final National Instruments (*.pdf)
WinW3S database exported for IEA DPC
(MICS<country>1.mdb)(MICS<country>2.mdb)
Student and Teacher Tracking Forms (paper copies or scanned)
26 3rd NRC Meeting, 9-12 June 2008, Windsor
Material to be submitted
Send Material to IEA DPC Sampling
Send Material to ISC at ACER
Send Material toIEA DPC
WinDEM data files (*.dbf) + WinDEM codebooks (*.sdb)
(ICGICSM2.DBF; *SDB)
(ITGICSM2.DBF; *SDB)
(ISGICSM2.DBF; *SDB)
(ISAICSM2.DBF; *SDB)
(ISRICSM2.DBF; *SDB)
(ISEICSM2.DBF; *SDB)
(ISLICSM2.DBF; *SDB)
(ISSICSM2.DBF; *SDB)
Occupation data file(ISOICSM2.mdb)
27 3rd NRC Meeting, 9-12 June 2008, Windsor
Material to be submitted
Data and Documents have to be submitted
by 27-Mar-2009 latest for SH countries and
by 29-Aug-2009 latest for NH countries
IEA DPC Sampling
ISC at ACER
IEA DPC
28 3rd NRC Meeting, 9-12 June 2008, Windsor
File-Exchange using the IEA DPC WebFTP-Server
The IEA DPC has setup secure FTP sites to exchange materials between countries and the - ISC at ACER and/ or- IEA Secretariat and/ or - IEA DPC
Can easily be accessed using a browser (e.g. Internet Explorer)
Individual country logins from FT are still valid
FT folders and documents have been moved to a folder “0_Old_Material_from_Field_Trial “ please do not upload any documents into that folder any more
29 3rd NRC Meeting, 9-12 June 2008, Windsor
File-Exchange using the IEA DPC WebFTP-Server
MS Materials should be uploaded using a predefined folder structure1_Instruments_and_NAF_for_Adaptation_Verification_ACER
2_Instruments_and_NAF_for_Translation_Verification_IEA_Secretariat
3_Instruments_and_NAF_for_Layout_Verification_ACER
4_Final_Instruments_and_NAF_for_ACER _and_IEA_DPC
5_SurveySystem_Definition_Files_for_IEA_DPC
6_Codebooks_and_Data_Files_for_IEA_DPC
7_Exported_WinW3S_Database_for_IEA_DPC
8_Other_Materials
9_To_Country
30 3rd NRC Meeting, 9-12 June 2008, Windsor
File-Exchange using the IEA DPC WebFTP-Server
Please inform the recipients via email as soon you have uploaded any materials
Updated WebFTP-Server Manual will be sent to countries
31 3rd NRC Meeting, 9-12 June 2008, Windsor
MS Data Processing at the IEA DPC
32 3rd NRC Meeting, 9-12 June 2008, Windsor
Data Processing at the IEA Data Processing at the IEA DPCDPC
Data Processing of the MS data at the IEA DPC basically follows the same prcedures as for the FT
But: Any problems and/or inconsitencies will be reported back to countries and need correction and/or approval
Countries need to be available after data submission
33 3rd NRC Meeting, 9-12 June 2008, Windsor
Data Processing at the IEA Data Processing at the IEA DPCDPC
Country Databas
eNew Data
Structure
Reports, Statistic
s & Docu.
INP
UT
OU
TP
UTStructur
e Check
Background
Cleaning
IEA DPC
Country
ID Cleaning
Linkage Cleanin
g
Data & Codebook
sInstrument
s & Forms Cleaning
34 3rd NRC Meeting, 9-12 June 2008, Windsor
Thank you very much for your attention.