Pennsylvania Inspired Leadership Program
Student Occupational Competency
Assessment
Analyzing Data for Student Achievement
Pennsylvania Association of Career & Technical Administrators
October 7, 2008
Pennsylvania Association of Career & Technical Administrators Pennsylvania Inspired Leadership Program
Student Occupational Competency Assessment
Presentation Outline
1. Review and Analysis of NOCTI Results (12:30 p.m. – 1:30 p.m.) Page #
a. Begin with the End in Mind ................................................................................................. 3 b. Data Collection
i. Participant Score Report........................................................................................ 8 ii. Test Coordinator’s Score Report, Section 1 .......................................................... 9 iii. Test Coordinator’s Score Report, Section 2 ........................................................ 10
c. Data Compilation i. Annual Spreadsheet
1) Program .................................................................................................. 11 2) Student.................................................................................................... 12
ii. Historical Spreadsheet 1) Program .................................................................................................. 13 2) Student.................................................................................................... 14
d. Data Analysis i. Program Level Analysis
1) Local Norm v. National Norm.................................................................. 15 2) Percentage of Students by Proficiency Level ......................................... 16
ii. Competency Level Analysis 1) National Score ........................................................................................ 17 2) Local Score 3) Standard Error 4) Analysis Range—Assignment #2.......................................................... 18
e. Data Presentation i. Tables .................................................................................................................. 19 ii. Charts................................................................................................................... 21
2. Identification of Program Strengths & Deficiencies (1:30 p.m. – 2:15 p.m.)
a. Program Level Analysis i. Local Norm to National Norm
1) Above ...................................................................................................... 23 2) Below ...................................................................................................... 24 3) Long-term Trend—Rising, Declining, Up-and-down............................... 25
ii. Percentage of Students by Proficiency Level ...................................................... 26 b. Competency Level Analysis
i. Deficient Competencies ....................................................................................... 27 ii. Declining Competencies ...................................................................................... 27
c. Competency Analysis Relative to NOCTI Blueprint Percentages .................................... 28
Break (2:15 p.m. – 2:30 p.m.)
Page 1 of 40
Pennsylvania Inspired Leadership Program Student Occupational Competency Assessment Presentation Outline
3. Root Cause Analysis of Strengths and Deficiencies (2:30 p.m. – 3:15 p.m.) Page #
a. Techniques........................................................................................................................ 29 i. Ask ‘Why’ Five Times ii. Cause-and-Effect Diagram (Fishbone) iii. Interrelationship Diagram
b. Potential Sources of Variance (Deficiency) i. Student-Related Variables ................................................................................... 30
1) Number of Students 2) Number of Students with Disabilities
ii. Staff-Related Variables ........................................................................................ 31 iii. Curriculum-Related Variables
1) Alignment with NOCTI Blueprint a. Competency to Competency ..................................................... 33 b. Scope to Scope.......................................................................... 35
2) Sequence................................................................................................ 38
4. Strategies to Address Curricular & Instructional Deficiencies (3:15 p.m. – 4:00 p.m.)
a. Strategies Related to Curriculum i. Expand Curriculum—Drafting & Design ii. Reduce Curriculum—Drafting & Design iii. Re-sequence Curriculum—Health Assistant iv. Discount Deficiencies—Facilities Maintenance v. Change CIP—Electrical Engineering................................................................... 39
b. Strategies Related to Instruction i. Review ‘Older’ Competencies ii. Alter Instructional Methods
1) Projects 2) Group Activity 3) Theory or Laboratory 4) Integrated Academics
iii. Instructional Resources 1) Guest Speakers 2) Field Trip 3) Textbooks 4) Work-based Activity
5. Creating the Strategic Initiative for Each School (4:00 p.m. – 4:45 p.m.)
Page 2 of 40
Page 1
Auto Body RepairWritten
1999
Dat
a, 4
5.5
1999
Dat
a, 4
7.7
2000
Dat
a, 5
1.7
2000
Dat
a, 4
7.8
2001
Dat
a, 5
4.1
2001
Dat
a, 4
7.8
2002
Dat
a, 5
4.5
2002
Dat
a, 4
8.1
2003
Dat
a, 5
4.5
2003
Dat
a, 4
8.6
2004
Dat
a, 5
2.5
2004
Dat
a, 4
8.6
2005
Dat
a, 5
9.2
2005
Dat
a, 4
8.9
2006
Dat
a, 6
0.2
2006
Dat
a, 4
8.9
2007
Dat
a, 6
4.9
2007
Dat
a, 5
9.9
2008
Dat
a, 5
2.2
2008
Dat
a, 5
5.8
0
10
20
30
40
50
60
70
Local Norm National Norm
NOCTI Scores
Local norms above national
norms andrising over time!
Page 3 of 40
Page 1
Automotive TechnologiesWritten
-10
0
10
20
30
40
50
60
70
80
90
100
Competencies Tested
Perc
ent S
core
Group Norm--2006 54.9 54.7 53.1 58.0 53.2 78.5
Sig. Diff.--2006 0 0 0 0 0 0
Group Norm--2007 60.1 55.4 55.8 56.4 50.6 75.0
Sig. Diff.--2007 0 0 0 0 0 0
Group Norm--2008 61.5 90.0 85.0 76.0 73.9 84.2
Sig. Diff.--2008 0 0 0 0 0 0
Engine Repair Suspension Brakes Electrical Engine Perf Customer Rels
Competency scoreson the rise
and above the national norm
Page 4 of 40
Erie County Technical SchoolNOCTI Analysis
Percent of Students At or Above the National Norm
83%
82%
71%
100%
89% 93%
62% 71
%
89%
43% 50
%
50%
43%
82%
65%
56%
33%
0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Auto Bod
y Rep
air
Automoti
ve Tec
hnolo
gies
Early C
hildh
od E
duca
tion
Art & D
esign
for B
usine
ss
Compu
ter In
formati
on S
ystem
s
Constr
uctio
n Trad
esCos
metolog
y
Culina
ry Arts
(Tes
t #09
90 A
CF)Draf
ting &
Des
ign
Electric
al Eng
ineeri
ngElec
tronic
s
Facilit
y Main
tenan
ce Tec
hnolo
gies
Graphic
Com
munica
tions
Health
Ass
istan
t
Hospit
ality
Manag
emen
tMeta
l Fab
ricati
on
Network
ing Tec
hnolo
gies
Tool &
Die
Programs
Perc
ent o
f Stu
dent
s
Page 5 of 40
Page 1
Drafting & DesignWritten
-10
0
10
20
30
40
50
60
70
80
90
100
Competencies Tested
Perc
ent S
core
Group Norm--2007 72.6 62.8 56.3 68.9 54.7 45.1
Sig. Diff.--2007 0 0 0 0 0 0
Group Norm--2008 72.6 62.8 56.3 68.9 54.7 45.1
Sig. Diff.--2008 0 0 0 0 0 0
Prepare to Draw- Basic Dimensioning
24%
Geometric Construction19%
Drawing Techniques13%
Applied Mathematics9%
Dimensioning Skills21%
Supplementary Views14%
Lowest Performing Competency Area
Supplementary Views
Page 6 of 40
Page 7 of 40
Participant Written Assessment Score Report - Score reports are provided for each test taker and include the individual’s subpart/job scores and total scores along with comparative data for the group, site cumulative, site, state and nation. While separate reports are provided for written and performance assessments, both provide the same data.
2/1/2007 Michigan Test Taker A
A B C
3
D E F G
SecondaryNOCTI Test Site, 9999
Page 8 of 40
Test Coordinator’s Written Report - The Test Coordinator Report is a summary of the assessment results for all test takers within a particular group. Participants are listed alphabetically with individual sub-scores and total scores displayed. This report contains a summary for the written assessment results for the designated group of test takers.
Test Taker A
Test Taker B
Test Taker E
Test Taker D
Test Taker C
A
B C D
2/1/2007
NOCTI Test Site, 9999
Michigan
5
Secondary
Page 9 of 40
Test Coordinator’s Written Assessment Comparative Data Report - The comparative data report provides a wealth of information that can be helpful in making inferences about the observed assessment results/scores. The report displays group, sitecumulative, site, state and nation user-based normative information for both the written and performance assessments.
G
F
E
D
C
B
A
NOCTI Test Site, 9999
7
2/1/2007
Secondary Michigan
Page 10 of 40
Program Analysis Erie County Technical SchoolNOCTI Analysis
Class of 2008
9/24/2008
Program 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 TotalAuto Body Repair NEW TEST FOR 2007
Written Test n= 6Group Norm 50.6 54.6 39.6 70.6 53.9National Norm 56.9 55.2 46.1 67.1 56.3Std. Error 4.8 5.3 5.2 5 5.1Range--Low 52.1 49.9 40.9 0 51.3Range--High 61.7 60.5 51.3 72.1 61.4Within or Above Range FALSE TRUE FALSE TRUE TRUEVariance from Mean -6.3 -0.6 -6.5 3.5 -2.5
Sig. Diff.--National -6.3 0 -6.5 0 0.0
Automotive TechnologiesWritten Test n= 11
Group Norm 61.5 90 85 76 73.9 84.2 78.4National Norm 62.5 51.9 46.7 46.5 52.8 50.6 51.8Std. Error 4.9 5.9 5.2 5.1 5.9 5.7 5.5Range--Low 57.6 46 41.5 41.4 46.9 44.9 46.4Range--High 67.4 57.8 51.9 51.6 58.7 56.3 57.3Within or Above Range TRUE TRUE TRUE TRUE TRUE TRUE TRUEVariance from Mean -1.0 38.1 38.3 29.5 21.1 33.6 26.6
Sig. Diff.--National 0 0 0 0 0 0 0.0
Early Childhood Education NEW TEST FOR 2007Written Test n= 17
Group Norm 75 90 70.8 81.3 83.8 75.7 75.7 75 86.3 79.2 77.5 67.3 90.6 79.1National Norm 75.4 81.5 58.1 78 78.4 70.7 69 71.7 78.3 69.1 67.9 68.6 84 73.1Std. Error 4.1 4.3 5.7 3.9 4.7 4.9 5.6 6.1 5.7 6.7 6.7 6 6.2 5.4Range--Low 71.3 77.2 52.4 74.1 73.7 65.8 63.4 65.6 72.6 62.4 61.2 62.6 77.8 67.7Range--High 79.5 85.8 63.8 81.9 83.1 75.6 74.6 77.8 84 75.8 74.6 74.6 90.2 78.6Within or Above Range TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUEVariance from Mean -0.4 8.5 12.7 3.3 5.4 5.0 6.7 3.3 8.0 10.1 9.6 -1.3 6.6 6.0
Sig. Diff.--National 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Competency Area Number
Page 1Page 11 of 40
Student Analysis Erie County Technical SchoolNOCTI Analysis
Class of 2008
9/24/2008
ProgramStudent Scores Written Written Written Written Written Written Written
1 56.9 79.2 82.1 63.4 32.6 48.3 70.9 2 52.0 59.2 71.1 77.7 65.1 64.9 74.6 3 22.8 60.8 73.7 79.4 45.0 70.9 60.5 4 57.7 70.0 54.7 74.9 62.8 76.2 79.9 5 61.8 44.2 81.1 74.3 62.8 54.3 76.9 6 61.8 58.3 83.7 73.7 62.0 68.2 59.7 7 61.7 89.5 69.1 53.5 72.2 70.9 8 66.7 86.8 63.6 60.3 59.0 9 50.8 80.5 58.9 55.6 73.9
10 65.0 71.6 26.5 76.9 11 53.3 67.4 47.7 80.6 12 87.4 65.6 56.0 13 77.9 66.2 45.5 14 68.4 62.9 15 68.4 16 74.7 17 67.9
ADV 65.0 54.0 83.6 65.7 48.2 71.0 73.2 COM 60.0 49.0 78.6 60.7 43.2 66.0 68.2 BAS 50.0 39.0 68.6 0.1 33.2 56.0 58.2
Local Norm 52.2 60.8 75.7 73.2 56.3 60.0 68.1National Norm 55.8 51.1 73.9 63.3 50.7 54.8 69.3Std. Error 4.4 4.5 3.9 4.2 4.7 4.3 3.7Range--Low 51.4 46.6 70.0 59.1 46.0 50.5 65.6Range--High 60.2 55.6 77.8 67.5 55.4 59.1 73.0n= 6.0 11.0 17.0 7.0 9.0 14.0 13.0
# At or Above Nat. Ave. 5.0 9.0 12.0 7.0 8.0 13.0 8.0# Advanced 0 8 4 6 7 2 6# Competent 2 2 3 1 1 3 2
# Basic 4 1 10 0 1 9 5Advanced & Competent 2 10 7 7 8 5 8
% Above Nat. Ave. 83% 82% 71% 100% 89% 93% 62%
Auto Body Repair (2083)
Automotive Technologies
(4008)
Early Childhood Education
(3016)Art & Design
(3019)Construction Trades (3015)
Computer Information
Systems (3023)Cosmetology
(2082)
Page 1Page 12 of 40
Program Analysis Erie County Technical SchoolNOCTI Analysis--2008
9/24/2008
Program 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Competency Area Number
Auto Body Repair
Ref
inis
hing
Det
ailin
g
Cor
rosi
on
Safe
ty/E
nviro
n
Written TestGroup Norm--2006 53.6 71 70 59 57.1 84.6 40 37.5 40 70National Norm--2006 48 59 48.1 49 44.8 62.4 48 40.4 22.8 44.3Sig. Diff.--2006 0 0 0 0 0 0 -8 0 0 0Group Norm--2007 65.8 48.8 54.5 78.1National Norm--2007 54 42.2 47.7 64.7Sig. Diff.--2007 0 0 0 0Group Norm--2008 50.6 54.6 39.6 70.6National Norm--2008 56.9 55.2 46.1 67.1Sig. Diff.--2008 -6.3 0 -6.5 0
Automotive Technologies
Engi
ne R
epai
r
Susp
ensi
on
Bra
kes
Elec
trica
l
Engi
ne P
erf
Cus
tom
er R
els
Written TestGroup Norm--2006 54.9 54.7 53.1 58.0 53.2 78.5National Norm--2006 55.3 51.5 52 55.6 46.2 74.1Sig. Diff.--2006 0 0 0 0 0 0Group Norm--2007 60.1 55.4 55.8 56.4 50.6 75.0National Norm--2007 55 51.5 52.2 55.2 45.7 74.6Sig. Diff.--2007 0 0 0 0 0 0Group Norm--2008 61.5 90.0 85.0 76.0 73.9 84.2National Norm--2008 62.5 51.9 46.7 46.5 52.8 50.6Sig. Diff.--2008 0 0 0 0 0 0
Page 13 of 40
Student Analysis Erie County Technical SchoolNOCTI Analysis--2008
Percentage of Students Performing 'At or Above' the National Norm
9/24/2008
ProgramStudent Scores Written Written Written Written Written Written Written
Art & Design for Business
Auto BodyRepair
Automotive Technologies
Construction Trades
Early Childhod Education Cosmetology
Computer Information
2004 DataLocal Norm 52.5 75.3 70.5 48.8 68.6 74.1National Norm 48.6 70.4 0.0 46.9 63.5 68.3
# At or Above Nat. Ave. 5 0 8 10 7 15 10% At or Above Nat. Ave. 83% 89% n/a 78% 88% 91%
2005 DataLocal Norm 59.2 55.2 70.8 61.7 53.0 81.0 67.8National Norm 48.9 52.8 70.3 62.5 45.8 63.4 68.3
# At or Above Nat. Ave. 8 10 8 7 10 18 9% At or Above Nat. Ave. 100% 77% 80% 78% 83% 100% 75%
2006 DataLocal Norm 60.2 56.3 72.7 72.0 60.4 70.6 72.9National Norm 48.9 53.0 70.1 62.5 46.0 63.2 68.2
# At or Above Nat. Ave. 1 13 7 16 9 9 7% At or Above Nat. Ave. 100% 72% 78% 100% 100% 75% 70%
2007 DataLocal Norm 64.9 56.2 78.4 67.2 61.9 48.2 70.6National Norm 59.9 53.0 70.1 62.5 46.0 43.2 68.2
# At or Above Nat. Ave. 45 20 6 11 10 16 7% At or Above Nat. Ave. 0% 74% 86% 79% 100% 89% 54%
2008 DataLocal Norm 52.2 60.8 75.7 73.2 56.3 60.0 68.1National Norm 55.8 51.1 73.9 63.3 50.7 54.8 69.3
# At or Above Nat. Ave. 5 9 12 7 8 13 8% At or Above Nat. Ave. 83% 82% 71% 100% 89% 93% 62%
Page 14 of 40
Page 1
Auto Body RepairWritten
1999
Dat
a, 4
5.5
1999
Dat
a, 4
7.7
2000
Dat
a, 5
1.7
2000
Dat
a, 4
7.8
2001
Dat
a, 5
4.1
2001
Dat
a, 4
7.8
2002
Dat
a, 5
4.5
2002
Dat
a, 4
8.1
2003
Dat
a, 5
4.5
2003
Dat
a, 4
8.6
2004
Dat
a, 5
2.5
2004
Dat
a, 4
8.6
2005
Dat
a, 5
9.2
2005
Dat
a, 4
8.9
2006
Dat
a, 6
0.2
2006
Dat
a, 4
8.9
2007
Dat
a, 6
4.9
2007
Dat
a, 5
9.9
2008
Dat
a, 5
2.2
2008
Dat
a, 5
5.8
0
10
20
30
40
50
60
70
Local Norm National Norm
NOCTI Scores
Page 15 of 40
Erie County Technical SchoolNOCTI Analysis
Percent of Students At or Above the National Norm
83%
82%
71%
100%
89% 93%
62% 71
%
89%
43% 50
%
50%
43%
82%
65%
56%
33%
0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Auto Bod
y Rep
air
Automoti
ve Tec
hnolo
gies
Early C
hildh
od E
duca
tion
Art & D
esign
for B
usine
ss
Compu
ter In
formati
on S
ystem
s
Constr
uctio
n Trad
esCos
metolog
y
Culina
ry Arts
(Tes
t #09
90 A
CF)Draf
ting &
Des
ign
Electric
al Eng
ineeri
ngElec
tronic
s
Facilit
y Main
tenan
ce Tec
hnolo
gies
Graphic
Com
munica
tions
Health
Ass
istan
t
Hospit
ality
Manag
emen
tMeta
l Fab
ricati
on
Network
ing Tec
hnolo
gies
Tool &
Die
Programs
Perc
ent o
f Stu
dent
s
Programs PerformingAt or Above
Your Benchmark
Page 16 of 40
Page 17 of 40
Assignment #2
More than Numbers: Working with the Score Report
Competency Area Titles
(Record below)
% of NOCTI Test for
Competency Area
Nation Mean for
Competency Area
Standard Error
for Competency
Area
Nation Mean minus
Standard Error
Nation Mean plus
Standard Error
Range for Competency
Area
Group Mean for
Competency Area
Analysis: Group Mean
Within, Above or
Below Range?
Practice #1 (from Workplace Readiness Sample) Communications
24%
Practice #2 Problem Solving and Critical Thinking
19%
Competency 1:
Competency 2:
Competency 3:
Page 18 of 40
Program Analysis Erie County Technical SchoolNOCTI Analysis--2008
9/24/2008
Program 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17Competency Area Number
Auto Body Repair
Ref
inis
hing
Det
ailin
g
Cor
rosi
on
Safe
ty/E
nviro
n
Written TestGroup Norm--2006 53.6 71 70 59 57.1 84.6 40 37.5 40 70National Norm--2006 48 59 48.1 49 44.8 62.4 48 40.4 22.8 44.3Sig. Diff.--2006 0 0 0 0 0 0 -8 0 0 0Group Norm--2007 65.8 48.8 54.5 78.1National Norm--2007 54 42.2 47.7 64.7Sig. Diff.--2007 0 0 0 0Group Norm--2008 50.6 54.6 39.6 70.6National Norm--2008 56.9 55.2 46.1 67.1Sig. Diff.--2008 -6.3 0 -6.5 0
Automotive Technologies
Engi
ne R
epai
r
Susp
ensi
on
Bra
kes
Elec
trica
l
Engi
ne P
erf
Cus
tom
er R
els
Written TestGroup Norm--2006 54.9 54.7 53.1 58.0 53.2 78.5National Norm--2006 55.3 51.5 52 55.6 46.2 74.1Sig. Diff.--2006 0 0 0 0 0 0Group Norm--2007 60.1 55.4 55.8 56.4 50.6 75.0National Norm--2007 55 51.5 52.2 55.2 45.7 74.6Sig. Diff.--2007 0 0 0 0 0 0Group Norm--2008 61.5 90.0 85.0 76.0 73.9 84.2National Norm--2008 62.5 51.9 46.7 46.5 52.8 50.6Sig. Diff.--2008 0 0 0 0 0 0
Page 19 of 40
Student Analysis Erie County Technical SchoolNOCTI Analysis--2008
Percentage of Students Performing 'At or Above' the National Norm
9/24/2008
ProgramStudent Scores Written Written Written Written Written Written Written
Art & Design for Business
Auto BodyRepair
Automotive Technologies
Construction Trades
Early Childhod Education Cosmetology
Computer Information
2004 DataLocal Norm 52.5 75.3 70.5 48.8 68.6 74.1National Norm 48.6 70.4 0.0 46.9 63.5 68.3
# At or Above Nat. Ave. 5 0 8 10 7 15 10% At or Above Nat. Ave. 83% 89% n/a 78% 88% 91%
2005 DataLocal Norm 59.2 55.2 70.8 61.7 53.0 81.0 67.8National Norm 48.9 52.8 70.3 62.5 45.8 63.4 68.3
# At or Above Nat. Ave. 8 10 8 7 10 18 9% At or Above Nat. Ave. 100% 77% 80% 78% 83% 100% 75%
2006 DataLocal Norm 60.2 56.3 72.7 72.0 60.4 70.6 72.9National Norm 48.9 53.0 70.1 62.5 46.0 63.2 68.2
# At or Above Nat. Ave. 1 13 7 16 9 9 7% At or Above Nat. Ave. 100% 72% 78% 100% 100% 75% 70%
2007 DataLocal Norm 64.9 56.2 78.4 67.2 61.9 48.2 70.6National Norm 59.9 53.0 70.1 62.5 46.0 43.2 68.2
# At or Above Nat. Ave. 45 20 6 11 10 16 7% At or Above Nat. Ave. 0% 74% 86% 79% 100% 89% 54%
2008 DataLocal Norm 52.2 60.8 75.7 73.2 56.3 60.0 68.1National Norm 55.8 51.1 73.9 63.3 50.7 54.8 69.3
# At or Above Nat. Ave. 5 9 12 7 8 13 8% At or Above Nat. Ave. 83% 82% 71% 100% 89% 93% 62%
Page 20 of 40
Page 1
Construction Trades--Written
1999
Dat
a, 5
6.2
1999
Dat
a, 8
1.2
2000
Dat
a, 4
9.4
2000
Dat
a, 4
9.0
2001
Dat
a, 6
5.6
2001
Dat
a, 6
3.1
2002
Dat
a, 6
3.8
2002
Dat
a, 6
3.3
2003
Dat
a, 7
3.1
2003
Dat
a, 6
3.4
2004
Dat
a, 6
8.6
2004
Dat
a, 6
3.5
2005
Dat
a, 8
1.0
2005
Dat
a, 6
3.4
2006
Dat
a, 7
0.6
2006
Dat
a, 6
3.2
2007
Dat
a, 4
8.2
2007
Dat
a, 4
3.2
2008
Dat
a, 6
0.0
2008
Dat
a, 5
4.8
0
10
20
30
40
50
60
70
80
90
Local Norm National Norm
NOCTI Scores
Page 21 of 40
Page 1
Facilities Maintenance TechnologiesWritten
-30
-20
-10
0
10
20
30
40
50
60
70
80
Competencies Tested
Perc
ent S
core
Group Norm--2006 42.1 79.0 65.0 60.0 55.7 66.7 72.5 73.3 44.3
Sig. Diff.--2006 -9.5 0 0 0 0 0 0 0 0.0
Group Norm--2007 36.7 48.5 56.7 38.3 33.3 38.9 40.0 51.1 36.4
Sig. Diff.--2007 -14 0 0 -19.7 -9.9 -20.3 -14.5 -8.7 -11.8
Group Norm--2008 46.2 71.1 58.0 55.6 42.9 63.6 60.4 65.3 43.8
Sig. Diff.--2008 0 0 0 0 0 0 0 0 0
General Maintenance- Welding, Glazing Electrical Plumbing Painting/Wall Covering
HVAC/Sheet Metal; Power Plant/Boiler Masonry
Grounds & Equipment Maint;
Hydraulic/PneumaticSafety and Tools Carpentry
Page 22 of 40
Page 1
Auto Body RepairWritten
1999
Dat
a, 4
5.5
1999
Dat
a, 4
7.7
2000
Dat
a, 5
1.7
2000
Dat
a, 4
7.8
2001
Dat
a, 5
4.1
2001
Dat
a, 4
7.8
2002
Dat
a, 5
4.5
2002
Dat
a, 4
8.1
2003
Dat
a, 5
4.5
2003
Dat
a, 4
8.6
2004
Dat
a, 5
2.5
2004
Dat
a, 4
8.6
2005
Dat
a, 5
9.2
2005
Dat
a, 4
8.9
2006
Dat
a, 6
0.2
2006
Dat
a, 4
8.9
2007
Dat
a, 6
4.9
2007
Dat
a, 5
9.9
2008
Dat
a, 5
2.2
2008
Dat
a, 5
5.8
0
10
20
30
40
50
60
70
Local Norm National Norm
NOCTI Scores
Local NormAbove
National Norm
Page 23 of 40
Page 1
Networking Technologies--Written
2005
Dat
a, 5
5.9
2005
Dat
a, 6
2.9
2006
Dat
a, 4
6.3
2006
Dat
a, 6
3.2
2007
Dat
a, 4
6.3
2007
Dat
a, 6
3.2
2008
Dat
a, 5
3.5
2008
Dat
a, 5
7.7
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
Local Norm National Norm
NOCTI Scores
Local NormBelow
National Norm
Page 24 of 40
Page 1
Drafting & Design--Written
1999
Dat
a, 6
2.0
1999
Dat
a, 7
5.7
2000
Dat
a, 5
2.4
2000
Dat
a, 5
5.0
2001
Dat
a, 5
7.1
2001
Dat
a, 5
6.2
2002
Dat
a, 5
8.0
2002
Dat
a, 5
5.1
2003
Dat
a, 5
2.7
2003
Dat
a, 5
4.7
2004
Dat
a, 5
7.0
2004
Dat
a, 5
4.6
2005
Dat
a, 5
4.0
2005
Dat
a, 5
4.6
2006
Dat
a, 5
9.6
2006
Dat
a, 5
4.5
2007
Dat
a, 6
0.5
2007
Dat
a, 5
4.5
2008
Dat
a, 6
8.8
2008
Dat
a, 5
9.6
0
10
20
30
40
50
60
70
80
Local Norm National Norm
NOCTI Scores
Up-and-DownLong-term
Trend
Page 25 of 40
Erie County Technical SchoolProficiency Level Percentage
2007-2008
0%
25%
50%
75%
100%
Auto B
ody R
epair
(208
3)
Automoti
ve Tec
hnolo
gies (
4008
)
Early C
hildh
ood E
duca
tion (
3016
)
Art & D
esign
(301
9)
Compu
ter In
formati
on S
ystem
s (30
23)
Constr
uctio
n Trad
es (3
015)
Cosmeto
logy (
2082
)
Culi
nary
Arts (4
136 L
evel
2) KIP
S
Culi
nary
Arts (0
990)
ACFF
Draftin
g & D
esign
(303
8)
Electric
al Eng
ineeri
ng (3
030)
Electro
nics
(303
5)
Facilit
ies M
ainten
ance
Techs
(201
2)
Graphic
Com
munica
tions
(414
2)
Health
Ass
istan
t (305
8)
Touris
m & H
ospit
ality
Mgmt. (
2080
)
Metal F
abric
ation
(307
2)
Network
ing Tec
hnolo
gies (
2414
)
Tool &
Die-
PRECIS
ION M
ACHININ
G
Technical Training Programs
Perc
ent P
rofic
ient
% Advanced % Competent % Basic
Page 26 of 40
Page 1
Facilities Maintenance TechnologiesWritten
-30
-20
-10
0
10
20
30
40
50
60
70
80
Competencies Tested
Perc
ent S
core
Group Norm--2006 42.1 79.0 65.0 60.0 55.7 66.7 72.5 73.3 44.3
Sig. Diff.--2006 -9.5 0 0 0 0 0 0 0 0.0
Group Norm--2007 36.7 48.5 56.7 38.3 33.3 38.9 40.0 51.1 36.4
Sig. Diff.--2007 -14 0 0 -19.7 -9.9 -20.3 -14.5 -8.7 -11.8
Group Norm--2008 46.2 71.1 58.0 55.6 42.9 63.6 60.4 65.3 43.8
Sig. Diff.--2008 0 0 0 0 0 0 0 0 0
General Maintenance- Welding, Glazing Electrical Plumbing Painting/Wall Covering
HVAC/Sheet Metal; Power Plant/Boiler Masonry
Grounds & Equipment Maint;
Hydraulic/PneumaticSafety and Tools Carpentry
Declining Competency
Deficient Competencies
Page 27 of 40
Page 1
Drafting & DesignWritten
-10
0
10
20
30
40
50
60
70
80
90
100
Competencies Tested
Perc
ent S
core
Group Norm--2007 72.6 62.8 56.3 68.9 54.7 45.1
Sig. Diff.--2007 0 0 0 0 0 0
Group Norm--2008 72.6 62.8 56.3 68.9 54.7 45.1
Sig. Diff.--2008 0 0 0 0 0 0
Prepare to Draw- Basic Dimensioning
24%
Geometric Construction19%
Drawing Techniques13%
Applied Mathematics9%
Dimensioning Skills21%
Supplementary Views14%
Competency AreasRelative to
NOCTI Blueprint Percentages
Page 28 of 40
Root Cause Analysis
• Ask “Why?” Five Times • Cause-and-Effect Diagram (Fishbone)
• Interrelationship Diagram
You are looking for ‘causes of variance’
from the norm or the expected
Page 29 of 40
Page 1
Facilities Maintenance TechnologiesWritten
-30
-20
-10
0
10
20
30
40
50
60
70
80
Competencies Tested
Perc
ent S
core
Group Norm--2006 42.1 79.0 65.0 60.0 55.7 66.7 72.5 73.3 44.3
Sig. Diff.--2006 -9.5 0 0 0 0 0 0 0 0.0
Group Norm--2007 36.7 48.5 56.7 38.3 33.3 38.9 40.0 51.1 36.4
Sig. Diff.--2007 -14 0 0 -19.7 -9.9 -20.3 -14.5 -8.7 -11.8
Group Norm--2008 46.2 71.1 58.0 55.6 42.9 63.6 60.4 65.3 43.8
Sig. Diff.--2008 0 0 0 0 0 0 0 0 0
General Maintenance- Welding, Glazing Electrical Plumbing Painting/Wall Covering
HVAC/Sheet Metal; Power Plant/Boiler Masonry
Grounds & Equipment Maint;
Hydraulic/PneumaticSafety and Tools Carpentry
Potential Sources of Deficiency--
Student-Related Variables
Page 30 of 40
Page 1
Drafting & Design--Written
1999
Dat
a, 6
2.0
1999
Dat
a, 7
5.7
2000
Dat
a, 5
2.4
2000
Dat
a, 5
5.0
2001
Dat
a, 5
7.1
2001
Dat
a, 5
6.2
2002
Dat
a, 5
8.0
2002
Dat
a, 5
5.1
2003
Dat
a, 5
2.7
2003
Dat
a, 5
4.7
2004
Dat
a, 5
7.0
2004
Dat
a, 5
4.6
2005
Dat
a, 5
4.0
2005
Dat
a, 5
4.6
2006
Dat
a, 5
9.6
2006
Dat
a, 5
4.5
2007
Dat
a, 6
0.5
2007
Dat
a, 5
4.5
2008
Dat
a, 6
8.8
2008
Dat
a, 5
9.6
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
Local Norm National Norm
NOCTI Scores
Potential Sources ofDeficiency--Staff-Related
Variables
Health Condition
Page 31 of 40
Page 1
Tool & Die--Written
1999
Dat
a, 4
8.6
1999
Dat
a, 5
1.6
2000
Dat
a, 6
0.0
2000
Dat
a, 5
1.9
2001
Dat
a, 6
1.7
2001
Dat
a, 5
1.6
2002
Dat
a, 5
8.4
2002
Dat
a, 5
2.1
2003
Dat
a, 6
6.5
2003
Dat
a, 5
2.6
2004
Dat
a, 6
7.9
2004
Dat
a, 5
2.6
2005
Dat
a, 6
7.9
2005
Dat
a, 5
3.1
2006
Dat
a, 7
4.4
2006
Dat
a, 6
7.6
2007
Dat
a, 6
5.6
2007
Dat
a, 6
7.6
2008
Dat
a, 4
8.2
2008
Dat
a, 6
0.8
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
Local Norm National Norm
NOCTI Scores
Potential Sources ofDeficiency--Staff-Related
VariablesNew Teacher
Page 32 of 40
Collision Repair And Refinishing Technology
Specific Competencies and Skills Tested in this Assessment:
Structural
• Select, setup, and adjust welders • Describe various types and uses of
welding processes • Use adhesive bonding procedures • Analyze primary and secondary
damage • Replace and/or repair structural
components Non-Structural
• Identify automotive plastics and proper repair procedures • Remove and replace automotive glass and trim • Remove, install, replace, and align non-structural panels • Remove, install, and replace components
Painting and Refinishing
• Identify and demonstrate surface preparation techniques • Identify and demonstrate paint and materials preparation techniques • Identify and demonstrate paint and materials application techniques • Identify painting and refinishing safety/environmental requirements
Mechanical and Electrical Systems
• Identify, remove, and install/replace components • Identify basic suspension geometry • Describe the terms voltage, current, and resistance (fundamental
electrical theory) • Perform basic and advanced electrical diagnostic operations
Safety and Environmental Issues
• Demonstrate knowledge of safety/environmental requirements in the collision repair industry
• Identify proper safety techniques for the use of shop equipment
Page 1 of 5 NOCTI Job Ready Assessment Copyright © NOCTI 2007 All Rights Reserved Page 33 of 40
Assignment #4 How Close Are They- NOCTI and my Curriculum?
Aligning Your Program with NOCTI Competencies
Auto Body Technology Test Code 2083 Check the NOCTI competencies below against your Duty/Task List; if your program's tasks are numbered, indicate the corresponding number in the column to the left of the NOCTI competency. Highlight any NOCTI competency on this list that you can not find in your curriculum. Specific Competencies and Skills Tested in this Assessment: Structural Select, setup, and adjust welders Describe various types and uses of welding processes Use adhesive bonding procedures Analyze primary and secondary damage Replace and/or repair structural components Non-Structural Identify automotive plastics and proper repair procedures Remove and replace automotive glass and trim Remove, install, replace, and align non-structural panels Remove, install, and replace components Painting and Refinishing Identify and demonstrate surface preparation techniques Identify and demonstrate paint and materials preparation techniques Identify and demonstrate paint and materials application techniques Identify painting and refinishing safety/environmental requirements Mechanical and Electrical Systems Identify, remove, and install/replace components Identify basic suspension geometry Describe the terms voltage, current, and resistance (fundamental electrical
theory) Perform basic and advanced electrical diagnostic operations Safety and Environmental Issues Demonstrate knowledge of safety/environmental requirements in the collision
repair industry Identify proper safety techniques for the use of shop equipment
Page 34 of 40
Collision Repair And Refinishing Technology
Written Assessment:
Administration Time: 3 hours Number of Questions: 150 Areas Covered:
Page 2 of 5 NOCTI Job Ready Assessment Copyright © NOCTI 2007 All Rights Reserved Page 35 of 40
Revision Date: February 2008
AUTO BODY
REPAIR
Program Guide
Content
Area Course Title Course Number Course Hours
00 Intro to Auto Body Orientation and Safety ABR101 45 Principles of Design Construction Lifting
and Supporting ABR102 45 Hand Tools ABR103 30 Power Tools ABR104 30 Mathematics for Automotive Body
Repair I ABR105 15 Vehicle Handling in the Shop ABR106 30 Mathematics for Automotive Body
Repair II ABR107 15 Applied Math II ABR201 15 Applied Math III ABR301 15 Leadership ABR109 15 Business Principles ABR209 15 Total Quality Concepts ABR309 15
10 Laboratory Practices Applied Practices I ABR111 15 Applied Practices II ABR112 15 LII Laboratory Practices I ABR211 30 LII Laboratory Practices II ABR212 30 LII Laboratory Practices III ABR213 30 LII Laboratory Practices IV ABR214 30 LIII Laboratory Practices I ABR311 30 LIII Laboratory Practices II ABR312 30 LIII Laboratory Practices III ABR313 30 LIII Laboratory Practices IV ABR314 30
20 Welding LI MIG Welding I ABR121 30 LI MIG Welding II ABR122 30 LII MIG Welding I ABR221 30 LII MIG Welding II ABR222 30 LII MIG Welding III ABR223 30
Page 36 of 40
Revision Date: February 2008
LIII MIG Welding I ABR321 30 LIII MIG Welding II ABR322 30 LIII MIG Welding III ABR323 30
30 Panel Repairs Sheet Metal Repair I ABR131 30 Sheet Metal Repair II ABR132 15 LII Panel Repair I ABR231 30 LII Panel Repair II ABR232 30 LII Panel Repair III ABR233 30
LII Panel Repair IV ABR234 30 LIII Panel Repair I ABR331 30 LIII Panel Repair II ABR332 30 LIII Panel Repair III ABR333 30 LIII Panel Repair IV ABR334 30
40 Paint and Refinishing Surface Prep ABR141 30 Sanding Techniques I ABR142 30 Refinish Equipment and Masking
Techniques ABR143 30 LII Refinish I ABR241 30 LII Refinish II ABR242 30 LII Refinish III ABR243 30 LII Refinish IV ABR244 30 LIII Refinish I ABR341 30 LIII Refinish II ABR342 30 LIII Refinish III ABR343 30 LIII Refinish IV ABR344 30
80 Business Concepts Estimating ABR181 30
Page 37 of 40
AUTO BODY REPAIR ECTS QUARTERLY COURSE SEQUENCE
Revision Date: December 2007 JK
YE
AR
I
1st Quarter ABR101 45 Orientation and Safety ABR102 45 Principles of Design, Construction Lifting and Supporting ABR103 30 Hand Tools TOTAL:120
2nd Quarter ABR104 30 Power Tools ABR121 30 MIG Welding I ABR131 30 Sheet Metal Repair I ABR105 15 Math for Automotive Body Repair I ABR109 15 Leadership TOTAL: 120
3rd Quarter ABR111 15 Applied Practices I ABR122 30 MIG Welding II ABR132 15 Sheet Metal Repair II ABR141 30 Surface Prep ABR142 30 Sanding Techniques TOTAL:120
4th Quarter ABR112 15 Applied Practices II ABR107 15 Math for Automotive Body Repair II ABR 106 30 Vehicle Handling in the Shop ABR143 30 Refinish Equipment and Masking Techniques ABR181 Estimating 30 TOTAL:120
YE
AR
II
1st Quarter ABR211 30 LII Laboratory Practices I ABR221 30 LII, MIG Welding I ABR241 30 LII Refinishing I ABR231 30 LII Panel Repair I TOTAL:120
2nd Quarter ABR212 30 LII Laboratory Practices II ABR201 15 Applied Math II ABR209 15 Business Principles ABR242 30 LII Refinishing II ABR232 LII Panel Repair II 30 TOTAL:120
3rd Quarter ABR213 30 LII Laboratory Practices III ABR222 30 LII MIG Welding II ABR243 30 LII Refinishing III ABR233 LII Panel Repair III 30 TOTAL:120
4th Quarter ABR214 30 LII Laboratory Practices IV ABR223 30 LII MIG Welding III ABR244 30 LII Refinishing IV ABR234 LII Panel Repair IV 30 TOTAL:120
YE
AR
III
1st Quarter ABR311 30 LIII Laboratory Practices I ABR321 30 LIII MIG Welding I ABR341 30 LIII Refinishing I ABR331 30 LIII Panel Repair I TOTAL:120
2nd Quarter ABR312 30 LIII Laboratory Practices II ABR309 15 Total Quality Concepts ABR301 15 Applied Math III ABR342 30 LIII Refinishing II ABR332 30 LIII Panel Repair II TOTAL:120
3rd Quarter ABR313 30 LIII Laboratory Practices III ABR322 30 LIII MIG Welding II ABR343 30 LIII Refinishing III ABR333 30 LIII Panel Repair III TOTAL: 120
4th Quarter ABR314 30 LIII Laboratory Practices IV ABR323 30 LIII MIG Welding III ABR344 30 LIII Refinishing IV ABR334 30 LIII Panel Repair IV TOTAL: 120
Page 38 of 40
Page 1
Electrical EngineeringWritten
-40
-30
-20
-10
0
10
20
30
40
50
60
70
Competencies Tested
Perc
ent S
core
Group Norm--2007 47.5 52.5 35.8 58.6 33.6 51.1 61.4 34.6 22.5 33.8 47.5 40 38.6 37.5 40 35 41.7
Sig. Diff.--2007 -9.8 -19.7 -19.4 -12.7 -18.3 -21.4 0 -16.6 -16.8 -17.6 0 -12.5 -14 -8.4 -8.8 0 -15.9
Group Norm--2008 28.6 40.5 35.3 60.2 29.9 50.8 38.8 27.5 32.1 34.8 39.3 27.5 28.6 25 38.1 42.9 40.5
Sig. Diff.--2008 -29.8 -30.1 -18.2 -12.4 -20.3 -21.7 -19.5 -22 0 -14.2 0 -24.5 -23.9 -18.9 -8.8 0 -12.1
General Information Safety
Meters, Measurements, Testing
Identification of Tools,
Materials, &
National Electric Code
Blueprints, Drawings, Diagrams,
Planning, Layout, & Estimating
AC Theory, Conductors, Magnetic
ElectronicsDC: Basic
Electric/Electron
Low Voltage Systems
Wiring, Circuits &
Installation
Raceway & Conduit Bending
Transformers
Motors, Motor
Control Lighting
Basic Computer
Skills
Page 39 of 40
Creating a
Strategic Improvement
Plan
to improve student achievement
Page 40 of 40
Top Related