William R. Veal, Ph.D.

College of Charleston

NSTA/NCATE SPA Coordinator

Objectives

See the relationship between NSTA 2003 Standards and the NCATE Assessment System

Understand the NSTA-specific items that make this report science-specific

Discuss exemplar assessments

Look at the 2012 NSTA Preservice Standards

Things to keep in mind for scoring guides

Do the criteria adequately cover the important defining elements of the desired performance?

Do the cell descriptors distinguish the different levels of performance in observable ways?

Are the cell descriptors likely to be interpreted in the same way by different evaluators? (inter-rater reliability)

Is there enough substance to the assessment to assure external reviewers that if the criteria are met, the candidate is prepared to teach to the standards?

Section I - Context

Describe state or institutional policies

Describe field and clinical experiences

Program of Study:

Most important for graduate program to show content alignment

Advising Sheet

Include 3 years of data on candidates

Disaggregate data in a table by licensure area (i.e., biology, chemistry, etc.) and program level (i.e., baccalaureate, post-bacc., masters, etc.)

Section II - NCATE Assessments

Maximum of 8 assessments

Minimum of 6 assessments, but NSTA requires 8

Numbers 1 through 5 are the same for all SPA’s

Assessments fall into 3 categories

Content

Pedagogy

K-12 Student Learning

NSTA/NCATE Alignment (2003)

Standards 1 2 3 4 5 6 7 8

Concepts & Principles

Unifying Concepts

Technology in Sc.

Research in science

Math in science

2. Nature of Science

3. Inquiry

4. Issues in Science

6. Curriculum

7. Science in the Community

8. Assessment

9. Safety & Welfare

1. S

c. c

on

ten

t

Section IV – Evidence for Meeting Stds.

Key assessments required for all candidates.

Data tables should reflect & align with the scoring guides

10 criteria in scoring guide = 10 data points in table

Narrative:

A. Description of the assessment and use in program

B. Alignment of assessment with stds. (scoring guide)

C. Brief data analysis

D. Interpretation and reflection on program

1 Content: Praxis II Test

Standards 1 2 3 4 5 6 7 8

Concepts & Principles

Unifying Concepts

Technology in Sc.

Research in science

Math in science

2. Nature of Science

3. Inquiry

4. Issues in Science

6. Curriculum

7. Science in the Community

8. Assessment

9. Safety & Welfare

1. S

c. c

on

ten

t

1 Content: Praxis II or state exam

Alignment of test with NSTA stds. if not Praxis II

Aggregated pass rates for all completers for the previous 3 years (includes most recent year) [2 years for resubmits]

Disaggregate data by year, program level, and licensure area

80% pass rate applies if more than 10 total (all science content tests for all 3 years)

Data include:

Mean scores

Range scores

Pass rates for state

Sub-scores if available

Year Number of

Completers Pass Rate (%)

2010-2011 6 100

2009-2010 5 100

2008-2009 7 87.5

Subscore 1 (Science and Technology): 278.5 (254-300) Subscore 2 (Life Science): 284.7 (269-300) Subscore 3 (Physical Science): 264.3 (238-279) Subscore 4 (Earth Systems &the Universe): 251.3 (222-279) Subscore 5 (Cell Biology, Heredity, &Evolution): 267.2 (240-283) Subscore 6 (Organismal Biology and Ecology): 274.8 (255-300)

Aggregated pass rates on ICTS Science: Biology test for past 3 years Mean (and range) of subscores on ICTS Science: Biology test for most recent year (2010-2011)

Assessment 1 Content: Praxis II Items to Consider

Are the data available?

How will you get the data?

Do you have 10 completers in years? If not, then do not have to meet 80%. Still have to show disaggregated data by program and level and subscores.

Reflections on data

Upon examination of the data, we were able to learn and improve our program. For example, the year that we dropped Evolutionary Biology from our required coursework, the subscores in that area dropped dramatically. We re-evaluated our program and brought the course back into the requirements.

2 Content: Science Content Alignment

NSTA 2003 Standards 1 2 3 4 5 6 7 8

Concepts &Principles

Unifying Concepts

Technology in Sc.

Research in science

Math in science

2. Nature of Science

3. Inquiry

4. Issues in Science

6. Curriculum

7. Science in the Community

8. Assessment

9. Safety & Welfare

1. S

c. c

on

ten

t

2: GPA and Content Exams

Grade distributions or GPAs must be disaggregated by licensure areas of the candidates (i.e., biology, general science, etc)

distinct levels of licensure (middle level, high school, secondary, etc

degree levels of preparation (undergraduate/postbaccalaureate/ masters)

Provide aggregated data from all science courses taken by candidates in the program.

Include science content alignment with the 2003 NSTA Standards using the Content Analysis Forms

GPA and Science Content

Year Program Number of

Completers

Mean

Major GPA

(4.0 scale)

Range of

Major GPAs

(4.0 scale)

2009-2010 Undergraduate 0 ---- ----

2008-2009 Undergraduate 3 3.11 2.60-4.0

2007-2008 Undergraduate 7 3.16 2.88-3.40

Table 1: Undergraduate Major GPA- Traditional Plan of Study

2: Content Analysis Forms Available on the NCATE website

Align the science content with the NSTA 2003 Standards

Must demonstrate a 90% alignment

Table 1: Unifying Concepts A. Core

Competencies

B. Required course number & name and

advising requirements

1. Multiple ways we

organize our

perceptions of the world

and how systems

organize the studies

and knowledge of

science.

CHE 161- Introduction to Teaching Science Seminar CHE140- General Chemistry I CHE 141- General Chemistry II BSC 197- Molecular and Cellular Basis for Life GEO 100 Earth Systems Science PHY 110- Physics for Engineering and Science Majors I PHY 111 Physics for Engineering and Science Majors II PHY 205- Origins of the Universe

2. Nature of scientific

evidence and the use of

models for explanation.

CHE 161- Introduction to Teaching Science Seminar CHE140- General Chemistry I CHE 141- General Chemistry II BSC 197- Molecular and Cellular Basis for Life GEO 100 Earth Systems Science PHY 110- Physics for Engineering and Science Majors I PHY 111 Physics for Engineering and Science Majors II PHY 205- Origins of the Universe

3 Pedagogy: Unit Plan

NSTA 2003 Standards 1 2 3 4 5 6 7 8

Concepts & Principles

Unifying Concepts

Technology in Sc.

Research in science

Math in science

2. Nature of Science

3. Inquiry

4. Issues in Science

6. Curriculum

7. Science in the Community

8. Assessment

9. Safety & Welfare

1. S

c. c

on

ten

t

3 Pedagogy: Unit Plan At least one major teaching unit

(not just a single lesson plan) includes requirements for activities with lesson plans and various assessments. The unit must include: Unifying concepts

Nature of science

Inquiry

Issues

Personal and technological applications

State and national (NSES) science standards

Assessment

Must be SCIENCE-specific (i.e., criteria must align with NSTA elements)

3 Pedagogy: Unit Plan

How does this assessment fit into what the rest of the Unit is doing?

Can you build on a Unit Plan assessment that is already a part of the program?

Are the scoring guides presently used at your institution specific to the 2003 NSTA standards?

Usually done with an Addendum.

Criteria / Sub-Standard

Inadequate –1 Adequate - 2 Target - 3

1a. Understand major science concepts.

One or more lessons contain scientific inaccuracies and/or are ineffective at conveying concepts or principles. The unit fails to teach how scientific theories are developed.

Lessons successfully and accurately convey to students concepts and principles of science.

Lessons very successfully and accurately convey to students concepts and principles of science. Creative planning is evident. The unit teaches how scientific theories are developed.

1b. Understand NSES unifying concepts.

The unit fails to teach the nature of scientific laws or to relate concepts to them.

The unit teaches the nature of scientific laws and relates concepts to them.

The unit teaches the nature of scientific laws and relates concepts to them in particularly effective ways.

1c. Understand technical applications of science.

The personal and technological applications of the scientific concepts and principles of the unit are inadequately addressed, or pedagogy is inadequate for student understanding.

The personal and technological applications of the scientific concepts and principles of the unit are effectively addressed and pedagogy is adequate to develop student understanding.

Candidates apply their understanding of the personal and technological applications of the scientific concepts and principles to their own life and surroundings.

2a. NOS Understanding of the historical and cultural development of science as related to the key ideas and major understandings of the unit are poorly addressed or pedagogy is inadequate to develop student understanding.

Understanding of the historical and cultural development of science as related to the key ideas and major understandings of the unit are explicitly addressed and pedagogy is adequate to develop candidate understanding.

Candidates use their understanding of historical and cultural development of science in multiple and creative ways in the lesson plans.

4 Pedagogy: Student Teaching Observation Form

NSTA 2003 Standards 1 2 3 4 5 6 7 8

Concepts & Principles

Unifying Concepts

Technology in Sc.

Research in science

Math in science

2. Nature of Science

3. Inquiry

4. Issues in Science

6. Curriculum

7. Science in the Community

8. Assessment

9. Safety & Welfare

1. S

c. c

on

ten

t

4 Pedagogy: Student Teaching Observation Form Addendum to regular form

Shows that candidates can implement safety knowledge

Include science specific areas

Safety elements 9a-d is a minimum to pass this Assessment.

4 Pedagogy: Example A

Standard Likert

9b. The candidate practices safe and proper techniques for the preparation, storage, dispensing, supervision, and disposal of all materials used in science instruction.

1 2 3 4 5

1 does not address standard 2 addresses but does not meet standard 3 meets standard at acceptable level 4 above minimum acceptable level 5 exceeds standard

Scoring guide needs improvement: The criteria do not differentiate different levels of performance in observable ways.

4 Pedagogy: Example B Unacceptable Acceptable Target

Safe techniques (NSTA 9b)

The intern fails to practice safe and proper techniques for the preparation, storage, dispensing, supervision, and disposal of all materials used in science instruction.

The intern practices safe and proper techniques for the preparation, storage, dispensing, supervision, and disposal of all materials used in science instruction.

The intern practices safe and proper techniques for the preparation, storage, dispensing, supervision, and disposal of all materials used in science instruction without additional prompting from mentor or university supervisor.

Candidate are rated using the following guidelines: Unacceptable: The candidate's performance is not improving and is not acceptable Acceptable: The candidate's performance meets the expectations for pre-service teachers Target: The candidate's performance meets expectations that would be characteristic of an experienced teacher

Scoring guide needs improvement: Criteria using operational terms are needed. More description in the criterion is needed.

5: Evidence of K-12 Student Learning

NSTA 2003 Standards 1 2 3 4 5 6 7 8

Concepts & Principles

Unifying Concepts

Technology in Sc.

Research in science

Math in science

2. Nature of Science

3. Inquiry

4. Issues in Science

6. Curriculum

7. Science in the Community

8. Assessment

9. Safety & Welfare

1. S

c. c

on

ten

t

5: Evidence of K-12 Student Learning

Candidate/Teacher Work Sample or Portfolio

Must be science specific and address each applicable NSTA standard and element.

Must have descriptors that are measurable.

2c. The candidate facilitates students learning the nature of science

Provides little to no evidence of student learning beyond memorization of at least one aspect of the nature of science.

Provides evidence of student learning beyond memorization of at least one aspect of the nature of science. Representative sample of student work included.

Provides evidence of student learning beyond memorization of two or more aspects of the nature of science and representative samples of student work. Includes a reflective analysis of students’ learning.

6 Content: Safety and Welfare

NSTA 2003 Standards 1 2 3 4 5 6 7 8

Concepts & Principles

Unifying Concepts

Technology in Sc.

Research in science

Math in science

2. Nature of Science

3. Inquiry

4. Issues in Science

6. Curriculum

7. Science in the Community

8. Assessment

9. Safety & Welfare

1. S

c. c

on

ten

t

This assessment shows and measures candidates knowledge of safety items 9a-d.

Usually involves a test of safety knowledge.

Test items must be specifically aligned with each NSTA element, or aggregated assignments and tests are aligned with each element.

Ex. 5. Hazardous chemicals enter the body the fastest by: (9a)

A. ingestion.

B unbroken skin absorption.

C inhalation.

D. broken skin absorption

Ex. Results of Flinn safety exams & chapters 12-18 (9b)

7 Content: Research in Science Discipline

NSTA 2003 Standards 1 2 3 4 5 6 7 8

Concepts & Principles

Unifying Concepts

Technology in Sc.

Research in science

Math in science

2. Nature of Science

3. Inquiry

4. Issues in Science

6. Curriculum

7. Science in the Community

8. Assessment

9. Safety & Welfare

1. S

c. c

on

ten

t

7 Content: Research in Science Discipline

Usually performed in the science content courses

A system for post degree or transfer candidates

Alternative course or

Evaluation of previous work

Action Research or science fair is not acceptable

What would your science departments deem acceptable?

Could be a short project

Example Scoring Guide Investigation parts Not acceptable (1) Acceptable (2) Excellent (3)

D. Data collection and organization (NSTA 1d)

Data is collected in a manner that is inconsistent with the science discipline. Data is poorly organized and/or displayed.

Data is collected consistent with the science discipline. Data is organized and presented using graphs, charts or tables with minor errors.

Data is collected consistent with the science discipline. Data is organized, clearly addresses research problem and presented with no errors.

F. Analysis and interpretation of data (NSTA 1d, 1e)

Inappropriate techniques used in data analysis and interpretation.

Appropriate techniques used in data analysis and interpretation with minor errors.

Appropriate techniques are used in data analysis and interpretation is clear and concise with no errors.

G. Final conclusions (NSTA 1d)

Conclusions not clearly related to problem or supported by data

Conclusions generally related to problem and supported by data

Conclusions clearly related to problem and fully supported by data

No part of this project should receive less than a “2” or it should be redone to the satisfaction of the instructor.

8 Content: Contextual Content

NSTA 2003 Standards 1 2 3 4 5 6 7 8

Concepts & Principles

Unifying Concepts

Technology in Sc.

Research in science

Math in science

2. Nature of Science

3. Inquiry

4. Issues in Science

6. Curriculum

7. Science in the Community

8. Assessment

9. Safety & Welfare

1. S

c. c

on

ten

t

8 Content: Contextual Content

Content understanding of topics important to science educators

Assessment #8 is usually addressed with a portfolio of appropriate assignments

If there are courses that help to meet this standard, there needs to be specific descriptions to tie the course directly back to the 2003 NSTA standards

Review: Things to keep in mind for scoring guides

Do the criteria adequately cover the important defining elements of the desired performance?

Do the cell descriptors distinguish the different levels of performance in observable ways?

Are the cell descriptors likely to be interpreted in the same way by different evaluators? (inter-rater reliability)

Is there enough substance to the assessment to assure external reviewers that if the criteria are met, the candidate is prepared to teach to the standards?

New 2012 NSTA Standards

These were approved by NSTA & SASB.

First submission in Fall 2012 as a pilot.

Programs can use either set of standards through Fall 2013.

Spring 2014 – All programs must use 2012 Standards

Discontinue Option C at this time.

NSTA Standard 1: Content Knowledge Effective teachers of science understand and articulate the knowledge and practices of contemporary science. They interrelate and interpret important concepts, ideas, and applications in their fields of licensure.

Preservice teachers will:

1a) Understand the major concepts, principles, theories, laws, and interrelationships of their fields of licensure and supporting fields as recommended by the National Science Teachers Association.

1b) Understand the central concepts of the supporting disciplines and the supporting role of science-specific technology.

1c) Show an understanding of state and national curriculum standards and their impact on the content knowledge necessary for teaching P-12 students.

NSTA Standard 2: Content Pedagogy Effective teachers of science understand how students learn and develop scientific knowledge. Preservice teachers use scientific inquiry to develop this knowledge for all students.

Preservice teachers will:

2a) Plan multiple lessons using a variety of inquiry approaches that demonstrate their knowledge and understanding of how all students learn science.

2b) Include active inquiry lessons where students collect and interpret data in order to develop and communicate concepts and understand scientific processes, relationships and natural patterns from empirical experiences. Applications of science-specific technology are included in the lessons when appropriate.

2c) Design instruction and assessment strategies that confront and address naïve concepts/preconceptions.

NSTA 2011 Standards and Elements Content Assmt

1 Stand. Tests

Content Assmt 2

Grades / Analysis

Forms

PCK Assmt 3

Unit Plan

PCK Assmt 4 Std.

Tchg. eval form

Std. Lrng. Assmt 5

Work Sample

PCK Assmt 6

Profess.

1a. Concepts, principles, theories, laws, and

interrelationships

1b. Supporting content

1c. State/Nat’l Standards

a. Variety of inquiry

2b. Active inquiry lessons

2c. Formative and summative assessment

2d. Instruction and assessment of naïve

/preconceptions

3a. Variety teaching strategies

3b. Inquiry, laboratories, field experiences,

NOS

3c. Assessment

3d. Safety

4a. Chemicals

4b. Procedures

4c. Living organisms

5a. Collect, organize, analyze, reflect on pre-

formative and summative evidence

5b. Distinguish science from other ways of

knowing

5c. Developmentally appropriate inquiries

6 a. Content Prof. Dev.

6b. Education Prof. Dev.

2003 NSTA Standard/Element

Assessment 2012 NSTA Standard/Element

1a Licensure Test 1a-c. Content Knowledge

1a Content Exams or Science Content GPA and Content Analysis Form

2a-c. Content Knowledge

1a-c, 2a-c, 3a-b, 4a-b, 6a-b, 7a-b, 8a-c

Unit Plan 3a-d. Learning Environments

9a-d Teaching Observation Instrument 4a-c. Safety

1a, 2b, 3b, 4b Teacher Work Sample or Portfolio 5a-c. Impact on Student Learning

9a-d Safety exams, papers, and/or projects

1d-e Science Research (subsumed in CAF)

2a-c, 3a-b, 4a-b Assignments, tests, and/or papers.

6a-b. Professional Knowledge and Skills

What do Programs have to do to adjust to 2012 Standards? Keep the same Assessments (1-5)

Alter the criteria and cell descriptors in the instruments to align with the new wording of the 2012 Standards.

Review the Rubrics online at NSTA.org/preservice for guidelines about how to interpret evidence.

New SASB guidelines state: Programs must meet Standards with a “preponderance of

evidence”. (subjective interpretation of reviewers)

Not all elements of a standard need to be met in order to pass the standard.