An Introduction to Research

& WESTOP Data Collection

Lee Fulmer

WESTOP Research and Technology Committee

Agenda

Why do research? What is research? Types of research What does DOE say about research? Exercise WESTOP data collection

Why do Research?

To find out what is effective Accountability Mathematica

What is Research?

Definitions

– Information gathering

– Collecting and analyzing new information in order to increase our understanding

What Research is

Attempt to achieve systematically and with the support of data the answer to a question, the resolution to a problem, or the greater understanding of a phenomenon

Generation of new information and testing of ideas

Characteristics of Formal Research

l Starts with a question or problem

l Requires a clear articulation of a goal

l Follows a specific plan or procedure

l Often divides the principal problem into more manageable subproblems

Characteristics of Research Cont’d

6. Specify assumptions you’re making

7. Collection and interpretation of data

Types of Research

A man from remote village is the first in his village to fly

Friend 1: How far did you go? How high did you fly?

Friend 2: How did you feel moving so fast? What was it like being above the clouds?

Types of Research

Quantitative: answer questions about data that can be measured in order to explain and predict

Qualitative: answer questions about nature of phenomena in order to describe phenomena and understand it from the participant’s point of view

What the Dept. of ED says

No Child Left Behind

Requires that teachers use techniques and methods that have been shown by “rigorous” research to be effective

Randomized control trial

Randomized Control Trial

Definition: people randomly assigned to an intervention group or to a control group, in order to measure the effects of the intervention

Unique feature: Best chance of objectively demonstrating cause and effect

Superior to other designs in measuring intervention’s effect

Potentially Misleading Designs

Pre-post study: Examine whether participants improve or regress during intervention

– Even Start

– Summer Training and Education Program

Potentially Misleading Designs

Comparison Group:Comparison group chosen by other than randomization

Hormone replacement therapy

Well-matched comparison groups can suggest trends, but gold standard is control group

Responses to DOE

American Evaluation Association– http://www.eval.org/doestatement.htm

American Educational Research Association– http://www.eval.org/doeaera.htm

National Education Association– http://www.eval.org/doepage.htm

Uses of WESTOP Data

Illustrate program effectiveness– Within your institution or agency– To potential funders– In grant proposals

Research on program effectiveness

Data Collection

Two stages

– Initially collect data on prior experience– illustrate program effectiveness

– Eventually collect additional data that may be relevant for research

Why Prior Experience?

Aggregate data

Speaks to legislative intent

Why Two Stages?

Prior experience data a modest data set and perhaps the most important

Work out bugs

Gauge participation level

Second stage will require specifying which data to collect

Discussion

Is the collection of regional prior experience data useful to you?

What other types of regional data would you like to see collected?

References

Identifying and Implementing Educational Practices Supported by Rigorous Evidence: A User Friendly Guide. 2003. US. Dept of Education Institute of Education Sciences National Center for Education Evaluation and Regional Assistance. http://www.ed.gov/rschstat/research/pubs/rigorousevid/index.html Downloaded 1/5/05.

References

Practical Research Planning and Design. (1997) Leedy, Paul with contributions by Timothy J. Newby and Peggy A. Ertmer. Prentice Hall, Inc. Sixth Edition.