How Do We Know What We Know?

EDU 8603

Day 1

Purposes of Course

1. Consumers of research

2. Creators of research (action research)

“Curious Skeptic”

Key Question for Course

How do we know what we know?How do we know what we know?

???

How Do We Know What We Know

How Do We Know What We Know

How Do We Know What We KnowHospital Mortality Rates for 2 Hospitals

Mortality Rates for 2 Hospitals by Condition on Arrival

Sources of Knowledge

• Tradition/ tenacity

• Authority

• A priori or intuition

• Direct experiences

• ResearchResearch Validity

Reliability

According to NEA

Role of Educator: “The educator strives to help each student realize his or her potential.”

How do you know how to go about doing this?

Must know how to critically examine knowledge claims.

Joel Best, Damned Lies and Statistics (2001), “Most of the time, most people simply accept statistics without question” (p. 4).

Curious Skeptic:

1 eyebrow raised in interest

1 eyebrow squinted in skepticism

A Nation at Risk

“Our Nation is at risk. Our once unchallenged preeminence in commerce, industry, science, and technological innovation is being overtaken by competitors throughout the world. This report is concerned with only one of the many causes and dimensions of the problem, but it is the one that undergirds American prosperity, security, and civility. We report to the American people that while we can take justifiable pride in what our schools and colleges have historically accomplished and contributed to the United States and the well-being of its people, the educational foundations of our society are presently being eroded by a rising tide of mediocrity that threatens our very future as a Nation and a people. What was unimaginable a generation ago has begun to occur--others are matching and surpassing our educational attainments.”

A Nation at Risk

Indicators of the Risk• Average achievement of high school students on

most standardized tests is now lower than 26 years ago when Sputnik was launched.

• The College Board's Scholastic Aptitude Tests (SAT) demonstrate a virtually unbroken decline from 1963 to 1980. Average verbal scores fell over 50 points and average mathematics scores dropped nearly 40 points.

A Nation at Risk

Indicators of the Risk

• College Board achievement tests also reveal consistent declines in recent years in such subjects as physics and English.

• Average tested achievement of students graduating from college is also lower.

Average SAT by year

Average SAT subset scores

Who Takes the SAT?

Verbal Scores

Percentage of students taking SAT by class rank

Average SAT Scores by High School Rank

Average SAT Scores by High School Rank

Simpson’s Paradox

The aggregate group shows one trend or pattern, but the subgroups show a different trend or pattern, usually the reverse.

Summary

• Numbers can be used to support different argument.

• Knowledge claims cannot just be accepted at face value.

• As a curious skeptic, you must consider how to make the most informed decision.

Are we here?...

…or here?

A Tale of 2 Paradigms:Qualitative/Quantitative

Research Problems, Hypotheses, and Intro to Action Research

Why do Research?

Rather than just guessing,

the scientific process gives us a way to systematically answer questions which gives credibility to the results.

Why do Research?

• Develop knowledge!– Describe phenomena – Examine empirical relationships between or

among phenomena– Test whether such relationships are causal in

nature

(qualitative or descriptive)

(correlational)

(causal)

A Secret about Science

• We never really know anything!

• Therefore, we have THEORIES

Theories

• Get us closer to the truth

• Set of propositions that explain the relationships among phenomena

• Theory and practice tightly coupled

What are some theories you have heard related to education?

• Examples of learning theories

Different Kinds of Theories

• A priori theory - interrelated set of concepts that is used to explain a body of data and to make predictions about the results of future experiments.

• Grounded theory - inductively derived from the study of the phenomenon it represents.

Educational Research

• Lack of a single, appropriate methodological approach to studying education

• Two paradigms– Quantitative– Qualitative

How to Choose?

A tale of two parties…

Basic Premises

• Three assumptions that differentiate qualitative and quantitative studies

– Epistemology (Qualitative -or- Quantitative)

1. ___________ researchers believe there are multiple realities represented by the participants’ perspectives.

2. _____________ researchers believe a single, objective reality exists.

Basic Premises

• Three assumptions that differentiate qualitative and quantitative studies

– Context (Qualitative -or- Quantitative)

1. __________ researchers do not believe context is an important factor.

2. __________ researchers believe context is critical to understanding the phenomena being studied.

Basic Premises

• Three assumptions that differentiate qualitative and quantitative studies

– Researcher Bias (Qualitative -or- Quantitative)

1. __________ researchers believe researcher bias is controlled through the control of internal validity threats.

2. __________ researchers believe the researcher’s biases and perspectives must be understood to interpret the results.

Parts to a Research Study

• Introduction– Brief background on topic– Why important to study– Research question or Research problem;

Hypothesis if quantitative

• Literature Review– Background on topic– Patterns and gaps in research– Justifies why current research is necessary

Parts to a Research Study

• Methods– Participants/Subjects– Type of data collected (interviews,

observations, surveys, questionnaires, etc.)– Analysis

• Results– What the analysis shows; factual

Parts to a Research Study

• Discussion/ Conclusion– Interpretation of results– Relationship to extant literature– So what?

Research Problems/Questions

• One or more sentences indicating the goal, purpose, or overall direction of the study

• General characteristics– Implies the possibility of empirical investigation– Identifies a need for the research– Provides focus– Provides a concise overview of the research

Research Problems/Questions

• This study examines the differences between males’ and females’ attitudes toward mandated high school drug testing programs.

• What are the differences between males’ and females’ attitudes toward mandated high school drug testing programs?

Research Problems

• Which of the following are non-researchable problems? Why?

• Is democracy a good form of government?• Should value clarification be taught in public

schools?• Can crime be prevented?• Should physical education classes be dropped

from the high school curriculum?– Non-researchable problems include explanations

of how to do something, vague propositions, and value-based concerns

Your Turn…

• What is something you have been wondering about related to counseling or education?

• Write this in the form of a research question or problem.

• Share with your neighbor.• Which paradigm is most suited for this

question and why?

Research Problems

• Quantitative problems– Specific– Closed– Static– Outcome oriented– Use of specific

variables

• Qualitative problems– General– Open– Evolving– Process oriented

Research Problems

• Sources of research problems– Personal interests and experiences (professional

journal)– Deductions from theory– Replication of studies– Clarification of contradictory results– Further research suggested by completed study– Unique phenomenon

Hypotheses

• Tentative statements of the expected relationships between two or more variables

• Hypotheses with variables in quantitative studies only – Students will show significantly improved

levels of motivation toward reading after participating in a 2 month Reading/Writing Workshop.

Your Turn

• Take the topic you wrote about earlier in class and write a hypothesis about a relationship between 2 variables.

Hypotheses

• Criteria for evaluating– Stated in declarative form– Consistent with known facts, prior research, or

theory– Logical extension of the research problem– States an expected relationship between two or

more variables– Can be tested– Is clear and concise

Null Hypothesis

• What is the point of having a “null” hypothesis?

Action Research Purpose

• “A systematic process of solving educational problems and making improvements” (Tomal, 2003, p. 5)

• Context specific

• Limited to participants/ subjects in your setting (classroom or school)

• Can be qualitative or quantitative or mixed methods

Action Research Questions

• Reflecting in-action, on-action, for-action

• Process of funneling– “Why don’t students like to read?”– “Does a Reading/Writing workshop affect

students’ motivation to read in language arts class?”

Action Research Questions

• Don’t try to solve the world!• What is interesting/ troubling/ challenging/

compelling to you??• Focus question as reading literature• Should be action- or outcome-oriented

– How will you change based on getting closer to the truth about this question?

• KISS

Thinking Ahead

• Continue reflecting on research problem/ question.

• Consider: How would you go about studying it?

• Next week… Meet in Griffin Room of Falvey Library.

• Next week’s readings…Literature Reviews!