The Nature of Argument Among Freshmen College Students,

Science Teachers, and Practicing Scientists

Issam Abi-El-MonaRowan University

Department of Teacher Education

Fouad Abd-El-KhalickUniversity of Illinois at Urbana-ChampaignDepartment of Curriculum and Instruction

AIMHE Annual MeetingSeptember 24-26, 2008

Issues; Prior Research in Argumentation:

Rarely researchers explicate what they mean by, or how they assess the validity or “goodness” of arguments

Rarely are prior perceptions of nature of an argument from participants considered

Analysis of student arguments using analytical frameworks

Formalized and generic in nature (e.g.Toulmin) Not necessarily derived from studying scientific

practice

Purpose of study

This study aimed to: Elucidate college freshmen science students,

secondary science teachers, and practicing scientists’ perceptions of argument

Characterize the criteria deployed by participants to assess the “validity” or “goodness” of argument

Examine how such perceptions and criteria compare to those of (or consistent with) Toulmin’s (1958) framework

Research questions

What structural elements are evident in arguments about a socio-scientific issue generated by freshmen science students, secondary science teachers, and practicing scientists? How do these elements compare and contrast?

What are freshmen science students, secondary science teachers, and practicing scientists’ perceptions of argument? How do these perceptions compare and contrast?

Research questions

What criteria do freshmen science students, secondary science teachers, and practicing scientists draw upon when judging the “validity” of arguments? Are these criteria consistent among and/or different across the three groups?

To what extent do participants’ perceptions of arguments draw upon elements that are characteristic of those of Toulmin?

Participants

Three groups of participants (N = 30; 50% female) in a large, Midwestern University and neighboring communities: 10 Freshmen college students, 10 secondary science teachers, and 10 practicing scientists

Method

Exploratory and qualitative in nature In-depth, semi-structured interviews Data collection occurred in two phases

Arguments constructed based on issue (global warming) led to construction of maps (following Horn, 2003)

Participants in each group assessed and provided feedback on Phase I arguments

Interviewees identified features of a good argument (assessed “goodness”,“validity” and justified assessments)

Data analysis: Phase I

Transcripts generated argument maps (via a team of three graduate students)

Maps analyzed using Toulminian and non-Toulminian frameworks

Analysis derived participants’ perceptions of the nature of argument. Later compared and contrasted both within and across the three groups

Data analysis: Phases II & III

Phase II transcripts analyzed to Characterize participants’ perceptions of the nature

of arguments Derive criteria deployed by participants to assess

the “validity” or “goodness” of arguments Criteria were compared and contrasted both within

and across the three groups Phase III: Comparing criteria derived from Phase

II with those derived from Toulmin’s framework

Results: Non-Toulminian analysis

Elements noted % Maps of

S T SciClaims-Prior Content Knowledge 30 60 90

Claims-Conclusive 50 50 70

Claims-Counterclaims (TOC) 30 60 70

Facts-Examples 20 20 40

Results: Toulminian analysis

0

10

20

30

40

50

60

70

80

90

C D W R B Q

Toulmin Element

To

tal

Qu

anti

ty P

rese

nt

S

T

Sc

Common perceptions of argument

Data used shows clear support to claims made Explains methods used to collect data

presented Demonstrates use of various types of

quantitative (scientific) data Demonstrates knowledge of scientific facts Is unbiased, clear, comprehensive Is both credible and verifiable

Differing perceptions of argument

Students: A product; persuasive power Students and Teachers: Visual representation;

does not impose claims; avoids bias; addresses ethical issues; is relevant

Teachers: Justification based on ethical issues Teachers and Scientists: A process (~scientific

method); holistic view; reproducible; contains elemental structures (hypotheses-conclusions)

Scientists: logical coherence

Major Criteria for Assessment of Argument

0

2040

60

80100

120

Common Features of Criteria

Per

cen

tag

es S

T

Sc

Minor Criteria for Assessment of Argument

05

1015202530354045

Thi

nkin

g

Com

plet

enes

s

Ana

lysi

s/In

terp

reta

ti

Per

suas

ive

Hol

sitic

Con

clus

ive

Goo

d po

ints

Dig

ress

ion

Goo

d R

easo

ning

Just

ifica

tion

Per

sona

l Lim

its

Use

of

lang

uage

Incl

inat

ion

bias

Dat

a ex

agge

ratio

n

Em

otio

nally

driv

en

Kno

wle

dge

quan

tity

Common (Minor) Features of Criteria

Per

cen

tag

es S

T

Sc

Peer/Others Perceptions of Argument

0

10

20

30

40

50

60

70

Argument PartialArgument

No Argument BestArgument

Overall Judgement

Pe

rce

nta

ge

S

T

Sc

Peer/other assessment of arguments

Contrary to prior research assumptions, teachers were perceived to have the most complete or better arguments by all participant groups

Elements used in assessment go beyond the field-invariant elements emphasized by Toulmin: Use of other (minor) criteria, which correspond to field-dependent elements specific to the context of science

Peer/other assessment of arguments

Possible explanation: Teachers could be more skilled in forms of persuasive, holistic, and/or comprehensive discourse that are characteristic of the pedagogical practices of science teaching

Participants’ perceptions vs. Toulmin

Correspondence with Toulmin in two major respects Major assessment criteria (except “comprehen-

sibility”) map well onto Toulmin’s basic elements

Minor criteria deployed by smaller groups of participants when assessing arguments relate to Toulmin’s dimension of field-dependent elements albeit more elaborately

Discussion

Product vs. Process Bias and ethical issues vs. Logical coherence Data as most essential element in argument

structure Use of elements-counterclaims Common elements with Toulmin were claims but

unlike Toulmin data, and NOT warrants determine the “validity” of an argument

Conclusions

Need to further understand perceptions of the specific nature of elements that compose an argument (e.g., what is considered to be credible data and what is not)

Teaching practice needs to emphasize instruction on models that help identify how to develop structural elements (e.g., using counter claims) within inquiry based contexts

Conclusions

Research efforts need to focus on developing context dependent criteria for analyzing and identifying arguments based on common perceptions of argument held by learners

Phase II interviews

Examined Argument By

Examined Argument By

Examined Argument By

Scientists

Teachers Students

Map Geography Across Groups

Maps S% T% Sc%

Number of cycles 40 50 60

Web design (total) 50 50 70

Linear design (total) 20 30 40

Results: Toulminian analysis

Elements S(f) T(f) Sc(f)

Claims 68 66 83

Data 25 37 46

Warrants 5 12 9

Backings 0 0 2

Rebuttals 0 0 1

Qualifiers 2 2 1

Maps showing Toulmin based Complete arguments (n=10)

4 5 6

Sample scientist map showing Toulmin excerptSample scientist map showing Toulmin excerpt

Sample student map showing Toulmin excerpt

Common assessment criteria of argument as designated by partcipants

Major Minor

Data (mainly as support) Way of thinking

Clarity of position Completeness of data

Comprehensibility Analysis of data

Logical coherence Persuasive power

Holistic views

Conclusive

Good points made