So how well does this stuff work? Extensive research validates use of visual tools…
* Reading comprehension* Vocabulary acquisition* Writing fluency & ideation* Content-area learning
High-achievingTypical-achievingLow-achievingLearning Disabilities
TM
differentiated visual tools
Large N quantitative studies (true- & quasi-experimental designs) Qualitative studies Program Evaluations
Typical measures include…General performance on high-stakes testsPerformance in specific skills (writing) and vocabularyDepth / Breadth / Accuracy of new content knowledgeSocial validity (teacher & student satisfaction)
Fidelity & factors that affect it
For example….
TM
differentiated visual tools
Teaches same 2nd mini-unit using traditional guided note-taking / discussion instruction
Teaches 1st mini-unit using traditional guided note-taking / discussion instruction
Teacher A
Changes in students’ knowledge about the mini-unit topic are measured at end of each mini-unit
Teacher B
32 Typical Achieving 32 Low Achieving 16 Students w/LD
WEEK 1 WEEK 2
32 High Achieving
Teaches 2nd mini-unit using
Teaches same 1st mini-unit using
TM
differentiated visual tools
American History
HA
First, we measured how much new knowledge of history High Achieving students typically gain when teachers use traditional content instruction methods.
This allowed us to establish the “high water” line.
Text-based, guided note-taking / class discussion
TM
TM
differentiated visual tools
American History
HAHA
TA
HA
Then, we measured how much new knowledge of history Typical Achieving students usually gain when teachers use traditional content instruction methods.
This allowed us to establish the “typical amount” line.
21% more Gap between High & Typical Achievers
High Achieving students tend to gain 21% more knowledge than do Typical Achieving students from the same lesson.
TM
differentiated visual tools
American History
HAHA
TA
HA
Then, we measured how much new knowledge of history Low Achieving students typically gain.
LA
Typical-achievers typically gain 29% more knowledge from a traditional lesson than do Low Achievers.
29% more Gap between Typical & Low Achievers
TM
differentiated visual tools
American History
HAHA
TA
HA
Then, we measured how much new knowledge of history students with LD typically gain from the same ”traditional” lesson.
LA
Typical-achievers typically gain 27% more knowledge from a traditional lesson than do students with LD.
27% more Gap between Typical & Students with LD
LD
TM
differentiated visual tools
American History
HAHA
TA
HA
LA
So now we know what to expect in terms of differences in gains in knowledge about a topic among different types of students when business-as-usual teaching techniques are used on the same history lessons.
LD
TM
differentiated visual tools
American History
HAHA
TA
HA 4 % lessWhen DVTs is used, Typical Achievers increase their learning to within 4% of what High Achievers learned when traditional instruction is used.
TM
differentiated visual tools
American History
HAHA
TA
HA
LA
4 % less
When DVTs is used, Low Achievers increase their learning to within 9% of what Typical Achievers learned when traditional instruction is used.
9 % less
TM
differentiated visual tools
American History
HAHA
TA
HA
LAWhen DVTs is used, Students with LD increase their learning to within 11% of what Typical Achievers learned when traditional instruction is used.
11 % less
LD
TM
differentiated visual tools
American History
HAHA
TA
HA
LA
At first glance, it seems like DVTs is a powerful tool for “reducing the achievement gap.”
11 % less
LD
9 % less
4 % less
The reality is that ALL students greatly enhanced their knowledge when teachers used DVTs
TM
differentiated visual tools
American History
HAHA
TA
HA
LA LD
So the effects of DVTs on the “gap” is to “raise the bar…”
All students are significantly increasing their knowledge of history…
But the gaps remain!
…and even get wider!
TM
differentiated visual tools
American History
This study compared the relative impact of….
Generic Graphic Organizers
Text Resources
“Business as Usual”
VS.
…on depth, breadth, and accuracy of new history knowledge
96 11th grade students…
32 high-achieving, 32 typical achieving, 16 low-achieving, & 16 low-achieving w/ LD
VS.
Essential Understandings
VS.
Generative Idea
TM
differentiated visual tools
depth / breadth / accuracy of knowledge
CLASS 1
PHASE 1 PHASE 2 PHASE 3 PHASE 4
Teacher employed a different strategy during each phase of instruction.
TEXT RESOURCES
GENERIC GRAPHIC ORGANIZERS
GENERATIVE IDEA ESSENTIAL UNDERSTANDINGS
TM
differentiated visual tools
depth / breadth / accuracy of knowledge
Pre & Post measures of students’
Breadth of knowledge
Depth of knowledge
Accuracy of knowledge
CLASS 1
PHASE 1 PHASE 2 PHASE 3 PHASE 4
Learning measured for each phase
TEXT RESOURCES
GENERIC GRAPHIC ORGANIZERS
GENERATIVE IDEA ESSENTIAL UNDERSTANDINGS
TM
differentiated visual tools
depth / breadth / accuracy of knowledge
CLASS 1
PHASE 1 PHASE 2 PHASE 3 PHASE 4
2nd teacher used same strategies, but in a different order
CLASS 2
TEXT RESOURCES
GENERIC GRAPHIC ORGANIZERS
GENERATIVE IDEA Visual Tools
ESSENTIAL UNDERSTANDINGS
TEXT RESOURCES
GENERIC GRAPHIC ORGANIZERS
TM
GENERATIVE IDEAESSENTIAL UNDERSTANDINGS
TM
differentiated visual tools
depth / breadth / accuracy of knowledge
CLASS 1
PHASE 1 PHASE 2 PHASE 3 PHASE 4
CLASS 2
CLASS 3
TEXT RESOURCES
GENERIC GRAPHIC ORGANIZERS
GENERATIVE IDEA Visual Tools
ESSENTIAL UNDERSTANDINGS
Visual Tools
ESSENTIAL UNDERSTANDINGS
Visual Tools
TEXT RESOURCES
GENERIC GRAPHIC ORGANIZERS
GENERATIVE IDEA Visual Tools
CLASS 4
GENERATIVE IDEA Visual Tools
GENERIC GRAPHIC ORGANIZERS
TEXT RESOURCES
ESSENTIAL UNDERSTANDINGS
Visual Tools
TEXT RESOURCES
GENERIC GRAPHIC ORGANIZERS
GENERATIVE IDEA Visual Tools
ESSENTIAL UNDERSTANDINGS
Visual Tools
3rd & 4th teachers used same strategies, also in different orders
TM
differentiated visual tools
depth / breadth / accuracy of knowledge
Analysis of pre- and post-mini-unit student generated concept maps relative to pre-established criterion maps provided 3 scores (Ruiz-Primo, Shavelson & Shultz, 1997):
Accuracy of Relational Understanding ScoreBreadth of Relational Understanding ScoreDepth of Relational Understanding Score
Students orally explained links between the terms & asked to elaborate on concepts relating to the proposition identified.
3 types of oral prompts were provided to encourage elaboration: Prompts to summarize important ideas about the topicPrompts to relate or apply ideas Prompts to think about the idea in a new way (what if…)Each proposition scored 0-3, depending on whether the proposition appeared on the criterion map, whether the student’s explanation was accurate, factually correct and complete, and the degree of elaboration provided.
TM
differentiated visual tools
depth / breadth / accuracy of knowledge
ANOVA statistical analysis was used to determine the relative impact of each of the four instructional approaches relative to level of student achievement.
Generic G.O.s
EU
GI
Text Resources
High Achieving
Typical Achieving
Low Achieving
Learning Disabilities
Tukey’s HSD was used for post hoc analysis.
TM
TM
differentiated visual tools
depth / breadth / accuracy of knowledge
Tukey’s HSD was used for post hoc analysis.In other words, if a treatment worked well with one type of student, it worked equally well with the other types (& vice versa).
ANOVA of Accuracy of Knowledge Scores by Level of Student Achievement and Form of Instruction
Generic G.O.s
EU
GI
Text Resources
High Achieving
Typical Achieving
Low Achieving
Learning Disabilities
… no significant differences
TM
TM
differentiated visual tools
depth / breadth / accuracy of knowledge
Tukey’s HSD was used for post hoc analysis.
Generic G.O.s
EU
GI
Text Resources
High Achieving
Typical Achieving
Low Achieving
Learning Disabilities
TM
ANOVA of DEPTH of Relational Understanding Scores by Level of Student Achievement and Form of Instruction
Significant contrasts:
Scores Using Scores UsingEU Visual Tools > GI Visual ToolsEU Visual Tools > Text ResourcesGeneric Gos > Text Resources
EU Visual Tools had the greatest impact on all students depth of relational understanding.
TM
differentiated visual tools
depth / breadth / accuracy of knowledge
Tukey’s HSD was used for post hoc analysis.
Generic G.O.s
EU
GI
Text Resources
High Achieving
Typical Achieving
Low Achieving
Learning Disabilities
TM
ANOVA of BREADTH of Relational Understanding Scores by Level of Student Achievement and Form of Instruction
Significant contrasts:
Scores Using Scores UsingEU Visual Tools > GI Visual ToolsEU Visual Tools > Text ResourcesGeneric Gos > Text Resources
EU Visual Tools had the greatest impact on all students depth of relational understanding.
TM
differentiated visual tools
depth / breadth / accuracy of knowledge
Source Sum of Squares df Mean Square F pLevel of Ach. (A) 7990.415 2 3995.207 159.078 .000Form of Inst. (B) 740.245 3 246.748 9.825 .000A X B Interaction 48.095 6 8.016 .319 .927A X B within error 9041.321 360 25.113
Post Hoc Analysis of Depth of Relational Understanding Scores- Tukey’s HSD
Significant contrasts:
Scores Using Scores UsingEU Visual Tools > GI Visual ToolsEU Visual Tools > Text ResourcesGeneric GOs > Text Resources
ANOVA of Depth of Relational Understanding Scores by Level of Student Achievement and Form of Instruction
Significant differences were found between the 4 techniques relative to students with and without LD. For students with LD, the EU & Generic Graphic Organizers had the greatest impact on their depth of relational understanding.
Use of traditional-text-based instruction was the least effective.
Depth, breadth, & accuracy of new 11th grade history knowledge
Source Sum of Squares df Mean Square F pLevel of Ach. (A) 1481.321 2 740.6607 126.603 .000Form of Inst. (B) 227.494 3 75.831 12.962 .000A X B Interaction 7.553 6 1.259 .215 .927A X B within error 9041.321 360 5.850
Post Hoc Analysis of Depth of Relational Understanding Scores- Tukey’s HSD
Significant contrasts:
Scores Using Scores UsingEU Visual Tools > GI Visual ToolsEU Visual Tools > Text ResourcesGeneric GOs > GI Visual ToolsGeneric GOs > Text Resources
ANOVA of Breadth of Relational Understanding Scores by Level of Student Achievement and Form of Instruction
Significant differences were found between the 4 techniques relative to students with and without LD. EU Visual Tools had the greatest impact on breadth of relational understanding. Use of GI Visual Tools & traditional-text-based instruction were the least effective.
Depth, breadth, & accuracy of new 11th grade history knowledge
* Essential Understanding Visual Tools* Generative Idea Visual Tools* Traditional Guided Note-taking
Qualitative data, via semi-structured interviews of teachers and students were collected and analyzed.
Teachers taught history units employing instruction featuring each of the four instructional approaches.
* Generic Graphic Organizers
Qualitative analysis of history teachers’ & students perceptions of different types of visual tools
Sample comments:Teacher: “I could use these (Generic GOs) every day. They really help organize the material. When the students used these to take notes, it really helped them see what was important and how things relate”
Student: “The (Generic GOs) point to the main topics instead of going into long, long notes. It is better than pages of notes.”
Generic Graphic Organizers with embedded prompts about the information structure (hierarchic, compare/contrast, cause/effect, and/or sequence) were perceived as useful by teachers because they helped differentiate the curriculum and organize material.
Emergent themes…
Qualitative analysis of history teachers’ & students perceptions of different types of visual tools
Sample comments:Teacher: “These (EU Visual Tools) make the information real and personal to the student. It reminds them that history is interconnected and can even apply to their own lives.”
Student: ““These (EU Visual Tools) make it so that you don’t forget what it’s all about. You look at the questions, then the whole topic comes back to you and you say ‘oh yea, I remember that from our notes’.”
Emergent themes…
Qualitative analysis of history teachers’ & students perceptions of different types of visual tools
Essential Understanding Both teachers and students perceived that the EU Visual Tools with embedded prompts related to topic-specific essential understandings and prompts to engage in specific information processing skills help make the concepts addressed during the history instruction applicable for the students.
Generative IdeaTeachers and students viewed the Generative Idea Visual Tools somewhat differently.
Teachers perceived them as complex and perceived that their students did not understand the “big ideas”.
Sample comment:Teacher: “Maybe they would be good for 12th grade second semester students; but my students did not understand these. They did not get the big picture that was trying to be expressed.”
Qualitative analysis of history teachers’ & students perceptions of different types of visual tools
Sample comments:Teacher: “The concepts in these (GI Visual Tools) were not difficult, they were just new. We’ve never taught anything like this.”
Teacher: “I personally don’t have time to teach concepts not on the graduation exam.”
Teacher: “They (the students) needed more experience working with these (GI Visual Tools) and these concepts.”
Qualitative analysis of history teachers’ & students perceptions of different types of visual tools
Generative Idea Teachers’ value of the GI Visual Tools seemed to be impacted by:
• The novelty of teaching generative ideas;
• Perceived misalignment between generative ideas and what they perceived to be objectives in the state course of study.
Generative Idea Students, however, valued the GI Visual Tools and appreciated the direct instruction in the generative ideas related to a topic.
Sample comment:Student: “These (GI Smart-sheets) tell you what is important to understand. It spells it right out.”
TM
Qualitative analysis of history teachers’ & students perceptions of different types of visual tools
97%
POST
81%
PRE
11TH grade typical achievers
71%
PRE
11th grade students w/LD
17 pt gain 26 pt gain
98%
POST
TM
differentiated visual tools
11th grade history vocabulary
Typical 8th graderN=20
117 words
CONTROL group of students w/LD
N = 28
EXPERIMENTAL group of students w/LD
N = 28
FLUENCY
Pretest Post-test Pretest Post-testTypical 8th grader produces an average of 117 words on high-stakes essays.
8th students with LD typically produce about 27 words on the same task.
Experimental Group Students w/LD
29 words
26 words
Control Group Students w/LD
TM
differentiated visual tools
Writing fluency: 8th grade students with LD
Typical 8th graderN=20
117 words
CONTROL group of students w/LD
EXPERIMENTAL group of students w/LD
FLUENCY
Pretest Post-test Pretest Post-test
29 words
26 words
24 words
126words
+97 words more than pretest
+9 words more than typical 8th grader
Business-as-usual DVTs
TM
differentiated visual tools
Writing fluency: 8th grade students with LD
9 Schools Extremely low performance
20.6%before DVTs
8 Schools Low performance
38.83%before DVTs
7 Schools Moderate
performance
58.39%before DVTs
2 Schools Good
performance
73.02%before DVTs
% of students meeting or exceeding standards BEFORE schools started implementing DVTs
TM
differentiated visual tools
Impact on AYP Writing Assessment
DVTs
49.24%
DVTs
62.41%
DVTs
74.81%
DVTs
82.68%% of students meeting or exceeding standards AFTER schools started implementing DVTs
20.6%before DVTs
38.83%before DVTs
58.39%before DVTs
73.02%before DVTs
+ 23.58 pts. +16.42 pts.+ 28.64 pts. +9.66 pts.
9 SchoolsExtremely Low Performing
8 Schools Low Performing
7 Schools Moderate Performing
2 SchoolsHigh Performing
TM
differentiated visual tools
Impact on AYP Writing Assessment
So how well do these work?
Results consistently show that DVTs …FAR better than “business as usual” (control groups)
Significantly better than generic graphic organizers
Teachers and students HIGHLY value them
TM
differentiated visual tools
Impact on AYP Writing Assessment
DVTs implementation Year 1
51.4336.1
22.0
28.08
10.0 62.24
81.0
71.0
% students meeting or exceeding standards
+ 23
+ 52
+30School #1
School #2
Year 2 Year 3 Year 4Business-as-usual
+ 9
TM
differentiated visual tools
Semi-rural 5th grade Alabama Writing Assessment
Non-DVTs High School 53%
DVTs High School 77%
26 pt gain
2 pt gain
DVTs implementation
51%
51%
Both groups performed at the same levels in Year 1
Year 1 Year 2
% students meeting or exceeding standards
TM
differentiated visual tools
Semi-rural 5th grade Alabama Writing Assessment
Year 1 Year 2 Gains
32.71 57.84 +25.13 Rural 7th GradeSuburb 7th Grade 38 61 +23.00
% students meeting or exceeding standards
Business-as-Usual DVTs
TM
differentiated visual tools
7th grade Alabama Writing Assessment
Top Related