Evidence-Based Practice: Yes, but with Cautions & Recommendations

Steve Graham

Vanderbilt University

• I will use Writing, my area of expertise to illustrate the 5 points that I plan to make.

POINT 1

• The use of Evidence-Based practices is more trustworthy than the use of practices based on professional (expert) judgment or teacher experience

2 Sources for Effective Practice

• Professional Writers – many draw on their experience to give advice on either how to teach writing or facilitate writing development

• This advice ranges from useful to simplistic to ridiculous

Don’t use words too big for the subject.

C.S. Lewis

When you catch an adjective kill it.

Mark Twain

• Have something to say.

• Say it.

• Stop when you have said it.

• Give it an accurate title.

Rules for Writing a Paragraph

Set type for as long as you can hold your breath without getting blue in the face

Then put in a comma

When you yawn put in a semicolon, and when you sneeze, that’s time for a paragraph.

Sources for Effective Practice

• Experienced Writing Teachers

This can involve the insights they have acquired from teaching or insights acquired from others who study them

LIMITATIONS

Difficult to separate “wheat from shaft”

Often no direct evidence on effectiveness (validity -- testimonials)

Evidence is often selective(generalizability -- possible bias)

Evidence often based on 1 or a few teachers(reliability – can’t predict its effectiveness)

Sources for Effective Practice

• Scientific Studies -- Should be more trustworthy than insight and experience

Collect evidence

Present findings for all participants

Replicability

Strength of Impact

Point 2

• Make the best of the available evidence. The data-base is currently very thin for some areas.

I illustrate this with writing instruction for students with LD – This is based on a recent chapter by myself, Natalie Olinghouse, and Karen Harris

FINDINGS – 4 or more studies

• 1. Teach students with LD strategies for planning, revising, and editing their compositions (strong positive impact).

• 2. Use direct instruction to teach grammar skills to students with LD (moderate positive impact).

FINDINGS – 2 to 4 studies

3. Have students with LD work cooperatively with other struggling writers to plan, draft, revise, and edit their compositions (strong positive impact).

4. Explicitly teach students with LD strategies for producing written summaries of reading material (strong positive impact).

FINDINGS – 2 to 4 studies

5. Make the process writing approach more effective for students with LD by explicitly teaching them strategies for carrying out the processes of planning, revising, and editing (strong positive impact).

6. Set clear and specific goals for what students with LD are to accomplish in their writing (moderate to strong positive impact).

This is IT!!!

We can only make 6 recommendations on writing instruction for students with LD that have some scientific backing

POINT 3 – The What Works Clearinghouse Syndrome

In other words – only certain types of studies will be included in determining what is or is not evidence-based

They have adopted a best evidence synthesis approach

As a result, they are often referred to as:

THE WHAT DOESN’T WORK CLEARINGHOUSE

RECOMMENDATION

• We need to take a careful look at our definitions of evidence-based practice

• Are they too stringent are they too lax?

• More pointedly, we need to systematically study the process and criteria used to identify evidence-based practices.

POINT 4

• Until our data base is more sufficiently developed, we will need to supplement our evidence-based recommendations with recommendations taken from areas other than experimental and single-subject design research

We will need to continue to draw on the practices of effective teachers.

For example, we located 5 qualitative studies that looked at the writing instructional practices of effective teachers and schools. We identified those practices employed by a majority of teachers as promising

Some Examples

• Dedicate time to writing, including writing across the curriculum.

• Involve students in various forms of writing over time.

• Have students plan, draft, revise, edit, and share their work.

• Keep students engaged and on-task by involving them in thoughtful activities (such as planning a composition) versus activities that require little thoughtfulness (such as completing a workbook page that can be finished quickly).

Some More Examples

• Model, explain, and provide guided assistance when teaching.

• Provide just enough support so that students can make progress or carry out writing tasks and processes, but encourage students to act in a self-regulated fashion, doing as much as they can on their own.

• Be enthusiastic about writing and create a positive environment, where students are constantly encouraged to try hard, believe that what they are learning will permit them to write well, and attribute success to effort and what they learned.

• Set high expectations for students, encouraging them to surpass their previous efforts or accomplishments.

POINT 5

Be Cautious About What Is Promised, as there is no guarantee that evidence-based practices travel easily or as expected

RTI Is A Case In Point

Based in part on the premise that the implementation of effective practices (evidence-based) in the regular classroom will reduce the number of students who are misidentified as LD (in other words, eliminate educational casualties)

While this may be the case, this is a huge burden for Evidence-Based practices, as we have some schools where the writing of three quarter or more of the students is not strong enough to meet classroom demands.

In addition, much can go wrong in the implementation of any educational approach

Case in Point 2

The National Reading Panel Recommendations

MAY BE A CASE OF OVER-SELL THAT WILL COME BACK AND LITERALY BITE US IN THE BUTT.

WE MUST BE REASONABLE AND CAUTIOUS IN HOW WE SELL EVIDENCE-BASED PRACTICES TO THE EDUCATIONAL COMMUNITY