Surviving (and Thriving) in a “Perfect Storm” Environment Ray T. Clifford 22 February 2007.

Surviving (and Thriving) in a “Perfect Storm” Environment

Ray T. Clifford

22 February 2007

The Perfect Storm

• Too little time.

• Limited funding.

• Not enough qualified teachers.

• Rising expectations.

Some Recent Headlines

• Court Creates Demand for Translators

• Home Invasions Show Need for Translators

• On the Table: A Cure for Medicare Language Gap

• Built to be Bilingual: Researchers Pinpoint How the Brain Switches Between Languages

• Untying U.S. Tongues

• FBI Agents Still Lacking Arabic Skills

• U.S. has a Strategic Need for Multilingual Citizens

• Learning Second Language Changes the Brain: Bilingual People have More Grey Matter in Key Region

• Iraq Panel: Put Forward Officials who Speak to Arabs in Arabic

• First Ears, Then Hearts and Minds: Facing Shortage of Arabic Interpreters, Pentagon Seeks a Technological Solution

High Demand, Low Supply

• More than 80 federal agencies employ individuals with skills in more than 100 languages.

• Degrees granted in 2002:– 2, 396 French– 13 Arabic

• Only one certified Arabic teacher in U.S. public schools.

And Yet…

• Our nation suffers from delusions of linguistic adequacy!

Overview

• What do Americans “know” about languages?

• Why aren’t foreign languages required subjects in U.S. schools?

• How can the U.S. respond to changing national language needs?

Limited Experience Leads to Naiveté

• The general public believes:

– Every English word has an exact match in the second language.

– Second language learning is simply a

re-lexification process.

– High levels of language competence can be attained in two years of school classes…or perhaps in a few weeks with a computer program.

Limited Experience Leads to Naiveté

• Even teachers often believe:

– Students’ failure to attain meaningful levels of proficiency is primarily the result of poor teaching.

– There is a single, best teaching method.

Needs + Naiveté = Nonsense

• The Reality of FL Learning– “Time on Task,” not teaching method, is the

best predictor of success in second language learning.

– Of course, how that time is spent does make a difference.

• Nothing clarifies the debate about teaching methods like a definition of goals.

The Major ACTFL Proficiency Levels

ILR Proficiency Level Summary[With language “text type” highlighted in red]

LEVEL FUNCTION/TASKS CONTEXT/TOPICS ACCURACY

All expected of an educated NS [Books]

All subjects Accepted as an educated NS

Tailor language, counsel, motivate, persuade, negotiate [Chapters]

Wide range of professional needs

Extensive, precise, and appropriate

Support opinions, hypothesize, explain, deal with unfamiliar topics

[Multiple pages]

Practical, abstract, special interests

Narrate, describe, give directions [Multiple paragraphs]

Concrete, real-world, factual

Intelligible even if not used to dealing

with non-NS

Errors never interfere with

communication & rarely disturb

Q & A, create with the language [Multiple sentences] Everyday survival

Intelligible with effort or practice

Memorized [Words and Phrases] Random Unintelligible

The ACTFL and ILR Scales Represent Significant, Substantive

Levels of Accomplishment

• Language is the most complex of human behaviors.

• Language learning is an educational process which slowly, but inexorably reveals the limitations of any monolingual view of the world.

Language Teachers are Professional Educators.

• The study of second language acquisition is a professional discipline.– Graduate degrees are offered in the field.– The discipline has professional journals,

organizations, and conferences.– The field conducts an ever-expanding body of

research.

• Being a “native speaker” is not enough.

Bloom’s Taxonomy• Evaluation and persuasion through refined use of

professional, literary, and rhetorical skills.• Synthesis of concepts to produce and comprehend

abstract ideas and hypothetical situations.• Analysis and definition of factual relationships in

paragraph length communications.• Application of skills to create and understand new

communications.• Comprehension and use of words and phrases.• Memorization of facts.

Language and Cognition• Levels 4 & 5, evaluation and persuasion through

refined use of professional rhetorical skills.• Level 3, synthesis of ideas to produce and

comprehend abstract comments and hypothetical situations expressed in essays, chapters, etc.

• Level 2, analysis and definition of relationships expressed in multiple interrelated paragraphs.

• Level 1, application of skills to create and understand sentence length communications.

• Level 0+, comprehension and use of words and phrases.

• Level 0, memorization of facts.

Language and Perceived Intelligence

It is good [for students] to become acquainted with languages, for they may have to go abroad, and should be able to talk to people, and not look like fools. I care not how much intelligence you have, if you cannot exhibit it, you look like an ignoramus.

John Taylor, 1852

A Continuing Problem• 1975: The International Association for the

Evaluation of Educational Achievement published the results of a study titled, The Teaching of French as a Foreign Language in Eight Countries.– In the U.S., the researchers could not find enough 12th

grade students with four years of language study to complete the study as designed.

– The U.S. students ranked last in competence.– The study found that “…the primary factor in the

attainment of proficiency in …any foreign language… is the amount of instructional time provided.”

A Continuing Problem

• 1979: The President’s commission on Foreign Language and International Studies reported,

“Americans’ incompetence in Foreign Languages is nothing short of scandalous, and it is becoming worse.”

• 1983: The Commission on Excellence in Education– Heard testimony that in the U.S., FL instruction

had yet to attain mediocrity.– Published the report A Nation at Risk, which

recommended longer course sequences for foreign language programs.

• 2004: The Modern Language Association survey of foreign language enrollments in higher education reported: – Total foreign language enrollments have

increased slightly.– However, as a percentage of total enrollments,

foreign language enrollments are decreasing.• In 1960 they were 16.1% of total enrollments.

• In 2002 they were only 8.3% of total enrollments.

And Now…• The original 2007 federal budget figures from

the Office of Management and Budget showed the following planned reductions:

Defense – 8.7%

Education – 24.4%

• The forecast: A competitive fiscal environment.

How can language programsbecome competitive?

• Assess the current status of our programs.

• Tell the real story.

• Support our story with test results.

What do tests have to do with Teaching and Learning?

• It helps to begin with the end in mind.

• You can’t teach skills you can’t measure.

(Even if you could, how could you prove you’ve done it?)

• Education is undergoing a paradigm shift from instructional objectives to learner outcomes.

Changes in Accreditation

• There is an unprecedented move to replace process reviews with outcome reviews.

• Accreditation and Student Learning Outcomes: A proposed Point of Departure– Knowledge outcomes– Skills outcomes– Affective outcomes– Abilities (the integration of KSA outcomes)

Peter T. Ewell , Council for Higher Education Accreditation, September 2001

What will be the effect of these accreditation requirements?

• More testing will take place.– Some beneficial– Some detrimental

• These tests will influence learning.– The “Will that be on the test?” phenomenon.– The temptation to “teach the test” instead of

teaching the skills necessary to pass the test.– Every testing decision creates a washback effect.

“Washback” Effects

• Testing has a negative impact when:– Educational goals are reduced to those that are

most easily measured. – Testing procedures do not reflect course goals, for

instance…• Giving multiple choice tests in speaking classes.

• Using grammar tests as a measure of general proficiency.

– The test results don’t provide useful information.

The National Debate onSchool Testing

• One formula for evaluating school performanceSchool score = (((((X23*100)*Y23) + ((X24*100)*Y24) + ((X25*100)*Y25) + ((X26*100)*Y26) + ((X27*100)*Y27) + ((X28*100)*Y28) + ((X29*100)*Y29) + ((X30*100)*Y30) / ((X23 + X24 + X25 + X26 + X27 + X28 +X29 + X30)*100)) + ((((Z23*100)*Y23) + ((Z24*100)*Y24) + ((Z25*100)*Y25) + ((Z26*100)*Y26) + ((Z27*100)*Y27) + ((Z28*100)*Y28) + ((Z29*100)*Y29) + ((Z30*100)*Y30)) /((Z23 + Z24 + Z25 + Z26 + Z27 + Z28 + Z29 + Z30)) / ((Z23 + Z 24 + Z25 + Z26 + Z27 + Z28 + Z29 + Z30)*100))) / 2

The Wall Street Journal, May 21, 2001, page A24

• What would be the washback effect of this evaluation formula?– Perhaps Confusion?– Perhaps Frustration?– Perhaps “teaching the test” in a desperate

attempt to improve results?

Washback Effects of Tests

• Testing has a positive impact when:– Tests reinforce course objectives.

(For students, nothing defines course objectives like a test.)

– Tests can act as change agents for improving teaching and learning.

Language Testing and Motivation

• Appropriate tests can motivate learners to improve their skills.

• Appropriate tests can motivate teachers to refine their teaching to match their students’ needs.

“Some considered projections” from

THE NATIONAL INTEREST AND FOREIGN LANGUAGES

Eight- and ten-year sequences of foreign language study will become common in the public schools.The better colleges and universities will require demonstrated proficiency (not high school “units”) in a foreign language for entrance, and demonstrated proficiency in a second foreign language (often non-Western) for graduation.

WILLIAM RILEY PARKER for

THE U.S. NATIONAL COMMISSION FOR UNESCO, DEPARTMENT OF STATE, 1961.

• What might have been the washback effect if students’ language abilities had been used to evaluate program effectiveness in 1961?– Perhaps clearer statements of objectives?– Perhaps more realistic learner expectations?– Perhaps more accurate course descriptions?– Perhaps qualitative feedback that would have

justified longer course sequences?

After four decades, why are Parker’s projections not yet a reality?

• Lasting improvements are only sustainable if there is a recognition of need for them.

• Norm-referenced grading gave little evidence of real-world ability gaps.

• Accreditation was focused on process rather than on outcomes, and provided no incentive to change.

• Administrators didn’t read the research results showing that no improvements in teaching methods or curricula can compensate for the current lack of “time-on-task” in our educational system.

International Research on Learning French

• John B. Carroll, The Teaching of French in Eight Countries (International Studies in Evaluation V) John Wiley & Sons, New York, 1975.– “… the primary factor in the attainment of

proficiency in French (and presumably, any foreign language) is the amount of instructional time provided.” [Page 276]

International Research on Learning English

• G. Bonnet, et al. The Assessment of Pupils’ skills in English in Eight European Countries: 2002. European Network of Policy makers for the Evaluation of Education Systems, 2004.– Students from those nations where there is more

contact with (and more time spent using) English have higher levels of competence in English.

– In a language-rich environment, time spent using the language is more important than the teaching methods used in the classroom.

Assessing FL Learner Outcomes• FL profession has made much progress in the

area of assessment and standards.• Our ability to test students’ language

proficiency puts us ahead of many other disciplines.

• But proficiency tests are not the only way to measure learner outcomes.

• Which test is “best” will depend on:– The purpose of the test.– Each program’s instructional goals.

Matching Tests with Your Instructional Purposes

• Some typical FL testing purposes.– Assigning course grades in a beginning class.– Placement of students into a sequence of courses.– Certifying a general level of ability for teachers.– Screening of job applicants.

• Some general instructional purposes.– Achievement – knowledge of contributing language

elements found in a specific curriculum.– Performance – ability to communicate in specific

settings.– Proficiency – demonstration of unrehearsed ability.

If Tests Are to bePositive Motivators

• We have to select the right type of test for each testing purpose.

The 3 Major Types of Tests• Achievement = Memorized responses

using the content of a specific textbook or curriculum.

• Performance = Rehearsed ability to communicate in specific, familiar settings.

• Proficiency = Unrehearsed general ability to accomplish communication tasks across a wide range of topics and settings.

Some CommonTesting Purposes

• Assigning grades in a class.

• Placing students into a sequence of courses.

• Selecting an applicant for a job with limited, static language requirements.

• Screening employees for future jobs with broad, general language requirements.

Aligning testing programs with our testing purposes requires

knowing what makes proficiency tests different fromperformance tests

and achievement tests.

The ILR/ACTFL Proficiency Scales are Different, Because They:

1. Represent a geometric progression.

2. Use threshold rating criteria.

3. Do not use compensatory rating procedures.

4. Are a multidimensional measure of ability.

5. Apply Guttmann scaling criteria.

6. Are based on a wide range of “real world” communication needs.

7. Assess unrehearsed language abilities.

1. The ILR/ACTFL scales represent a geometric progression.

• Advancing from one level to the next becomes increasingly difficult as one moves up the scale.

• For instance, moving from 0 to 1 requires less time and effort than moving from 2 to 3.

What are the Major Levels?

How are They Defined?

2. The ILR/ACTFL scales use “threshold” rating criteria.

• Candidates must meet all of the stated criteria for a given level.

• If they meet most, but not all of the criteria, some award a “plus” designation to the next lower level.

• For example, someone who meets the criteria for Level 2 almost all of the time, could be rated 1+.

3. The ILR/ACTFL Ratings Are Not Based On Total, Average, or Other “Compensatory” Scores.

• Ratings may be limited by one or more of the task, context, and accuracy requirements.

• For example, a large vocabulary does not compensate for inaccurate usage.

Implications of Points 2 and 3: Assessment Criteria-

Speaking

4. The ILR/ACTFL scales representa complex set of abilities.

• The ILR/ACTFL scale is a multi-dimensional scale.– The scale is not uni-dimensional like a rope that has been

cut into segments.

– It is multi-dimensional, because each level of the scale represents a different constellation of supporting and enabling skills.

• The ILR/ACTFL scale (and other complex scales) cannot be represented by a one-dimensional statistical model.

Bloom’s Cognitive Taxonomy• Evaluation and persuasion through refined use of

professional, literary, and rhetorical skills.

• Synthesis of concepts to produce and comprehend abstract ideas and hypothetical situations.

• Analysis and comparisons of factual relationships in complex situations.

• Application of skills to apply facts and understand new applications of those facts.

• Comprehension and facts learned.

• Memorization of facts.

Cognition and Language Taxonomy

• Levels 4 & 5, evaluation and persuasion through refined use of professional rhetorical skills.

• Level 3, synthesis of concepts to produce and comprehend abstract ideas and hypothetical situations.

• Level 2, analysis and description of factual relationships in paragraph length communications.

• Level 1, application of skills to create and understand new communications. (Performance)

• Level 0+, comprehension and use of words and phrases. (Achievement / Performance)

• Level 0, memorization of facts. (Achievement)

5. ILR/ACTFL Scale Ratings Apply Guttmann Scaling Criteria.

• To qualify for any given rating, a person must meet the criteria for that level and also satisfy the criteria for all lower ratings.

• For instance, a Level 2 (Advanced) speaker must also be able to satisfy Level 1 (Intermediate) task, topical, and accuracy communication requirements.

6. The ILR/ACTFL ratings are based on a range of “real world” communication needs.• The candidate must demonstrate sustained

ability across a wide range of topics and real world settings.

• For instance, proficiency ratings can not be assigned based on one’s:– knowledge of a textbook.

– performance of a specific job task or set of tasks.

7. The ILR/ACTFL scalesassess unrehearsed abilities.

• Broad elicitation techniques are used to verify sustainable ability versus limited performance.

• For instance,– At level 1 (Intermediate), test takers must create new utterances, not

just recite dialogs.

– At level 2 (Advanced), they must handle new work requirements, not just routine communications.

– At level 3 (Superior), they must hypothesize and defend opinions in subjects beyond their personal interests and areas of specialization.

• “Proficiency” indicates transferable skills – not just rehearsed performance in familiar areas.

A Summary of the Contrasts Between Proficiency And Rehearsed Performance

Unrehearsed Rehearsed

Proficiency Performance

Task: A wide range of abilities Specific Job Skills

Context: Broad, in-depth, variable Focused, restricted

Accuracy: Ascending expectations Situation dependent

So what?

Review of Major Test Types

• Achievement = Memorized responses using the content of a specific textbook or curriculum.

• Performance = Rehearsed ability to communicate in specific, familiar settings.

• Proficiency = Unrehearsed general ability to accomplish communication tasks across a wide range of topics and settings.

A Self Assessment Quiz

• You will be asked about 6 different testing purposes.

• For each of those test purposes, which type of test would you choose?

a. Achievement.

b. Performance.

c. Proficiency.

Which type of test would you choose:Achievement, Performance, or Proficiency?

1. To assess students’ language learning after Chapter 3 of a beginning language course?

2. To place students into a university’s sequence of courses?

3. To test students completing a year-long, intensive language course?

Which type of test would you choose:Achievement, Performance, or Proficiency?1. To assess students’ language learning after Chapter 3 of a beginning

language course?

4. To screen job applicants for a specific job with well-defined, repetitive tasks?

Which test type would you choose:Achievement, Performance, or Proficiency?

5. To select someone to be the NATO spokesperson on a news show with a “hostile” moderator?

Which test type would you choose:Achievement, Performance, or Proficiency?1. To assess students’ language learning after Chapter 3 of a beginning

language course?

5. To select someone to be the NATO spokesperson on a news show with a “hostile” moderator?

6. To document a person’s language ability in that person’s official personnel file?

Achievement, Performance, or Proficiency

Which test type did you choose?

5. To select someone to be the NATO spokesperson on a “hostile” news show?

6. To document a person’s language ability in that person’s official personnel file?

Solving Testing Problems• “The solutions to our problems should be as

simple as possible, but no simpler.”

Albert Einstein

• There is no answer for the overly simple question of “Which test is best?”

• There is an answer to the question, “Which type of test is best for a given purpose?”

Which test is best?

• The test that matches the purpose for which the results will be used.– Use achievement tests for testing mastery of

lessons in a textbook.– Use performance tests for checking rehearsed

abilities within specific contexts.– Use proficiency tests for determining general,

unrehearsed ability in real-world situations.

Testing Second Language Skills

Superior1600+ hours

Advanced800+ hours

Intermediate400+ hours

Novice100+ hrs

Incremental skill development moving from achievement to rehearsed performance andbuilding a foundation for increased proficiency by using purposefully designed curricula to:

1. Learn relevant topical content.2. Master level-appropriate communication tasks.3. Document progress using ability profiles, test results, portfolios, etc.

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Superior1600+ hours

Advanced800+ hours

Novice100+ hrs

ties a

ssessed u

g infr

equent

Why are programs not assessed?

• It would be too much work.

• It is not “my thing.”

• We’d rather not know the results.

Why assess our programs?

• In today’s environment – if you don’t measure it, you can’t get money for it.

• Assessment will help us articulate our objectives.– For ourselves.– For the students.– For program sponsors.

• Formative assessment allows informed decision making.

The Improvement Process

• Set goals.

• Decide how to measure progress toward those goals.

• Collect data.

• Analyze the results.

• Make informed changes.

• Repeat.

It won’t be easy, but if we are willing to work together and

learn from the past…

then William Riley Parker’s vision of second language programs in the U.S. might

yet become a reality!

In the field of second language teaching, the U.S. has a long history

of missed opportunities.

• Only if we are willing to document the status of our programs, can we hope to improve them.

• With limited resources, collaboration will be more productive than competition.

Conclusion

• The past has not been perfect,

• and the future is conditional

• on whether the present is progressive.

Surviving (and Thriving) in a “Perfect Storm” Environment Ray T. Clifford 22 February 2007.

Documents

Transcript of Surviving (and Thriving) in a “Perfect Storm” Environment Ray T. Clifford 22 February 2007.