DIALANG Plus
INTUIT Closing conference
30 January 2008
Kevin HainesHans VeenkampLanguage Centre, University of Groningen
Putting the Plus into
The beginning...
- DIALANG is the first major language assessment system to be based on the Council of Europe's "Common European Framework of reference", which has become established throughout Europe as the most widely recognized frame of reference in the field of language learning.
www.dialang.org
- Curriculum: “no immediate connection of piloting with local pedagogy/syllabus” (Alderson 2005: 60).
- Beta version: “accept its limitations in order to appreciate its beauty…?” (KH, Utrecht, January 2006)
Putting the Plus into
Initial evaluation: recommendations
Pre-test training or acclimatization is neededFollow-up is needed to make advice “real” and relevant to contextPractical exercises are needed to deal with the deficiencies defined in the testThe test should take place within the context of a course rather than preceding it – a systematic lead from DIALANG into the tasks on the courseThe link made to the Common European Framework should be consolidated
Putting the Plus into
The first pilot: June 2006
– Group: 18 x 2nd Year Law students
– Purpose: screening for 3rd Year
– Materials: DIALANG + ‘Finland’* materials: scales and self-assessment material
– Exercise: DIALANG Reading and Writing and follow-up group discussion
*Thanks to Ari Huhta of Jyvaskyla University, Finland
Putting the Plus into
The first pilot: Conclusions:
DIALANG technically sound and user-friendly; occasional in-built errors are a minor irritation rather than a major problem
Feedback generally good; students liked reviewing questions; some students would like more detailed explanation of mistakes
Students liked hints and tips for moving to the next level of CEFR
Some students: awareness: confirms information already known
Some students: encouraging advice: practice more, practice often
Follow-up as catalyst for wider discussion of learning and evaluative discussion of course content
Putting the Plus into
Embedding DIALANG in Blackboard
The University of Groningen Nestor matrix
– Make effective link between the interfaces through test instructions in Blackboard
– Create preparation and follow-up procedures
– Create instructions for students AND teachers
– Make link to the test through the Course Organizer matrix
Putting the Plus into
The second pilot: The Set-up
350 students
14 teachers
Non-teaching weeks
Supervision
Putting the Plus into
Having now supervised my second DIALANG Plus test
group, I must say I'm rather pleased with the way
things are going. It's also clear, however, that, human
nature being what it is, students tend to rely on the
supervisor telling them what to do rather than just
following the written instructions laid down in
DIALANG Plus. Still, as a step-by-step guide to working
with DIALANG, it's very useful.
Weblog Hans Veenkamp, 5th October 2007
Putting the Plus into
We discover that students find it quite complicated to
follow the instructions until they get the idea that
they need to use the Blackboard screen as a backdrop
behind the DIALANG screen. Partly the problems lie in
the connections between our information and partly in
the human tendency not to read instructions.”
Weblog Kevin Haines, 2nd October 2007
Putting the Plus into
It was easy to refer back to the DIALANG Plus pages during the DIALANG test.
Results question 8
0
20
40
60
80
100
120
140
Frequency
strongly disagree
slightly disagree
slightly agree
strongly agree
Putting the Plus into
Personally, I think the DIALANG level indication may very well
be right, but despite this, there are some parts on which I
have to disagree with the DIALANG score sheet. One of these
things is that if I were to believe DIALANG, my punctuation
would be consistent. Unfortunately, my punctuation is not
flawless, as I see it. Another point which differs from the
DIALANG outcome is the fact that I hardly ever use a
dictionary but instead try to find the meaning of a word by
looking at its context. Nice as a C1 score might be, there is
still room for improvement. To reach the next level I think I
need to expand my vocabulary, work on my punctuation and
improve my pronunciation.
Putting the Plus into
I was able to keep a good record of the feedback given during the DIALANG Test in order to write my reflective report later.
Results question 9
0
20
40
60
80
100
120
Frequency
strongly disagree
slightly disagree
slightly agree
strongly agree
Putting the Plus into
Recommendations
– Structure DIALANG into the wider learning context to encourage meaningful interpretations:
• Preparation » Test » Follow-up
– Make full use of the formative function of DIALANG to turn feedback into learning goals:
• Encourage students to make their own judgments about the meaning of the DIALANG feedback and advice
• Encourage teachers to engage with students to discuss the relevance of DIALANG feedback in context
– Use DIALANG to increase awareness of the CEFR amongst students and teachers, providing a catalyst for the discussion of language performance
Putting the Plus into
The future
– extend use to a variety of settings, such as• other large-scale courses• individual formative assessments• indicative screening procedures
– make use of an institutional version• will increase test security• will improve reporting• will promote autonomous use
– fine-tune through continual evaluation so that DIALANG Plus remains fit for purpose
Top Related