Writing ePlatform: Developing an innovative feedback tool for students’ writing to expand...

Writing ePlatform:Developing an innovative feedback tool for students’ writing to expand assessment for, of, as learning.

Outline

(1)Introduction: What is the Writing ePlatform

(2)Theory behind design: Assessment of, for, and as Learning Technology Enhanced Learning

(3) The eLab & eTutor: What are they?

(4) Planning design, and Pilot.

(5)Unintentional Outcome: Learning Analytics & Assessment for Learning.

(6)What’s Next & Conclusion

Writing ePlatform

The Writing ePlatform is a set of tools that assist KS3 students with their writing, allowing them to take a discovery-based approach to accurate and fluent English.

Writing ePlatformTake advantage of online resources to help KS3 students and teachers shift from a machine- or teacher-centered pedagogy to one that puts the KS3 students at the center of the writing process (Milton, 2006, 2011).

Guided by three key principles based on education and educational technology researchers such as Anderson, Chickering and Ehrmann, Siemens, and the TBLT Group (2004, 1996, 2009, and 2012).

Writing ePlatformThe platform should:

(1) assist with constructing the lexico-grammatical and discoursal/rhetorical knowledge of the target language and the skills required to access and apply that language;

Writing ePlatform

(2) encourage reflection and metacognition, where students are encouraged in independent learning and self-confidence; and

Writing ePlatform

(3) develop ‘cognitive apprenticeship’, where coaching and modeling occur, and where scaffolding is provided to support language learning.

Assessment of, for, and as Learning

Using Classroom Assessment for Differentiating Learning

Assessment As Learning (Earl, 2003)

Features of Assessment of, for, and as Learning


Assessment of Learning

What role would the ePlatform play in summative assessment?How will it certify learning and report to students and teachers students’ progress in writing?

Assessment as learning

How can the ePlatform encourage students to become active, engaged, and critical assessors of their writing?

In what ways can the ePlatform encourage independent learning?

Assessment for learning

How will the ePlatform enable/enhance formative assessment?

How can the ePlatform be interactive, providing assistance as part of/during the assessment?

In what ways could the ePlatform assist teachers with collecting data so that they can modify the learning work for their students?

Technology Enhance Learning

PC T

A teacher’s knowledge of technology

A teacher’s knowledge of pedagogy

A teacher’s knowledge of the content to be taught

Content, Pedagogy, and Technology KnowledgeContent, Pedagogy, and Technology Knowledge

P

C

T

P

C

T

Mirsha, P. & Koehler, M. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6).

Technological pedagogical content knowledge

ICT integration and curriculum design

E-learning vision

ICT integration and curriculum design

Pedagogical Content Knowledge

Technological Content Knowledge

Technological Pedagogical Knowledge

Technological Pedagogical Content Knowledge

Assessment Roles and Goals with Writing ePlatform

eLab &

eTutor

eLabThe eLab is a web-based platform where students can submit their writing to receive feedback and suggestions on how to improve their written English

eLab and the writing process

Student writes a draftStudent writes a draft

Student submits draft to eLabStudent submits draft to eLab

eLab analyses student’s texteLab analyses student’s text

eLab identifies problems in writingeLab identifies problems in writing

Student receives feedbackStudent receives feedback

Student rewrites/adjusts draftStudent rewrites/adjusts draft

Student sample

Nowadays, English is more and more important. It is very difficult to learn. How can I speaking English well?

Nowadays, English is more and more important. It is very difficult to learn. How can I speaking English well?

Using the wrong ending is a very common error. Be careful with words ending in –ing.

A common mistake may look like this:

How can I speaking English well?

The correct way is:

How can I speak English well?

eLab and the writing process

eTutor

The eTutor is a web-based tool that offers comprehensive advice on writing accurate and fluent English.

Planning, Design,& Pilot


Compiled rules based on previous research at UST + rules based on research data provided by EDB.

Example: Although + but in a sentence

Although the food was not great but I felt very happy that I was able to cook the meal by myself.

Although the food was not great, I felt very happy that I was able to cook the meal by myself.


Rule:

!{well|such|known}{as|since|although}<{(CC)|(IN)}> *<0-6> (CC)<{but}>


1.Piloted program in 7 schoolsStudent survey

Teacher observation report

2.Analyzed a learner corpus4 Topics

400 papers from each topic


Student Survey (Anonymous)

336 responses

6 schools (1 school could not complete pilot due to time constraints).


Responses to the question ‘What could be done to improve the Writing eLab?’ can be categorized in the following ways:

FALSE POSITIVES

ERROR IDENTIFICATION

INSTANT PROMPTS

INSTRUCTIONS

VOCABULARY

PERFORMANCE

Analysis of Learner Corpus

Although + but in a sentence (start with a capital, end in a full stop)

Hits: 22

Issues: None.

Other error type?: None.

False positives: None.

Correct Identification (samples):

{Although they were so old but }they were still very strong.

{Although it looked like a high-class hotel but }I paid fifthty-five dollars only.

{Although it is quite expensive, but }it is a fun way to learn English as we can know more about the culture of the place at the same time.

{Although we can't said Japan's language, but }we can said english with they.

Analysis of Learner Corpus

Error Rule 1: A sentence / clause has more than 2 verbs

Hits: 23874

Issues: This rule is very problematic – too many false positives. Detection issues.

Other error type?: None.

False positives:

During the past summer holiday, my family and I {went to a village in the Mainland China to visit }my grandparents.

When we arrived, grandpa and grandma {were in excitement since we had}n't visit them for a long period of time.

They {took out a watermelon from the fridge to greet }us in ecstasy.

eLab

A simple interface

Choice for different end-users

Access to rich feedback

eLab and feedback

eLab and Word Tag

eLab and vocabulary profile

Unintentional Outcome:

Learning Analytics&

Assessment for Learning

Learning Analytics

What is Learning Analytics?

Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs.

-- Wikipedia

Back to Assessment

1. A

n E

njo

yab

le T

rip

2. F

un

Way

s to

Imp

rove

En

glis

h

3. G

ive

Ad

vice

Ab

ou

t a

Tee

nag

er's

Mis

beh

avio

ur

4. M

y T

hre

e-d

ay C

amp

ing

Tri

p

Des

crib

ing

Giv

ing

ad

vice

To

tal

New Rule Errors Identified 330 141 49 62 392 190 582Old Rule Errors Identified 1406 1317 482 600 2006 1799 3805Total Errors Identified 1736 1458 531 662 2398 1989 4387

2 49 82 43 13 62 125 187promblematic: error in classification I {think my mum do} had a great meeting with them.15 97 66 5 14 111 71 182accurate I took {many photo }because I love Hello Kitty so much.17 58 19 54 18 76 73 149accurate I'm so glad that I {can had }that happy trip and I think it might be my most unforgetable memory.18 5 5 11 0 5 16 21accurate The beach is clean and beautiful, it {can makes }people relax.19 6 312 13 5 11 325 336accurate Finally, we {can learning }English and playing game togather.23 2 11 1 0 2 12 14accurate After you joined it, you must feel your speaking level a much {more improve}.27 8 36 10 3 11 46 57accurate Also, we can watch movies or tv are talking {about improve }english.31 1 1 15 0 1 16 17accurate Also, If {he have }problem in homework, you should tech him, help he to get the good result.33 9 13 0 0 9 13 22accurate {It have }many ways, I will give you some idea.36 76 4 8 29 105 12 117accurate We went there {to bought }some souvenirs and dolls.49 5 23 7 0 5 30 35accurate It's just like having the writing {everyday}.50 2 11 6 0 2 17 19accurate You can write blog {everyday or }every week.58 6 20 2 3 9 22 31accurate Thirdly, you can improve {you English language }skills by writing more.80 0 11 0 0 0 11 11accurate You just need to pratice {more and more }in you daily life!84 52 30 10 6 58 40 98accurate I've already heard that Macau have so {many delicious food}.85 6 1 0 1 7 1 8 accurate {It located }in countryside.

Total 382 645 185 92 474 830 1304

10 2 0 0 0 2 0 2 accurate Also, that trip was about sightseeing, it could {let me learn }more about an other country things !18 3 1 0 2 5 1 6 accurate {Although they were so old but }they were still very strong.32 7 5 0 1 8 5 13accurate I am looking forward to going to {another places }tomorrow.41 5 0 0 0 5 0 5 accurate I love Japan {very very }much.56 0 0 3 0 0 3 3 accurate David’ s poor result {may due to }the lack of study.58 4 1 0 1 5 1 6 accurate {She just like }a friend I've made for many years.70 3 1 0 0 3 1 4 accurate and I will go to {the another }country to take photos.91 80 16 0 2 82 16 98accurate {There had a lot }of exciting games, my family and I had so much fun in there.

100 103 58 5 15 118 63 181accurate I took {many photo }because I love Hello Kitty so much.101 47 23 11 6 53 34 87accurate There are {many big museum }and old buildings there.135 20 3 2 14 34 5 39accurate I am {very enjoy }this trip.136 4 0 0 0 4 0 4 accurate When we had a spa, I {felt very relax }too.137 6 0 0 2 8 0 8 accurate I {am very enjoy }this trip.138 7 0 0 5 12 0 12accurate It {was very enjoy}.155 10 0 1 1 11 1 12accurate I can even eat the whole chicken soup as it was {very delicious}.158 18 25 15 7 25 40 65accurate There were some funny mask {and also }some delicious snack which sold on the street.

319 133 37 56 375 170 545

No

tes

Exa

mp

le

New Rules

To

pic

s ca

teg

ori

zed

into

tw

o g

rou

p t

ypes

Des

crib

ing

= 1

+ 4

Giv

ing

Ad

vice

= 2

+ 3

Old Rules

eLab and Assessment for Learning

There is a lot of data.


1. A

n E

njo

yab

le T

rip

2. F

un

Way

s to

Imp

rove

En

glis

h

3. G

ive

Ad

vice

Ab

ou

t a

Tee

nag

er's

Mis

beh

avio

ur

4. M

y T

hre

e-d

ay C

amp

ing

Tri

p

Des

crib

ing

Giv

ing

ad

vice

To

tal

No

tes

Exa

mp

le

76 4 8 29 105 12 117 accurate We went there {to bought }some souvenirs and dolls.

To

pic

s ca

teg

ori

zed

into

tw

o g

rou

p t

ypes

Des

crib

ing

= 1

+ 4

Giv

ing

Ad

vice

= 2

+ 3

To + 'ed': to{(VBD)|(VBN)}


An

En

joy

ab

le T

rip

Fu

n W

ay

s t

o Im

pro

ve

En

glis

h

Giv

e A

dv

ice

Ab

ou

t a

Te

en

ag

er'

s M

isb

eh

av

iou

r

My

Th

ree

-da

y C

am

pin

g T

rip

De

sc

rib

ing

Giv

ing

ad

vic

e

To

tal

School #1 1 12 12 1 13School #2 0 6 6 0 6School #3 0 0 0School #4 0 1 0 1 1School #5 11 8 19 0 19School #6 8 1 0 8 1 9School #7 0 1 1 0 1

Total 19 1 2 27 46 3 4949

To + 'ed': to{(VBD)|(VBN)}

To

pic

s ca

teg

ori

zed

into

tw

o g

rou

p t

ypes

Des

crib

ing

= 1

+ 4

Giv

ing

Ad

vice

= 2

+ 3

What’s Next?

What’s Next?

• Analyse additional sample writings from KS3 students to develop a better understanding of what type (and scope) of feedback is required for students when using the program;

• Provide more accurate and meaningful feedback prompts (and vocabulary profiles) for both students and teachers to use;

What’s Next?

Expand the project to create a user-friendly interface to generate reports for learning analytics.

Thank You.Sean McMinn

[email protected]

Writing ePlatform: Developing an innovative feedback tool for students’ writing to expand...

Documents

Transcript of Writing ePlatform: Developing an innovative feedback tool for students’ writing to expand...