UTS,    Jurgen Schulte 1NSW  Learning   Analytics   Working   Group3  February   2016

.    .    .

NSW  Learning  Analytics  Working  Group

Experiences  with  an  Adaptive  Learning  Product

adaptive  testingconfidence  and  proficiency

analytics

Jurgen SchulteSciMERITSchool  of  Mathematical  and  Physical  SciencesUniversity  of  Technology  Sydney

UTS,    Jurgen Schulte 2NSW  Learning   Analytics   Working   Group3  February   2016

Computerise Adaptive  Testing

Presentation  of  educational  material

according  to  students'   learning  needs

Adaptive Learning

AI 1970s (SCHOLAR,  adaptive  &  intelligent  learning  systems)1980s (Intelligent  Tutoring  Systems)1990s (Adaptive  Learning  Technology)2000s (AutoTutor,  passed  Turing  test)

UTS,    Jurgen Schulte 3NSW  Learning   Analytics   Working   Group3  February   2016

www.wileyplus.com

B  Engineering  HonoursBE  DipEngPracBEngSci LLBBE  (Hons)   BA  Int StBE  (Hons)    BMedScBE  (Hons)    BSciBEng  (Hons)    DipProfEngPracBEng  (Hons)    DipProfEngPracBEng  (Hons)    BBusBEng  (Hons)    BCIInnStudy  Abroad  Undergraduate

…  various  sub-­‐majors

Adaptive Learning

UTS,    Jurgen Schulte 4NSW  Learning   Analytics   Working   Group3  February   2016

www.wileyplus.com

Adaptive Learning – Student’s View

UTS,    Jurgen Schulte 5NSW  Learning   Analytics   Working   Group3  February   2016

quantitative  problem

skills  required  to  solve  problemno  multiple  guess  choice

confidence  leveltime  to  solveproficiency,  not  performance

level  of  challenge

track  record  on  this  topic

disciplinary  skill

Next round  of  problems  depends  on  how  this  round  of  10  problems  has  been  completed    

Adaptive Learning – Student’s View

UTS,    Jurgen Schulte 6NSW  Learning   Analytics   Working   Group3  February   2016

www.wileyplus.com

Adaptive Learning – Student’s View

UTS,    Jurgen Schulte 7NSW  Learning   Analytics   Working   Group3  February   2016

Week  3 Week  8 Week  12

Adaptive Learning – Manager View

UTS,    Jurgen Schulte 8NSW  Learning   Analytics   Working   Group3  February   2016

Adaptive  Learning    2D

Knowledge  maturity,  personal  timeline

d-­‐analysis motion forces statics momentum energy thermal electricity fluids waves optics

Adaptive Learning

UTS,    Jurgen Schulte 9NSW  Learning   Analytics   Working   Group3  February   2016

Approach   (adaptive  learning)

flexible  low  to  high  level  entry

24/7  access,  all  semester  long

continuous  incremental  work

start  and  work  on  preferred  topics

continue  any  time  where  left  off  last

‘Adaptive  Learning’    (2D)

Adaptive  to  …

• learning  ability

• learning  load

• personal  circumstances

• personal  interest

• building  up  skills

Adaptive Learning

UTS,    Jurgen Schulte 10NSW  Learning   Analytics   Working   Group3  February   2016

Data  extraction,   preparation,   analytics  …

…  feed  into  weekly  Personal  Support  Actions

Adaptive Learning Analytics

UTS,    Jurgen Schulte 11NSW  Learning   Analytics   Working   Group3  February   2016

12 3 4 5 6 7 8 910111214151617181921222426273334

0

2000

4000

6000

8000

10000

12000

2 3 4 5 6 7 8 9 10 11 12 13 14 15

Chapter

No.5S

tudents

Week

Cumulative5Questions5 Correct5Weekly5 (Friday)

Adaptive Learning Analytics

UTS,    Jurgen Schulte 12NSW  Learning   Analytics   Working   Group3  February   2016

0.0

2.0

4.0

6.0

8.0

10.0

12.0

28/07/20

15

29/07/20

15

30/07/20

15

31/07/20

15

1/08

/201

5

2/08

/201

5

3/08

/201

5

4/08

/201

5

5/08

/201

5

6/08

/201

5

7/08

/201

5

8/08

/201

5

9/08

/201

5

10/08/20

15

11/08/20

15

12/08/20

15

13/08/20

15

14/08/20

15

15/08/20

15

16/08/20

15

17/08/20

15

18/08/20

15

19/08/20

15

20/08/20

15

21/08/20

15

22/08/20

15

23/08/20

15

24/08/20

15

25/08/20

15

26/08/20

15

27/08/20

15

28/08/20

15

29/08/20

15

30/08/20

15

31/08/20

15

1/09

/201

5

2/09

/201

5

3/09

/201

5

4/09

/201

5

5/09

/201

5

6/09

/201

5

7/09

/201

5

8/09

/201

5

9/09

/201

5

10/09/20

15

11/09/20

15

12/09/20

15

13/09/20

15

14/09/20

15

15/09/20

15

16/09/20

15

17/09/20

15

18/09/20

15

19/09/20

15

20/09/20

15

21/09/20

15

22/09/20

15

23/09/20

15

24/09/20

15

25/09/20

15

26/09/20

15

27/09/20

15

28/09/20

15

29/09/20

15

30/09/20

15

1/10

/201

5

2/10

/201

5

3/10

/201

5

4/10

/201

5

5/10

/201

5

6/10

/201

5

7/10

/201

5

8/10

/201

5

9/10

/201

5

10/10/20

15

11/10/20

15

12/10/20

15

13/10/20

15

14/10/20

15

15/10/20

15

16/10/20

15

17/10/20

15

18/10/20

15

19/10/20

15

20/10/20

15

21/10/20

15

22/10/20

15

23/10/20

15

24/10/20

15

25/10/20

15

26/10/20

15

27/10/20

15

28/10/20

15

29/10/20

15

30/10/20

15

31/10/20

15

1/11

/201

5

2/11

/201

5

3/11

/201

5

4/11

/201

5

5/11

/201

5

6/11

/201

5

7/11

/201

5

S S M T W T F S S M T W T F S S M T W T F S S M T W T F S S M T W T F S S M T W T F S S M T W T F S S M T W T F S S M T W T F S S M T W T F S S M T W T F S S M T W T F S S M T W T F S S M T W T F S S2 3 4 5 6 7 8 9 10 11 12 13 14 15

Total5Time5Spent5(Da

ys,5equvilant5to524hrs)

Date,/,Week

Time5Students5Spent5on5Orion5throughout5the5Semester5(Daily)

Adaptive Learning Analytics

UTS,    Jurgen Schulte 13NSW  Learning   Analytics   Working   Group3  February   2016

Analytics  triggered  PSA

P. S. A.

• Weekly    personalised emails  

• Status  report

• Expectations

• Encouragement  /  Caution  

• Progress  support

• Early  warning  flags

UTS,    Jurgen Schulte 14NSW  Learning   Analytics   Working   Group3  February   2016

Adaptive Learning Analytics

Pros

• informative  dash  boards

• links  with  WileyPLUS gradebook

• proficiency,  confidence,  diagnostic

• extends  WileyPLUSproblem  set

• local  technical  support

• free  of  charge  add-­‐on

ORIONCons

• separate  development  team

• long  time  for  support  to  response

• limited  access  to  visible  data

• data  format  non-­‐standard

• external  product

• more  a  trial  than  a  solid  product

UTS,    Jurgen Schulte 15NSW  Learning   Analytics   Working   Group3  February   2016

What  did  you  value  most  about  the  assessment  structure  in  this  subject  (50%  Orion,  30%  prac,    20%  exam)?

“Worked  well  for  me.  Engaging.”

“I  think  Orion  is  a  fantastic  tool  but  I  would  have  used  it  more  if  the  punishment  for  answering  incorrectly  was  not  so  bad.”

“I  liked  that  I  could start  where  I  had  left  off.”

“Made  it  easier  to  let  me  do  it  at  my  own  pace.  Since  it  was  online  I  could  do  more  of  it  from  home.”

“Get  marked on  questions  you  do  at  home  [opposed]  to  other  subjects  [where]  you  get  marked  mostly  on  test  and  assignments.”

“If  something  was  due  in  another  subject  I  could  focus  on  that  then  come  back  to  Orion  knowing  it’s  not  due  yet.”

“It  worked  very  well  for  me  because  I  tend  to  stress  when  a  final  exam  has  a  huge  percentage  weight.”

Adaptive Learning

UTS,    Jurgen Schulte 16NSW  Learning   Analytics   Working   Group3  February   2016

In  your  own  opinion  and  from  experience  you  may  have  had  in  other  subjects,  do  you  believe  you  had  deeper  learning  experience  this  way  [adaptive]  compared  to  an  exam-­‐based  assessment  ?

Other  subject  you  won’t  do  many  questions  for  each  chapter  but  for  this  subject  I  was  able  to  do  many  questions  on  all  chapters  which  gave  me  a  good  depth  of  knowledge.  

It  was  different  because  I  was  able  to  see  improvements  on  a  weekly  basis.  I  didn’t  do  physics  at  school.  So  this  assessment  task  really  helped  me  because  I  was  able  to  do  heaps  of  questions  and  track  my  progress.

Also,  having  a  big  focus  on  [problem  solving]  practise skills  is  much  more  valuable  for  overall  learning.

I  found  I  was  much  more  involved  with  this  style  of  assessment.

Were  there  any  other  skills  you  developed  throughout  the  process  of  this  homework  assessment  task?

Time  management  Solve  problems  quickerConfidence  when  problem  solvingVisualise and  solve  problems  quicker

Adaptive Learning

UTS,    Jurgen Schulte 17NSW  Learning   Analytics   Working   Group3  February   2016

Individualised just-­‐in-­‐time   feedback

“I  liked  that  Dr Jurgen  was  able  to  send  weekly  progress  reports  to  not  only  give  us  the  proficiency  scores  but  where  we  are  sitting  in  the  subject  in  order  to  get  a  pass.      It  was  a  source  of  motivation  !!”

P S A

UTS,    Jurgen Schulte 18NSW  Learning   Analytics   Working   Group3  February   2016

Q    &    A