readySTEMgo: Descriptive Results · readySTEMgo Project • EU Erasmus + Strategic Partnership •...

readySTEMgo: ResultsUniversity of Birmingham

Maarten Pinxten & Kamel Hawwash

ReadySTEMgo Erasmus+ Strategic Partnership 2014-1-BE02- KA200-000462

readySTEMgo Project

• EU Erasmus + Strategic Partnership

• Duration: 3 years

• Start date: 1st of October 2014

• 6 Universities + network partner

• Coordinator: KU Leuven

readySTEMgo Erasmus+ Strategic Partnership 2014-1-BE02- KA200-000462

International partnersKatholieke Universiteit Leuven (Belgium)

Technical University Hamburg-Harburg (Germany)Prof. Christian Kautz

University of Žilina (Slovakia)Prof. Peter Hockicko

University of Birmingham (UK)Prof. Kamel Hawwash

Aalto University (Finland)Prof. Katrina Nordström

Budapest University of Technology and Economics (Hungary)Prof. Pal Pacher


Prof. Greet Langie

Outline academic year

Project objectives

Objective 1

Identification of key STEM

competences

Focus on first year

Cognitive and non-cognitive aspects

Objective 2

Diagnostic instrument inventory

Validated measures

Predictive power on achievement

Objective 3

Identification of at risk students and target intervention

Focus on targets for improvement

Feedback to student


(~ Privacy issue !!!)

Difference #1: University entranceHighly selective

institutionsOpen-admission

institutions

Strict selection criteria (entrance exam, matura exams, interviews,…)

More homogeneous inflow of incoming students (at least in termsof prior achievement)

Substantially lower drop-out rates after first year (+/- 5%)

Early decision-making

No or minimal entrance requirements

More heterogenous inflow of incoming students

Higher drop-out rates after the first year (+/- 30%)

Later decision-making process


University of Birmingham KU Leuven

321

Transition to university in Flanders


Secondary School System – Grade 7 – Grade 12 Higher education

University

University Colleges

Labour market

Cycle 1: Grades 7-8

Cycle 2: Grades 9-10

Cycle 3: Grades 11-12

Common core

Academic track

Technical track

Vocational track

Technical track

Academic track

Vocational track

18 years

Transition to university in Flanders• Formal requirement: diploma secondary school

National school leaving examination (e.g., A-level exams)

Entrance exam

Selection interview

• 3 important supporting iniatives in decision-making process

1. Advice of secondary school teacher board

2. Supporting initiatives for decision-making KU Leuven

3. Diagnostic test - Positioning test (organised since 2013)Non-obligatory test at the end of secondary school

Result is non-binding: students can enroll in any case


Difference #2: Engineering Programmes

University of Birmingham (UK) KU Leuven (BE)

• From Day 1: Engineering Specialisation

• Civil Engineering, MechanicalEngineering, Chemical Engineering,…

• 3 Engineering programmes:1. Engineering Science

2. Engineering Technology

3. Bioscience Engineering

• Semester 1-3: general engineering

courses

• From Semester 4: specialisation


Difference #3: Student mobilityBelgium: LowUK: High

* Not to scale

Despite our differences…

We have one common goal:

Deliver the best Engineers in theworld!

Participants University of Birmingham

Study programme N % Female

Chemical Engineering 148 29%

Civil Engineering 37 22%

Mechanical Engineering 96 24%

Sciences 209 31%

Other 9 43%

Total 499 29%


Data collected during the first semester (November 2015)

Background

Parents Degree higher education

32%

65%

3%

No Yes Missing

Does one of your parents

have a degreein higher

education?


1/3rd of the incoming

students at UoB indicates

that both parents do not

hold a higher education

degree

Certain choice study programme

1%5%

21%

47%

25%

0%5%

10%15%20%25%30%35%40%45%50%

Not certain atall

Not certain More or lesscertain

Certain 100% certain

Pro

po

rtio

no

f st

ud

en

ts

“How certain are you for the choice of your current study programme?”


The majority of students are pretty sure about their choice

for their respective studyprogramme

A-level mathematics

57%

11%21%

47%

32%

59%60%

25%

2%

8%

8%12%

1%9%

22%8% 15%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Chemicalengineering

Civil engineering Mechanicalengineering

Sciences

A+ A B C D Missing


A-level Physics

17%3% 4%

22%

34%

27%33%

24%

11%

27%

35% 22%3% 11%

13%4%

1%

1%35% 32%14%

28%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Chemical engineering Civil engineering Mechanical engineering Sciences

A+ A B C D Missing


A-level Chemistry

29%

3% 3%14%

46%

8% 14%

26%

15%

11%17%

28%

4%

2%

10%

78%63%

29%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Chemical engineering Civil engineering Mechanical engineering Sciences

A+ A B C D Missing


Effort levels secondary school


3% 11%

24%36%

25%

0%5%

10%15%20%25%30%35%40%

Not at all typical ofme

Not typical of me Somewhat typicalof me

Fairly typical of me Very typical of me

“I had to study hard to obtain my A-levels in secondary school”

12%

43%32%

11% 2%0%5%

10%15%20%25%30%35%40%45%50%

Not at all typical ofme

Not typical of me Somewhat typicalof me

Fairly typical of me Very typical of me

“I had to study hard for my obtained study results in secondary education”

Birmingham (N=499)

KU Leuven (N=1521)

61% High effort

13% High effort

14% Low effort

55% Low effort

Relation Effort – achievement: KU Leuven


40% 28% 19% 18% 13%

14%9%

8% 10% 13%

17%18%

18% 18% 17%

29%45% 54% 54% 57%

0%10%20%30%40%50%60%70%80%90%

100%

Very low effortlevel

Low effortlevel

Average effortlevel

High effortlevel

Very high effortlevel

% credits obtained after the first year

<30% CSE of dropout 30-50% CSE 50-80% CSE >80% CSE

Higher effort levels in secondary school Better first-year achievement

Planning on taking a student job?

46%58%

42% 46% 38%

19%17%

25% 17%20%

35%25% 33% 37% 42%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Total Chemicalengineering


Sciences

No Yes I Don't know yet


Course content?

2% 3% 2% 1%16% 21%

14% 9% 15%

36% 35%34% 40% 36%

41% 38% 51% 42% 42%

5% 3% 6% 6%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%



Sciences

"I have clear knowledge of the content of all courses in the first year of my current study programme."

Not at all Not More or less Good idea Very good idea


Confident succesful first year?

2% 5% 2%10% 16% 14%3% 8%

25%27% 19%

24%25%

42% 35% 46%49%

44%

21% 20% 14% 24% 21%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%



Sciences

"I feel very confident to successfully complete the first year"

Not confident at all Not confident More or lessConfident Very confident


Learning and studystrategies

Learning and Study Strategies Inventory

• Scientific measurement of learning and study strategies of incoming students

• 77 items

• 10 scales


Learning and study strategies inventory• Attitude• Motivation – persistence• Time management• Anxiety• Concentration• Information processing• Selecting main ideas• Study aids• Self-testing• Test strategies


Research at KU

Leuven, BME & UniZa

has shown that these

4 variables are

statistically related to

first-year students’

achievement at the

end of the year

17%

22%

26%

19%

17%

Very good Time management Good time managment

Average time management Weak management

Very weak time management

Time management: Birmingham


36% of the incomingstudents reported (very) weak time management skills at the start of the

academic year

15%

19%

31%

19%

16%

Very good timemanagement skills Good time management skills

Average time management skills Weak time management skills

Very weak time management skills

Time management (KU Leuven – N=1521)

16% 25% 29% 35% 40%

44%43% 40%

41% 40%

40% 33% 31% 24% 20%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Very goodTM

GoodTM

AverageTM

WeakTM

Very weakTM

% of credits obtained after first year(SE)

<30% SE or dropout 30-80% SE >80% SE


Concentration: Birmingham


21%

19%

25%

19%

14%

Very good Concentration Good Concentration Average Concentration

Weak Concentration Very weak Concentration

33% of the incomingstudents reported (very) weak time concentrationlevels at the start of the

academic year

Concentration (KU Leuven – N=1521)

19% 27% 30% 34% 45%40%

41% 41% 46% 37%40% 33% 29% 21% 18%

0%10%20%30%40%50%60%70%80%90%

100%




18%

24%

30%

18%

10%

Very good concentration Good concentration Average Concentration

Weak Concentration Very weak concentration

25%

26%26%

15%

8%

Very good Motivation Good Motivation Average Motivation

Weak Motivation Very weak

Motivation – persistence: Birmingham


23% of the incomingstudents reported difficultiesin persisting with confronted

with challenging tasks

15%

26%

27%

20%

12%

Zeer goede Motivatie Goede Motivatie Gemiddelde Motivatie

Zwakke Motivatie Zeer zwakke Motivatie

Motivation – Persistence (KU Leuven)

22% 21% 29% 36% 44%

36% 46% 39%40%

43%

42% 32% 32% 23% 13%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Very goodmotivation

Goodmotivation

Averagemotivation

Weakmotivation

Very weakmotivation




European comparison: Time management


71%72% 73%

82% 84%

66%

77% 79%78%

82%

44%46%

53%59% 64%

30%

40%

50%

60%

70%

80%

90%

Very weak Timemanagement

Weak timemanagement

Average timemanagement

Good timemanagement

Very Good timemanagementM

ean

% c

redi

ts o

btai

ned

afte

r exa

ms J

anua

ry 2

016

Average SE in function of Time management skills start academic year

UniZa (N=814) BME (N=990) KU Leuven (N=1521)

Hypothesized curve Birmingham

European comparison: Motivation


72%74%

77%84%

92%

69%

78%76%

80%86%

42%46%

54%59% 62%

30%

40%

50%

60%

70%

80%

90%

100%

Very weakmotivation

Weakmotivation

Averagemotivation

Goodmotivation

Very goodmotivation

Mea

n %

cre

dits

obt

aine

d af

ter e

xam

s Jan

uary

201

6

Average SE in function of Motivation start academic year

UniZa (N=814) BME (N=990) KU Leuven (N=1521)

Hypothesized curve Birmingham

Conclusion• Similar positive relation in three different institutions

• Average levels of achievement linked to entrance criteria university

• KU Leuven, on average, lower percentages of credits are obtained

• BME & UniZa (entrance criteria apply), higher averagepercentages of credits are obtained >>> ceiling effects


Follow-up Project in UoB• Questionnaire in second semester:

Participation (+grades) Math diagnostic test start Year 1 Foundation year BTEC Visit Math Support Center Summer placement between Year 1 and Year 2 …

• Attempt: make the connection with 2015-2016 database using students’ ID


Some salient findingsin FlandersResults KU Leuven

1. Prior grades secondary school

• Weighted GPA score of three subject (Mathematics, Chemistry & Physics)

• Math has higher weight in accordance with importance in curriculum

• 1 score calculated ranging between 1 (very low score) and5 (excellent score)

• Results of academic year 2016-2017 (Exams January)



63%52%

40% 35%24%

15% 11%

16%15%

19%14%

17%16%

8%

11%22%

19%23%

20%25%

13%

9% 11%23% 28%

39% 44%68%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1-1,5(N=97)

1,5-2(N=132)

2-2,5(N=394)

2,5-3(N=227)

3-3,5(N=338)

3,5-4(N=135)

>4(N=183)

Behaalde studieresultaten na Januari 2017<30% credits of dropout 30-50% credits 50-80% credits >80% credits

Data 2016-2017 – Exam results January 2017 (N=1512)

Weighted GPA (math – Physics – Chemistry) Secondary school



Strong relation withacademic performance

Students with a GPA below 2 are at seriousrisk: +/- 60% obtain lessthan 30% of the creditsafter the first examination period

This is a direct result of an open-admissionsystem to university

1. Advice teacher board secondary school


51%

21%

20%

7%

Full positive advice(N=776)Partially positive advice (N=322)No advice (N=303)Negative advice (N=113)

Academic year 2015-2016 (N=1521)

Academic year 2016-2017 (N=1509)

48%

23%

16%

13%

Full positive advice (N=725)Positive advice (N=344)No advice (N=236)Negative advice (N=200)

34%48%

30%13%

12%

19%

13%

6%

19%

18%

20%

16%

36%15%

37%

65%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

No advice(N=303)

Negative advice(N=113)

Partially positiveadvice (N=322)

Full positive advice(N=776)

<30% credits or Drop-out 30-50% credits 50-80% credits >80% credits


Data 2015-2016 – Exam results after resits (N=1521)

46%58%

33%21%

14%13%

21%

14%

13%17%

22%

21%

26%14%

24%44%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

No advice(N=240)

Negative advice(N=200)

Partially positiveadvice (N=344)

Full positive advice(N=725)

<30% credits or Drop-out 30-50% credits 50-80% credits >80% credits


Data 2016-2017 – Exam results January 2017 (N=1512)

3. Advice teacher board

• Strong indicator of studysuccess

• 48% of the studentswith a negative advicedo not obtain 30% of his/her credits after thefirst year

• Students with a full positive advice: confident


3. Math diagnostic test

• Prior to the start of the academic year (June)

• Last week of secondary school

• Voluntary participation

• Non-binding results: even if a student fails, s/he can start

• Participation rates vary between programs


3. Math diagnostic test (N=450)


55%

23%

10%

37%

14%

20%

9%12%11%

30%26%

14%

9%

22%

49%

16%12%

5% 6%

22%

0%

10%

20%

30%

40%

50%

60%

Failed(N=57)

Passed 10/20 - 13/20(N=208)

Passed >13/20(N=134)

Not participated(N=51)

% S

tude

nts

Result Math Diagnostic Test – Engineering Science

<30% credits 30-50% credits 50-80% credits >80% credits Dropout

At risk???

We succesfully manage to identify a risk population and and a population

of high-performing students

3. Math diagnostic test & Secondary school math grades


50%35% 28%

15% 8%

42%53%

55%

44%33%

8% 12% 17%41%

58%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

<60% (N=12) 60-70% (N=34) 70-80% (N=71) 80-90% (N=70) >90% (N=11)M a t h g r a d e s s e o n d a r y s c h o o l

Math test score between 10/20 and 13/20

<30% SE of dropout 30-80% SE >80% SE

* Diagnostic test – Engineering Science 2015

Conclusion• Based on a math diagnostic test and secondary school

grades we manage to identify an at-risk population

• Non-cognitive factors help us to further refine the picture


More information?

www.iiw.kuleuven.be/readystemgo

[email protected]

readySTEMgo: Descriptive Results · readySTEMgo Project • EU Erasmus + Strategic Partnership •...

Documents

Transcript of readySTEMgo: Descriptive Results · readySTEMgo Project • EU Erasmus + Strategic Partnership •...