The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the...

58
The European Science Fairs Evaluation Framework – pilot study An Intel & EUN initiative. September 2013

Transcript of The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the...

Page 1: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

The European Science Fairs Evaluation Framework – pilot studyAn Intel & EUN initiative.

September 2013

Page 2: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

Authors ÀguedaGras-Velázquez,JonK.Price,PřemyslVelek,MichałDzoga&IzaPastuszynska

Collaborators CarolineBergaud,SaraPasquali,EmmaBluck,MichelaSaputi

Formoreinformationonthereport,pleasecontactDrÀguedaGras-Velázquez([email protected])

orIzabelaK.Pastuszynska([email protected]).

Page 3: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

EXECUTIVE SUMMARY

Page 4: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

4

Pilot evaluation of Science Fairsin Europe

Executive summary

EXECUTIVESUMMARY

Inthisreportwepresentandanalysethedata obtainedfromthepilotevaluationoffourEuropeansciencefairs.Theaimoftheevaluationexercisewastoshowthedatathatcanbeobtainedfromtheevaluationtooldeveloped.TheevaluationtoolisplannedtobeadministeredinallIntel-affiliatedfairsandprovideusefuldataforanalysis,ensuringcross-countryandcross-yearcomparability.

Thestudents’andteachers’datafromthefourfairswerecombinedandareprovidedintableorgraphformatinseparatesections,withshortdescriptionsandhighlightsforeachgraph.Asahelpingsimpletool,averagestatisticalprofilesofparticipatingstudentsandteachersareprovidedattheendofeachdatasection.Thesestatisticalprofilescanbeascomprehensiveasisrequired.

Attheendofthereport,amoredetailedanalysisof thestudents’andteachers’dataispresentedtogetherwithsomeconclusions,whilethedataforeachfairisprovided inaseparatereportperfair.

Main findings from students

• Theyarenewcomers(butloyaltyeffect).

• Malepredominance.

• Genderdifferencesintheprojectareas,withmales’interestsplitbetweenscienceandtechnology&engineering,whilewomenfocusmainlyonscience.

• Abalancebetweenindividualandteamprojects.

• Hardwork(highdegreeofstudentcommitmentto theprojects).

• Greaterconfidenceinidentifyingproblems,findingsolutionsandhowtousethescientificmethodorevenhadfuninthefair.

• Improvementincommunicatingandteamskills(studentsfindmanyvaluesinparticipation).

• Difficultiesingettingholdofmaterialsandequipment,aswellascarryingoutthedifferentstepsinthescientificmethod(facing andovercomingthesedifficultiesresultsin “anincreaseinconfidence”).

• Internshipsfor20%oftheparticipants(afurthershow ofcommitment).

• Essentialfactorsforsuccess:creativityandcommunicationskills,followedbyintelligence.Collaborationandleadershipcameinlast.

• Gendereffectinpersonaltraits:intelligenceis theoneskillwheretherewasalargerthan15%genderdifference.

• Voluntaryparticipationfor80%students.Theywereequallyinfluencedbytheteacher(s).(Bigteacherinfluence.)

• Satisfactionfor90%oftheparticipants.(Thismakes thefairsaneducationalsuccess.)

Main findings from teachers

• Slightgender(female)predominance.

• Longteachingexperience,over10yearsfor36% andover20yearsfor40%.

• Repeatschoolparticipationfor66%oftheschools. (Goodreceptivityofthefairsamongschools).

• Evenhigherteacherrepetition:notthefirstfairfor75%ofthem.(Thismaybeduetoteachers’permanencein agivenschool).

• Largestudents/teacherratio:75%bringupto10. (Highteachercommitmentwiththefairs).

• Teachers’areas:Science(38%),nextEngineering(19%). (Thefigurescorrelatewiththetopicsinthestudents’projects).

• SupervisedprojectsarepredominantinScience;TechnologyandEngineeringaresimilar.(Variedcharacteroftheschoolsactivities).

• Out-of-schoolactivitiesfor50%ofthestudents.They arenotin21%schools.(Studentsmakeextraefforts toparticipateinthesefairs).

• Voluntarytutoringfor99%oftheteachersand aschoolcoordinatorexistsin60%schools.(This indicatesafeltneedaswellasaguarantee ofsuccess).

Page 5: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

5

Pilot evaluation of Science Fairsin Europe

Executive summary

• TheprojectclearlylinkedwiththenationalSTEMcurriculumin40%ofthecases.(So,eithertheprojectshaveabroaderaimthanthecurriculum,oritisnoteasytodevelopsufficientlyinterestingprojectsbykeepinginsidethecurriculumboundaries).

• Schoolsupportforparticipationinthefairin75%of thecases.55%supportfromcolleagues,parentsandexperts.Leastsupportcomesfromlocalauthorities,localmediaoreventheNationalMinistryofEducation.(Thoseitemsrequiremoreeffortfromthefairandschoolmanagement).

• Mostusefulsupportcomesfrommentorsorscienceexpertsaswellastheschoolmanagement(60and46%,respectively).

• Students’personaltraitsforsuccessarecreativity,communicationskillsandintelligence(50%).Collaboration,scientificliteracyandworkethicareconsideredonlyin25%oftheanswers.Thecapacityofleadershipwasnotconsidered.(Itwouldbeinterestingtoanalysetheseresultsinmoredetailinthefuture).

• Mainreasonforteachers’commitment:“Mystudents”(94%)andprestige(74%).Lessimportant:theprospectofpromotion(29%),andtheeffectsoftheircolleagues

(43%).(Theseanswersmayhelpindesigningfutureannouncementsandrulesforthedevelopmentoffairs).

• Inquiry-basedmethodologychangesfrom25%to35%. For2/3oftheparticipatingteacherstheuseofIBSME(Inquiry-BasedScienceandMathematicsEducation)isstillanunknownfactorintheteachingequation.(ThustheScienceFairinitiativehelpstoimprovingschooleducationasregardsthenecessarytransformationandupdatingofteachers’methodologies,sothatthestudents’educationisimprovedintermsoftransferableknowledge,skillsandcompetences).

• 86%oftheteachersaresatisfiedandwouldrepeattutoring.(Thisisaremarkableresultofthewholeprogrammeand,inconjunctionwithstudents’opinions,onemayconcludethatbothforteachersandstudents,participatinginthesciencefairsisanenjoyableandpositiveeducationalexperience).

Overallthekeymessagesarethat70%ofthestudentsthatparticipateinsciencefairsareafterwardsmorelikelytoconsiderstudyingaSTEMdegree;65%ofthestudentsafterparticipatinginasciencefairaremorelikelytoconsideraSTEMcareerandmorethan80%oftheteachersconsideredtheyhadlearntalotthroughparticipatingin thesciencefair,thusfindingsciencefairsaneffectivewayofobtainingprofessionaldevelopment.

Page 6: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

TABLE OF CONTENTS

Executivesummary. ......................................................................................................................................3

TableofContents ............................................................................................................................................6

Listoftables ...............................................................................................................................................7

Listoffigures .............................................................................................................................................8

Introduction ....................................................................................................................................................10

EvaluationMethodology ..........................................................................................................................12

Questionnaires .......................................................................................................................................14

Likertscalesandstatisticalanalysis............................................................................................16

Overallresults .................................................................................................................................................18

Students’ questionnaires ..........................................................................................................................20

Teachers’ questionnaires ...................................................................................................................30

AnalysisandConclusions..........................................................................................................................41

OpenDiscussionandFuturework ......................................................................................................45

Appendix ............................................................................................................................................................47

Science Fairs – teachers’ questionnaire ....................................................................................47

Science Fairs – students’ questionnaire ....................................................................................53

Page 7: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

List of tables

7

Pilot evaluation of Science Fairsin Europe

Table1:Aspectsevaluatedineachsectionofthestudents’(SQ) andteachers’(TQ)questionnaires ....................................................................................................................................15

Table2:Likertscale&Redefinedscalesforanalysis ............................................................................................17

Table3:Numberofrespondentstothequestionnairesperevent ...............................................................19

Table4:RatioofScientific to Technological to Engineering to Maths projectsbygenderandaverage ........................................................................................................................................23

Table5:Usefulnessofvarioussourcesofprojectsupport(students) ........................................................26

Table6:Mostessentialcontributionstotheproject’ssuccess(students) ................................................27

LISTOFTABLES

Page 8: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

List of figures

8

Pilot evaluation of Science Fairsin Europe

Figure1:Ageofrespondents(students) ......................................................................................................................21

Figure2:Gendersplitamongtherespondents(students) .................................................................................21

Figure3:Numberofparticipationsinasciencefairincludingthepresentone(students) .............22

Figure4:Majorfocusoftheprojects(students) ......................................................................................................22

Figure5:Majorfocusoftheprojectsdependingongender(students) ......................................................22

Figure6:Individualversusteamparticipationinthefair(students) ............................................................23

Figure7:Timespentondevelopingtheproject(students) ...............................................................................23

Figure8:Timepreparingthedisplayandthepresentationoftheproject(students) .......................24

Figure9:Participationinanafter-schoolscienceclub,programmeoractivity(students) ..............24

Figure10:Agreementwithdifferentstatements(students) ...........................................................................24

Figure11:Advantagesofdoingasciencefairproject(students) ..................................................................25

Figure12:Difficultiesencountered(students) .........................................................................................................25

Figure13:Participationininternshipsrelatedtotheproject(students) ...................................................26

Figure14:Isitcompulsoryintheschooltodoasciencefairproject?(students) ................................27

Figure15:Factorsinfluencingthedecisiontocompeteinthesciencefair(students) .....................28

Figure16:Willingnesstorepeatasciencefairandrecommendtheexperience(students) ..........28

Figure17:Averageparticipantprofile(students) ...................................................................................................29

Figure18:Gendersplitamongparticipants(teachers) ........................................................................................31

Figure19:Yearsofteachingexperience(teachers) ..............................................................................................31

Figure20:Highestdegreeheld(teachers) ..................................................................................................................32

Figure21:Numberofschoolparticipationsinthefair(teachers) .................................................................32

Figure22:Numberofstudentsfromtheirschoolswhoarecompeting(teachers) .............................32

Figure23:Firsttimeparticipantsinfairs(Yes)versusrepeatparticipants (No)(teachers) .............................................................................................................................................................................33

Figure24:Mainsubjecttaught(teachers) ...................................................................................................................33

Figure25:Mainfocusoftheprojectssupervised(teachers)............................................................................33

LISTOFFIGURES

Page 9: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

List of figures

9

Pilot evaluation of Science Fairsin Europe

Figure26:Schoolswherethestudentsparticipatedinafter-schoolscienceclubs, etc.(teachers) ...............................................................................................................................................................................34

Figure27:Schoolswhere8)itiscompulsorytoparticipateinscience fairsand9)thereisapersoninchargeoftheparticipation(teachers) ......................................................34

Figure28:Itwaseasytointegratethecompetitionwiththecurricula(teachers) .............................34

Figure29:Impactofparticipatinginthesciencefairsonbothteachers andstudents(teachers) .........................................................................................................................................................35

Figure30:Advantagesofdoingasciencefairprojectoveraregular scienceclass(teachers) ..........................................................................................................................................................36

Figure31:Difficultiesencounteredinpreparingaprojectandparticipating inthefair(teachers). .................................................................................................................................................................36

Figure32:Difficultiesstudentsfoundinactivitiesincarryingoutaproject andparticipatinginafair(teachers). ................................................................................................................................37

Figure33:Supportobtainedfromdifferentpeopleororganizations(teachers)....................................37

Figure34:Organizationsorpeoplethatwereinstrumentaltocarryingout thestudents’projects(teachers). ......................................................................................................................................38

Figure35:Essentialissuescontributingtothesuccessofstudents’sciencefair projects(teachers). .....................................................................................................................................................................38

Figure36:Factorsorpeoplethatinfluencedthedecisiontoleadparticipants inthefairandmadethetaskeasier(teachers). ........................................................................................................39

Figure37:UseofIBSMEbeforethesciencefairandafterwards(teachers). ...........................................39

Figure38:Wouldyousuperviseaprojectforasciencefairagain?(teachers). .......................................39

Figure39:InterestinbecomingaScienceFairAmbassadorfortheircountry(teachers). ...............40

Figure40:Averageparticipantprofile(teachers). ....................................................................................................40

Page 10: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

INTRODUCTION

Page 11: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

11

Pilot evaluation of Science Fairsin Europe

Introduction

INTRODUCTION

InthisreportwepresentandanalysethedataobtainedfromthepilotevaluationoffourEuropeanScienceFairs.Theaimoftheevaluationexercisewastoshowthedatathatcanbeobtainedfromtheevaluationtooldeveloped.TheevaluationtoolisplannedtobeadministeredinallsciencefairsaffiliatedtotheIntelInternationalScienceandEngineeringFairs(IntelISEF)andprovideusefuldataforanalysis,ensuringcross-countryandcross-yearcomparability.

ThebasicideasbehindthevaluationmethodologyandstrategyforitswidespreadinfairsacrossEuropeare

discussedinGras-Velázquez,À.,Price,J.K.,Velek,P.,Dzoga,M.&Pastuszynska,I.K.(2013):Designing a Science Fair Evaluation Strategy: The European Model.

Herewejustprovidetheminimuminformationrequiredtobeabletofollowtherestofthereport.Thedatacontainedin the students’ and teachers’ questionnaires are listed in twoseparatesectionsingraphortabularform,followedbyadetailedanalysisandconclusions.Thefullcontentsofthestudents’andteachers’questionnairesaregiveninanappendix.

Page 12: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

EVALUATION METHODOLOGY

Page 13: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

13

Pilot evaluation of Science Fairsin Europe

Evaluation methodology

EVALUATIONMETHODOLOGY

Theevaluationofthesciencefairswasfocusedonthreeaspectsinrelationtotwotargetgroups:studentsandscienceteachers.Thethreemainaspectsofthefairsthatwereevaluatedareimpact,supportandsustainability.Theaimswerethefollowing:

• Impact–Theeffectofparticipatinginsciencefairsonstudents and teachers.

• Support–Thehelpreceivedforparticipatinginsciencefairs(students)orfortutoringstudents’scienceprojects(teachers).

• Sustainability–Themotivationofstudentsandteacherstoparticipateinsciencefairs.

Weexploredthosethreeaspectsbydevelopingtwoquestionnaires(oneforteachersandaseparateoneforstudents,seeAppendix)tobefilledinbytheparticipantsatthefiveselectedsciencefairs.

Page 14: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

QUESTIONNAIRES

Page 15: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

15

Pilot evaluation of Science Fairsin Europe

Questionnaires

QUESTIONNAIRES

Theitemsinbothquestionnaires(forstudentsandteachers)aredividedintoseparatesectionsinorderto

obtaininformationaboutthevariousitemsofinterest,asshowninTable1.

ThequestionnaireitemsmentionedinSection1providetheusualdemographicdatathatservethepurposeofcharacterizingthesampleofstudentsandteacherswhoparticipatedinthefivesciencefairsthathavebeenevaluated.

TheparticularcomponentsinvestigatedinSection2(Impact)ofthestudentquestionnaireincludestudents’(self-assessed)knowledge,skillsandattitudetowardsscience,theirlearningexperience,andtheirmotivationtostudyscienceandchooseacareerinscience.Section2intheteachers’questionnaireadditionallycontainsquestionsregardingtheeducationalvalueoftheirstudent’sparticipation,andtheimpactthatthesciencefairhadontheirschoolandontheirteachingpractice.

Section3(Support)aimstoidentifyproblematicorchallengingaspects(e.g.funding,accesstomaterialsandequipment)ofthoseparticipatinginsciencefairsfromthepointofviewofbothstudentsandteachers;andhowandbywhomtheyweresupportedtoovercomethem.

Section4(Sustainability)focusesonthemotivatinganddiscouraging(intrinsicandextrinsic)factorsforstudentsandteacherstoparticipateinsciencefairs.Theteachers’questionnairealsocontainsitemsoninquiry-basedteachingandthepossiblerolethatsciencefairsmayplayinpromotingit.

The questions contained in the teachers’ and students’ questionnairesarepresentedinunambiguous,jargon-freestatementswithaFlesch-Kincaidreadability. Gradelevelis7.3forthestudents’questionnaire,suitableforGrades7–8(i.e.12–14yearsold),and8.4fortheteachers’questionnaire.Additionally,exceptinthecaseofthefewopen-endedquestionsincluded,allthequestionshadalimitedrangeofanswers,eitherintheformofaLikert-typescale,asamultiple-choice orasasingle-choicequestion.WeshallgiveafewmoredetailsabouttheLikertscalesandtheiranalysisin thefollowingsection.Insomeitems,respondentscouldaddadditionalcommentsorexplanatoryremarks totheiranswers.

TABLE 1: ASPECTS EVALUATED IN EACH SECTION OF THE STUDENTS’ (SQ) AND TEACHERS’ (TQ) QUESTIONNAIRES.

SECTIONITEMS IN...

TO OBTAIN DATA ON THE...SQ TQ

1 1-7 1-10 Demographic dataandotherbasiccharacteristicsofparticipants.

216-20 22-27

Impactofsciencefairsontheparticipants.8-9 11-12

3 10-13 13-17 Supportreceivedbytheparticipants.

4 14-15 18-21 Sustainabilityandmotivationofparticipantsinthesciencefair.

Page 16: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

LIKERT SCALES AND STATISTICAL ANALYSIS

Page 17: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

17

Pilot evaluation of Science Fairsin Europe

Likert scales and statistical analysis

LIKERTSCALESANDSTATISTICALANALYSIS

Asmentionedabovewehaveusedvarious4-levelLikertscalestoratethevariousoptionsprovidedinsomeanswers,seecolumn1inTable2.However,thedifferencebetweenrespondentssaying“completelyagree”orjust“agree”ishighlyinfluencedbypersonalitytraitsandthedifferencessignifylittleintermsof

statistics.Thereforefortheanalysisofdatalevels1+2and3+4havebeencombinedandredefinedasLowandHighdegreeoffulfilment,respectively(seeTable2,column2,RedefinedscaleA),or,whenevermoreappropriate,asNoandYes,respectively(seeTable2column3,RedefinedscaleB).

Concerningthestatisticalanalysisofthequestionnairedata,andasageneralcommentonthedatatobediscussedinthisreport,letusmentionthatduetotheintrinsicaccuracyofthedataobtainedfromthequestionnaires,whichis

roughlywithinthe±10%range,weshallonlysingleoutgenderdifferencesorotherdifferencesinanyoftheitemscontained in the students’ and teachers’ questionnaires whenthesedifferencesarelargerthanabout15%.

TABLE 2: LIKERT SCALE & REDEFINED SCALES FOR ANALYSIS.

LIKERT SCALEREDEFINED SCALE

A B

1–Notatall/VerydifficultLow No

2-

3-High Yes

4–Completelyagree

Page 18: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

OVERALL RESULTS

Page 19: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

19

Pilot evaluation of Science Fairsin Europe

Overall results

OVERALLRESULTS

Therespondentsofthequestionnaireswereparticipants–studentsandtheirteachers(tutors)–attheselectedsciencefairs.Intotal,129teachersand464students

completedthequestionnaires.ThebreakdownpersciencefairisindicatedinTable3.

TheeventsinPortugalandtheCzechRepublicwerenationalevents,whiletheeventintheNetherlandswasinternational.TheHungarianevent,whilenational,includedHungarianexpatcommunitieslivinginother

countrieslikeSerbia.Throughouttheremainingof thereport,wewillrefertotheeventsbythecountrywheretheywerehosted,regardlessofthenationality oftheparticipants.

TABLE 3: NUMBER OF RESPONDENTS TO THE QUESTIONNAIRES PER EVENT.

NAME OF EVENT LOCATION STUDENTS TEACHERS

ConcursoJovensCientistaseInvestigadores Portugal 114 37

TheInternationalEnvironment&SustainabilityOlympiad(INESPO)

Netherlands 78 2

Ifjúságitudományosésinnovációstehetségkutatóverseny Hungary 51 16

Středoškolskáodbornáčinnost CzechRepublic 221 74

Totals 464 129

Page 20: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

STUDENTS’ QUESTIONNAIRES

Page 21: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

21

Pilot evaluation of Science Fairsin Europe

Students’ questionnaires

STUDENTS’QUESTIONNAIRES

The464studentswhofilledinthesciencefairsstudent’squestionnaireweremostlybetween15and20yearsofage,withanaverageageofover17year,

Figure1.Asplitof~40%womenversus60%menisalsoobserved(seeFigure2).

Figure1:Ageofrespondents(students).

Figure2:Gendersplitamongtherespondents(students).

0%

5%

10%

15%

20%

25%

30%

>21212019181716151413121110<10

0% 0% 0% 0% 0%1%

6%

12%

24%

29%

23%

3%

1% 1%

Female41%

Male57%

Not answered2%

Page 22: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

22

Pilot evaluation of Science Fairsin Europe

Students’ questionnaires

Althoughmoststudentswerenewcomers(72%oftheparticipantswereparticipatingintherespectivefairfor

thefirsttime)asignificant10%hadbeentothreeormoreofthesefairs(Figure3).

ScienceandTechnologywerethemostcommontopicsoftheparticipants’projects(~80%,seeFigure4)withMathsbeinglessthan5%.Still,sciencetopics

predominatebyafactorofthreeovertechnologyprojectsandafactorofalmostfouroverengineeringprojects.

Figure3:Numberofparticipationsinasciencefairincludingthepresentone(students)*.

Figure4:Majorfocusoftheprojects(students).

Whenthedatainthepreviousfigurearediscriminatedbygender,wefindthatfemalesfavoursciencetopicsovertechnologyorengineeringtopicsbyafactorof10

to1(seeFigure5)whereasmalesonlydo68%morescientificthantechnologicalprojects,and47%moretechnologicalthanengineeringprojects.

Figure5:Majorfocusoftheprojectsdependingongender(students)*.

*Themissing/above%infigureswherethetotalshouldbe100%,isresultofroundingthefigures.

0% 20% 40% 60% 80% 100%

1

2

3

More than 3Overall

72% 17% 6% 4%

0% 20% 40% 60% 80% 100%

Science

Technology

Engineering

MathOverall

61% 20% 14% 5%

0% 20% 40% 60% 80% 100%

Female

Male

Overall

Science

Technology

Engineering

Math

81% 8% 7% 3%

47% 28% 19% 6%

62% 20% 14% 5%

Page 23: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

23

Pilot evaluation of Science Fairsin Europe

Students’ questionnaires

Thefollowingshowstheratiooftheprojecttopicsdiscriminatedbygender.

TABLE 4:

RATIO OF SCIENTIFIC TO TECHNOLOGICAL TO ENGINEERING TO MATHS PROJECTS BY GENDER AND AVERAGE.

GENDER RATIO OF SCIENCE TO TECHNOLOGY TO ENGINEERING TO MATHS PROJECTS

Males 1.7 1.5 3.2 1

Females 10.1 1 2.5 1

Average 3.1 1.4 2.8 1

Theoverallparticipationintheeventsappearstobesplitequallybetweenindividualprojectsandteamprojects,Figure6.

Figure6:Individualversusteamparticipationinthefair(students).

Analysingthenextthreefigurestogether:over80%oftheparticipantstookseveralmonthstodeveloptheproject(Figure7),whilethepresentationparttookatime

thatvariedbetweenafewhours,for17%,andaweekfor37%(Figure8).Morethanhalfoftheparticipantsattendedafter-schoolscienceactivities(Figure9).

Figure7:Timespentondevelopingtheproject(students)*.

0% 20% 40% 60% 80% 100%

individual

Team

Overall

49% 51%

0% 20% 40% 60% 80% 100%

2-10 h

1 day

1 month

several monthsOverall

1% 13% 84%

*Themissing/above%infigureswherethetotalshouldbe100%,isresultofroundingthefigures.

Page 24: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

24

Pilot evaluation of Science Fairsin Europe

Students’ questionnaires

Figure8:Timepreparingthedisplayandthepresentationoftheproject(students).

Figure9:Participationinanafter-schoolscienceclub,programmeoractivity(students).

InFigure10weshowthepercentageofstudentswhoagreedwithdifferentstatementsontheeffectofthesciencefairsonthemselves.Thefullstatementsare:

8.1. Now,Iammoreconfidentatidentifyingproblems. 8.2. Now,Iamabletocommunicatebetter. 8.3. Now,Iambetteratteamwork. 8.4. Now,Iambetteratfindingsolutionsorcomingup withnewideas. 8.5. Ihadfuncompetinginthesciencefair.

8.6. Ilearntalotthroughcompetinginthesciencefair. 8.7. Now,Iknowmoreaboutthescientificmethod. 8.8. Now,Iammoreexcitedaboutscience. 8.9. Now,Icanseebetterhowscienceisrelevanttomy everydaylife. 8.10. Now,Iammorelikelytoconsiderstudying ascientificsubject. 8.11. Now,Iammoreattractedtotheprospectof ascientificcareer.

Figure10:Agreementwithdifferentstatements(students).

0% 20% 40% 60% 80% 100%

1-2 hours

3-5 hours

6-10 hours

1 day

1 week

Overall

12% 24% 10% 17% 37%

0% 20% 40% 60% 80% 100%

Yes

No

Overall

54% 46%

0% 20% 40% 60% 80% 100%

8.11. Now, I am more attracted to the prospect of a scientific career.

8.10. Now, I am more likely to consider studying a scientific subject.

8.9. Now, I can see better how science is relevant to my everyday life.

8.8. Now, I am more excited about science.

8.7. Now, I know more about the scientific method.

8.6. I learnt a lot through competing in the science fair.

8.5. I had fun competing in the science fair.

8.4. Now, I am better at finding solutions or coming up with new ideas.

8.3. Now, I am better at team work.

8.2. Now, I am able to communicate better.

8.1. Now, I am more confident at identifying problems. 79%

85%

62%

87%

87%

89%

86%

77%

76%

70%

66%

Page 25: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

25

Pilot evaluation of Science Fairsin Europe

Students’ questionnaires

Ahighpercentage,over80%,reportedanincreaseinconfidenceinidentifyingproblems,findingsolutionsandknowingmoreabouthowtousethescientificmethod,orevenhadfunparticipatinginthefair.Aslightlysmallerpercentageofthestudents,about60%,haveimprovedinteam-workingcapabilitiesorfeltmorelikelytostudyaSTEMdegree(around65%).

InFigure11weshowthepercentageofstudentswhoselectedeachofthestatementsasadvantagesofcarryingoutasciencefairproject.Themain

advantagesselected(basedonmorethan50%responses)werethepossibilityofworkingonaprojectoftheirown(64%),andhavingachancetoworkwithpeopleoutsidetheschoolwalls(58%).Themostimportantfactoristhechancetoshowtheirworktosocietyatlarge(78%).

Thefactthatnomarksareinvolvedindevelopingaprojectisonlyrelevantfor14%ofthestudents.Veryfewstudents(belowafew%)checkedthe“Idonotknow”or“Noneofthese”options.

Figure11:Advantagesofdoingasciencefairproject(students).

WeseeinFigure12that,ontheotherhand,around48%founditdifficulttogetholdofmaterialsandequipmentneededfortheproject.Atasimilarlevelofdifficultyforthestudents(40-50%)cameworkingwiththevarious

componentsofthescientificmethod(formulatingandtestinghypotheses,analysingdata,etc.).Therelativelyleastdifficultpartwastochooseatopictoworkonfortheproject(36%).

Figure12:Difficultiesencountered(students).

0% 10% 20% 30% 40% 50% 60% 70% 80%

Don't know

None of these

You are part of a mixed group of students from various...

You get to work with people from out of school e.g....

No marks are involved

You get to work with teachers from various subjects

Mistakes allowed

You get the chance to show your work to the outside world

You feel like a real scientist working on a real project

You are the one in charge

You have a procect of your own 63%

30%

42%

78%

19%

27%

13%

58%

17%

2%

2%

0% 10% 20% 30% 40% 50%

10.8 Displaying and communicating your project in a clear and compelling way

10.7 Drawing conclusions from project data

10.6 Analysing project data (including statistical analysis)

10.5 Testing your project hypothesis

10.4 Formulating your project hypothesis

10.3 Finding funding for the project

10.2 Getting hold of the materials and equipment needed for the project

10.1 Coming up with a topic 37%

49%

45%

47%

45%

45%

39%

39%

Page 26: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

26

Pilot evaluation of Science Fairsin Europe

Students’ questionnaires

InTable5weshowthepercentageofstudentswhofoundtheknowledgeandexperiencereceivedfromdifferentsourcesuseful.Wehavehighlightedingreenthosepercentageshigherthan50%.TheInternetprovestobethemostsuccessfulresource,followedbythe

teacherandmentorsinthefield.Familymemberscomein4thplace.About50%findschoolclassesuseful.

Advicefrompreviousfairparticipantsandafter-schoolscienceclubsareimportantforathirdofthestudents.

TABLE 5: USEFULNESS OF VARIOUS SOURCES OF PROJECT SUPPORT (STUDENTS).

OVERALL

11.1Teacher(s) 77%

11.2Mentor/Expertinthefield 75%

11.3Afterschoolscienceclub/programme 32%

11.4Schoolclasses 48%

11.5Formersciencefairparticipantsfromyourschool 34%

11.6Internet/Onlinesupport(webpages,chats,forums) 85%

11.7Familymembers 54%

Only20%oftheparticipantsparticipatedonaverageinaninternshiprelatedtotheproject(seeFigure13).

Figure13:Participationininternshipsrelatedtotheproject(students).

AsseeninTable6,over60%ofthestudentsfeltcreativitywasthemostessentialfactor,followedbycommunicationskills(58%).Thethirdskillwouldbe

intelligence(42%).Collaboration(11%)andleadership(7%)cameinlast.Scientificliteracywasplacedatthesamelevel(about23%)asworkethic.

0% 20% 40% 60% 80% 100%

Yes

No

Overall

21% 79%

Page 27: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

27

Pilot evaluation of Science Fairsin Europe

Students’ questionnaires

TABLE 6: MOST ESSENTIAL CONTRIBUTIONS TO THE PROJECT’S SUCCESS (STUDENTS).

OVERALL

Intelligence 42%

Workethic 24%

Criticalthinking 37%

Communicationskills 58%

Scientificliteracy 22%

Curiosity 32%

Creativity 61%

Collaboration 11%

Leadership 7%

Whendiscriminatedbygender,thedataintheprevioustableshowthatintelligenceprovedtobetheonlyskillwheretherewasalargerthan15%differenceinresponsesbetweengenders.50%ofthemalestudentsconsideritanessentialskill,comparedto32%ofthewomen.

Overall80%ofthestudentscarriedoutthescienceprojectsonavoluntarybasis,withouttheschoolforcingthem(Figure14)butalmost80%wereinfluencedbytheteacher(s),asseeninFigure15).

Figure14:Isitcompulsoryintheschooltodoasciencefairproject?(students).

Furthermore,accordingtoFigure15,additionalfactorsthatinfluencetheparticipationofstudentsintheseprojectsaretheopportunitytodosomethingscience-relatedthatisdifferentfromwhatisdoneinschool(77%)andtheprestigeorrecognitionfromcompetinginthefair(78%),aswellasthe

opportunitytoimproveone’sapplicationtouniversity(78%).

Theinfluenceoffriendsparticipatinginthefairranksatonly30%.Itisfollowedbyparents’influence,whichisstressedin50%ofthecases.

0% 20% 40% 60% 80% 100%

Yes

No

Overall

17% 83%

Page 28: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

28

Pilot evaluation of Science Fairsin Europe

Students’ questionnaires

Figure15:Factorsinfluencingthedecisiontocompeteinthesciencefair(students).

Finally,almost90%oftheparticipantswishtorepeatandrecommendtheexperiencetoafriend(Figure16).

Onlyaround10%wouldnotparticipateinasciencefaireventagaininthefuture.

Figure16:Willingnesstorepeatasciencefairandrecommendtheexperience(students).

0% 10% 20% 30% 40% 50% 60% 70% 80%

14.10 The opportunity to improve my application to...

14.9 The personal attention I received from my...

14.8 The opportunity to do something science-related...

14.7 The opportunity to do a real science project like a...

14.6 My friends were competing in the science fair too

14.5 The prestige / recognition from competing in the fair

14.4 The opportunity to win prizes / awards

14.3 The opportunity to attend our national science fair

14.2 My parents

14.1 My teacher(s) 78%

50%

68%

57%

78%

30%

67%

77%

66%

71%

0% 20% 40% 60% 80% 100%

15.B Would you recommend competing in the fair to a friend?

15. Would you like to compete in a science fair again- irrespective of whether it's compulsory your school or not? 89%

91%

Page 29: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

29

Pilot evaluation of Science Fairsin Europe

Students’ questionnaires

Takingintoaccountonlythemajorcharacteristicsindicatedinthestudents’repliestothequestionnaires,onefindstheaveragepicturedepictedinFigure17.This

providesasimplehelpingtoolwhichcontainstheaveragestatisticalprofilesofparticipatingstudents.

Conclusion: Average students’ profile

Figure17:Averageparticipantprofile(students).

Thediagramabovecanbeeasilyexpandedtomakeitascomprehensiveasdesired,byincludingthemostsalientanswerstootheritemsinthequestionnaires.Inthisway,

cross-year,cross-countryandcross-faircomparisonsarequiteeasytomake.

72%newcomers to a fair

54%participate in after-school science activities

61%volunteer to do a Science project

50/50%Working alone or in a team

60/40%

Older than 17

Male - Femal

Page 30: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

TEACHERS’ QUESTIONNAIRES

Page 31: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

31

Pilot evaluation of Science Fairsin Europe

Teachers’ questionnaires

TEACHERS’QUESTIONNAIRES

Theanalysisofthequestionnairesfilledin bytheteachersparticipatinginthefivescience fairsthatarereportedonhereshowsthefollowing results:

First,femaleteacherspredominateslightly,Figure18,andtheyhavealongteachingexperience,over10yearsfor36%ofthemandover20yearsfor40%.Only5%havelessthan5years’teachingexperience(Figure19).

Figure18:Gendersplitamongparticipants(teachers).

Figure19:Yearsofteachingexperience(teachers)*.

Female54%

Male46%

0% 20% 40% 60% 80% 100%

<5 years

5-10 years

11-20 years

>20 years

5% 18% 36% 40%

*Themissing/above%infigureswherethetotalshouldbe100%,isresultofroundingthefigures.

Page 32: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

32

Pilot evaluation of Science Fairsin Europe

Teachers’ questionnaires

Teachersarealsowellqualified,withalmost75%holdingamaster’sdegree,Figure20.Thereiseven

asignificant8%ofPhDholders.

Figure20:Highestdegreeheld(teachers).

Morethan66%oftheparticipatingteachersworkinschoolsthathaveparticipatedinmorethanthree

sciencefairs,Figure21,andforonlyabout20%oftheschoolsisthistheirfirstfair.

Figure21:Numberofschoolparticipationsinthefair(teachers)*.

Almost50%oftheteachersareaccompanyingbetween1and4students,Figure22,whileatotal

of75%bringupto10.Aquarteroftheteachersarebringinginbetween11and50students.

Figure22:Numberofstudentsfromtheirschoolswhoarecompeting(teachers).

0% 20% 40% 60% 80% 100%

School leaving certificate

Bachelor's degree

Master's degree

DoctorateOverall

4% 14% 74% 8%

0% 20% 40% 60% 80% 100%

1

2

3

More than 3

Don't know

Overall

19% 8% 7% 66% 1%

0% 20% 40% 60% 80% 100%

1-4

5-10

11-50

More than 50

Don't know

Overall

48% 26% 23% 1% 2%

*Themissing/above%infigureswherethetotalshouldbe100%,isresultofroundingthefigures.

Page 33: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

33

Pilot evaluation of Science Fairsin Europe

Teachers’ questionnaires

WeseeinFigure23thatformorethan75%oftheteachersthiswasnottheirfirstexperiencewithasciencefair.

Figure23:Firsttimeparticipantsinfairs(Yes)versusrepeatparticipants(No)(teachers).

AccordingtoFigure24thelargestgroupareScienceteachers,at38%,andthenextnumerousgroupteaches

Engineering(19%)whileTechnologycomesthirdat12%.Mathsteachersaccountfor6%.

Figure24:Mainsubjecttaught(teachers)*.

Accordingtotheaccompanyingteachers,Figure25,theratioofSciencetoTechnologytoEngineeringprojectspresentedis2.25:1.1:1.Mathsprojectsare

justafew(1%).Animportant25%oftheteachersareaccompanyingprojectsfromacombinationoftheareasmentionedabove.

Figure25:Mainfocusoftheprojectssupervised(teachers).

0% 20% 40% 60% 80% 100%

Yes

No

Overall

23% 77%

0% 20% 40% 60% 80% 100%

Science (Phisics, Chemistry, Biology)

Technology

Engineering

Math

A combination of the above

Overall

38% 17% 19% 1% 25%

*Themissing/above%infigureswherethetotalshouldbe100%,isresultofroundingthefigures.

0% 20% 40% 60% 80% 100%

Science (Phisics, Chemistry, Biology)

Technology

Engineering

Math

Other (e.g.: astronomy, informatics, languages)

Overall

38% 12% 19% 6% 24%

Page 34: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

34

Pilot evaluation of Science Fairsin Europe

Teachers’ questionnaires

Around45%ofthestudentsparticipatedinafter-schoolactivities,accordingtotheteachers,Figure26,whileover

25%didnotandin21%ofthecasestheseextracurricularoptionswerenotavailableintheirschools.

Figure26:Schoolswherethestudentsparticipatedinafter-schoolscienceclubs,etc.(teachers).

Itisnotcompulsoryforteachersinagivenschooltoparticipateinafairprojectin99%ofthecases,Figure27,

andtheschooldesignatesacoordinatorin60%ofthecases.

Figure27:Schoolswhere8)itiscompulsorytoparticipateinsciencefairsand9)thereisapersonincharge oftheparticipation(teachers).

ThereisaclearlinkbetweenthenationalSTEMcurriculumandthecontentsofthefairprojectsin

morethan40%ofthecases,Figure28.

Figure28:Itwaseasytointegratethecompetitionwiththecurricula(teachers).

0% 20% 40% 60% 80% 100%

Yes

No

There is no after school science club / programme at our school

Don't know

Overall

45% 27% 21% 7%

0% 10% 20% 30% 40% 50% 60%

9. Is there in your school a person in charge of coordinating the participation in the science fair?

8. Is it compulsory in your school to do a science fair project? 1%

58%

0% 10% 20% 30% 40% 50%

Overall 42%

Page 35: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

35

Pilot evaluation of Science Fairsin Europe

Teachers’ questionnaires

Theteacherswerealsoaskedtoratevariousaspectsofstudents’participationinthefair.AccordingtoFigure29formorethan90%oftheteacherstheirstudentslearntalotfromparticipatingandknowmoreaboutthe

scientificmethod.Ontheotherhand,forabout53to60%,participationinthefairhashadabigimpactonthewaytheyteachandrewardedtheirstudentsforexcellenceinSTEM.

11. To what extent do you agree or disagree with each of the following statements now that you and your students have participated in the science fair?

Figure29:Impactofparticipatinginthesciencefairsonbothteachersandstudents(teachers).

62%

0% 20% 40% 60% 80% 100%

11.16 Now, those who participated in the science fair are more likely to consider studying a scientific subject

11.15 Now, those who participated in the science fair are better able to see how science is relevant to their everyday life

11.14 Now, those who participated in the science fair are more excited about science

11.13 Now, those who participated in the science fair are better communicators

11.12 Now, those who participated in the science fair are betterat teamwork

11.11 Now, those who participated in the science fair are better able to apply the scientific method

11.10 Now, those who participated in the science fair know more about the scientific method

11.9 My students learnt a lot through participating in the science fair

11.8 My students had fun participating in the science fair

11.7 Participating in the science fair has had a big impact on the way I teach

11.6 I as a teacher learnt a lot through participating in the science fair

11.5 Participating in the science fair motivated my students to pursue science, technology, engineering and maths careers

11.4 Participating in the science fair promoted project-based science in my school

11.3 Participating in the science fair promoted scientific inquiryin my school

11.2 Participating in the science fair rewarded my studentsfor excellence in STEM

11.1 Participating in the science fair encouraged my studentsto pursue excellence in STEM 77%

53%

62%

61%

72%

81%

60%

75%

94%

90%

87%

77%

89%

81%

83%

80%

Page 36: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

36

Pilot evaluation of Science Fairsin Europe

Teachers’ questionnaires

Theadvantagesofpartakinginasciencefairare,fortheteachers,thattheyinteractwiththeirstudentsdifferently(74%)andstudentsaremoremotivated(69%),

Figure30.Ontheotherhand,itisagoodopportunityforcollaborationacrossschoolyearsfor23%,andstudentsstartbehavinglikerealscientistsfor34%oftheteachers.

12. In your opinion, what are the advantages of doing a science fair project over regular science classes?

Figure30:Advantagesofdoingasciencefairprojectoveraregularscienceclass(teachers).

Theteachersmayexperiencesomedifficultiesinprovidingassistancetotheparticipatingstudents.Themainproblemsarefindingfundingandgettingstudentsonboard,ormaterials,andgettingexperts(about60%),Figure31,

butsupervisingstudentsinareasotherthantheteachers’areasofexpertiseisaproblemforjust26%.Only4%oftheteachersfinddifficultiesinfillinginapplicationformsforthecompetition.

Figure31:Difficultiesencounteredinpreparingaprojectandparticipatinginthefair(teachers).

0% 20% 40% 60% 80% 100%

Don’t know

This is a good opportunity for collaboration across school years

This is a good opportunity to engage with the outside world

This is a good opportunity to work in an interdisciplinary approach

Students start behaving like actual scientists

As a teacher, I get to interact with students in a different context

Students are the leaders of their learning

Students are more motivated 69%

43%

74%

34%

56%

65%

23%

1%

0% 20% 40% 60% 80%

13.6 Supervising students whose projects were not in my area of expertise

13.5 Filling in application forms for the competition

13.4 Finding funding for the project

13.3 Getting hold of the materials and equipment needed for the project

13.2 Getting mentors / experts on the topic on board

13.1 Getting students on board 58%

50%

57%

60%

4%

26%

Page 37: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

37

Pilot evaluation of Science Fairsin Europe

Teachers’ questionnaires

Theteachersalsogavetheiropinionaboutwhatdifficultiesthestudentsmetinpreparingtheprojectsforthefair.Intheiropinion,Figure32,ontopofthesametwoitemsmentionedinthepreviousfigure(findingfundingandthematerialsfortheproject,bothatabout55%),teachersadd

thedifficultyforstudentstoapplythescientificmethod(at55%).Theleastdifficultylayinfillingintheformtoparticipateintheevent(difficultforonly8%),followedbythedifficultyforstudentstocommunicatetheirprojectsclearly(40%).

Figure32:Difficultiesstudentsfoundinactivitiesincarryingoutaprojectandparticipatinginafair(teachers).

Now,concerningsupportforparticipationinthefair,theschoolprovidessupportforalmost75%oftheteachers,Figure33,withsupportfromcolleagues,parentsandexpertscomingsecondatabout55%

average.Universitiesranknextat42%.Leastsupportcomesfromlocalauthorities,localmediaoreventhenationalMinistryofEducation(alloftheminthe15-22%range).

Figure33:Supportobtainedfromdifferentpeopleororganizations(teachers).

0% 20% 40% 60%

14.6 Filling in application forms for the competition

14.5 Displaying and communicating their project in a clear and compelling way

14.4 Analysing project data (including statistical analysis)

14.3 Applying the scientific method

14.2 Finding funding for the project

14.1 Getting hold of the materials and equipment needed for the project 54%

56%

55%

51%

40%

8%

0% 20% 40% 60% 80%

15.10 Universities / Research institutes / Partnerships

15.9 The Intel team

15.8 Your national Ministry of Education

15.7 Local/national science experts

15.6 Your local media

15.5 Your local authority

15.4 Mentors / Experts

15.3 Parents

15.2 Your colleagues

15.1 The senior management of your school 71%

53%

50%

57%

15%

22%

32%

17%

9%

42%

Page 38: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

38

Pilot evaluation of Science Fairsin Europe

Teachers’ questionnaires

Asregardstheusefulnessofthesupport,mentorsorscienceexpertsaswellastheschoolmanagementwereconsideredtobethemosteffective,withabigdifference

(60and46%,respectively),Figure34.Colleaguesandparentscomenext,athalftherelativeimportance. Theimportanceofothersourcesforsupportisbelow5%.

Figure34:Organizationsorpeoplethatwereinstrumentaltocarryingoutthestudents’projects(teachers).

Aquestionlookedintothepersonaltraitsoftheparticipatingstudentsthat,intheopinionoftheaccompanyingteachers,contributedmosttothesuccessoftheprojectundertaken.Teachersrankatabout50%ormorecreativity,

communicationskillsandintelligence,inthatorder,Figure35.Factorslikecollaboration,scientificliteracyandworkethicareconsideredoflessimportance,selectedinabout25%oftheanswers.Leadershipisnotconsideredatall.

Figure35:Essentialissuescontributingtothesuccessofstudents’sciencefairprojects(teachers).

Itisimportanttoknowthereasonswhyteachersmakethecommitmenttoorientandevenhelpstudentstoparticipateinthefair.AccordingtothedatainFigure36“Mystudents”standsoutat94%,andthe

prestigethatcomesfromthecompetition(74%).Leastimportanceisattributedtotheprospectofpromotion(29%)followedbytheeffectoftheirowncolleagues(43%).

0% 15% 30% 45% 60%

Don’t know

The Intel team

Your national Ministry of Education

Mentors / science experts

Your local media

Your local authority

Parents

Your colleagues

The senior management of your school 59%

30%

27%

4%

2%

46%

5%

2%

9%

0% 15% 30% 45% 60%

Leadership

Collaboration

Creativity

Curiosity

Scientific literacy

Communication skills

Critical thinking

Work ethic

Intelligence 48%

27%

37%

53%

25%

31%

58%

21%

0%

Page 39: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

39

Pilot evaluation of Science Fairsin Europe

Teachers’ questionnaires

Figure36:Factorsorpeoplethatinfluencedthedecisiontoleadparticipantsinthefairandmadethetaskeasier(teachers).

RegardingtheextenttowhichsciencefairscontributetowardsextendingtheuseofInquiry-BasedScienceandMathematicsEducation(IBSME),asFigure37shows,participationinscience

fairsresultsina7percentagepointincreaseintheuseofinquiry-basedteaching,since28%oftheteachersalreadyuseIBSMEintheirclassesbeforeattendingthefair.

Figure37:UseofIBSMEbeforethesciencefairandafterwards(teachers).

Itisapositiveexperiencefortheteacherstosupervisestudents’projectsforthefaircompetition:86%ofthem

woulddoitagaininthefuture,Figure38.

Figure38:Wouldyousuperviseaprojectforasciencefairagain?(teachers).

0% 20% 40% 60% 80% 100%

18.8 The access to labs from other organizations

18.7 The prospect of promotion

18.6 The prestige / recognition from competing in the fair

18.5 The opportunity to win prizes / awards

18.4 The opportunity to attend our national science fair

18.3 My colleagues

18.2 My students

18.1 My school principal 55%

94%

43%

70%

60%

74%

29%

47%

0% 5% 10% 15% 20% 25% 30% 35%

21. How much do you use inquiry-based teaching in your classes nowadays?

20. Did you use inquiry-based teaching in your classes before the first time you participated in the science fairs? 28%

35%

0% 20% 40% 60% 80% 100%

Overall 86%

Page 40: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

40

Pilot evaluation of Science Fairsin Europe

Teachers’ questionnaires

RegardingthedatainFigure39,askedwhethertheywouldbeinterestedinbecomingaScienceFairAmbassador,42%agree,26%donot,and32%didnotknowyetatthetimeofthefair.Thereasonsforthenegativeanswersweresplitequallybetween:

• Iwouldliketotryoutotheractivitiesinthefuture.• Ittookuptoomuchofmytime.• Ididn’tenjoyitasmuchasIthoughtIwould.• Ireceivedtoolittlesupport.• Itinvolvedtoomuchpaperwork.

Figure39:InterestinbecomingaScienceFairAmbassadorfortheircountry(teachers).

Takingintoaccountonlythemajorcharacteristicsindicatedintheteachers’repliestothequestionnaires,onefindstheaveragepicturedepictedinFigure40.

ThesamecommentsmadebeforeandafterFigure17,concerningtheaveragestudents’profile,alsoholdhere.

Conclusion: Average teachers’ profile

Figure40:Averageparticipantprofile(teachers).

0% 20% 40% 60% 80% 100%

Yes

No

Don't know

26% 42% 32%

2+2=

?

Female - Male54/46%

66%schools participated in 3+ fairs

74%hold Master’s degrees

58%schools have

a Fair coordinator

38%practice IBSME

86%would repeat a Fair,

as Ambassadors26%

77%teachers repeat Fair

48%bring 1-4 students

Page 41: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

ANALYSIS AND CONCLUSIONS

Page 42: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

42

Pilot evaluation of Science Fairsin Europe

Analysis and conclusions

ANALYSISANDCONCLUSIONS

Themainconclusionsofthepreviousoverallanalysisofthesciencefairswillbesplitinthefollowingsectionintostudents’ and teachers’ results.

Main students’ results

Newcomers: Moststudentsarefirst-timersatafair(72%).• Asignificant10%hadparticipatedinthreeormoreofthesefairs.

◊ Thisshowsanimportantloyaltyeffect.

Gender differences in the project areas:Malesfocusonscienceandengineering&technologyinequalmeasure.• Femalesfavourscientificprojectsoverallotherareas.

◊ Thislargedifferencedeservesfurtherinvestigation.

Balance: Thereisagoodnaturalbalanceinparticipationbetweenindividualprojectsandteamprojects.

◊ Thisagreeswiththespiritofthesciencefairs.

Hard work: Theparticipantsworkedhardfortheprojectstheypresented.• Forover80%ittookseveralmonthstodeveloptheprojectwhilethepresentationpartvariedbetweenafewhoursandaweek.

• Andmorethanhalfoftheparticipantsattendedafter-schoolscienceactivities.

◊ Thesefiguresshowahighdegreeofstudentcommitmenttotheprojects,sustainedovermonths,whichisanevidenteducationalandpersonaldevelopmentvaluethatoriginatesfromthesciencefaircompetition.

Greater confidence: Over80%noticedanincreaseinconfidenceinidentifyingproblems,findingsolutionsandusingthescientificmethodorevenhadfunparticipatinginthefair.• Aslightlysmallerpercentageofthestudents,about60%,haveimprovedinteam-workingcapabilitiesorfeltmorelikelytostudyaSTEMdegree(around65%).

◊ Itwouldbeofinteresttoascertainwhetherthemissing35%wereinitiallyconvincedtodoaSTEMdegree,ortheworkinthesciencefairprojectsomehowdiscouragedthem.Inthefirstcase,itmaybethatstudentsareagainststudyingaSTEMdegreeorwantedtodoitfromthestart,andparticipationinthesciencefairdidnotcontributetothis(astheywerealreadyconvincedinitially).Inthesecondcase,i.e.someonewhoisinterestedinaSTEMdegreebutafterparticipatingfeelsresearchisnotforthemandthereforetheydo

notwanttopursueaSTEMdegreeanymore,itmaybethattheyarenotawareofothercareeroptionsinSTEMthanresearch,aknowledgewhichisprovidedbyEUNprojectslikeinGenious,ideasfromwhichmightbeconsideredforinclusioninthesciencefairsprogramme.

Skills:Havingfunduringtheprojectandhavingimprovedtheircommunicationskillsstandoutaspossibleoutcomesofparticipationintheprogramme.• Themostimportantfactoristhechancetoshowtheirworktothesocietyatlarge.Otheradvantagesofcarryingoutasciencefairprojectwerethepossibilityofworkingonaprojectoftheirownandhavingachancetoworkwithpeopleoutsidetheschoolwalls.

◊ Thisshowsthatstudentsfindmanypossiblevaluesindevelopingtheseprojectsandthatgainsotherthanscientificskillsarealsoinvolved.

Difficulties: 40%ofthestudentsfounditdifficulttogetholdofmaterialsandequipmentneededfortheprojectandincarryingoutthedifferentstepsinthescientificmethod.• Therelativelyleastdifficultpartwastochooseatopictoworkon.

• Applyingthevariouscomponentsofthescientificmethod(formulatingandtestinghypothesis,analysingdata,etc.)comeatasimilarlevelofdifficultytogettingmaterialsfortheproject.

◊ Obviously,facingandovercomingthesedifficultiesresultsin“anincreaseinconfidence[in]howtousethescientificmethod”.

◊ Maybeinfuturestudiesonemightcheckhowmanyfailedattemptsstudentshavemade,thatis,whethertheprojectdevelopedwastheonefinallypresented.

Internships:20%oftheparticipantsonaveragetookpartinaninternshiprelatedtotheproject.

◊ Thisfactreinforcesthecommitmentfactormentionedabove:studentsparticipatewillinglyandaremotivatedtospendtimeontheprojectforaperiodofmonths,withouttheneedforamoreformallinklikeaninternship.

Essential factors for success: Oneofthemostinterestingresultsfoundwastheitemsparticipantsfoundessentialtothesuccessoftheirproject.Over60%oftheparticipantsfeltcreativitywasthemostessentialfactor,followedbycommunicationskills.

◊ Thethirdskillwouldbeintelligence.Collaborationandleadershipcameinlast.Scientificliteracycameoutatthesamelevel(about23%)asworkethic.

Page 43: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

43

Pilot evaluation of Science Fairsin Europe

Analysis and conclusions

◊ Termslikeleadership,collaborationandworkethicmaynotbetypicalingredientsofstandardschooleducation,andbelongmoretothestandardworkplaceculture.

Gender effect in personal traits:Whendiscriminatedbygender,thedatashowthatintelligenceprovedtobetheonlyskillwheretherewasalargerthan15%genderdifference.• 50%ofthemalestudentsconsideritanessentialskill,comparedtothe32%ofwomen.

◊ Thisfactdeservesfurtherconsideration.

Voluntary participation: Overall80%ofthestudentscarriedoutthescienceprojectsonavoluntarybasis.• Almost80%wereinfluencedbytheteacher(s).

◊ Theseresultsshowtheperhapsexpectedinfluenceofteachersinorientingstudents’activities,especiallytheextracurricularprojects.

Satisfaction: Almost90%oftheparticipantswishtorepeatandtorecommendtheexperiencetoafriend.

◊ Thismakesthefairsaneducationalsuccess.

Main teachers’ results

Slight gender predominance:Femaleteacherspredominateslightly.• Participantshavealongteachingexperience,above10yearsfor36%ofthemandover20yearsfor40%.Only5%havelessthan5years’teachingexperience.

◊ Contrarytothestudents’relativemalepredominance,femaleteachersareslightlymorepresentthanmalecounterparts.

Repeat school participation: Morethan66%oftheparticipatingteachersworkinschoolsthathaveparticipatedinmorethanthreesciencefairs.• Foronlyabout20%oftheschoolisthistheirfirstfair.

◊ Thisdemonstratesthegoodreceptionthatthefairsmeetwithamongschools.

Even higher teacher repetition: Teachersareonaveragemoreassiduousparticipantsinthefairsthanstudents:formorethanfor75%ofthemthiswasnottheirfirstexperiencewiththesciencefair.

◊ Thisis,ofcourse,onlynaturalasteachersstayintheschoolformoreyears.

Students/teacher: Teachersbringmanystudentstothefair:almost50%ofthemaccompanybetween1and4studentswhileatotalof75%bringupto10.• Andafourthoftheteachersarebringinginbetween11and50students.

◊ Thelargeratioofstudents/teachershowhighteachercommitmentwiththefairs.

Teachers’ areas:MostareScienceteachers,at38%,andthenextnumerousgroupteachEngineering(19%).• NextcomeTechnology(12%)andMaths(6%).

◊ Thesepercentagescorrelatewiththetopicspresentedbythestudents.

Supervised projects:TheratioofSciencetoTechnologytoEngineeringprojectspresentedis2.25:1.1:1.• Animportant25%oftheteachersareaccompanyingprojectsfromacombinationoftopics.

◊ Thisstressesthevariedcharacteroftheschools’activities,whichpromoteprojectsinallpossiblefields.

Out-of-school activities:Almost50%ofthestudentsparticipatedinextracurricularactivities,accordingtotheteachers.• Over25%didnotandin21%ofcasestheseextracurricularoptionswerenotavailable.

◊ Thefiguresindicatethatanimportantproportionofstudentsmakeextraeffortstoparticipateinthesefairs,aswehavealreadyfoundinthestudents’data.Itwouldbeinterestingtocheckwhetherthisproportiondoesindeedincreaseinthenextyears.

Voluntary tutoring and support, although it is not compulsoryforteachersinagivenschooltoparticipateinafairprojectin99%ofthecases.• Forparticipatingteacherstheschooldesignatesacoordinatorin60%ofthecases.

◊ Thisindicatesafeltneedaswellasaguaranteeofsuccessforprojectdevelopment.

Curriculum and project: ThereisaclearlinkbetweenthenationalSTEMcurriculumandthecontentsofthefairprojectsinslightlymorethan40%ofthecases.

◊ Thisshowsthateithertheprojectshaveabroaderaimthanthecurriculum,orthatitisdifficulttodevelopsufficientlyinterestingprojectswhilekeepinginsidethecurriculumboundaries.

Learning from participation: Studentslearnalot(94%),theylearntocommunicatebetter(89%)andmoreaboutthescientificmethod(90%).• Also,rewardingstudentsforexcellenceinSTEM(53%)andimpactingthewaytheyteach(60%).

◊ Thesensationofgettingimportantpersonalandprofessionalgainsfromparticipatinginthefairisimportantforteachersandstudents.

Advantages of fairs over classes: Differentwayofinteractingwithstudents(74%)andthestudents’motivation(69%).• Opportunitiestoengagewiththeoutsideworldareimportant(65%).

◊ Theseareskillsthatarenotusuallyassociatedwithregularclassroomactivities.

Page 44: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

44

Pilot evaluation of Science Fairsin Europe

Analysis and conclusions

Difficulties encountered: Getting students on board is difficultfor58%oftheteachers.• Alsofindingfunding,materials(about58%).

◊ Difficultiesingettingstudentsandfundingrateequally,butarenotinthe80-90%rangelikeotheranswersinthe questionnaire.

Difficulties students encountered:Gettingfunding,applyingthescientificmethodandthematerialsfortheprojectallratesimilarlyatabout55%.• Supervisingstudentsoutsidetheteacher’sareaisonlydifficultfor25%.Fillingintheapplicationformforthefairisdifficultfor8%.

◊ Again,onlyabout50%ofthestudentsorlesshavesomedifficulties.

School support:Participationinthefairgetsschoolsupportin75%ofthecases.• Supportfromcolleagues,parentsandexpertscomesecondatabout55%average.Universitiesranknextat42%.Leastsupportcomesfromlocalauthorities,localmediaoreventhenationalMinistryofEducation.Thefactoris2-3timeslesssupport.

◊ Thelastitemsmayrequiremoreeffortfromthefairandschoolmanagement.

Usefulness of the support:Mentorsorscienceexpertsaswellastheschoolmanagementwereconsideredthemosteffective,withabigdifference(60and46%,respectively).• Colleaguesandparentscomenextbutathalftherelativeimportance.Theimportanceoftheothersourcesforsupportisalmostinsignificant.

◊ ThesupportfromtheIntelteamshouldprobablyincreaseinthefuture.

Students’ personal traits and success: intheopinionoftheaccompanyingteachers,creativity,communicationskillsandintelligence,inthatorder,standoutat50%ormore.• Perhapssurprisingly,students’factorslikecollaboration,scientificliteracyandworkethicareconsideredoflessimportanceinabout25%oftheanswers.

◊ Strikingly,thecapacityofleadershipgotstrictlyzeroresponses.Itwouldbeinterestingtoanalysetheseresultsinmoredetailinthefuture.

Reasons for teachers’ commitment: “Mystudents”predominates(at94%)andtheprestigethatcomesfromthecompetition(74%).• Leastimportanceisattributedtotheprospectofpromotion(29%)followedbytheeffectsofcolleagues(43%).

◊ Theseanswersmayhelpindesigningfutureannouncementsandrulesforthedevelopmentoffairs.

Methodology changes:Participationinsciencefairsresultsina7%increaseintheuseofinquiry-basedteachingwith35%ofthemusingIBSMEnowadays.• However,thedatashowsthatstillfor2/3oftheparticipatingteacherstheuseofIBSMEisstillanunknownfactorintheteachingequation.

◊ Thesciencefairinitiativehastheambitiontohelpimproveschooleducationasregardsthenecessarytransformationandupdatingofteachers’methodologies,sothatthestudents’educationisimprovedintermsoftransferableknowledge,skillsandcompetences.OneofthekeyinnovatingTeachingandLearningmethodologies(T&L)isInquiry-BasedScienceandMathematicsEducation(IBSME),anditisinterestingtochecktheextenttowhichsciencefairscontributetowardsextendingitsuse.

Teachers’ satisfaction: For86%oftheteachersitisaquitepositiveexperiencetosupervisestudents’projectsforthefaircompetition.• Theywoulddoitagaininthefuture.

◊ Thisisaremarkableresultofthewholeprogrammeand,inconjunctionwithstudents’opinions,onemayconcludethatbothforteachersandstudents,participatinginthesciencefairsisanenjoyableandpositiveeducationalexperience.

Teachers’ further commitment: 26%oftheteacherswouldbeinterestedinbecomingaScienceFairAmbassador.• 32%wouldhavetogiveitfurtherconsideration.

◊ Again,thisresultspeaksofaremarkablesuccessofthesciencefairsamongtheteachers,whowouldconsideramajorinvolvementinpossibly58%ofcases.

Page 45: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

OPEN DISCUSSION AND FUTURE WORK

Page 46: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

46

Pilot evaluation of Science Fairsin Europe

Open Discussion and Future work

OPENDISCUSSIONANDFUTUREWORK

InadditiontotheresultshighlightedintheAnalysis andConclusionssection,theauthorsnotedanumber ofissuesduringtheanalysisstillopentointerpretation.Manyoftheseissuesopenthedoortofurtherdiscussionsandcouldbeexploredinfurtherstudies.

Forexample,inTable5wefoundthat85%of thestudentsfoundtheInternet/Onlinesupportusefulforthedevelopmentoftheirprojects,whileonly20%ofthemhadparticipatedonaninternship(Figure13).Itisinterestingtonotethetechnologypeakispresentevenwithsciencefairsprojects.Resultsfrompreviousstudiesindicatealmosttwothirdsoffinalistspointedouthavingamentoras“ImportanttoVeryImportant”(p.15)totheirsciencefairparticipation(Rilleroetal,2005).Thestrongresultsoftoday’sstudentparticipantsuseofinternetresourcesmayindicatethatthewaysamentoringrelationshipwasonceunderstoodarebeingreplaced bythelargenumberofonlineresources,communities andaccesstolesspersonalexpertrelationships.

Additionally,inTable6wefindthatCreativityisconsideredthemostessentialcontributiontothesuccessofaprojectbythestudents.Itisimportanttorememberthesearesciencestudents,carryingoutresearchprojectsforsciencefairs.Oftenthestigmaattached tosciencefairsisoneofaneventattractivetothetypeofstudentmostoftenfoundinthelibrarysurrounded bybooks.Incontrast,thestudentparticipantssuggestthatwhatisreallyneededareattributesnotoftenconsideredwhenthinkingaboutcreativity,suchasimagination,criticalthinkingskillsandnon-traditionalapproachestoproblemsolving.

Whilewehavementionedtheincreaseduseoftheinternetintheimplementationoftheprojects,teacherscontinue tobethekeyfactorforstudents’participationinthesefairs.Teacherscannotbereplacedbycomputersandplay acriticalroleinchallengingthestudentsandprovidingthemwithresources(seeFigure15).Ontheotherhand,parent’sinfluenceappearstobelessfeltbythestudents.

Representingtheyearsofteachingexperience,Figure19,illustratesthat40%ofcurrentteachershaveover20years’inthefield,andalmost80%over10years.Thisreallyindicatesnewteachersaregoingtobeinneedofsupportinordertogetthemtojointhesciencefairswiththeirstudents.

Figures29:Impactofparticipatinginthesciencefairs onbothteachersandstudents(teachers)and30:Advantagesofdoingasciencefairprojectoveraregularscienceclass(teachers)highlightedtwoimportantissues.Ononehand,therelativelylowresponsesto“Participatinginthesciencefairpromotedscientificinquiryinmyschool”and“Now,thosewhoparticipatedinthesciencefairknowmoreaboutthescientificmethod”(~60%)compared tothealmost90%to“Now,thosewhoparticipated inthesciencefairarebetterabletoapplythescientificmethod”showsthatwhileasignificantimpactisperceivedinthestudent’sabilities,itappearstostayatanindividuallevelwithlimitedimpactintheschoolasawhole.Thisissimilarto,ontheotherhand,theteachers’impressionwith“ParticipatinginthesciencefairhashadabigimpactonthewayIteach”onlytruefor60%oftherespondentscomparedto“Asateacher,Igettointeractwithstudents inadifferentcontext”withalmost75%positiverespondents.Overallitappearstheeffectsremainatstudents/teachersparticipatinginthefairanddoesnotgettransferredtotheschooloreventheclassroomlevel.

Evenmoredisconcertingisfinallytheresultfound inFigure33,whichshowshowlittleinvolvementappears tocomefromthelocalmediaandauthorities(including theMinistriesofeducation).

Overallitappearsthatwhiletheimpactofthefairsisclearforthoseparticipatinginthefairs,furtheractionsshould bemadeto1)spreadthesebeneficialeffectsto theschoolingeneral;2)gettheinvolvementofotherpartsofthesocietywhicharecurrentlyunaware oruninterestedinthefairs,astheywouldafterallreapthebenefitsofhavingmoreSTEMstudentsandmoreSTEMliteratecitizensasawhole.

Rillero,P.,Zambo,R.&Haas,N.(2005).IntelInternationalScienceandEngineeringFair2005EvaluationReport. Retrievedfrom:http://download.intel.com/education/EvidenceOfImpact/ISEF-2005-Report.pdf

Page 47: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

APPENDIX

SCIENCE FAIRS – TEACHERS’ QUESTIONNAIRE

Page 48: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

48

Pilot evaluation of Science Fairsin Europe

Appendix

APPENDIXSCIENCEFAIRS–TEACHERS’QUESTIONNAIRE

SCIENCE FAIRS – TEACHERS’ QUESTIONNAIRE

Q QUESTIONS ANSWER CATEGORIES

1 How many times has your school participated in the fair including this year?

123Morethan3Don’tknow

2 What is the approximate number of students from your school who competed in the science fair this year?

1–45-1011–50Morethan50Don’tknow

3 Is this the first year that you as a teacher participate in your national science fair?

YesNo

4 What subject do you teach primarily? Science(Physics,Chemistry,Biology)TechnologyEngineeringMathsOther,pleasespecify

5 Have many years have you been teaching? <5years5-10years11-20years>20years

6 What is the major focus of the student projects you supervised?

Science(Physics,Chemistry,Biology)TechnologyEngineeringMathsAcombinationoftheaboveOther,pleasespecify

7 Did students who competed in the science fair participate in an after-school science club / programme / activity?

YesNoThereisnoafter-schoolscienceclub/programmeat our schoolDon’tknow

8 Is it compulsory in your school to do a science fair project? YesNo

9 Is there in your school a person in charge of coordinating the participation in the science fair?

YesNo

Page 49: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

49

Pilot evaluation of Science Fairsin Europe

Science fairs – teachers’ questionnaire

Q QUESTIONS ANSWER CATEGORIES

10 How easy is it to link competing in the national science fair to your national STEM curriculum?

1–Verydifficult234–VeryeasyThereisnonationalSTEMcurriculum

11 To what extent do you agree or disagree with each of the following statements now that you and your students have participated in the science fair?

IfullyagreeIratheragreeIratherdisagreeIfullydisagreeDon’tknow/Notapplicable

11.1 Participatinginthesciencefairencouragedmystudentstopursueexcellenceinscience,technology,engineeringandmaths

11.2 Participatinginthesciencefairrewardedmystudentsforexcellenceinscience,technology,engineeringandmaths

11.4 Participatinginthesciencefairpromotedproject-basedscienceinmyschool

11.5 Participatinginthesciencefairmotivatedmystudentstopursuescience,technology,engineeringandmathscareers

11.6 Iasateacherlearntalotthroughparticipatinginthesciencefair

11.7 ParticipatinginthesciencefairhashadabigimpactonthewayIteach

11.8 Mystudentshadfunparticipatinginthesciencefair

11.9 Mystudentslearntalotthroughparticipatinginthesciencefair

11.10 Now,thosewhoparticipatedinthesciencefairknowmoreaboutthescientificmethod

11.11 Now,thosewhoparticipatedinthesciencefairarebetterabletoapplythescientificmethod

11.12 Now,thosewhoparticipatedinthesciencefairarebetteratteamwork

11.13 Now,thosewhoparticipatedinthesciencefairarebettercommunicators

11.14 Now,thosewhoparticipatedinthesciencefairaremoreexcitedaboutscience

11.15 Now,thosewhoparticipatedinthesciencefairarebetterabletoseehowscienceisrelevanttotheireverydaylife

11.16 Now,thosewhoparticipatedinthesciencefairaremorelikelytoconsiderstudyingascientificsubject

Page 50: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

50

Pilot evaluation of Science Fairsin Europe

Appendix

Q QUESTIONS ANSWER CATEGORIES

12 In your opinion, what are the advantages of doing a science fair project over regular science classes? [MULTIPLE ANSWERS POSSIBLE]

*Studentsaremoremotivated*Studentsaretheleadersoftheirlearning*Asateacher,Igettointeractwithstudentsinadifferentcontext*Studentsstartbehavinglikeactualscientists*Thisisagoodopportunitytoworkinaninterdisciplinaryapproach*Thisisagoodopportunitytoengagewiththeoutsideworld*Thisisagoodopportunityforcollaborationacrossschoolyears*Don’tknow*Other,pleasespecify

13 How difficult did you as a teacher find the following activities in particular?

1–Verydifficult2–3–4VeryeasyN/A

13.1 Getting students on board

13.2 Gettingmentors/expertsonthetopiconboard

13.3 Gettingholdofthematerialsandequipmentneededfortheproject

13.4 Findingfundingfortheproject

13.5 Fillinginapplicationformsforthecompetition

13.6 Supervisingstudentswhoseprojectswerenotinmyareaofexpertise

13.7 Other,pleasespecify

14 And how difficult did your students find the following activities?

1–Verydifficult2–3–4VeryeasyN/A

14.1 Gettingholdofthematerialsandequipmentneededfortheproject

14.2 Findingfundingfortheproject

14.3 Applyingthescientificmethod

14.4 Analysingprojectdata(includingstatisticalanalysis)

14.5 Displayingandcommunicatingtheirprojectinaclearandcompellingway

14.6 Fillinginapplicationformsforthecompetition

14.7 Other,pleasespecify

15 How much support did you receive from the following people / organisations in particular?

AlotofsupportSomesupportNotalotofsupportNosupportatallDon’tknow/Notapplicable

15.1 Theseniormanagementofyourschool

15.2 Yourcolleagues

Page 51: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

51

Pilot evaluation of Science Fairsin Europe

Science fairs – teachers’ questionnaire

Q QUESTIONS ANSWER CATEGORIES

15.3 Parents

15.4 Mentors/Experts

15.5 Yourlocalauthority

15.6 Yourlocalmedia

15.7 Local/nationalscienceexperts

15.8 YournationalMinistryofEducation

15.9 TheIntelteam

15.10 Universities/Researchinstitutes/Partnerships

16 Whose support proved most instrumental in your opinion to carry out the projects with your students? [UP TO 3 ANSWERS]

*Theseniormanagementofyourschool*Yourcolleagues*Parents*Yourlocalauthority*Yourlocalmedia*Mentors/scienceexperts*YournationalMinistryofEducation*TheIntelteam*Don’tknow*Other,pleasespecify

17 Among the following issues, please select the 3 that are in your opinion most essential to the students’ success with their science fair project? [UP TO 3 ANSWERS]

IntelligenceWorkethicCriticalthinkingCommunicationskillsScientificliteracyCuriosityCreativityCollaborationLeadership

18 How important were the following people / factors in your decision to lead students competing in the science fair or made it easier to carry out the projects?

VeryimportantFairlyimportantNotveryimportantNotimportantatallDon’tknow/Notapplicable

18.1 Myschoolprincipal

18.2 Mystudents

18.3 Mycolleagues

18.4 Theopportunitytoattendournationalsciencefair

18.5 Theopportunitytowinprizes/awards

18.6 Theprestige/recognitionfromcompetinginthefair

18.7 Theprospectofpromotion

18.8 Theaccesstolabsfromotherorganizations

18.9 Other,pleasespecify

Page 52: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

52

Pilot evaluation of Science Fairsin Europe

Appendix

Q QUESTIONS ANSWER CATEGORIES

19 Do you feel like supervising students competing in the science fair again – irrespective of whether it’s compulsory at your school or not?

DefinitelyMaybeProbablynotDefinitelynotDon’tknow/Notapplicable

19A [IF NO TO Q19] Why not? [MULTIPLE ANSWERS POSSIBLE] IwouldliketotryoutotheractivitiesinthefutureIttookuptoomuchofmytimeIdidn’tenjoyitasmuchasIthoughtIwouldIreceivedtoolittlesupportItinvolvedtoomuchpaperworkDon’tknowOther,pleasespecify

20 Did you use inquiry-based teaching in your classes before the first time you participated in the science fairs?

1-Notatall2-Notmuch3-Quiteabit4-Alwaysx-Don’tknowwhatInquiry-basedteachingis

21 How much do you use inquiry-based teaching in your classes nowadays?

1-Notatall2-Notmuch3-Quiteabit4-Alwaysx-Don’tknowwhatInquiry-basedteachingis

22 Is there anything else you would like to share with us or recommend?

[OPEN-ENDED]

23 Would you be available for a 15-minute follow-up interview over the phone?

YesNoDon’tknow

24 Would you be interested in becoming a Science Fair Ambassador in your country?

YesNoDon’tknow

25 What is the highest degree you hold? DoctorateMaster’sdegreeBachelor’s degreeSchoolleavingcertificateOther,pleasespecify

26 Gender MaleFemale

27 Country Listofcountriesparticipatingintheevaluation.

Page 53: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

APPENDIX

SCIENCE FAIRS – STUDENTS’ QUESTIONNAIRE

Page 54: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

54

Pilot evaluation of Science Fairsin Europe

Appendix

APPENDIXSCIENCEFAIRS–TEACHERS’QUESTIONNAIRE

SCIENCE FAIRS – STUDENTS’ QUESTIONNAIRE

Q QUESTIONS ANSWER CATEGORIES

1 How many times have you participated in the fair including this one?

123Morethan3

2 What is the major focus of your project? ScienceTechnologyEngineeringMathsOther,pleasespecify

3 Did you submit your project as an individual or as a team? IndividualTeam

4 How much time did you spend developing the project? Closedquestion:2–10h1day1week1monthseveralmonths

5 Approximately how many hours did you work on the display and presentation part of your project?

1–2hours3–5hours6–10hours1day1week

6 Did you participate in an after-school science club / programme / activity?

YesNo

7 Is it compulsory at your school to do a science fair project? YesNo

8 To what extent do you agree or disagree with each of the following statements?

IfullydisagreeagreeIdisagreeIagreeIfullyagreeDon’tknow/Notapplicable

8.1 Now,Iammoreconfidentatidentifyingproblems

8.2 Now,Iamabletocommunicatebetter

8.3 Now,Iambetteratteamwork

8.4 Now,Iambetteratfindingsolutionsorcomingupwithnewideas

Page 55: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

55

Pilot evaluation of Science Fairsin Europe

Science fairs – students’ questionnaire

Q QUESTIONS ANSWER CATEGORIES

8.5 Ihadfuncompetinginthesciencefair

8.6 Ilearntalotthroughcompetinginthesciencefair

8.7 Now,Iknowmoreaboutthescientificmethod

8.8 Now,Iammoreexcitedaboutscience

8.9 Now,Icanseebetterhowscienceisrelevanttomyeverydaylife

8.10 Now,Iammorelikelytoconsiderstudyingascientificsubject

8.11 Now,Iammoreattractedtotheprospectofascientificcareer

9 In your opinion, what are the advantages of doing a science fair project? [MULTIPLE ANSWERS POSSIBLE]

*Youhaveaprojectofyourown*Youaretheoneincharge*Youfeellikearealscientistworkingonarealproject*Yougetthechancetoshowyourworktotheoutsideworld*Mistakesareallowed*Yougettoworkwithteachersfromvarioussubjects*Nomarksareinvolved*Yougettoworkwithpeoplefromoutofschoole.g.experts,students,etc.*Youarepartofamixedgroupofstudentsfromvariousclassesandlevels*Noneofthese*Don’tknow

10 How difficult did you find the following activities in particular?

1–Verydifficult234–Veryeasy

10.1 Comingupwithatopic

10.2 Gettingholdofthematerialsandequipmentneededfortheproject

10.3 Findingfundingfortheproject

10.4 Formulatingyourprojecthypothesis

10.5 Testingyourprojecthypothesis

10.6 Analysingprojectdata(includingstatisticalanalysis)

10.7 Drawingconclusionsfromprojectdata

10.8 Displayingandcommunicatingyourprojectinaclearandcompellingway

11 How useful were the following sources of knowledge and experience in providing support with your science project?

1-Notusefulatall2-Notveryuseful3-Fairlyuseful11.1 Teacher(s)

Page 56: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

56

Pilot evaluation of Science Fairsin Europe

Appendix

Q QUESTIONS ANSWER CATEGORIES

11.2 Mentor/Expertinthefield

4-VeryusefulDon’tknow/Notapplicable

11.3 After-schoolscienceclub/programme

11.4 School classes

11.5 Formersciencefairparticipantsfromyourschool

11.6 Internet/onlinesupport(webpages,chats,forums)

11.7 Familymembers

11.8 Other,pleasespecify

11A If you had a mentor or an expert in the field to support you, what was her/his profession and how did you meet/find her/him?

[OPEN-ENDED]–Optional

12 Did you participate in an internship related to your project?

YesNo

13 Among the following items, please select the 3 that are in your opinion most essential to the success of a science fair project? [UP TO 3 ANSWERS]

IntelligenceWorkethicCriticalthinkingCommunicationskillsScientificliteracyCuriosityCreativityCollaborationLeadership

14 How important were the following people/factors in your decision to compete in the science fair?

1-Notimportantatall2-Notveryimportant3-Fairlyimportant4-VeryimportantDon’tknow/Notapplicable

14.1 Myteacher(s)

14.2 Myparents

14.3 Theopportunitytoattendournationalsciencefair

14.4 Theopportunitytowinprizes/awards

14.5 Theprestige/recognitionfromcompetinginthefair

14.6 Myfriendswerecompetinginthesciencefairtoo

14.7 Theopportunitytodoarealscienceprojectlikearealresearcher

14.8 Theopportunitytodosomethingscience-relatedthat’sdifferentfromwhatwedoatschool

14.9 ThepersonalattentionIreceivedfrommymentor/teacher

14.10 Theopportunitytoimprovemyapplicationtouniversity

Page 57: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

57

Pilot evaluation of Science Fairsin Europe

Science fairs – students’ questionnaire

Q QUESTIONS ANSWER CATEGORIES

15 How useful were the following sources of knowledge and experience in providing support with your science project?

1Definitelynot2Probablynot3Maybe4DefinitelyDon’tknow/Notapplicable

15A [IF NO TO Q15] Why not? [MULTIPLE ANSWERS POSSIBLE] • Iwouldliketotryoutotheractivitiesinthefuture

• Ittookuptoomuchofmyfreetime• Ididn’thavesomuchfun• Igotdiscouragedbecausemostofmyexperimentsfailed

•Mostofmyfriendsarenotinterestedinjoining•Other,pleasespecify

15B [IF YES TO Q15] Would you recommend competing in the fair to a friend?

1Definitelynot2Probablynot3Maybe4DefinitelyDon’tknow/Notapplicable

16 Is there anything else you would like to share with us or recommend?

[OPEN-ENDED]

17 Would you be available for a 15-minute follow-up interview?

YesNoDon’tknow/Notapplicable

18 Age <10,10,11,12,1314,15,16,17,1819,20,21,>21

19 Gender MaleFemale

20 Country Listofcountriesparticipatingintheevaluationproject

Page 58: The European Science Fairs Evaluation Framework – pilot study · science fairs affiliated to the Intel International Science and Engineering Fairs (Intel ISEF) and provide useful

PublishedinSeptember2013.TheviewsexpressedinthispublicationarethoseoftheauthorsandnotnecessarilythoseofEUNPartnershipAISBLorIntelCorporation.Reproductionisauthorisedprovidedthesourceisacknowledged.