Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in...
-
Upload
polly-ellis -
Category
Documents
-
view
214 -
download
1
Transcript of Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in...
![Page 1: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/1.jpg)
Standing on the Shoulders of Chess Masters:
Using RCTs to Produce Evidence aboutTeaching Chess in Italian Elementary Schools
Gianluca Argentin Gianluca Argentin
Alberto MartiniAlberto Martini
Barbara RomanoBarbara [email protected]
![Page 2: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/2.jpg)
Why should chess be taught in school?
According to the European Parliament for
several reasons
![Page 3: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/3.jpg)
The EU Parliament claims that
“Chess can improve children’s concentration, patience and persistence
and can develop sense of creativity, intuition, memory, analytic and decision-
making skills. Chess also teaches determination, motivation and
sportsmanship”
![Page 4: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/4.jpg)
The evidence on the effects of learning chess on cognitive outcomes is literally scattered all over the world: from chess association websites to academic journals in a variety of disciplines.
However
Rarely this evidence is based on rigorous evaluation designsuch as randomized controlled trials
![Page 5: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/5.jpg)
We will present:
1. the results of a simple RCT to measure the effects of learning chess on math achievement (SAM)
2. the design of a more complex RCT to answer several questions that arise when incorporating chess in the regular curriculum (SC3)
Our true long run objective is promoting the use of random assignment in evaluation and
education research in Italy.
![Page 6: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/6.jpg)
PROGETTO SAM
Scacchi e Apprendimento
della Matematica
![Page 7: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/7.jpg)
We recruited 33 schools that had been involved in chess teaching before
(a non probabilistic sample, yet nicely spread all over Italy)
![Page 8: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/8.jpg)
Each school agreed to exclude at least one 3° grade class, chosen
randomly
All the other 3° grade classes were taught 30 hours of chess by
professional instructors during 2009-10 school year
![Page 9: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/9.jpg)
A total of 113 classes - about 2000 pupils - participated either
as treatment or control
All pupils were pre-tested and post-tested using modified versions of INVALSI tests
High level of compliance to the design
![Page 10: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/10.jpg)
0 20 40 60 80 100
Gruppo di controlloGruppo sperimentale
0 20 40 60 80 100
Gruppo di controlloGruppo sperimentale
Clearly something is happening to the experimentals
ImpactPre-test Score distribution Post-test Score distribution
![Page 11: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/11.jpg)
Estimated effect on math scores(in standard deviations)
ITT
Mod. 0: nessun controllo, tutti i casi
0.28
p value .01***
Effect size 0.34
P-value .01***
A third of a standard deviation is an exceptionally good result
This makes people (including us) doubt it
![Page 12: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/12.jpg)
Is it because you included questions too favorable for chess players?
a. Draw the snail path
b. Which shape corresponds to the snail path?
![Page 13: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/13.jpg)
We found an effect on almost each item on the test
Differential probability of correct response for each item
![Page 14: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/14.jpg)
IT MUST BE THE EFFECT OF CHEATING!
![Page 15: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/15.jpg)
-10
12
Med
ia
.4 .6 .8 1 1.2 1.4Dev. St.
Mandati Controlli
CHEATING SHOULD SHOW UP HERE, BUT IT’S EMPTY
![Page 16: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/16.jpg)
We need to prove it with an RCT based on a probabilistic
sample of schools
Chess does work!
![Page 17: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/17.jpg)
Sc3
SPERIMENTAZIONE CONTROLLATA DI
SCACCHI A SCUOLA
(Springer = Knight in German)
![Page 18: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/18.jpg)
Does learning chess at school (at some point between 3rd and 4th grade) affects achievement in mathematics and reading measured in 5th grade?
How persistent are the effects of learning chess? Does any beneficial effect remain one or two years after training? Which fraction of pupils continue to play chess and possibly benefit from it, after it is no longer taught in class?
Does chess taught by professional instructors obtain better or similar results than when based on web teaching tools with the support of regular teachers? Which solution is more cost-effective and sustainable at scale?
![Page 19: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/19.jpg)
First question: does learning chess improve achievement in math and reading, as measured by INVALSI standardized tests
80 control schools
160 treated schools
Fifth grade
INVALSINational
test
Fifth grade
INVALSINational
test
Second grade
INVALSINational
test
Second grade
INVALSINational
test
240 schools selected with probability prop. to sizeout of 18,000 primary
schools in Italy
RARA
![Page 20: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/20.jpg)
The second question:External instructor or regular teacher+web?
80 schools“teaching by external instructor”
Fifth gradeINVALSINational
test
Fifth gradeINVALSINational
test
Second grade
INVALSINational
test
Second grade
INVALSINational
test80 schools
“web-based with support by regular teachers”
RARA
![Page 21: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/21.jpg)
The third question:Do school-based activities increase persistency of the
effect of learning chess?
80 schools promote chess in 5° grade
Test of knowledge
of chess and
frequencyof use (late In
fifth grade)
Test of knowledge
of chess and
frequencyof use (late In
fifth grade)
80 schoolsdo not promote chess in 5° grade
Test of knowledge
of chess and
frequencyof use
(early in third
grade)
Test of knowledge
of chess and
frequencyof use
(early in third
grade)
RARA
![Page 22: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/22.jpg)
The two dimensions crossed represent a factorial design
![Page 23: Standing on the Shoulders of Chess Masters: Using RCTs to Produce Evidence about Teaching Chess in Italian Elementary Schools Gianluca Argentin Alberto.](https://reader030.fdocuments.in/reader030/viewer/2022032722/56649cdb5503460f949a6550/html5/thumbnails/23.jpg)
We are looking for sponsor/supporter
And for other countries willing to “play (it again) SAM”
Anyone interested?