A Perceptually-Motivated Harmonic Compatibility Method for ...
supp.apa.orgsupp.apa.org/.../edu0000145/zcz005162798so1.docx · Web viewSupplemental Materials...
20
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 1 Supplemental Materials Supporting Students in Making Sense of Connections and in Becoming Perceptually Fluent in Making Connections among Multiple Graphical Representations by M. A. Rau et al., 2016, Journal of Educational Psychology http://dx.doi.org/10.1037/edu0000145 Appendix Table 1A. Topics covered by the Fractions Tutor. Topic Description Introduction Learning how each graphical representation depicts fractions as parts of a unit 1. Naming fractions Naming unit fractions and proper fractions, given a graphical representation, comparing fractions with like numerators and like denominators 2. Making fractions Making representations given
Transcript of supp.apa.orgsupp.apa.org/.../edu0000145/zcz005162798so1.docx · Web viewSupplemental Materials...
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 1
Supplemental Materials
Supporting Students in Making Sense of Connections and in Becoming Perceptually Fluent
in Making Connections among Multiple Graphical Representations
by M. A. Rau et al., 2016, Journal of Educational Psychology
http://dx.doi.org/10.1037/edu0000145
Appendix
Table 1A. Topics covered by the Fractions Tutor.
Topic Description
Introduction Learning how each graphical representation
depicts fractions as parts of a unit
1. Naming fractions Naming unit fractions and proper fractions,
given a graphical representation, comparing
fractions with like numerators and like
denominators
2. Making fractions Making representations given symbolic unit
fractions and proper fractions, comparing
fractions with like numerators and like
denominators
3. Reconstructing the unit Reconstructing the unit of a given fraction
4. Naming improper fractions Naming improper fractions and mixed
numbers, given a graphical representation
5. Making improper fractions Making representations given symbolic
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 2
improper fractions and mixed numbers
6. Concepts of equivalent fractions Learning about what makes fractions
equivalent
7. Procedures with equivalent
fractions
Finding several fractions equivalent to given
unit fractions and proper fractions
8. Fraction comparison Comparing fractions with unlike numerators
and denominators
9. Fraction addition Adding fractions with like denominators and
unlike denominators
10. Fraction subtraction Subtracting fractions with like denominators
and unlike denominators
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 3
Table 2A. Covariance parameter estimates for HLM models.
Covariance
parameterSubject Estimate Standard Error Z Value Pr > |Z|
Conceptual
knowledge1
UN(1,1) School -.0002 .00052 -.38 .7033
UN(1,1) Class(School) .003152 .001659 1.9 .0574
UN(1,1) Student(Class) .01793 .002245 7.99 <.0001
Residual .01905 .001408 13.53 <.0001
Procedural
knowledge2
UN(1,1) School .000302 .000558 .54 .589
UN(1,1) Class(School) .000785 .000764 1.03 .3044
UN(1,1) Student(Class) .01313 .001319 9.96 <.0001
Residual .006815 .000504 13.53 <.0001
1 The intra-class correlation coefficient is computed as ICC = .01905 / (-.0002 + .003152 + .01793 + .01905) = .4771
2 The intra-class correlation coefficient is computed as ICC = .006815 / (.000302 + .000785 + .01313 + .006815) = .3240
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 4
Table 3A. Estimates and standard errors of effects in HLM models for the conceptual knowledge
scale.
Effect EstimateStandard
Errort-value (df) p-value
Intercept .075 .025 2.99 (58.6) .0041
Immediate posttest -.047 .010 -4.65 (366) <.0001
Final posttest 0 - - -
No sense-making -.085 .030 -2.82 (341) .0050
Sense-making with linked GRs -.068 .031 -2.20 (342) .0284
Sense-making with analogous examples
0 - - -
No fluency-building -.0724 .043 -2.30 (345) .0218
Fluency-building 0 - - -
No sense-making + no fluency-building
.1166 .043 2.70 (347) .0073
No sense-making + fluency-building
0 - - -
Sense-making with linked GRs + no fluency-building
.0967 .045 2.18 (341) .0301
Sense-making with linked GRs + fluency-building
0 - - -
Sense-making with analogous examples + no fluency-building
0 - - -
Sense-making with analogous examples + fluency-building
0 - - -
Pretest .6094 .087 7.00 (349) <.0001
Pretest + no sense-making -.081 .103 -.78 (345) .4349
Pretest + sense-making with linked .1120 .109 1.03 (343) .3029
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 5
GRs
Pretest + sense-making with analogous examples
0 - - -
Pretest + no fluency-building -.042 .088 -.49 (343) .6279
Pretest + sense-making with linked GRs
0 - - -
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 6
Table 4A. Estimates and standard errors of effects in HLM models for the procedural knowledge
scale.
Effect EstimateStandard
Errort-value (df) p-value
Intercept .017 .020 .84 (26.2) .4078
Immediate posttest -.010 .006 -1.56 (366) .1185
Final posttest 0 - - -
No sense-making -.018 .023 -.77 (341) .4428
Sense-making with linked GRs -.001 .024 -.024 (341) .7491
Sense-making with analogous examples
0 - - -
No fluency-building -.030 .024 -1.20 (349) .2299
Fluency-building 0 - - -
No sense-making + no fluency-building
.045 .034 1.34 (347) .1823
No sense-making + fluency-building
0 - - -
Sense-making with linked GRs + no fluency-building
.028 .035 .81 (340) .4210
Sense-making with linked GRs + fluency-building
0 - - -
Sense-making with analogous examples + no fluency-building
0 - - -
Sense-making with analogous examples + fluency-building
0 - - -
Pretest .9997 .071 14.09 (354) <.0001
Pretest + no sense-making -.130 .080 -1.62 (345) .1066
Pretest + sense-making with linked -.051 .091 -.56 (347) .5761
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 7
GRs
Pretest + sense-making with analogous examples
0 - - -
Pretest + no fluency-building -.016 .070 -.23 (343) .8215
Pretest + sense-making with linked GRs
0 - - -
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 8
Table 5A. Implied covariance matrix for the causal path analysis model that tests the
understanding hypothesis.
FL pre post delpost place1Error SE-
Error
FL 0.25
pre 0 0.0459
post 0.0195 0.0252 0.0486
delpost 0.0049 0.0326 0.0376 0.0669
place1Error 0.2351 -0.1176 -0.4038 -0.3324 20.5252
SE-Error 1.4155 -0.7084 -1.3058 -1.459 28.5992 172.223
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 9
Table 6A. Standardized parameter estimates with significance tests for the causal path analysis
model that tests the understanding hypothesis.
Edge from… to… Type Parameter
estimate
Standard
error
t-value p-
value
SE equivalenceError Edge Coef. -15.6951 3.4057 -4.6085 0
SE improperMixedE
rror
Edge Coef. -15.2752 2.0985 -7.2789 0
SE nameCircleMixe
dError
Edge Coef. 6.571 1.7245 3.8103 0.0002
equivalenceError delpost Edge Coef. -0.0012 0.0008 -1.4756 0.1425
equivalenceError improperMixedE
rror
Edge Coef. 0.1173 0.0512 2.2905 0.0236
equivalenceError post Edge Coef. -0.0017 0.0009 -1.9154 0.0576
improperMixedErr
or
delpost Edge Coef. -0.0019 0.0012 -1.6422 0.103
improperMixedErr
or
post Edge Coef. -0.0046 0.0013 -3.6517 0.0004
nameCircleMixedE
rror
delpost Edge Coef. -0.0023 0.0015 -1.5189 0.1312
nameCircleMixedE
rror
equivalenceError Edge Coef. 0.6033 0.1652 3.6525 0.0004
nameCircleMixedE
rror
post Edge Coef. -0.0032 0.0017 -1.9244 0.0565
post delpost Edge Coef. 0.4842 0.0795 6.088 0
pre delpost Edge Coef. 0.3268 0.0831 3.9339 0.0001
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 10
pre nameCircleMixe
dError
Edge Coef. -13.3078 4.0656 -3.2733 0.0014
pre post Edge Coef. 0.4525 0.0839 5.3954 0
SE SE Std. Dev. 0.4967 0.0306 8.0722 0
pre pre Std. Dev. 0.2107 0.0053 8.3811 0
post post Std. Dev. 0.184 0.0039 8.6261 0
delpost delpost Std. Dev. 0.1636 0.003 9.0035 0
equivalenceError equivalenceError Std. Dev. 18.4632 42.2822 8.0623 0
improperMixedErr
or
improperMixedE
rror
Std. Dev. 11.2387 15.6666 8.0623 0
nameCircleMixedE
rror
nameCircleMixe
dError
Std. Dev. 9.4912 11.1734 8.0623 0
SE SE Mean 0.4427 0.0434 10.2021 0
pre pre Mean 0.3445 0.0184 18.7124 0
post post Mean 0.4399 0.021 20.9394 0
delpost delpost Mean 0.501 0.0223 22.4887 0
equivalenceError equivalenceError Mean 34.2137 1.7787 19.2348 0
improperMixedErr
or
improperMixedE
rror
Mean 11.4962 1.2373 9.2912 0
nameCircleMixedE
rror
nameCircleMixe
dError
Mean 7.2672 0.8948 8.1211 0
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 11
Table 7A. Implied covariance matrix for the causal path analysis model that tests the sense-
making hypothesis.
SE pre post delpost equivalence-
Error
improper-
MixedError
nameCircle-
MixedError
SE 0.2467
pre 0 0.0444
post 0.0185 0.0228 0.0531
delpost 0.0166 0.0274 0.0382 0.0596
equivalence-
Error
-2.8943 -0.3564 -1.41 -1.5565 410.4891
improperMixed
-Error
-4.1082 -0.0418 -1.0257 -0.96 92.3517 199.8921
nameCircle-
MixedError
1.6212 -0.5908 -0.5912 -0.7366 40.0696 -20.0652 108.5975
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 12
Table 8A. Standardized parameter estimates with significance tests for the causal path analysis
model that tests the fluency hypothesis.
Edge
from…
to… Type Parameter
estimate
Standard
error
t-value p-value
F selfExplErro
r
Edge Coef. 5.6624 2.3249 2.4356 0.0164
F post Edge Coef. 0.1159 0.0303 3.8302 0.0002
selfExplError delpost Edge Coef. -0.0032 0.0014 -2.2492 0.0264
selfExplError place1Error Edge Coef. 0.1661 0.0283 5.8614 0
selfExplError post Edge Coef. -0.0047 0.0013 -3.6036 0.0005
place1Error post Edge Coef. -0.0118 0.0037 -3.1734 0.0019
post delpost Edge Coef. 0.4823 0.0975 4.9487 0
pre selfExplErro
r
Edge Coef. -15.4224 5.4235 -2.8436 0.0053
pre delpost Edge Coef. 0.3953 0.0939 4.2084 0.0001
pre post Edge Coef. 0.4455 0.0717 6.2158 0
F F Std. Dev. 0.5 0.0328 7.6249 0
pre pre Std. Dev. 0.2143 0.0058 7.8963 0
post post Std. Dev. 0.1553 0.0028 8.7424 0
delpost delpost Std. Dev. 0.1767 0.0038 8.2495 0
place1Error place1Error Std. Dev. 3.9719 2.0715 7.6158 0
selfExplError selfExplErro
r
Std. Dev. 12.3807 20.1269 7.6158 0
F F Mean 0.4957 0.0462 10.7246 0
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 13
pre pre Mean 0.3771 0.0198 19.0334 0
post post Mean 0.4786 0.0207 23.1475 0
delpost delpost Mean 0.5342 0.0241 22.2004 0
place1Error place1Error Mean 3.1795 0.4185 7.5978 0
SE-Error SE-Error Mean 25.2051 1.2086 20.8541 0
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 14
Figure 1A. Example test items from the conceptual knowledge test.
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 15
Figure 2A. Example test items from the procedural knowledge test.
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 16
Figure 3A. SAS code for the HLM used to investigate learning gains.
TITLE1 "Procedural: learning gains";TITLE2 "All tests completed, all tutor problems completed ";TITLE3 "Nested times within students within class within schools + RI for schools, class(school), student(school) + random slopes for school";PROC MIXED DATA=WORK.dataset_centered COVTEST NOITPRINT NOCLPRINT ORDER = INTERNAL NOBOUND ;CLASS student condition school test condition district class;MODEL procedural_centered = test condition test*condition/ SOLUTION DDFM=KR OUTPRED=resid_proc ;
RANDOM INTERCEPT / SUBJECT=school TYPE=UN;RANDOM INTERCEPT / SUBJECT=class(school) TYPE=UN;RANDOM INTERCEPT / SUBJECT=student(class) TYPE=UN;
LSMEANS test*condition;LSMEANS test*condition / slice = test;ESTIMATE "learning: post minus pre across conditions" test -1 1 0;ESTIMATE "learning: delpost minus pre across conditions" test -1 0 1;
RUN;
TITLE1 "Conceptual: learning gains";TITLE2 "All tests completed, all tutor problems completed ";TITLE3 "Nested times within students within class within schools + RI for schools, class(school), student(school) + random slopes for school";PROC MIXED DATA=WORK.dataset_centered COVTEST NOITPRINT NOCLPRINT ORDER = INTERNAL NOBOUND ;CLASS student condition school test condition district class;MODEL conceptual_centered = test condition test*condition/ SOLUTION DDFM=KR OUTPRED=resid_conc ;RANDOM INTERCEPT / SUBJECT=school TYPE=UN;RANDOM INTERCEPT / SUBJECT=class(school) TYPE=UN;RANDOM INTERCEPT / SUBJECT=student(class) TYPE=UN;
LSMEANS test*condition;LSMEANS test*condition / slice = test;ESTIMATE "learning: post minus pre across conditions" test -1 1 0;ESTIMATE "learning: delpost minus pre across conditions" test -1 0 1;
RUN;
SENSE MAKING AND PERCEPTUAL FLUENCY IN CONNECTION MAKING 17
Figure 4A. SAS code for the HLM used to investigate research question 1.
TITLE1 "Procedural: factorial design, between condition contrasts";TITLE2 "All tests completed, all tutor problems completed ";TITLE3 "Nested times within students within class within schools + RI for schools, class(school), student(school) + random slopes for school";PROC MIXED DATA=WORK.dataset_centered COVTEST ASYCOV NOITPRINT NOCLPRINT ORDER = INTERNAL NOBOUND;CLASS student sense fluency school test district class;MODEL procedural_centered = test sense fluency sense*fluency procedural_pre_centered procedural_pre_centered*sense procedural_pre_centered*fluency/ SOLUTION DDFM=KR OUTPRED=resid_proc;RANDOM INTERCEPT / SUBJECT=school TYPE=UN;RANDOM INTERCEPT / SUBJECT=class(school) TYPE=UN;RANDOM INTERCEPT / SUBJECT=student(class) TYPE=UN;
LSMEANS sense*fluency;LSMEANS sense*fluency / slice = fluency;
ESTIMATE "sense 3 minus 1 for fluency" sense -1 0 1 fluency*sense 0 -1 0 0 0 1;ESTIMATE "sense 3 minus 2 for fluency" sense 0 -1 1 fluency*sense 0 0 0 -1 0 1;ESTIMATE "sense 2 minus 1 for fluency" sense -1 1 0 fluency*sense 0 -1 0 1 0 0;ESTIMATE "sense 3 minus 1 for no-fluency" sense -1 0 1 fluency*sense -1 0 0 0 1 0;ESTIMATE "sense 3 minus 2 for no-fluency" sense 0 -1 1 fluency*sense 0 0 -1 0 1 0;ESTIMATE "sense 2 minus 1 for no-fluency" sense -1 1 0 fluency*sense -1 0 1 0 0 0;
LSMEANS sense*fluency;LSMEANS sense*fluency / slice = sense;
ESTIMATE "fluency 2 minus 1 for SE" fluency -1 1 fluency*sense 0 0 0 0 -1 1;ESTIMATE "fluency 2 minus 1 for SL" fluency -1 1 fluency*sense 0 0 -1 1 0 0;ESTIMATE "fluency 2 minus 1 for no-sense" fluency -1 1 fluency*sense -1 1 0 0 0 0;RUN;
TITLE1 "Conceptual: factorial design, between condition contrasts";TITLE2 "All tests completed, all tutor problems completed ";TITLE3 "Nested times within students within class within schools + RI for schools, class(school), student(school) + random slopes for school";PROC MIXED DATA=WORK.dataset_centered COVTEST ASYCOV NOITPRINT NOCLPRINT ORDER = INTERNAL NOBOUND;CLASS student sense fluency school test district class;MODEL conceptual_centered = test sense fluency sense*fluency conceptual_pre_centered conceptual_pre_centered*sense conceptual_pre_centered*fluency / SOLUTION DDFM=KR OUTPRED=resid_conc;RANDOM INTERCEPT / SUBJECT=school TYPE=UN;RANDOM INTERCEPT / SUBJECT=class(school) TYPE=UN;RANDOM INTERCEPT / SUBJECT=student(class) TYPE=UN;
LSMEANS sense*fluency;LSMEANS sense*fluency / slice = fluency;
ESTIMATE "sense 3 minus 1 for fluency" sense -1 0 1 fluency*sense 0 -1 0 0 0 1;ESTIMATE "sense 3 minus 2 for fluency" sense 0 -1 1 fluency*sense 0 0 0 -1 0 1;ESTIMATE "sense 2 minus 1 for fluency" sense -1 1 0 fluency*sense 0 -1 0 1 0 0;ESTIMATE "sense 3 minus 1 for no-fluency" sense -1 0 1 fluency*sense -1 0 0 0 1 0;ESTIMATE "sense 3 minus 2 for no-fluency" sense 0 -1 1 fluency*sense 0 0 -1 0 1 0;ESTIMATE "sense 2 minus 1 for no-fluency" sense -1 1 0 fluency*sense -1 0 1 0 0 0;
LSMEANS sense*fluency;LSMEANS sense*fluency / slice = sense;
ESTIMATE "fluency 2 minus 1 for SE" fluency -1 1 fluency*sense 0 0 0 0 -1 1;ESTIMATE "fluency 2 minus 1 for SL" fluency -1 1 fluency*sense 0 0 -1 1 0 0;ESTIMATE "fluency 2 minus 1 for no-sense" fluency -1 1 fluency*sense -1 1 0 0 0 0;
RUN;
