1

Milwaukee Mathematics Partnership

Program EvaluationYear 6 Results

Carl HanssenHanssen Consulting, LLC

Cindy WalkerUniversity of Wisconsin-Milwaukee

November 2009 MTL Meeting

2

Where are we?

MMP efforts have been ongoing since 2003

This is the 6th year we have been conducting the evaluation

We anticipate continuing effortfor several more years

3

MMP Support

Original funding from NSF Started in 2003-04 Currently in 7th year

MMP Phase II funding from NSF Awarded last fall for 3 years Focus on research and evaluation

State of Wisconsin Provides funding for released MTL positions

4

Evaluation Goals

Help the MMP better serve its constituents and improve its effectiveness

Serve the broader mathematics education community through documentation and dissemination of MMP success factors

5

Key Evaluation Question

What are the critical factors or conditions promoted by the MMP that are related to student achievement?

6

Agenda

1. MMP Involvement

2. Social Network Analysis

3. Learning Team and Math Meeting Observations

4. Mathematical Knowledge for Teaching

5. Conclusions

6. Next Steps

7

Thank You!

142 schools provided data for the MMP Online Survey producing a record response rate!

114 schools provided social network analysis data!

25 schools allowed us to observe meetings!

Many others have provided data for a variety of other measures!

8

1. MMP Involvement

Metrics:

MMP Involvement Attendance at MTL Meetings MMP Courses

WKCE Student Achievement Growth from 2005-2008 2008 Achievement

9

MMP Involvement & GrowthSchools that have been more involved over time demonstratedhigher student achievement growth from 2005 to 2008:

Ave

rag

e %

Gro

wth

Number of Schools: high (n=34), medium (n=91), low (n=26), none (n=27)

10

MMP Involvement & 2008 Achievement

Schools that are more involved demonstrated higherstudent achievement in 2008:

Ave

rag

e %

Pro

fici

ent

Number of Schools: high (n=34), medium (n=97), low (n=31), none (n=36)

11

2. Social Network Analysis

We asked you to list individuals with whom you communicated about mathematics education in the past several months.

For each individual:1. Do they work at your school?2. Their role3. Frequency of communication4. Extent of collaboration

12

Key

Maps identify

MTL or MTS or Teachers Principal Literacy Coach Others in school Others outside school

Statistics

Network density In-School density

MTL In-Degree MTL Betweeness MTS IN-Degree

Benefit—a graphical AND statistical description ofschool-based networks.

13

Hi Density

School 09G

08 Students 7608 Passing 5508 Percent 72.4%5 year trend 20.1%SNA Respondents 16SNA Total Named 34Network Density 17.3%In School Density 35.8%MTL In Degree 23.48MTL Betweeness 5.50MTS In Degree 1.13

14

Lo Density

School 09F

08 Students 13008 Passing 4608 Percent 35.4%5 year trend -11.2%SNA Respondents 10SNA Total Named 35Network Density 3.4%In School Density 4.8%MTL In Degree 20.00MTL Betweeness 0.00MTS In Degree 2.35

15

Hi MTL In-Degree

School 09E

08 Students 14608 Passing 6908 Percent 47.3%5 year trend -31.8%SNA Respondents 23SNA Total Named 37Network Density 13.8%In School Density 24.2%MTL In Degree 49.44MTL Betweeness 40.73MTS In Degree 1.66

16

Lo MTL In-Degree

School 09I

08 Students 41808 Passing 22408 Percent 53.6%5 year trend 2.1%SNA Respondents 19SNA Total Named 57Network Density 4.6%In School Density 6.5%MTL In Degree 6.07MTL Betweeness 10.32MTS In Degree

17

Hi MTL Betweeness

School 09J

08 Students 33508 Passing 16408 Percent 49.0%5 year trend 6.2%SNA Respondents 32SNA Total Named 54Network Density 5.1%In School Density 7.5%MTL In Degree 38.05MTL Betweeness 49.22MTS In Degree 0.63

18

Hi MTS In-Degree

School 09C

08 Students 20208 Passing 7308 Percent 36.1%5 year trend -0.4%SNA Respondents 22SNA Total Named 34Network Density 16.4%In School Density 24.3%MTL In Degree 55.15MTL Betweeness 37.41MTS In Degree 24.24

19

So what?

A combination of factors…all are necessary but nosingle factor is sufficient

Math Focus

LT Quality

NetworkDensity

MTL In-Degree

MTLBetweeness

Predicts

StudentSuccess

Discriminant function analysis Indicates that this combination of factors predicts achievement quartiles

20

3. Meeting Observations

Overall, LT meetings tended to focus more on ‘administration’ than ‘learning’ though higher performing teams retain more of an emphasis on learning

Math meetings more focused, yet less team oriented

Common observations from math meetings: Real work is typically accomplished (e.g., scoring CR items)

Substantive mathematical discussions take place

Implications for classroom practice are rarely discussed as a basis for wrapping up

21

4. Mathematical Knowledge for Teaching

Two groups of teachers took the MKT assessment in 2008-09

Math Teacher Leaders Self-selecting Math Teachers

22

MTL MKT—2008-09 ResultsGrade 8-9 MTLs consistently score higher than the K-7 MTLs on both pre-tests and post-tests…especially in Geometry and Prob & Stat.

0.170.24

0.36

0.47

0.24

0.35

-0.17

0.29

0.420.47

0.71

0.52

0.90

0.46

-0.40

-0.20

0.00

0.20

0.40

0.60

0.80

1.00

Pre Post Pre Post Pre Post Pre Post

Number Algebra Geometry Prob & Stat

K-7 Mean 8-9 Mean

*

*

23

MTL Compared with Classroom TeachersMTLs consistently score higher than classroom teachers in all areas of the MKT assessment.

-0.33

0.19

-0.08

-0.25

0.29

0.540.50

0.01

-0.40

-0.30

-0.20

-0.10

0.00

0.10

0.20

0.30

0.40

0.50

0.60

Ab

ilit

y E

stim

ate

Teachers MTLs Number Algebra Geometry Prob & Stat

24

5. Overall Conclusions

Schools that have more heavily participated in MMP-sponsored activities are demonstrating greater student achievement growth.

There appears to be a set of necessary conditions for improving student achievement—no single factor is sufficient.

25

Overall Conclusions

Productive work is taking place at math-focus meetings in the context of grade-level meetings, collaborative planning time, or other special sessions.

A shortcoming of these sessions is often not considering implications for classroom practice based on the good work being done.

26

Overall Conclusions

MTLs have regularly demonstrated math content knowledge gains during the academic year, and there is some evidence to suggest these gains are being sustained over time.

MPS Math Teachers MKT results are generally lower than that of the MTLs. Slight improvements over time have been observed.

27

6. Evaluation 2009-10

MMP Online survey in May 2010 Continue to focus on 25 case study schools for the next 2 years Math meeting observations SNA School honorarium

Implement SNA in mostschools across the district

28

Small Group Discussion

1. What implications does the information presented have for MTLs as they promote PD efforts in their building?

2. What insights have you gained about the work of the MTLs? How will that information impact your work in your building?

3. What are some areas you can enhance or strengthen in your work with teachers that will make an impact on student achievement?

4. According to M. Fullan, “The role of the leader is to ensure that the organization develops relationships that help produce desirable results.”

As you reflect on Fullan’s work about building relationships, what connections can you make to your work and the information just shared?