1

PRELIMINARY RESULTS JUNE 6 , 2011

ARIANNA LEGOVINIMIRIAM BRUHN

BILAL ZIA

WORLD BANK

Impact Evaluation of Brazil’s School Based Financial

Education Program

2

Impact Evaluation Objectives (Students)

Measure the impact of a three semester long program that teaches high school students financial concepts Does the program improve students’ financial knowledge?

The program also includes exercises and case studies related to the students’ daily life (e.g. making a budget) Does the program change attitudes about financial decisions,

as well as actual financial behavior and decision making?

3

Impact Evaluation Objectives (Parents)

Some exercises are meant to be done at home, jointly with the parents Is this enough to also improve parents’ financial

knowledge and to change their attitudes and behavior?

Can we achieve a greater impact by also providing a financial literacy workshop directly to parents?

4

Methodology

Pilot test the financial education program in close to 900 schools in various states in Brazil

Schools that were interested in participating in the pilot were randomly divided into two groups Treatment group receives financial education text books for

free AND receives teacher training for how to implement the material

Control group will receive the same two school years later (possibly with modified material and training, based on evaluation results)

Half of the parents in treatment schools will also randomly be selected for participation in a parent financial education workshop in schools

5

Pilot Sample Size

Total of 891 schools in six states (439 treatment and 452 control schools)

Approximately 26,000 students and parents (one class per school)

State Number of schools

São Paulo 372

Rio de Janeiro 270

Ceará 122

Distrito Federal 64

Tocantins 34

Minas Gerais 29

6

Timeline

Time Activity

April – May 2010 Complied list of interested schools with help of State Ministries of Education; Randomly divided schools into treatment and control group

May – July 2010 Teacher training

Early August 2010 Baseline survey

Mid August 2010 Teachers started teaching financial education material, continuing until November 2011

Late November 2010 First follow-up survey

Spring 2011 Parent workshops

November 2011 Second follow-up survey

7

Results Presented Today

Use data from August 2010 baseline survey and November 2010 follow-up survey

Students had received about 1 semester of financial education before the November 2010 follow-up survey

Full program takes 3 semesters – the pilot ends in November 2011

Results presented today are preliminary since they only cover the first out of three semesters of financial education

8

Baseline Characteristics: Balance across Treatment and Control Group

9

Follow-up Data: Impact Analysis

At baseline, average student and parent characteristics, as well as test scores, were the same in the control and treatment group This is because the treatment was randomly assigned Therefore the control group is a valid comparison

group for the treatment group

Thus, any differences in averages between treatment and control group at follow-up are CAUSED BY the financial education program

10

To measure students’ financial knowledge, the survey firm (CAEd) developed a test tailored to the program’s material

Based on this test, CAEd calculated the level of financial proficiency for each student, ranging from 0 to 100

Students’ Financial Proficiency

11

In the follow-up survey, the average level of financial proficiency was significantly higher in the treatment group (60.4) than in the control group (56.1)

Impact on Financial Proficiency

Baseline Follow-up40.0

45.0

50.0

55.0

60.0

65.0

49.8

56.1

50.2

60.4

CONTROLTREATMENT

Avera

ge L

evel

of

Fi-

nan

cia

l P

rofi

cie

ncy

12

Statistical Analysis of Impact on Financial Proficiency

The statistical test of these differences shows that the financial education program increased the level of proficiency on the financial knowledge test by 3.6 points (compared to 56 points in the control group)

13

How large is the effect on financial proficiency?

The 3.6 point increase corresponds to a 6.4% increase compared to the average in the control group (in one semester) According to CAEd, other school programs they have worked with in

Brazil tend to improve test scores by 2.5% per school year In Mexico, providing cash transfers to parents conditional on children’s

school attendance improved test scores by 8-9% for 6-14 year olds (measured 6 years after the start of the transfers)

The effect size (0.25 of a standard deviation) is similar to the effect of A program in Colombia that provided low-income children with

vouchers for half the cost of private secondary school (0.2 std. dev. after 6 years)

A program in India that provided remedial education to students lagging behind in school (0.28 std. dev. after two years, the effect of the same program after one year was 0.14 std. dev)

14

Effect Size (est.)

0

3

6

9

Percentage change in learning achievement

15

Students’ Self-Reported Knowledge Ratings

The surveys also asked students to rate their knowledge about different financial concepts On a scale from 0 to 10, how would you rate your

knowledge about Interest rates Loans/financing Insurance Income tax Minimum payment for credit cards

16

Impact on Students’ Self-Reported Knowledge Ratings

(Follow-Up Data)

3.0

4.0

5.0

6.0

7.0

5.1

6.0

5.0 5.0

6.45.9

6.8

6.0 5.7

7.2

ControlTreatment

At follow-up, average self-reported knowledge ratings for all these concepts were higher in the treatment group than in the control group

18

Two Questions Often Used in the Economics Literature to Measure Financial Literacy

1. Suppose you are interested in taking a loan of R$50,000. Bank “A” offers you a loan of R$50,000 payable in one year with an annual interest rate of 15%. Bank “B” offers you a loan of the same amount, R$50,000, but you will have to pay $60,000 within one year. Which loan would you prefer?a) Bank Ab) Bank Bc) Don’t know

2. Suppose the interest rate on your savings account is 1% per year and inflation is 2% per year. After a year, if you had not made any deposits or withdrawals in this account, would you be able to buy more than, exactly the same as, or less than today with the money in this account?d) More than todaye) The same as todayf) Less than todayg) Don‘t know

19

Impact on Students’ Correct Answers to Financial Literacy Questions Used in Economics Literature

(Follow-Up Data)

20%

30%

40%40%

33%

42%36%

ControlTreatment

At follow-up, a higher percentage of students in the treatment group than in the control group gave the correct answer to the financial literacy questions often used in the economics literature

21

Student Financial Autonomy

Autonomy index, designed by CAEd, based on 3 dimensions of Autonomy Reflexive Autonomy e.g. “ I like to think carefully before

deciding to buy something” Emotional Autonomy e.g. “I am prepared to talk about

money with my parents” Functional Autonomy e.g. “I always try to save some

money to do things I really like”

These three components were combined into one overall index of financial autonomy, where higher values indicate more financial autonomy

22

Impact on Student Financial Autonomy (Follow-up Data)

Control Treatment47.00

48.00

49.00

50.00

51.00

52.00

48.86

50.91

Financial Autonomy Index

At follow-up, students in the treatment group obtained higher scores on the autonomy index than students in the control group

24

Students’ Intention to Save

Intention to save index, designed by CAEd, based on 3 components Attitudes towards behavior e.g. “In my opinion, saving

money every month extremely beneficial Subjective norms and expectations e.g. “My family has the

habit of saving money every month” Perception of capacity of controlling one’s behavior e.g. “I

believe I can save a little money every month”

These three components were combined into one overall index of intention to save, where higher values indicate higher intention to save

25

Impact on Students’ Intention to Save (Follow-Up Data)

Control Treatment92949698

100102104

97

103

Intention to Save Index

At follow-up, students in the treatment group had a higher measured intention to save than students in the control group

27

Students’ Saving Behavior

In order to assess whether intention to save is also reflected in savings behavior, the survey asked students Which fraction of their income they save

About 61% of students have non-zero income (from work or from parents)

About 37% of students work Whether they save to be able to finance future

projects

28

Impact on Students’ Saving Behavior (Follow-Up Data)

Q53: Pct of income saved is non-zero

Q65: I save money for future projects

0%

10%

20%

30%

40%

50%

60%

44%

17%

50%

19%ControlTreatment

At follow-up, a higher percentage of students in the treatment group than in the control group Saved at least some of their income Saved to able to finance projects in the future

30

Students’ Spending Behavior

The survey asked a number of questions to assess students’ spending behavior Do students make a list of expenses every month? How do students pay for purchases?

Do they use a credit card? Do they use a booklet/installment plan in the store?

31

Impact on Students’ Spending Behavior (Follow-Up Data)

0%6%

12%18% 13% 11% 10%

16%9% 9%

ControlTreatment

At follow-up, a higher percentage of students in the treatment group than in the control group made a list of expenses every month

A lower percentage of students in the treatment group than in the control group Often shopped with a credit card Often shopped with a booklet/installment plan in the store

33

Other Measures of Financial Behavior

Apart from saving and spending behavior, the survey asked about several other features of students financial behavior Did the student learn to manage money in school? Does the student negotiate how to pay when making a

purchase? Does the student search for other brands and model when

deciding which product to purchase (i.e. do the compare options?)

These are often thought to be savvy/smart financial behaviors

34

Impact on Students’ Financial Savvy (Follow-Up Data)

0%

30%

60%

2%

63% 66%

15%

67% 68%

ControlTreatment

At follow-up, a higher percentage of students in the treatment group than in the control group Say they learned to manage money in school Negotiate how to pay Search similar model/brands when deciding what to buy

36

Parent Outcomes

Parents have not received financial education directly

But the students’ financial education material includes exercises to be completed with parents (e.g. make a household budget)

This can have positive “spillovers” on parents’ financial knowledge To measure parent’s financial knowledge, the parent survey

included the same two financial literacy questions often used in the economics literature that were also included in the student survey

37

Impact on Parent Outcomes (Follow-Up Data)

0%20%40%60%80% 67%

48% 44%33%

70%52% 45%

35%

ControlTreatment

At follow-up, a higher percentage of parents in the treatment group than in the control group Say they talk to the student about finances Say the student helps organize the household budget

But, there are only small differences in the percentage of parents who answer the financial literacy questions correctly

38

Statistical Analysis of Impact on Parent Outcomes

The program significantly increased the % of parents who Say they talk to the student about finances (by 3.6% compared to 67% in

the control group) Say the student helps organize the household budget (by 4.9% compared

to 48% in the control group)So far, we see no statistically significant impact of the

program on parents’ financial knowledge

39

Conclusions

The first semester of the high school financial education program increased students’ financial knowledge

The program also led to improvements in students’ financial attitudes

And, the program changed financial behavior – due to the program, students are more likely to Save and manage their expenses Engage in “smart” financial behaviors Students are also more likely to talk to their parents about

financial matters and to help with organizing the household budget

“Spillover” effects on parents’ financial knowledge are small so far need for the parent workshop

40

Next Steps

Implement the parent workshop Currently working on this Have received updated student lists from close to 150

schools and have randomly allocated parents in these schools into two groups One will watch financial literacy video Other will watch health video

Still need to receive student lists from the remaining treatment schools

Start planning the November 2011 survey

41

World Bank Team & Contact Info

TTL Rogelio Marchetti (LCSPF) rmarchetti@worldbank.org

Researchers Miriam Bruhn (DECFP) mbruhn@worldbank.org Bilal Zia (DECFP) bzia@worldbank.org

Field coordinator Luciana Leão (LCSPF) lululeao@terra.com.br

Advisor Arianna Legovini (DECOS) alegovini@worldbank.org