Bala Janaagraha Impact Assessment,...

Bala Janaagraha Impact Assessment, 2014-2015Janaagraha Center for Citizenship and Democracy

Bangalore, 2015

1

Program Overview

The Bala Janaagraha program is designed to educating the futurecitizens about roles and responsibilities of citizenship, theprocesses and institutions of governance

The Bala Janaagraha Program runs from June/July toFebruary/March

The Bala Janaagraha program has all categories of schools onboard (Government, Private-Aided and Private)

The classes are delivered in the medium of English, Hindi andregional languages such as Kannada and Malayalam

The delivery of the program is managed by a Team of 54Facilitators spread across the 24 cities

2

Program Overview

Increase (in percentage) in Bala Janaagraha students between academic years: 118

Increase (in percentage) in the number of Bala Janaagraha schools between academic years: 108

Increase (in percentage) in the number of city outreach between academic years: 118

Bala Janaagraha Program 2013-2014 2014-2015

Total No. of Cities 11 24

Total No. of Schools 233 485

Total No. of Students 16, 439 36, 000

3

Donor Partnerships for Bala Janaagraha Program

8

4 4

2 2 2

1 1

0

1

2

3

4

5

6

7

8

9

Dell HSBC Pirojsha GodrejFoundation

Mindtree BPCL GMR Group Sharekhan TPG Capital

Year

s o

f P

artn

ersh

ip

Donor Partnership for Bala Janaagraha Program

Donors Partnership (In Years)

Donors

4

The Bala Janaagraha Program has expanded largely due to the financial support from like-

minded donors and institutions, who believe in the work areas of Janaagraha and have

partnered with Janaagraha from program inception

Theory of Change: Bala JanaagrahaActivity/Input Output Intermediate Outcomes Long-term Outcomes

Ob

jecti

ves

-Bala Janaagraha Resource Book.

-12 Curriculum Sessions delivered by

the Facilitators

-4 E-module sessions

-Increase in Civic Literacy Score

-Increase in Civic Awareness and

Civic Engagement

-Civic Fest

-Children actively take part in the development of

the neighborhood/city. E.g. Cleanliness Drives,

Planting Trees, Raising Awareness

-Children have adequate knowledge of the urban

governance that directly impacts the quality of

lives. E.g., Going to Councilors to lodge

complaints about public infrastructure

failure/repairs

-Children’s own attitude towards the city,

community and the Government is changed and

they influence the attitude of their family members

Active Citizenship

Ind

icato

rs

# of Bala Janaagraha Resource books,

translated into multiple languages, and

provided (for free) to the students

# of Facilitators delivering the

curriculum sessions

# of computer games delivered either

through individually handled computers

or as a classroom activity

-Percentage change in civic literacy

scores between pre/post surveys

-Photos taken by students in the

Bala Janaagraha Photo Project

-Interview excerpts from FGD

-Civic Project

-% increase in Civic Literacy Scores

-Increased involvement of Bala Janaagraha students

in civic campaigns

Increase in citizens engagement beyond

voting

Data

So

urc

e

-Project MIS

-Pre/Post Survey (Civic Literacy

Test )

-Focus Group Discussions

-Bala Janaagraha Photography

Project

-# of Civic Campaigns.

-I Change My Street/I Change My City

-Future Civic Projects

-Updated Student Information Database

-Updated Student Information

Database

-I Change My City website

-Janaagraha-Brown Citizenship Index

Ass

um

pti

on

s

-Schools allow Facilitators to hold the

Bala Janaagraha Curriculum Sessions.

-Schools have functional desktop/laptop

to deliver the E-Module sessions.

-Monitoring whether students are taking the test

without help from the friends/peers.

-Students are supported by friends/families in

raising awareness

-Updated information on civic campaigns and

volunteers involved in the campaign

Schools continue to take part in the

Bala Janaagraha Program and allow the

Janaagraha Research Team to use

quantitative and qualitative methods to

conduct Impact Assessment.

-Bala Janaagraha Resource Books

become available in multiple languages.

More Schools are encouraged to join

the Bala Janaagraha program.

-Integration of Bala Janaagraha database

with the institutional database that also

is connected to I Change My City,

allowing for longitudinal monitoring of

existing Bala Janaagrahis/alumnus of the

program

5

Impact Assessment : Different Variants

Bala Janaagraha

Program (2014-2015)

School-level Data Collection

Pre-Survey Civic Literacy Test

Pre

-Pro

gram

Sta

ge

Bala Janaagraha

Photography

Project

Post-Survey Civic Literacy Test

Focus Group Discussions

Po

st-Pro

gram Stage

Impact Assessment Research Methodology

School-level Data Collection Quantitative

Pre/Post Civic Literacy Test Quantitative

Focus Group Discussions Qualitative

Bala Janaagraha Photography Project Qualitative

6

Pre/Post Civic Literacy Test: SurveyBrief Excerpts

7

Pre/Post Civic Literacy Test: Respondent Mix

Population Sample Pre Civic Literacy Test Post Civic Literacy Test

Total # of Respondents (English) 24, 465 19, 355

Total # of Respondents (Kannada) 1,133 238

Total # of Respondents (Hindi) 629 867

Total # of Respondents 26,227 20,460

8

• For the Impact Assessment, we include all respondents who took part in the Pre/Post Civic Literacy Test. • The total sample size(n) for Impact Assessment =20,240

*Decline in # between pre/post surveys can be attributed to school dropouts, absentee students, etc

Themes for the Civic Literacy Test

9

Civic Knowledge

• Knowledge of three levels of Government; Local Government of the City; City Planning

• Responsibilities of the Local Government; Local Service Delivery Agencies; Civic Responsibilities & Citizenship Rights; Urban Planning themes and challenges

Civic Engagement

• Participating in civic activities

• Addressing civic problems

Civic Awareness

• Knowledge of civic issues specific to school locality and neighborhood

Key Findings: Survey

10

Pre/Post Literacy Test Scores

Schools/Cities Pre-Survey Test Score

(In Percentage)

Post-Survey Test Score

(In Percentage)

All Schools/Cities 39% 57%

11

Results: Civic Literacy Scores

*

12

0

10

20

30

40

50

60

Score

s (I

n P

erc

enta

ge)

Pre/Post Civic Literacy Test Scores-All Cities/Schools

Civic Literacy Scores (In Percentage): 2014-2015

Pre-Survey Test Score

Post-Survey Test Score


*

13

0

10

20

30

40

50

60

Score

s (I

n P

erc

enta

ge)

Pre/Post Civic Literacy Test Scores for All Cities/Schools

Civic Literacy Scores (In Percentage): 2014-2015

Pre-Survey Test Score

Post-Survey Test ScoreIncrease in civic

literacy scores by

18% (23points)


*

14

Civic Literacy Scores from Pre to Post Civic Literacy Test, increased in 19 out of 22

cities that took part in the pre/post surveys

*Schools from Bhubaneshwar and Ludhiana did not do pre/post surveys and Ludhiana did not take part in post surveys

0

10

20

30

40

50

60

70

80

AHM BHO BLR CHA CHE DEH DEL HYD JAI JOD KAN KOC KOL LUC MUM NAG PAT PUN RAI RAN TVM UDA

Sco

res

in P

erc

en

tage

Pre/Post Literacy Scores by Cities

Pre-Test Scores Post-Test Scores


*

15

Three cities (Ahmedabad, Kanpur and Kolkata) witnessed decline in Civic Literacy Scores in Post-Survey, marginally

Schools with more students scored lower than other schools

Post-surveys were started by students, but left incomplete (Kolkata, Kanpur)

*Schools from Bhubaneshwar and Ludhiana did not do pre/post surveys and Ludhiana did not take part in post surveys

0

10

20

30

40

50

60

70

80

AHM BHO BLR CHA CHE DEH DEL HYD JAI JOD KAN KOC KOL LUC MUM NAG PAT PUN RAI RAN TVM UDA

Sco

res

in P

erc

en

tage

Pre/Post Literacy Scores by Cities

Pre-Test Scores Post-Test Scores

3 cities witnessed decline in score

Results: Civic Engagement*Civic Activities Pre-Survey Post-Survey

Yes, I have

participated

No, I have not

participated

Yes, I have

participated

No, I have

not

participated

Save Water Campaign 38% 62% 61% 39%

Creating awareness on segregation of dry and wet waste 18% 82% 46% 54%

Save tree campaign 50% 50% 73% 27%

Cleaning the school campus 53% 47% 73% 27%

Safe and environment friendly festival (e.g., Diwali) 46% 55% 56% 44%

Traffic Police Day celebration 13% 87% 24% 76%

Demonstration of Rain Water Harvesting 25% 75% 43% 57%

Save electricity campaign/Switch off light campaign 41% 59% 59% 41%

Know your city campaign 24% 76% 50% 50%

16

*Respondents could select multiple activities

Results: Civic Engagement*Civic Activities Pre-Survey Post-Survey

Yes, I have

participated

No, I have not

participated

Yes, I have

participated

No, I have

not

participated










17

28% increase in generating

awareness on segregation

of dry and wet waste


Results: Civic EngagementCivic Activities Pre-Survey Post-Survey

Yes, I have

participated

No, I have not

participated

Yes, I have

participated

No, I have

not

participated










18

Overall increase in different forms of Civic Engagement through different activities

between the pre and post survey.

Top three civic activities that witnessed greater student engagement: Save Tree

Campaign, Cleaning the School Campus and Save Water Campaign (which

are highlighted in the table)


Results: Civic Engagement*

Visiting Civic Authorities Pre-Survey Post-Survey

Yes, I have

visited

No, I have

not visited

Yes, I have

visited

No, I have

not visited

Corporator of your ward 19% 81% 50% 50%

Police Station 32% 68% 40% 60%

Local Ward Office 17% 83% 39% 61%

Local Water Supply Office 25% 75% 40% 60%

Electricity Board 37% 66% 57% 43%

19

*All responses that were left blank were captured under the “no” category of responses

**Respondents were asked to identify one issue, but they identified multiple issues

Results: Civic Engagement

Visiting Civic Authorities Pre-Survey Post-Survey

Yes, I have

visited

No, I have

not visited

Yes, I have

visited

No, I have

not visited

Corporator of your ward 19% 81% 50% 50%

Police Station 32% 68% 40% 60%

Local Ward Office 17% 83% 39% 61%

Local Water Supply Office 25% 75% 40% 60%

Electricity Board 37% 66% 57% 43%

20

22% increase in the

number of visits to the

local ward office

• During the post-survey, greater percentage of students seem to have visited local

authorities/agencies post-survey

• Visiting the Local Ward Office witnessed the largest increase (22%) between pre/post

surveys

*All responses that were left blank were captured under the “no” category of responses

Results-Civic Awareness

Civic Issues Pre-Survey Post-Survey

Garbage Accumulation 26% 43%

Bad Roads 13% 32%

Frequent Power Cuts 6% 15%

Lack of Continuous Water Supply 5% 14%

Pollution 22% 29%

21

*Respondents identified more than one issue

Civic awareness increased in the post-survey period, since more students responded to the question on the main civic issues affecting their school locality

Garbage accumulation continued to be the top most concern for student in the pre and post survey

Key Findings: Focus Group Discussions

22

Focus Group Discussion Themes

I. Development of the City: Sustainable development will address

the future problems of a city .

Students received support from the Schools and Corporators to

conduct projects on sustainable development. In Pune procuring

support & cooperation was much easier than in Bangalore and Mumbai.

In Bangalore, the Corporator worked only due to unprecedented

pressure created by the Students.

II. Program Development: Bringing in real life examples and/or

have subject matter speakers with practical exposure make the

learning process much easier and more interesting.

Students suggests screening movies/short stories so as to aid the

learning process

23

Focus Group Discussions

III. Civic Engagement: Students continue generating awareness within

the larger community

In Pune, current group of Bala Janaagraha students started a tree

planting project that also involved 6th graders and 7th graders.

The intensions were twofold: First, engage 6th and 7th graders in civic

campaigns. Second, current 8th graders wanted to serve as potential

mentors for future Bala Janaagraha students.

IV. Program Learning: Greater awareness on themes of urban

planning, technology, maintaining and improving city life to behavioral such

as learning to converse with politicians and connecting with the local

people and helping people

24

Key Findings: The Bala Janaagraha Photography Project

25

Bangalore A Child’s View: Key Themes/Findings

26

Majority of children felt that they liked the parks in Bangalore. Some mentioned they enjoyed iconic green spaces in Bangalore, including LalBagh and Cubbon Park.

Bala Janaagrahis were worried about green spaces that had been destroyed or dirtied with garbage, and said they needed more parks.


27

.

Students identified electric hazards as one of the main challenges

Reasons for concern: loose electric wires and open transformers in parks and roads; power shortages affecting families and the economy


28

.

Students identified electric hazards as one of the main challenges

Reasons for concern: loose electric wires and open transformers in parks and roads; power shortages affecting families and the economy

Garbage: Every child who participated in the project is concerned about the garbage in Bangalore due to its implications on public health and environment.

Solutions: Government needs to do better job at dealing with garbage; People must take responsibility and clean up themselves


29

Children felt that the government and the citizens should grow more trees, plants and flowers. They were concerned about the impact of garbage, pollution on nature, health of Bangalorean and animals• Majority expressed concerns about wastage of environmental

resources, such as water


30

Children were concerned with the conditions of roads and footpaths in the city. Students noticed potholes, uneven roads and other things that could cause motor accidents.• Broken footpaths, open drainage and other obstacles make it hard for people to walk

around and could injure pedestrians


31

Students noted that roads need to be planned better with moresafety measures. Others criticized rash driving and said trafficissues create problems for pedestrians, including children andthe elderly.

Conclusion

32

Conclusion: Survey

From the pre to post survey, the civic literacy scores improved

by 18% (Score difference: 23 points)

The difference in scores indicate the impact of the Program in the improvement

of the civic learning among students of the Program.

Civic engagement and civic awareness for Bala Janaagraha

students increased in post survey

33

Conclusion: Focus Group Discussions & Photography Project

In Focus Group Discussions students’ expressed willingness to engage with

the program post completion. They continue to participate in various civic

activities (e.g., save tree campaigns; traffic police day celebrations)

Students engage with family and friends in raising civic awareness. But they

often face resistance from their families, when they try to participate in civic

activities

The Bala Janaagraha Photography Project reveal that garbage, environment,

parks and street/footpaths, were among the major themes of concern for

urban planning and development.

The solutions offered by a Bala Janaagraha students are more policy oriented than non

Bala Janaagraha students

Bala Janaagrahis are more willing to be participants of civic activities than non-Bala

Janaagraha students

34

Future Road map for Impact Assessment

Organizing workshops for the facilitators to inform on why/how Impact Assessment matters for the purposes of evaluating the Bala Janaagraha program evaluations

Surveys to be conducted by external survey agency that allows to remove bias in execution of surveys

Streamlining the surveying process with redesigning the survey and using Tabs to conduct surveys. By doing so, the analysis can be done more effectively & efficiently

Focus on the use of qualitative tools, including Focus Group Discussions and Structured Interviews to study civic learning among Bala Janaagraha students

35

Annexure: Methodology

36

Methodology- Survey

Sample Selection: All students who are part of the Bala Janaagraha program

take part in survey. However, only those students who have taken part in the

pre/post survey are included for Impact Assessment

Students were tracked using the Student UID that is assigned at the time of pre-

surveys

Total Sample# 20,240

Data Entry: Surveys took place in English, Kannada and Hindi. They were all

entered as is in Kannada and Hindi, then translated to English

Data Coding: Responses are both open and close-ended. Not all responses

receive points

37

Methodology: Focus Group Discussions (FGD)

Sample Selection: Two focus groups (12 participants/ group) were conducted for anhour, with students who were participants of the National Civic Fest, 2015

FGD Execution:

Two focus groups (12 participants/ group) were conducted for an hour, with studentswho were participants of the National Civic Fest, 2015.

The facilitators (Sayantani, Alia and Reema) guided the group through discussion andkept the group focused on the topic for discussion. The entire session was recordedand transcribed.

Focus Group Discussion Themes:

Nature of civic engagement before/after the Bala Janaagraha program;Conceptualizing the Ideal City, Role of Active Citizenship, Sustainable Development;Future engagement with the Bala Janaagraha program as Alumni.

Bala Janaagraha program-Themes, feedbacks, suggestions on The Resource Book, TheCivic Project, Post Program Engagement, etc.

38

Methodology: Bala Janaagraha Photography Project

Research Questions:

Does Bala Janaagraha shape how students perceive urban infrastructure challenges?

How do Bala Janaagraha students, as compared to non-Bala Janaagraha students, understand issues related to urban infrastructure?

Sample Selection: Five schools that had a Bala Janaagraha program and five schools that did not so that the

responses of the students.

All the students were 8th standard students.

Bala Janaagraha facilitators, who are tasked with running the program around Bangalore,

helped to identify schools that were and were not part of the Bala Janaagraha program

that would be willing to participate.

Schools included a mix of government and private schools with students from a variety of

socioeconomic backgrounds.

39

Methodology: Bala Janaagraha Photography Project

Data Collection:

Photo walk: Students then split into groups and walked around the area near their

school for 30 minutes to an hour

Each child was asked to fill out a worksheet in which they could write about up to

ten pictures

Students were asked to select pictures that they felt represented key positive and

negative issues for them and to write a few lines explaining what the picture showed

and why they had selected it

Student Interview on photography themes, city problems and solutions,

neighborhood

Data Cleaning and Analysis:

Once interviews and worksheets completed, respondent information was coded and

analyzed using the qualitative software, NVivo.

Responses were coded by themes and frequency: Garbage, Traffic, Environment, People,

Traffic, etc.

40

THANK YOU

41

Bala Janaagraha Impact Assessment,...

Documents

Transcript of Bala Janaagraha Impact Assessment,...