Implementation Completion and Results Report (ICR)...

Document of

The World Bank

FOR OFFICIAL USE ONLY

Report No: ICR00004122

IMPLEMENTATION COMPLETION AND RESULTS REPORT

CREDIT No. 5088-IN

ON A

CREDIT

IN THE AMOUNT OF SDR 322.4 MILLION

(US$500 MILLION EQUIVALENT)

TO THE

REPUBLIC OF INDIA

FOR THE

INDIA: SECONDARY EDUCATION PROJECT ( P118445 )

December 21, 2017

Education Global Practice

South Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective September 30, 2017)

Currency Unit = Indian Rupees (INR)

INR 65.32 = US$1

US$1.41 = SDR 1

FISCAL YEAR

April 1 – March 31

Regional Vice President: Annette Dixon

Country Director: Junaid Kamal Ahmad

Senior Global Practice Director: Jaime Saavedra Chanduvi

Practice Manager: Keiko Miwa

Task Team Leader(s): Sangeeta Dey

ICR Main Contributor: Mari Shojo

ABBREVIATIONS AND ACRONYMS

AWP&B Annual Work Plan and Budget BP Bank Policy CPS Country Partnership Strategy DEA Department of Economic Affairs DFID U.K. Department for International Development DP Development Partner DPO District Project Office EBB Educationally Backward Block EIRR Economic Internal Rate of Return EU European Union FA Financing Agreement FM Financial Management GER Gross Enrollment Ratio GoI Government of India GPI Gender Parity Index ICR Implementation Completion and Results Report INR Indian Rupees IO Intermediate Outcome ISR Implementation Status and Results Report JRM Joint Review Mission M&E Monitoring and Evaluation MHRD Ministry of Human Resource Development NAS National Achievement Survey NCERT National Council of Educational Research and Training NCSL National Centre for School Leadership NER Net Enrollment Ratio NITI National Institution for Transforming India NPV Net Present Value NUEPA National University of Educational Planning and Administration OP Operational Policy PAD Project Appraisal Document PDO Project Development Objective PISA Programme for International Student Assessment PMS Project Monitoring System RMSA Rashtriya Madhyamik Shiksha Abhiyan PPR Preliminary Project Proposal SC Scheduled Caste SDR Special Drawing Rights SEMIS Secondary Education Management Information System SFD Special Focus District SMDC School Management and Development Committee

SPO State Project Office SSA Sarva Shiksha Abhiyan ST Scheduled Tribe TSG Technical Support Group SWAp Sector-wide Approach UDISE Unified District Information System for Education UT Union Territory

TABLE OF CONTENTS

DATA SHEET .......................................................................................................................... 1

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5

A. CONTEXT AT APPRAISAL .........................................................................................................5

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION .............................................................. 10

II. OUTCOME .................................................................................................................... 11

A. RELEVANCE OF PDO .............................................................................................................. 11

B. ACHIEVEMENT OF PDO (EFFICACY) ........................................................................................ 12

C. EFFICIENCY ........................................................................................................................... 16

D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 17

E. OTHER OUTCOMES AND IMPACTS ......................................................................................... 17

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 20

A. KEY FACTORS DURING PREPARATION ................................................................................... 20

B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 20

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 23

A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 23

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 25

C. BANK PERFORMANCE ........................................................................................................... 26

D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 27

V. LESSONS AND RECOMMENDATIONS ............................................................................. 27

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 30

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 40

ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 42

ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 43

ANNEX 5. BORROWER COMMENTS ...................................................................................... 46

ANNEX 6. BORROWER’S COMPLETION REPORT .................................................................... 47

ANNEX 7. SUPPORTING DOCUMENTS .................................................................................. 55

The World Bank India: Secondary Education Project ( P118445 )

Page 1 of 55

DATA SHEET

BASIC INFORMATION

Product Information

Project ID Project Name

P118445 INDIA: SECONDARY EDUCATION PROJECT ( P118445 )

Country Financing Instrument

India Specific Investment Loan

Original EA Category Revised EA Category

Partial Assessment (B) Partial Assessment (B)

Organizations

Borrower Implementing Agency

DEA, Ministry of Finance MHRD

Project Development Objective (PDO) Original PDO

To help India to achieve increased and more equitable access to good quality secondary education through support of the Government‘s ongoing program for secondary education as delineated in the Rashtriya Madhyamik Shiksha Abhiyan (RMSA) Framework. PDO as stated in the legal agreement

To assist the Recipient in achieving increased and more equitable access to good quality secondary education by supporting its ongoing program for secondary education


Page 2 of 55

FINANCING

Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$)

World Bank Financing IDA-50880

500,000,000 500,000,000 475,874,311

Total 500,000,000 500,000,000 475,874,311

Non-World Bank Financing

Borrower 12,306,000,000 12,306,000,000 4,460,000,000

UK: British Department for International Development (DFID)

90,000,000 90,000,000 93,955,050

Total 12,396,000,000 12,396,000,000 4,553,955,050

Total Project Cost 12,896,000,000 12,896,000,000 5,029,829,361

KEY DATES

Approval Effectiveness MTR Review Original Closing Actual Closing

22-Mar-2012 07-Nov-2012 08-Sep-2015 30-Jun-2017 30-Jun-2017

RESTRUCTURING AND/OR ADDITIONAL FINANCING

Date(s) Amount Disbursed (US$M) Key Revisions

10-Jun-2014 176.65 Change in Disbursements Arrangements Change in Financial Management

KEY RATINGS

Outcome Bank Performance M&E Quality

Satisfactory Moderately Satisfactory Substantial


Page 3 of 55

RATINGS OF PROJECT PERFORMANCE IN ISRs

No. Date ISR Archived DO Rating IP Rating Actual

Disbursements (US$M)

01 17-Jul-2012 Satisfactory Satisfactory 0

02 07-Apr-2013 Satisfactory Satisfactory 52.58

03 28-Aug-2013 Satisfactory Satisfactory 82.60

04 26-Feb-2014 Satisfactory Satisfactory 82.60

05 07-Oct-2014 Satisfactory Moderately Satisfactory 163.38

06 11-Mar-2015 Satisfactory Moderately Satisfactory 249.19

07 09-Oct-2015 Moderately Satisfactory Moderately Unsatisfactory 249.19

08 31-Mar-2016 Moderately Satisfactory Moderately Unsatisfactory 322.40

09 06-Jun-2016 Moderately Satisfactory Moderately Satisfactory 322.40

10 16-Nov-2016 Moderately Satisfactory Moderately Satisfactory 322.40

11 09-Jun-2017 Moderately Satisfactory Moderately Satisfactory 322.40

12 29-Jun-2017 Satisfactory Moderately Satisfactory 322.40

SECTORS AND THEMES

Sectors

Major Sector/Sector (%)

Education 100

Secondary Education 100

Themes

Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Human Development and Gender 100

Education 100

Access to Education 50

Education Financing 50


Page 4 of 55

ADM STAFF

Role At Approval At ICR

Regional Vice President: Isabel M. Guerrero Annette Dixon

Country Director: N. Roberto Zagha Junaid Kamal Ahmad

Senior Global Practice Director: Amit Dar Jaime Saavedra Chanduvi

Practice Manager: Amit Dar Keiko Miwa

Task Team Leader(s): Tobias Linden Sangeeta Dey

ICR Contributing Author: Mari Shojo


Page 5 of 55

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES

A. CONTEXT AT APPRAISAL

Context

1. Country context. At the time of appraisal of the Secondary Education Project in 2012, India was a lower-middle-income country with a gross national income of US$1,330. Its economy was growing fast, with an economic growth rate of 6.6 percent. Rapid economic growth over the previous decade had contributed to reductions in poverty. The percentage of people living below the official poverty line of the Government of India (GoI) declined from 45 percent in the early 1990s to 30 percent in 2009–2010. However, while rural poverty had decreased significantly, a huge concentration of poor people remained in rural areas and structural inequalities based on gender, caste, and tribe persisted. Indicators of poverty and human development for these groups lagged behind those of the general population.

2. The rapid economic growth and reduced poverty rates did not translate into greater human development. India’s performance on human development indicators fell behind the world average and was on a downward trend. At appraisal, the country ranked 134 out of 187 on the 2011 Human Development Index scale. Persistently high rates of malnutrition contributed to poor health conditions and undermined productivity. Gender inequality remained high despite progress in education, health, maternal mortality, and fertility. The priorities of the GoI for the project period included sustaining economic growth while making it more inclusive. The 11th Five-Year Plan (2007–2012) clearly stated that education was the key to achieving national goals. Thus, the education sector was given high priority by the GoI.

3. Sector context. India has made significant progress in the education sector. Primary education has been largely universalized, with a gross enrollment ratio (GER) of 99 percent. The focus of the Government’s efforts has shifted to issues of the expansion of equitable access to and quality of secondary education. At the time of appraisal, the enrollment rate for grades 9–12 was just 40 percent; of those enrolled, approximately 15 percent dropped out and one-third failed their board examinations. Inequities in all dimensions, including interstate, caste, and gender, were a major concern. Research1 on learning outcomes suggests that students’ learning is modest. However, there was no reliable national learning assessment available before at the secondary education level, comparable across states and over time.

4. In 2009, Rashtriya Madhyamik Shiksha Abhiyan (RMSA), which means National Secondary Education Campaign, was launched as a GoI centrally sponsored program in partnership with the state governments with the main objective to make good quality secondary education available, accessible,

1 For instance, small-scale standardized assessments of student achievements in mathematics commissioned by the World Bank show that the quality of instruction and learning is modest at the secondary education level. Two Indian states, Himachal Pradesh and Tamil Nadu, participated in the Programme for International Student Assessment (PISA) 2009+ study, which measures 15-year-old students’ reading, mathematics, and science literacy. The mean scores for reading, mathematics, and science suggest that learning outcomes in India do not meet international standards.


Page 6 of 55

and affordable to all young persons. The program sought to enhance enrollment in grades 9 and 10 by providing a secondary school within a reasonable distance of every habitation; improve the quality of education imparted at the secondary level by ensuring that all secondary schools conform to prescribed/standard norms; remove gender, socioeconomic, and disability barriers; and achieve universal access to secondary level education by 2017 (that is, by the end of the 12th Five-Year Plan). The RMSA was supported with external funding by development partners (DPs): The World Bank, the United Kingdom’s Department for International Development (DFID), and the European Union (EU). The Secondary Education Project was part of the GoI’s RMSA framework, and the World Bank’s support was provided through a sectorwide approach (SWAp).

5. Rationale for the World Bank involvement. The World Bank’s Country Assistance Strategy for India for 2009–20122 focused on tackling the challenges to the achievement of rapid, inclusive growth; ensuring sustainable development; and improving service delivery. The project contributed to the first pillar of rapid, inclusive growth by reducing the human capital constraints to growth in both rural and urban areas. It also contributed to the third pillar of improved service delivery by supporting the development of secondary education services. The GoI’s 11th Five-Year Plan highlighted key priorities in education: (a) raising the minimum level of education to grade 10 and universalizing access to secondary education accordingly; (b) ensuring good quality secondary education with a focus on science, mathematics, and English; and (c) aiming toward a major reduction in gender, social, and regional gaps in enrollments, dropouts, and school retention. The project contributed to these priorities by supporting the GoI’s RMSA program.

Theory of Change (Results Chain)

6. The Project Appraisal Document (PAD) does not illustrate the results chain. In this Implementation Completion and Results Report (ICR), figure 1 was created to show the theory of change for the project. As shown in figure 1, the project’s activities were designed to promote the achievement of the objectives/outcomes of the project, and the design of the activities and results outputs were closely linked to the Project Development Objective (PDO).

Project Development Objectives (PDO)

7. The PDO stated in the Financing Agreement (FA) is “to assist the Recipient in achieving increased and more equitable access to good quality secondary education by supporting its ongoing program for secondary education.” Although there are some differences in wording, the statements in the PAD3 and the FA are consistent.

2 World Bank (2009). Country Strategy for India for the Period FY2009-2012. 3 The PDO stated in the PAD is “to help India to achieve increased and more equitable access to good quality secondary education through support of the Government’s ongoing program for secondary education as delineated in the RMSA Framework.”


Page 7 of 55

Figure 1. Theory of Change: India Secondary Education Project

RMSA

Part 1: Improving the quality of secondary education

Part 2: Improving equitable access to secondary education

Part 4: Developing innovative approaches to secondary education

Outputs Intermediate Outcomes PDO-level Outcomes

PDO

Access • New schools established • Existing schools upgraded • Main infrastructure features

available at school • Vocational education introduced • Additional teachers recruited

Equity • Girl’s hostels established in EBBs • New school established in

underserved areas • Teachers trained on gender

Quality • Learning materials provided

Curriculum framework adopted • National Program on School

Standards and Evaluation developed

• Qualified teachers appointed • Teaches trained • Head teachers/principals trained

for school leadership • Local institutions strengthened • NAS conducted • Staff positions at state, district

and sub-district levels filled • Innovation activities approved

and implemented • SMDCs established and trained • Official database improved

Access • GER • NER • Elementary to secondary transition • Gross secondary completion rate • Proportion of schools with all the main

infrastructure available as required

Equity

• Number of girls per 100 boys completing grade 10

• Share of SCs in secondary completers • Share of ST in secondary completers • GER for students from disadvantaged

social categories

Quality • Proportion of schools with all subject

specific teachers available in required number

• Proportion of teachers with professional qualification

• Proportion of schools with specific teachers available to teach for five core subjects

• Proportion of SDMCs received capacity/management training

• All staff positions at state, district and sub-district levels filled

• Number of innovative activities approved

• 25% of the completed innovation component projects which meet at least 75% of their agreed target

Access • Enrollment

of students in grades 9 and 10

Equity • Gender

parity Index in enrollment

Quality • New

system of national assessment of students’ achievement developed and used to inform the RMSA’s quality improvement policies

• Gross graduation rate

To assist the Recipient in achieving increased and more equitable access to good quality secondary education by supporting its ongoing program for secondary education

Part 3: Improving accountability of secondary education


Page 8 of 55

Key Expected Outcomes and Outcome Indicators

8. The ICR notes that the PDO consists of multiple outcomes. The following key outcomes were used to assess the achievement of the PDO: (a) increased access to secondary education, (b) improved equitable access to secondary education, and (c) enhanced quality of secondary education. For each outcome, the achievement of PDO-level Indicators was reviewed. Additionally, several intermediate outcome (IO) indicators were also taken into consideration for more robust assessment (see section II.B Achievement of PDO).

Outcome 1: Increased access to secondary education

9. Increased access was measured by the following indicators:

• Enrollment of students in secondary (grades 9 and 10) (PDO-level Indicator)

• Gross enrollment ratio (GER) (IO Indicator)

• Net enrollment ratio (NER) (IO Indicator)

Outcome 2: Improved equitable access to secondary education

10. Improved equitable access was measured by the following indicators:

• Gender parity index (GPI) in enrollment (in schools covered by RMSA) (PDO-level Indicator)

• Number of girls per 100 boys completing grade 10 (IO Indicator)

• Share of Scheduled Castes in secondary completers (IO Indicator)

• Share of Scheduled Tribe in secondary completers (IO Indicator)

• GER for students from disadvantaged social categories (IO Indicator)

Outcome 3: Enhanced quality of secondary education

11. Enhanced quality was measured by the following indicators:

• New system of National Assessment of Students’ Achievement developed and used to inform the RMSA’s Quality Improvement Policies (PDO-level Indicator)

• Proportion of schools with specific teachers available to teach for the five core subjects (IO Indicator)

• Gross graduation rate (in schools covered by RMSA) (PDO-level Indicator)

Components

12. The project supported all activities detailed in the GoI’s RMSA Framework, and the World Bank’s support (SDR 322.4 million) was provided through a SWAp. Therefore, there were no separate components. However, the project consisted of the following parts, as described in the FA.

Part 1: Improving the quality of secondary education

• Recruitment and appointment of additional teachers


Page 9 of 55

• In-service teacher professional development

• Building the capacity of relevant local institutions to support quality improvement in secondary schools and building professional development networks at the local level

• Provision of libraries and books, science laboratories and laboratory consumables, computer laboratories, and other needed school supplies

• Carrying out assessments of student learning outcomes and monitoring educational outcomes through secondary education management information systems (SEMIS), project management information systems, quality monitoring tools of the National Council of Education Research and Training, national assessment surveys, and third-party research and evaluation studies

Part 2: Improving equitable access to secondary education

• Expansion, repair, and renovation of existing government secondary schools, upgrading upper primary schools into secondary schools, and opening new secondary schools, mainly in underserved areas

• Provision of relevant learning materials, employment of qualified teachers, and support of outreach activities to communities

• Setting up girls’ hostels in educationally backward blocks (EBBs)

• Building separate toilets for girls and boys in secondary schools

Part 3: Improving accountability of secondary education

• Strengthening existing school management mechanisms and developing new mechanisms, including, among others, strengthening the capacity of the School Management and Development Committees (SMDCs) established in participating schools

Part 4: Developing innovative approaches to secondary education

• Supporting innovation in secondary education through, among others, the financing of pilot innovation sub-projects at state and/or sub-state level and the scaling up of such sub-projects at the national, state, or sub-state levels

13. As mentioned earlier, there are no separate components and hence the costs by component are not presented. There is no difference between the overall estimated and actual total project costs (SDR 322.4 million, IDA US$500 million).4

14. Counterpart funding from the GoI for the RMSA was much lower (US$4,460 million)5 than that estimated when the project was approved (US$12,306 million). There are a few reasons for this. First, the original estimated counterpart funds shown in the PAD were based on the amount of budgetary support for the RMSA that was stated in the 11th Five-Year Plan6. At that time, the GoI

4 The project was completed with full disbursement. There was some loss in U.S. dollars due to exchange rate fluctuation. The actual disbursed amount in U.S. dollars is US$475,874,311 (see Data Sheet). 5 The actual total counterpart funds from the GoI totaled INR 291,070 million. 6 The 11th Five-Year Plan (Planning Commission 2008) states the outlay for the 2007-12 period at INR 2.7 lakh crores for education of which 20 percent was to be allocated for secondary and higher secondary education. At the exchange rate of 44 in


Page 10 of 55

planned to gradually extend the RMSA program to higher secondary education (grades 11 and 12) and cover all government and government-aided schools. However, due to a paucity of funds, the RMSA scope was not expanded to cover higher secondary education during the project period7. Hence, the actual counterpart funding was much lower than that estimated at appraisal. Second, as is often the case, the budget approved by the Ministry of Finance was lower than the amount that the Ministry of Human Resource Development (MHRD) had requested. Third, the fund-sharing pattern between the center and the states for the RMSA was changed from 75:25 to 60:40 in 2015-2016 (for the North-Eastern states and for the hill states such as Himachal Pradesh, 90:10 sharing ratio continued to apply). With this devolution of funds, the center’s share had gone down; the Central Government’s expenditures and the budget allocation for the RMSA (counterpart funding) turned out to be lower.

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION

Revised PDO and Outcome Targets

15. There was no formal revision to the PDO, outcome targets, or PDO indicators.

Other Changes

16. Change in Part 4 of the component. The component was not formally revised. However, Part 4 of the component with regard to developing innovative approaches to pilot ideas or scale up proven interventions was not implemented due to lower allocations of counterpart funding by the GoI. Therefore, there was no progress on activities under Part 4. The following two IO indicators on innovations were not assessed during project implementation: (a) number of innovative activities approved and (b) 25 percent of the completed innovation component projects that meet at least 75 percent of their agreed targets.

17. Changes in financial management. A level two restructuring took place during the implementation of the project to make changes in financial management (FM). This restructuring was required to ensure smoother arrangements for withdrawals from the credit, without diluting the FM standards or arrangements, and bring this project in line with the rest of the India portfolio of the World Bank. The amendment to the FA was signed in June 2014. Changes included (a) deleting the requirement of preparation and submission of a consolidated report on audits summarizing all the state reports in the national program supported by the project and (b) deleting the requirement of submission of a consolidated report for the previous financial year before a withdrawal is made for a given financial year.

2007, this was equivalent to approximately US$12 billion, as shown in the PAD. 7 The 12th Five-Year plan (Planning Commission 2013) states the indicative gross budgetary support to the MHRD for the RMSA – INR 27,466 crores (US$4.2 billion at exchange rate of 65). The MHRD budget (revised budget) for the RMSA from 2012-2013 to 2016-2017 was INR 17,040 crores (US$2.6 billion, at exchange rate of 65). This is 62 percent of the indicative budget.


Page 11 of 55

Rationale for Changes and Their Implications on the Original Theory of Change

18. Rationale for change in Part 4 of the component. Part 4 was originally designed to promote innovative activities in secondary education, under which untied funds are allocated to states to pilot ideas or scale up proven interventions. The World Bank helped prepare a concept note on creating a separate Innovation Fund to fund proposals from states. This was done during project preparation, with some discussions occurring during the first year of the project. The MHRD opened the funding window for innovations and began encouraging and approving innovative practices through the Annual Work Plan and Budget (AWP&B) in early 2015. However, due to a paucity of funds and some resistance to this new approach, the MHRD were unable to take this initiative forward. As a result, the two IO indicators on innovations were not assessed during project implementation.

19. Rationale for change in FM. The original FA required that a consolidated report on audits, summarizing all the state reports in the national program supported by the project, be prepared and submitted to the World Bank every year. In addition, the FA required that, before a withdrawal is made for a given financial year, a consolidated report for the previous financial year be submitted to the World Bank. However, the consolidated report for the previous year is not actually due at the time of the first withdrawal application in a particular year. In addition, the World Bank’s dialogue on FM issues is driven by a review of each state’s financial statement rather than by the consolidated report. By the time the second withdrawal application for a financial year is due, the state financial statements are also due. Therefore, the submission of the state’s financial statement is a more fitting withdrawal condition than the submission of the consolidated report. In fact, the preparation of the consolidated report does not serve to strengthen the FM arrangements and has the inadvertent effect of delaying withdrawals. Therefore, it was proposed to amend the FA to delete this requirement and substitute it with the requirement to submit state financial statements as a withdrawal condition.

20. The changes discussed earlier did not affect the original theory of change or the originally expected outcomes.

II. OUTCOME

A. RELEVANCE OF PDO

Assessment of Relevance of PDO and Rating Rating: High

21. The project supported the RMSA through a SWAp and thus is directly relevant to the country’s key development priorities. Recognizing that a well-educated population, equipped with the relevant knowledge, attitudes, and skills, is essential for economic and social development in the 21st century, the GoI’s 12th Five Year Plan identified ‘access’, ‘equity’, ‘quality’, and ‘governance’ as the four main priorities for education policy. It also highlighted that secondary education should continue to receive a major thrust with the centrally sponsored schemes: RMSA and other schemes.

22. The project design is responsive to these priorities articulated in the 12th Five Year Plan. First, the project aimed to promote increased access by supporting expansion, repair, and renovation of the existing government secondary schools; opening new secondary schools; providing main


Page 12 of 55

infrastructures, relevant learning materials, and additional qualified teachers; and introducing vocational education in secondary schools. Second, the project supported improved equity by setting up girls’ hostels in educationally backward blocks (EBBs), establishing new schools in underserved areas, building separate toilets for girls and boys, and providing teacher training on gender sensitization. The project was also designed to promote enhanced quality by providing relevant learning materials to schools, supporting the curriculum framework 2005 adopted in the states, developing the National Program on School Standards and Evaluation, appointing qualified teachers, training teachers, strengthening local institutions, and developing the National Achievement Survey for grade 10. Finally, the project design supported increased accountability in secondary education by providing leadership training to head teachers/principals, aiding in the establishment and training of SMDCs, and strengthening staff capacity at the state, district, and sub-district levels.

23. The project is also well aligned with the World Bank Group’s current Country Partnership Strategy (CPS) for India for the period FY2013–20178. The CPS identifies three key engagement areas: (a) integration, (b) transformation, and (c) inclusion, which form the core of the World Bank Group’s support to India. The project directly supported Engagement Area 3 of the CPS, ‘inclusion’, which promotes human development and strengthens social programs for economic integration and spatial transformation to generate inclusive growth. Therefore, there were no shortcomings in the relevance to the CPS.

24. Based on the assessments, the rating for the relevance of PDO is High.

B. ACHIEVEMENT OF PDO (EFFICACY)

Assessment of Achievement of Each Objective/Outcome Rating: Substantial

25. Efficacy of the PDO is rated Substantial. Despite some challenges faced by the project with regard to low counterpart funding for the RMSA, the project made a major positive impact by supporting various activities under the RMSA Framework with a 100 percent disbursement ratio. Overall, the project was successful in attaining its stated PDO. All key performance indicators were highly relevant to assessing the outcomes. Students and teachers were included in the project beneficiaries and the total number of beneficiaries reached 24 million as opposed to the corresponding baseline of 11.6 million. This far exceeded the number of beneficiaries that it targeted to serve.

26. The PDO consists of the following three key outcomes: (a) increased access, (b) improved equitable access, and (c) enhanced quality of secondary education. The achievement of each outcome is summarized in the following paragraphs.

Outcome 1: Increased access to secondary education Sub-rating: Substantial

27. Increased access was measured by three outcome indicators: (a) enrollment of students in secondary education (PDO-level indicator), (b) gross enrollment ratio (GER) (IO indicator), and (c) net

8 World Bank (2013). Country Partnership Strategy for India for the Period FY2013-2017.


Page 13 of 55

enrollment ratio (NER) (IO indicator). Achievements made by other project interventions—for instance, improvements in school environments—are also presented below.

28. The project supported the RMSA program of opening new secondary schools/sections. The MHRD approved 12,682 new secondary schools, of which 11,310 schools have become functional, with a total enrollment of 1.47 million students. The establishment of new secondary schools has enhanced the percentage of habitations having a secondary school or section within the specified distance norm. To be precise, the gross access ratio—percentage of habitations having a secondary school or section within the specified distance norm (5 km from the habitation)—increased dramatically from 68.2 percent in 2011–2012 to 87.5 percent in 2016–2017. The RMSA program has provided funding support for the construction of additional classrooms in existing government schools to improve the student-classroom ratio. Of the 50,713 additional classrooms approved, 31,315 schools have completed the civil works. Consequently, the student-classroom ratio improved from 56 in 2010–2011 to 46 in 2015–2016. The project also supported the strengthening of infrastructure/facilities and increasing the proportion of schools having core infrastructure. Compared to the sanctioned numbers, civil works have been completed for 16,328 science labs, 11,295 computer rooms, 15,989 library rooms, and 17,900 art/craft/culture rooms. In addition, 14,424 toilet blocks and 8,647 drinking water facilities have been completed against the sanctioned numbers. Setting up girls’ hostels in EBBs helped female students remain in school.

29. These interventions resulted in substantial progress toward improving access to secondary education. At project closing, the enrollment in secondary education increased by 10 million students, increasing from the baseline of 28.3 million to 39.0 million9 at project closing. The GoI’s efforts to accommodate more students and reduce dropout rates in secondary education have contributed to this significant increase in access, which almost achieved the final target of 40.3 million. There was also a substantial improvement in the GER in secondary education. The GER reached 80 percent, an increase of 21 percentage points. It is commendable that the GER increased by 25 percentage points on average in 16 states/union territories (UTs),10 including low-income states.

30. The project’s Results Framework shows the same baseline and target values for GER and NER11. The GER is generally higher than the NER in countries where the education system accommodates over-aged and under-aged students because of early or late entrants, and grade repetition. As noted in the seventh Implementation Status and Results Report (ISR), the baseline value and projections were estimated based on the Census 2001 data that were available at project preparation. The ISR concluded that the baseline value set for NER at project preparation was less accurate and did not reflect the actual number of enrollment children ages 14 to 15. Therefore, the ICR looked at the data based on the updated census. The NER shows a positive trend, increasing from 45.6 percent to 51.6 percent 12 between 2013 and 2015–2016. This significant increase in these indicators shows a high degree of

9 This updated figure was obtained at ICR stage. 10 They are Sikkim, Tripura, Meghalaya, Bihar, Nagaland, Chhattisgarh, Jharkhand, Goa, Mizoram, Punjab, Dadra and Nagar Haveli, West Bengal, Assam, Delhi, and Lakshadweep. 11 While the GER for secondary education is defined as the number of students (of any age) who are enrolled in secondary education as a percentage of the total children of official school age population, the NER is the number of children of official secondary school age who are enrolled in secondary education as a percentage of the total children of the official school age population. 12 The NER value does not include data from Andhra Pradesh and West Bengal.


Page 14 of 55

participation of students in secondary education. Therefore, the achievement of outcome 1 is rated Substantial. However, additional effort is needed to tackle an issue of grade repetition and early or late entrants.

Outcome 2: Improved equitable access to secondary education Sub-rating: Substantial

31. The project supported the RMSA to close the gaps in the levels of participation of girls compared to those of boys and of children from disadvantaged social categories such as Scheduled Castes (SCs) and Scheduled Tribes (STs), compared to the general population. Improved equitable access was measured by five outcome indicators: (a) gender parity index (GPI) in enrollment (PDO-level indicator), (b) number of girls per 100 boys completing grade 10 (IO indicator), (c) share of scheduled castes (SCs) in secondary completers (IO indicator), (d) share of scheduled tribe (ST) in secondary completers (IO indicator), and (e) GER for students from disadvantaged social categories (IO indicator).

32. Over the life of the project there was tremendous progress in gender parity in secondary school enrollment. Gender parity is computed by comparing the gross enrollment of boys with the gross enrollment of girls in secondary schools. The GPI in GER increased from 0.94 at baseline to 1.03 in 2016–2017, exceeding the target identified for the project. There was also significant improvement in the number of girls per 100 boys completing grade 10, improving from 79.0 percent to 91.2 percent.

33. The establishment of girls’ hostels in EBBs has helped increase the enrollment of disadvantaged girls. By the end of the project, 1,148 girls’ hostels were constructed and made functional, with a total enrollment of 103,969 girls. The ICR mission notes that additional support was provided for the increased participation, performance, and welfare of girls. This support included self-defense training in martial arts, special adolescent well-being programs for young female students, career guidance, motivational camps, and the creation of boarding facilities. Further, to improve the in-school experience of girls, a special module on gender sensitization was developed by the National Council of Educational Research and Training (NCERT) and included in teacher training packages. Interviews with students and teachers, which were conducted during the ICR mission, reveal that students in the girls’ hostels were satisfied that they were able to participate and remain in school in a safe and improved learning environment. Other initiatives to promote girls’ participation include stipends for needy girls and ensuring that schools have adequate toilet facilities. Additionally, tools and guidelines were developed to promote greater gender awareness among education officials and teachers.

34. Civil works in special focus districts (SFDs) enabled increased access to secondary schools for students from disadvantaged social categories. By the end of the project, 5,028 new schools were constructed in SFDs, which amounted to 40 percent of all new school construction under the RMSA. Similarly, there was substantial strengthening of the existing schools in SFDs (development of school infrastructure), covering a total of 10,421 schools. These interventions resulted in significant improvement in the participation in secondary education of students from disadvantaged social categories. For instance, the proportion of SCs in secondary completers reached 18.4 percent, compared to the baseline of 16.8 percent. The share of ST in secondary completers increased from 6.0 percent at baseline to 7.7 percent. The GER for SC girls (86.9 percent) now exceeds that for SC boys


Page 15 of 55

(83.9 percent). The GER for ST girls (75.38 percent) is also higher than that for ST boys (73.7 percent). The achievement of outcome 2 is rated as Substantial.

Outcome 3: Enhanced quality of secondary education Sub-rating: Modest

35. Enhanced quality was measured by three outcome indicators: (a) a new system for the national assessment of students’ achievement was developed and used to inform the RMSA’s quality improvement policies (PDO-level indicator), (b) the proportion of schools with specific teachers available to teach the five core subjects (IO indicator), and (c) the gross graduation rate in schools covered by RMSA (PDO-level indicator). Other key project results for quality improvement are also presented in the following paragraphs.

36. The project emphasized the importance of monitoring student learning achievements. Before the project, there was no reliable national learning assessment available at the secondary level. One of the key contributions of the project was the development of a national system of student assessment at the secondary level, which would track progress in learning achievements over time and inform planning for quality improvement. Learning achievements in grade 1013 were measured for the first time in 2014–2015, with a sample of 277,000 students in 7,216 schools across 34 states/UTs. The National Achievement Survey (NAS) used the Item Response Theory, which enables comparisons across states and over time. The national report and the state summary reports have been shared with all participating states/UTs, providing evidence on which well-targeted remediation strategies could be developed. NAS results show, for instance, children’s social economic status, location, positive interaction in school and student communication together explain 8.5 percent of variance in student achievement in mathematics. Based on such NAS findings, some states developed learning improvement plans and conducted teacher training. The end target for this indicator was to complete two rounds of a national assessment survey in grade 10. The first round of NAS has established a good benchmark for the subsequent rounds in terms of preparation, implementation and utilization cycle. Preparation for the second round of assessment is currently under way (test item development and pilot survey were completed), with administration scheduled for early 2018. Thus, it is highly likely that the end target will be achieved in early 2018.

37. Several research studies had been carried out under the project, which provided evidence and informed policy decisions. Technical inputs have also been provided by the World Bank, the DFID, and the EU, including analytical reports and policy and technical assistance notes. The project focused on providing sufficient teachers in all the subjects, especially mathematics and science. The availability of subject teachers has improved during course of the project. In fact, teacher vacancies have gone down from 26.0 percent in 2012–2013 to 14.8 percent in 2016–2017. The proportion of schools with specific teachers available to teach the five core subjects has significantly increased from 62.2 at baseline to 87.7 at project closing. In addition to teacher recruitment, the project strengthened teacher and school leader training structures. For instance, in-service teacher training modules have been developed in core subjects (for example, mathematics, science, social studies, English, and Hindi) and rolled out in

13 The main objectives of the NAS were to (a) study the achievement levels of students of grade 10 in English, mathematics, science, social science, and one modern Indian language; (b) study the difference in achievement levels with regard to area, gender, social group, and school management; and (c) study the effect of intervening variables such as home, school, and teacher characteristics on students’ achievement levels.


Page 16 of 55

the states. Leadership training programs for principals and headmasters have been developed by the National Centre for School Leadership (NCSL) at the National University of Educational Planning and Administration (NUEPA). By the end of the project, 3,526 State Resource Group members and 26,332 secondary school headmasters/principals were trained. The ICR mission notes that teacher and school leadership training were well received by schools and have resulted in positive changes in teaching and school management.

38. Moreover, the project assisted the review of the curriculum and remedial courses to enable students to reach the grade 9 level. Many states have put processes in place to help children transition into the academic demands of grade 9. Approximately 2,241,800 students have benefitted from these programs. Almost all states/UTs have aligned their curricula, syllabi, and textbooks in accordance with the National Curriculum Framework 2005. Despite not implementing the Innovation Fund initiative, some states have implemented innovative approaches (for example, use of information and communication technology for teaching and teacher training and support for children with special needs). These innovative interventions also helped improve the quality of education. With regard to governance, SMDCs have been established and trained for supporting quality improvement and other school inputs.

39. The project established ambitious targets for the graduation rate (calculated as the proportion of students appearing in grade 10 exams in relation to the number of students enrolled in grade 9 in the previous year). This indicator has decreased from 74 percent at baseline to 69 percent at project closing. A decline in this indicator suggests that fewer students are appearing for the grade 10 board examination than those were enrolled in grade 9 in the previous year. Anecdotal evidence shows that a decline could indicate more students dropping out or leaving the public school system for other options (for example, open schools/vocational training) or more students being unprepared for taking the grade 10 board examination or being perceived to be unprepared by teachers. As discussed earlier, a large group of students from disadvantaged groups have entered the secondary education system, which may have negatively affected the graduation rate. Given these considerations, the achievement of outcome 3 is rated as Modest.

Justification of Overall Efficacy Rating

40. The overall efficacy rating is Substantial based on achievement of each outcome, as discussed above and provided in table 1.

Table 1. Efficacy Rating

Outcome 1 Outcome 2 Outcome 3 Overall

Rating Substantial Substantial Modest Substantial

C. EFFICIENCY

Assessment of Efficiency and Rating Rating: Substantial


Page 17 of 55

41. A cost-benefit analysis conducted at the time of the ICR shows a high economic rate of return for the project. Considering the stream of costs and benefits generated, the overall project’s net present value (NPV) of net cash flows from the RMSA is estimated to be US$3.5 billion in the base case scenario. Similarly, the economic internal rate of return (EIRR) is 16.1 percent. At appraisal, the NPV and EIRR for the RMSA were US$14 billion and 24.3 percent, respectively. The estimates at completion turned out to be lower than the estimates at appraisal due to conservative assumptions. However, it still can be said that the project has yielded positive economic returns and thus was a sound investment (see more details in Annex 4).

42. The project achieved its objectives efficiently within the original time frame with full disbursement. The implementation and design of the project were well thought out, and a number of noteworthy efficiency gains took place under the project: (a) the project used resources in an efficient manner for the certain level of outcomes, despite lower counterpart funds than those estimated at project preparation, (b) the drop-out rate for secondary education decreased from 17.9 percent in 2013-2014 to 17.0 percent in 2016-2017. This is an evidence of increased efficiency of internal system and reduced wastage of public and private resources, (c) streamlined administration procedures with use of information technology has resulted in increased program efficiency, (d) an online project monitoring system (PMS) has been developed and used for monitoring progress in the states, districts, and secondary schools. This online PMS enhanced efficiency in the management and implementation of the RMSA in the states/UTs, (e) better fiduciary procedures that improved planning, budgeting and timely release of funds have improved implementation efficiency, (f) the project leveraged the expertise of national (for example, NUEPA and NCERT) and state-level institutions to provide technical assistance on aspects such as teacher training, school leadership, and school standards, (g) despite not implementing the Innovation Fund initiative, some states have developed and implemented innovative interventions. For instance, satellite technology was used for beaming lectures/lessons of subject experts remotely to classrooms, especially in schools without mathematics and science teachers; and information and communication technology was used for teacher training, and (h) the MHRD supported states/UTs to share good practices on innovative interventions that have had successful outcomes at the state level. Both qualitative and quantitative aspects of efficiency gains were observed. Thus, the efficiency rating is Substantial.

D. JUSTIFICATION OF OVERALL OUTCOME RATING

43. The overall outcome rating of the project is Satisfactory based on high relevance, substantial efficacy, and substantial efficiency, as discussed above and provided in table 2.

Table 2. Overall Outcome Rating

Relevance Efficacy Efficiency Overall

Rating High Substantial Substantial Satisfactory

E. OTHER OUTCOMES AND IMPACTS

Gender


Page 18 of 55

44. The project focused on promoting equity by closing the gender gap and supporting children from disadvantaged social categories. Therefore, the project interventions have significantly benefitted girls, especially those from disadvantaged social categories, as discussed in Section II.B. Gender parity in the GER has been achieved. The GER for girls improved by nearly 15 percentage points, from 65.98 percent in 2012–2013 to 80.97 percent in 2015–2016. Meanwhile, the GER for boys increased from 68.60 percent to 79.16 percent. The GER for SC girls and for ST girls exceeds that for SC boys and ST boys, respectively. As discussed earlier, various interventions (for example, the establishment of girls’ hostels in EBBs, promoting gender sensitization through training programs, providing stipends for needy girls, and ensuring adequate toilet facilities for girls) were carried out and had direct positive impacts on promoting gender equity. At the end of the project, the number of female project beneficiaries reached 11,520,000. Moreover, with regard to equity on learning achievement, the NAS results for grade 10 shows that there is no difference between boys and girls in performing tasks at different levels of mathematics literacy.

Institutional Strengthening

45. The RMSA program has resulted in significant institutional strengthening. Examples of the main areas of improvements are listed below.

• Planning and budgeting. The planning, budgeting, and program management embodied in the Annual Work Plans and Budgets (AWP&Bs) has been strengthened and promoted to focus more on outcomes and processes. The system has become more streamlined. States now submit their plans online through the project monitoring system (PMS) developed by the MHRD and the Project Appraisal Board reviews the AWP&Bs before the beginning of the financial and academic year, which allows for timely fund releases to states.

• State and district capacity. The RMSA has developed program implementation, management, and monitoring structures. At the state and district levels, the State Project Office (SPO) and District Project Offices (DPOs) have been put in place to prepare AWP&Bs, as well as to supervise and manage educational districts and organize training programs. The planning and implementation capacity at the state and district levels has improved.

• School management. The RMSA has supported the establishment of SMDCs at the school level. SMDCs have been trained on the various aspects of the program for better school management. The ICR mission observed that training and empowering the SMDC members has had a positive social development impact (for example, more community participation, more activities planned and implemented for school improvement). School leadership training for headmasters/principals has also promoted effective school management.

Mobilizing Private Sector Financing

46. Not applicable.

Poverty Reduction and Shared Prosperity

47. The project did not include any specific impact measurement of poverty reduction or shared prosperity. However, the project has contributed to the greater acquisition of human capital by


Page 19 of 55

supporting the secondary education system. India is seeking to promote economic growth and reduce poverty. The expansion and development of human capital is vital for the achievement of this goal. The project is likely to have helped equip children with knowledge and competencies that make them more productive and capable of higher earnings.14 The knowledge and competencies obtained through the education system could then affect poverty reduction and shared prosperity at the country level.

Other Unintended Outcomes and Impacts

48. The following three positive impacts are noted:

• Joint Review Missions (JRMs) reviewed the overall progress of the RMSA program on a regular basis, identified the lessons learned, and provided implementation support. The assessments and recommendations made by the JRMs have been taken into consideration by the MHRD and states/UTs not only for the RMSA program improvement but also for whole education system improvement. Notably, due consideration is being given to (a) extending the Right to Education Act to include students up to grade 10, (b) including higher secondary grades (grades 11 and 12) within the RMSA, and (c) facilitating convergence between the national flagship programs at the elementary level (Sarva Shiksha Abhiyan) and the secondary level (RMSA). Furthermore, the MHRD has reintroduced the Innovation Fund with funds for the financial year 2017-18 at INR 100 crores (US$15.3 million). The MHRD has approved proposals from the states and allocated funds for the year to promote more innovative approaches at the state and sub-state levels15.

• The project contributed to the development of better fiduciary practices, including FM processes, manuals on FM, and procurement and training programs. These were used by the whole RMSA program and not just for World Bank funding.

• The project created huge awareness to look at various aspects of the learning and quality of secondary education. The project also generated substantial international interest in India’s secondary education sector and a large amount of high-quality research to inform reforms in the sector. The World Bank task team carried out analytical work and a number of research studies to inform the implementation. Part of the support for this was received from DFID in the form of a Trust Fund specifically set up for secondary education for US$900,000 for an Externally Funded Output.

14 Recent studies show that private rate of return to secondary education is higher than that to elementary education for India, which means that graduates of secondary education earn more than children who completed only elementary school. 15 The 2017–2018 Union budget highlights included one budget item on secondary education to set up an innovation fund of INR 100 crores (INR 1,000 million).


Page 20 of 55

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME

A. KEY FACTORS DURING PREPARATION

49. The World Bank’s preparation process was influenced by the following key factors:

• The project was aligned with the country’s long-term development strategy and priorities articulated in the 11th Five-Year Plan (2007–2012). The GoI recognized the need to strengthen the secondary education sector to enable the country to achieve national goals.

• The project was prepared on time. The Government demonstrated a high level of commitment to the objectives of the project during the project preparation.

• The activities and outputs supported by the project were closely linked to the PDO. The project’s Results Framework was well aligned with the operational objectives.

• The project supported the RMSA program through a SWAp. It facilitated an approach to address the issues in a holistic manner through collective dialogues with a focus on sector-wide results.

• Both the World Bank task team and the MHRD agreed that the project would use the existing government system for monitoring to strengthen the GoI’s ownership and sustainability. The project thus relied on the GoI’s projections regarding the Results Framework based on the education information database system that was available at appraisal.

• The task team leader and core team members at appraisal were based in Delhi. The World Bank team was in continuous policy dialogue with the GoI and DPs and led the SWAp preparation process. Project consultations with DPs ensured alignment of the proposed project activities with those of the other DPs.

• Key risks and appropriate mitigation measures were adequately identified during preparation. The PAD identified a weak fiduciary risk, inadequate implementation capacity, and challenges associated with improving the quality of secondary education and integrating the RMSA program with other centrally sponsored schemes. The risks were appropriately identified during the project design. The mitigation measures that were proposed were realistic and informed by lessons learned from education projects in India and other countries.

B. KEY FACTORS DURING IMPLEMENTATION

50. The project was completed on time and was fully disbursed. The project had some loss in U.S. dollars due to fluctuation in the exchange rate16 (actual disbursed amount US$475,874,311 versus

16 At appraisal, SDR 1 was equivalent to US$1.55 and US$1 to INR 50.05. At ICR, SDR was equivalent to US$1.41, and U.S. dollar to INR was 65.32 (see Data Sheet).


Page 21 of 55

original amount: US$500 million). However, there was ultimately some gain in Indian rupees (actual disbursed amount: INR 29,609,149,908 versus the originally estimated amount: INR 25,010,986).

51. Many factors contributed to successful project implementation.

• The Government’s strong and continued commitment was central to successful implementation of the RMSA program. The program, as a centrally sponsored scheme with high political commitment, served to mobilize stakeholders at all levels (authorities at the national, state, and district levels and communities).

• The RMSA Framework facilitated the states to prepare high-quality annual work plans. Further, the whole process of planning, appraisal, implementation, and monitoring was well organized. National and state governments provided technical support, established and monitored standards and targets, and facilitated the sharing of experiences. This helped mitigate the inadequacies in the capacity of implementing agencies, especially at the local level.

• The SWAp strengthened collaboration and coordination and minimized the transaction costs and reporting requirements for the Government. It also encouraged harmonization of the borrower’s and donor’s fiduciary system using country systems for FM and procurement activities. The project contributed to enhancing the Government’s own fiduciary system and building internal capacity and sustainability.

• Regular and sustained dialogue between the DPs and the national and state governments and stakeholders at the district and school level facilitated steady implementation progress in most aspects of the program. The JRMs for the RMSA were the hallmark of the partnerships with the Government as well as the DPs, and candid discussions took place. The Aide Memoires and associated documents were made available in hard and soft copy form to meeting participants and disseminated publicly.

• The continuity and local presence of the World Bank team and proactive follow-up on issues had a significant impact on implementation. Technical inputs from the World Bank, including analytical reports and policy and technical assistance notes, supported the implementation of key activities and helped improve the performance of the program.

52. It should be also noted that there were factors that slowed down implementation and hindered project progress.

• As discussed in Section 1.A, counterpart funding from the GoI for the RMSA was much lower than that originally envisaged at appraisal. Also with this devolution of funds, the center’s share has gone down; the Central Government’s expenditures and the counterpart funding allocation for the RMSA turned out to be lower.

• Several states faced implementation challenges due to limited human resources at the state and district levels. While the number of secondary schools has grown exponentially, there has not been a corresponding increase in program management personnel. The JRMs note that there have been a large number of vacancies, and the lack of management staffing has been particularly problematic in key areas such as FM. The lack of personnel


Page 22 of 55

at both the state and district levels undermined the quality of the program management.

• The World Bank operation co-financed the GoI’s RMSA, which is a much larger program and includes an enormous variety of activities and technical aspects. Any program of such scale and scope poses management and implementation challenges.

• There are other centrally supported schemes in addition to the RMSA. The achievement of project activities may have been affected by activities supported by such schemes.

• A level two restructuring in June 2014 led to some improvements in FM and contributed to smoother arrangements for withdrawals from credit, without diluting the FM standards or arrangements. While the bottleneck issue was removed by this restructuring, FM has remained challenging due to limited capacity of human resources through the end of the project (for example, delayed fund releases, quality issue with audit reports, as discussed in Section IV.B). This issue has been flagged by several of the JRMs.

• The midterm review, which was conducted in September 2015, addressed the weaknesses of the Results Framework and suggested that updating or modifying some targets through restructuring. Specific changes were: (a) dropping two IO indicators on innovations where there was no progress on activities (‘number of innovative activities approved’ and ‘25 percent of the completed innovation component projects that meet at least 75 percent of their agreed targets’); (b) modifying the target values for the indicators that were far behind and not on track to be achieved by the end of the project (‘gross graduation rate’, ‘NER’, and ‘proportion of schools with all subject-specific teachers available in required number’); (c) setting the baseline values for indicators that were not available at appraisal (‘gross secondary completion rate’, ‘proportion of schools with all the main infrastructure available’, ‘proportion of schools with all subject-specific teachers available in the required number’, ‘proportion of SMDCs receiving capacity/management training’, and ‘all staff positions at the state, district, and sub-district levels filled’); and (d) setting the target values for indicators that were not determined at appraisal (‘proportion of SMDCs receiving capacity/management training’ and ‘all staff positions at the state, district, and sub-district levels filled’).

• Although discussions on restructuring with the GoI started soon after the midterm review, the restructuring did not ultimately occur. There are the following reasons. First, the education information database system was not sufficiently mature, which resulted in a delay in making new projections regarding key indicators. Second, the GoI’s approval process for restructuring took time. The MHRD submitted a preliminary project proposal (PPR) for restructuring with additional financing to the Department of Economic Affairs (DEA) in February 2016 for covering the financing gap due to the GoI’s funding constraint till June 2017 (when the project closes) and beyond till 2020. The first GoI screening committee meeting, chaired by the DEA, took place in August 2016. The committee gave a decision for a new project with a different scope with a focus on supporting lagging states through a results-based financing approach and not for additional financing of the same project. Hence, the restructuring for changes in the Results Framework was not carried out as the project was due to close in June 2017. After the revised PPR was submitted by the MHRD, the second screening committee meeting was held in January 2017, wherein the committee approved the proposal in principle but suggested recasting


Page 23 of 55

it to align with suggestions made by the National Institution for Transforming India (NITI Aayog). At the third screening committee meeting, which took place in September 2017 (two months after the project was closed), the PPR for ‘Secondary Education in Focus States’ was approved. A request letter for the new project was then sent to the World Bank in early October 2017. The task team diligently followed up to support the process. For the above reason, however, they could not move to the restructuring during the project implementation.

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME

A. QUALITY OF MONITORING AND EVALUATION (M&E) Rating: Substantial

M&E Design

53. The links between the inputs, outputs, and outcomes and the PDO were adequate and sound. The PDO was clearly specified. Four PDO-level indicators and 15 IO indicators were identified for the project at appraisal to measure (a) increased access to secondary education, (b) improved equitable access to secondary education, and (c) enhanced quality of secondary education. The sources of data for the indicators were to be AWP&B, education information databases such as SEMIS and Unified District Information System for Education (UDISE), and project reports.

54. There were some challenges in the development of the Results Framework. As discussed earlier, the World Bank task team and the GoI agreed to use the existing government system for monitoring. This was done to strengthen the Government’s ownership of the Results Framework for monitoring the program overall and which would also in due time result in the database becoming more reliable. Thus, the project’s Results Framework was developed based on projections that the GoI made. At appraisal, the database system developed by the NUEPA was not sufficiently mature to allow the project to establish the targets for some indicators. As a result, the project end target values for five IO indicators were not determined at appraisal. These include (a) gross secondary completion rate; (b) proportion of schools with all the main infrastructure available, as required; (c) proportion of schools with all subject-specific teachers available in the required number; (d) proportion of SMDCs receiving capacity/management training; and (e) all staff positions at the state, district, and sub-district levels filled. Regarding indicators (d) and (e), the baseline values were also not available at appraisal, because these were never systematically tracked before at the national level through the M&E system. These indicators turned out to be difficult to make projections.

55. To strengthen the quality of M&E, the following strategies were proposed at appraisal: (a) reviewing the results periodically at the national, state, district, and sub-district levels using data- and field-based monitoring mechanisms; (b) supporting the MHRD and NUEPA to strengthen the education M&E information database and develop a range of additional monitoring tools to assess progress in both qualitative and quantitative terms (this included the monitoring of student learning outcomes and


Page 24 of 55

quality improvement processes); and (c) DPs reviewing the project progress with the Government every six months through a JRM, as well as at other times as necessary for effective implementation support.

M&E Implementation

56. The M&E strategies proposed above were carried out during project implementation. The JRMs were conducted and key results were regularly monitored. The project management and monitoring system has been strengthened. At the central level, a number of frameworks, guidelines, and tools were developed and used for the RMSA program management and oversight. The MHRD was supported by a National Technical Support Group (TSG) tasked with thematic and state responsibilities for oversight and monitoring progress. At the state and district levels, monitoring and implementation support systems have been put in place. The RMSA program instituted an external monitoring system by engaging 35 Monitoring Institutions (NGOs, State Universities, Foundations, etc.) to visit a sample of schools and monitor progress of RMSA implementation. Their reports are publicly available. The project also supported the establishment of SMDCs at the school level, which is a core governing body for school management and monitoring. Moreover, UDISE, the official database for key education indicators, and an online PMS have been put in place and used for monitoring progress in the states, districts, and secondary schools. Additionally, the NAS for grade 10 was conducted for monitoring student learning outcomes. Several research studies were also carried out, including classroom process studies in Meghalaya, Uttar Pradesh, and Rajasthan; curriculum studies in several states; time-on-task studies in Tamil Nadu and Madhya Pradesh; a study on teachers in nine states in the Indian education system; a study on the impact of teacher training on the quality of teaching in secondary education; a study on the dropout factors in secondary education; a study on the impact of self-defense training in schools; and a study on the impact of vocational education under the RMSA. The ICR notes that the project performance was regularly monitored during project implementation through various M&E arrangements.

57. Despite an improved project management and monitoring, the weaknesses of the Results Framework remained unchanged during project implementation. As discussed in the earlier section, the Government and the World Bank task team explored the possibility of restructuring to modify the Results Framework. However, due to the GoI’s prolonged approval process for PPR, the restructuring did not ultimately occur, and the Results Framework was not formally modified.

M&E Utilization

58. The results of monitoring and information from various sources (for example, UDISE, PMS, NAS, and findings from research studies) were shared and used at both the central and state levels. Such information was used to strengthen the implementation of interventions (for example, teacher training, curriculum, quality improvement activities) and for the development of strategy and decision making. For instance, online PMS was used for enhanced efficiency in the management and implementation of the RMSA in the states/UTs. In the PMS, comprehensive information from states regarding their physical and financial progress on all the RMSA components was captured and progress was monitored. UDISE, the official database, was used to monitor key school-, teacher-, and student-related indicators. Remedial actions were taken. The results of the NAS were analyzed and the summary report and state report cards were made available online. A large body of analytical work was supported by the project, such as: (a) analysis of funding allocation formula for states by the World Bank, (b) teacher recruitment


Page 25 of 55

policies and practices by the World Bank, (c) analysis of expenditure on in-service teacher training by the World Bank, (d) time-on-task in secondary schools by the World Bank and Educational Initiatives Pvt. Ltd, (e) international case studies on vocational education in secondary schools by Cambridge Education, UK, (f) study on government-aided schools by CfBT, UK, and (g) nine state teacher management study by the World Bank. For instance, the analysis on in-service teacher training informed policy makers to what extent the training program are implemented as intended. Based on the evaluation, in-service teacher training programs were institutionalized and guidelines for conducting 10 days training were developed. Time-on-task study addressed that instructional practice was mainly teacher-centric. On average, teachers spend about 60 percent of classroom time on lectures or instruction and assigning students classwork in mathematics classes. It also highlights that teachers are spending substantial class time on instructional activities but results from the NAS indicate that this is not necessarily leading to increased student learning outcomes. Such analytical work has led to new program design in the state level.

59. The project also benefitted from reviews and recommendations from the JRMs. The results of the monitoring and the information obtained through M&E proved to be useful. Some states have developed programs and taken actions for improving teaching and learning.

Justification of Overall Rating of Quality of M&E

60. Despite some weaknesses addressed earlier, the project collected a substantial amount of M&E data on performance and results progress, which was used to inform project management and decision making. M&E data collection and project monitoring have been strengthened over the years. Therefore, the enhanced quality of M&E improved overall project performance and provided sufficient evidence to assess the achievement of the objectives. The quality of M&E is thus rated Substantial.

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 61. Safeguards. The project was classified as Environmental Category B, as the proposed school construction and renovation may have had adverse social and environmental impacts. Environmental Assessment (OP/BP 4.01) and Indigenous Peoples (OP/BP 4.10) were triggered under the project. The environmental and social assessments were prepared during project preparation. The environmental management was assessed as Moderately Unsatisfactory because of the significant delay in updating the environment assessment. The World Bank task team flagged this issue in the JRMs and Management Letters. At later stages of the project, the rating was upgraded to Moderately Satisfactory, as the pending environmental assessment update was completed and the Environmental Management Framework was finalized. The World Bank’s assessment of safeguards concluded that there were no issues raised during project implementation regarding deviation from the agreed safeguard policies for the project.

62. Procurement. The procurement under the RMSA is handled at the national, state, and district levels. At the national level, a National TSG is responsible for overseeing and monitoring the project progress, including civil works. At the state and district levels, the SPO coordinating with DPOs monitors civil works activities. At the initial stage of the project implementation, the procurement rating was assessed as Moderately Satisfactory due to the slow progress of civil works and delays in the post-procurement review of contracts. However, the procurement rating was later upgraded to Satisfactory


Page 26 of 55

and was maintained as Satisfactory for the rest of the project implementation after the efficiency of the procurement implementation and management was improved.

63. The Procurement and FM manuals were revised during project implementation to account for changes in the local context. In most of the states, the e-tendering process was followed for all major procurements above INR 50 lakh, according to the agreement with the MHRD. Through the AWP&B review process, the MHRD cancelled civil works pending in 2015–2016 and sanctioned fresh works based on a revised schedule of rates in the states. This has improved the progress rate of new school construction and promoted better use of the limited resources. The MHRD also made the decision to release funds to states in advance, which has helped states to finalize civil works contracts expeditiously.

64. FM. The FM rating was changed from Satisfactory to Moderately Satisfactory at the fourth ISR completed in February 2014, and it was further downgraded to Moderately Unsatisfactory at the fifth ISR in September 2014 because fund releases were delayed and the quality of audit reports continued to be a concern. The JRMs reviewed the progress of FM and suggested that the capacity of auditors and state accountants should be strengthened, especially for improving the quality of audit reports and making remedial responses to audit observations. The MHRD agreed to complete a financial review of the RMSA program to look at various FM issues. However, sufficient actions were not taken during project implementation and the FM rating at project closing is Moderately Unsatisfactory. The World Bank task team raised a red flag with regard to this issue and the MHRD has taken steps for improvement. It should be noted that despite the financial inefficiencies, no significant impact on project activities was observed. The project was completed on time and achieved its objectives with full disbursement.

C. BANK PERFORMANCE Rating: Moderately Satisfactory

Quality at Entry

65. The PDO was highly relevant and remained so throughout the project. The objectives were aligned with the GoI’s development priorities and World Bank’s CPS. The design of the project was based on extensive consultations with various stakeholders at all levels and drew on the World Bank’s experience in education projects in India and other countries. The design of the activities and outputs supported by the project were closely linked to the PDO. Working closely with the GoI and DPs, the World Bank led the SWAp preparation process. The project design paid adequate attention to implementation arrangements to ensure compliance with the World Bank’s relevant policies and procedures. All significant risks were recognized and mitigation measures were formulated. The implementation arrangements were appropriate and well integrated in the project design. However, as discussed earlier, there were inadequacies in the design of the project’s Results Framework. Baseline and target values for some indicators were not available at appraisal due to data limitations in the GoI’s education information M&E system at that time. The World Bank task team planned to update these values during project implementation. It is also apparent that the baseline value for the NER was less accurate.


Page 27 of 55

Quality of Supervision

66. Sufficient budget and staff resources were allocated, and the project was adequately and regularly supervised and closely monitored. The JRMs were well planned, with experts in specific areas participating, providing reviews and recommendations, facilitating the sharing of good practices, and maximizing the impact of supervision. Aide Memoires were prepared regularly and alerted the GoI to issues with project implementation. The ISRs were detailed and provided management with a transparent and robust assessment of the project’s achievements and challenges. The continuity and local presence of the World Bank and sustained dialogue with the Government counterparts yielded positive results in the project activities. The amount of effort that the World Bank task team put forth was commendable and the World Bank team effectively coordinated with other DPs to expand the range of just-in-time technical inputs into the supervision process. The only unfortunate issue was that the Results Framework was not revised through restructuring to improve the project performance and update the baseline and target values for some of the indicators that were not available at appraisal. The World Bank task team continued to follow up to support the GoI’s process for restructuring though. For the reasons discussed earlier, the team could not move to the restructuring.

Justification of Overall Rating of Bank Performance

67. The project activities and outputs were well thought out and closely linked to the PDO. The World Bank task team was extremely proactive and effective. The project was fully disbursed, was closed on time, and made a major positive impact on outcomes by supporting various activities under the RMSA Framework because of enhanced supervision efforts. Despite a series of efforts made, the earlier mentioned weaknesses of the Results Framework posed difficulties in achieving targets for some indicators. Hence, the overall performance of the World Bank is rated Moderately Satisfactory.

D. RISK TO DEVELOPMENT OUTCOME 68. The GoI and state governments have been fully committed to continuing the RMSA program. The institutional development mechanisms are in place to ensure the program’s sustainability beyond the World Bank’s support. However, the continuation of reforms, especially those concerning improvements in the quality of education and fiduciary management, is likely to remain challenging in some states with inadequate governance environments. Furthermore, there are significant variations in capacities across states despite progress under the RMSA; planning and execution capacity remains weak in some lagging states and varies among administrative levels in the districts and schools. Addressing these risks and targeting interventions with a special focus on lagging states/UTs are the focuses of the program moving forward. The World Bank’s continued engagement, particularly through the new project, will ensure that the outcomes achieved under the RMSA are maintained and that the lessons learned are incorporated, to the extent possible, to further improve the secondary education system.

V. LESSONS AND RECOMMENDATIONS

69. The following key lessons have been identified.


Page 28 of 55

• Government ownership and commitment. Strong Government commitment and ownership is critical for successfully steering a sector-wide reform program and improving service delivery. Here, it enabled the mobilization of ownership and the commitment of stakeholders at all levels, leading to the achievement of the project objectives.

• Introducing new approach under the project. When introducing new approach such as Innovation Funds, there might be some resistance at the initial stage. Implementation may take longer than expected or fail to progress. It is important that there is an incentive structure to adapt innovative approaches and implementation support mechanism needs to be established. Strong Government buy-in and continuous commitment is a key to successful implementation of new approach.

• M&E. The availability of timely and reliable data and information is essential for planning, selecting, and setting the results indicators to measure the achievement of the development objectives and monitoring. In the RMSA program, data and information collected from various sources (for example, UDISE, PMS, NAS, findings from research studies) have been used at the both central and state levels and the monitoring system has also been strengthened.

• Results Framework. The project’s Results Framework plays an important role in monitoring project progress and improving project performance. The Results Framework needs to be carefully developed at project preparation and should be based on a realistic assessment regarding the data validity, reliability, accuracy and implementation capacity. Including indicators where no baselines exist at appraisal may pose a challenge. Projections will be extremely difficult.

• Timely restructuring. If implementation challenges are identified during project implementation, restructuring would be a great opportunity to make midcourse adjustments and enhance the project performance. It would be difficult to do timely project restructuring when the government approval process for restructuring takes time.

• Capacity building. Building the capacity of key stakeholders is important for project management and sustainability. Inadequate staffing and capacity constraints, especially at the state and local levels, led to challenges in areas such as fiduciary management and adversely affected project implementation. Continued capacity building of sub-national officials and key stakeholders at the state, block, and school levels is essential. The structure below the district level needs to be strengthened for supporting schools.

• Continued dialogue with the Government and proactive World Bank engagement. Continuous and intensive World Bank engagement and technical and advisory support can provide the needed support at critical points and help overcome challenges. The JRMs by the GoI and DPs contributed to effective project implementation and minimized the transaction costs.

• Value addition of the World Bank support to a large government program. The RMSA created and consolidated implementation procedures and increased good governance and capacity at all levels. It further demonstrated how a robust and comprehensive effort directed at the performance of the secondary education system as a coherent whole could


Page 29 of 55

achieve considerable success in increased access, improved equity, and enhanced quality of secondary education. SWAp approach can be used as a basis for a more encompassing secondary education reform. However, there are also some challenges in SWAp when the scale and scope of the supporting government program is very large while the World Bank’s financial contribution is small. In such case, more careful design of the project is needed. It is important to consider strategies on how to maximize the World Bank’s impact on a large national program. The World Bank has started a discussion with the GoI on a new project with PforR approach, which will enable the World Bank to better leverage its own financing and collaboration through pooling resources and focusing on supporting the government program and helping enhance efficiency, effectiveness and impact of the government program.

. .


Page 30 of 55

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS17

A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Increased access to secondary education

Indicator Name Unit of Measure Baseline Original Target Formally Revised

Target

Actual Achieved at Completion

Enrolment of students in Secondary (Grade IX and X)

Number 28.30 40.30 39.00

30-Sep-2009 30-Jun-2017 30-Jun-2017

Comments (achievements against targets): Almost achieved (97% achieved).

Objective/Outcome: Improved equitable access to secondary education


Target


Gender Parity Index (GPI) in enrolment (in schools covered by RMSA programs)

Percentage 94.00 98.00 100.03

01-Sep-2010 30-Jun-2017 30-Jun-2017

17 For the purpose of reporting to the World Bank Management, all PDO-level indicators and selected intermediate results indicators are included in the ISRs.


Page 31 of 55

Number of project beneficiaries

Number 11600000.00 16700000.00 24000000.00

01-Sep-2010 30-Jun-2017 30-Jun-2017

Proportion of project beneficiaries who are female

Percentage 45.60 48.00 47.28

01-Sep-2010 30-Jun-2017 30-Jun-2017

Comments (achievements against targets): Exceeded (102% achieved).

Objective/Outcome: Enhanced quality of secondary education


Target


Gross Graduation Rate (in schools covered by RMSA)

Percentage 74.00 88.00 69.00

01-Sep-2010 30-Jun-2017 30-Jun-2017

Comments (achievements against targets): Not achieved (78% achieved).


Target


New System of National Assessment of Students’ Achievement developed and used to inform the RMSA’s Quality Improvement Policies

Text No assessment Second round of assessment is carried out, data entry and compilation is conducted, and data analysis and draft report is prepared.

State/UT specific reports for the first round of NAS has been shared with 34 states/UTs. Preparation of second round of Grade 10 NAS is currently


Page 32 of 55

underway and will be administered in 2018.

01-Apr-2012 30-Jun-2017 30-Jun-2017

Comments (achievements against targets): Five out of six steps showed in the Results Framework were completed (83% achieved).The final step (the end target) was to complete two rounds of a national assessment survey in grade 10. Preparation for the second round of assessment is currently underway (test item development and pilot survey were completed), and will be administered in early 2018. Therefore, it is highly likely that the end target will be achieved in early 2018.

A.2 Intermediate Results Indicators

Component: There were no separate components since the project was part of the GoI’s RMSA Framework and the Bank’s support was provided through a Sector Wide Approach.


Target


Gross Enrolment Ratio (GER) Percentage 58.50 87.40 79.98

01-Sep-2010 30-Jun-2017 30-Jun-2017



Target


Net Enrolment Ratio (NER) Percentage 58.50 87.40 51.60

30-Sep-2009 30-Jun-2017 30-Jun-2017


Page 33 of 55

Comments (achievements against targets): Not achieved (59% achieved). The Project Results Framework shows the same baseline and target values for GER and NER. There is a clear difference between the two and the GER is generally higher than the NER in countries which have issues in access, early entry and repetition. As noted in the 7th ISR, the baseline value and projections were based on the Census 2001 data available at the time of project preparation. The task team concluded that the baseline value set for NER at project preparation did not reflect the actual population at that time. Actual data based on the updated Census suggest that the NER increased from 45.6 percent to 51.6 percent between 2013 and 2015-16. The final value at completion does not include data from Andhra Pradesh and West Bengal.


Target


Elementary to Secondary transition rate

Percentage 83.00 92.00 89.80

30-Sep-2009 30-Jun-2017 30-Jun-2017



Target


Gross Secondary Completion Rate

Percentage 55.40 0.00 64.52

30-Sep-2009 30-Jun-2017 30-Jun-2017

Comments (achievements against targets): The Project Results Framework in the PAD did not provide target value. The data collected during project implementation suggests that gross secondary completion rate increased from 55.4 percent in the baseline to 64.5 percent in 2016/17.


Target



Page 34 of 55

Number of girls per 100 boys completing grade X

Number 79.00 93.00 91.24

01-Sep-2010 30-Jun-2017 30-Jun-2017



Target


Share of Scheduled Castes in Secondary completers

Percentage 16.80 18.30 18.39

01-Sep-2010 30-Jun-2017 30-Jun-2017

Comments (achievements against targets): Achieved (100% achieved).


Target


Share of Schedule Tribe in Secondary completers

Percentage 6.00 7.40 7.67

01-Sep-2010 30-Jun-2017 30-Jun-2017

Comments (achievements against targets): Exceeded (104% achieved).


Target


Proportion of schools with all Percentage 21.00 40.00 24.60


Page 35 of 55

subject specific teachers available in required number

30-Sep-2009 30-Jun-2017 30-Jun-2017

Comments (achievements against targets): Not achieved (62% achieved).


Target


Proportion of teachers with professional qualification (B.Ed or equivalent or above)

Percentage 83.00 87.00 87.00

01-Sep-2010 30-Jun-2017 30-Jun-2017

Comments (achievements against targets): Achieved (100% achieved).


Target


Physical Infrastructure: Proportion of schools with all the main infrastructure features available as required

Percentage 0.30 0.00 0.50

01-Sep-2010 30-Jun-2017 30-Jun-2017

Comments (achievements against targets): The Project Results Framework in the PAD did not provide target value. The data collected during project implementation suggests that this increased from 0.3 in the baseline to 0.5 at completion.


Target



Page 36 of 55

Proportion of secondary schools/sections with at least the minimum number of teachers in position (5 subject teacher plus head teacher)

Percentage 62.20 0.00 87.70

30-Sep-2009 30-Jun-2017 30-Jun-2017

Comments (achievements against targets): The Project Results Framework in the PAD did not provide target value. The data collected during project implementation suggests that this increased significantly from 62.2 in the baseline to 87.7 at completion.


Target


Management, Accountability and Governance issues: Proportion of SDMCs received capacity/ management training (cumulative for past 2 years)

Percentage 0.00 0.00 79.00

01-Apr-2012 30-Jun-2017 30-Jun-2017

Comments (achievements against targets): The Project Results Framework in the PAD did not provide baseline or target values.


Target


Management, Accountability and Governance issues: All Staff positions at State, district and sub-district levels

Percentage 0.00 0.00 77.22

01-Apr-2012 30-Jun-2017 30-Sep-2016


Page 37 of 55

filled

Comments (achievements against targets): The Project Results Framework in the PAD did not provide baseline or target values.


Target


Innovations: Number of innovative activities approved

Number 0.00 75.00 0.00

01-Apr-2012 30-Jun-2017 30-Jun-2017

Comments (achievements against targets): This indicator was not assessed during project implementation due to change in activities under RMSA framework.


Target


Innovations: 25% of the completed innovation component projects which meet at least 75% of their agreed targets

Percentage 0.00 25.00 0.00

01-Apr-2012 30-Jun-2017 30-Jun-2017

Comments (achievements against targets): This indicator was not assessed during project implementation due to change in activities under RMSA framework.


Page 38 of 55

B. KEY OUTPUTS BY COMPONENT18

Objective/Outcome 1: Increased access to secondary education

Outcome Indicators 1. Enrollment of students in secondary (grades 9 and 10)

Intermediate Results Indicators

1. Gross enrollment ratio (GER) 2. Net enrollment ratio (NER) 3. Elementary to secondary transition 4. Gross secondary completion rate 5. Proportion of schools with all the main infrastructure available as required

Key Outputs by Component (linked to the achievement of the Objective/Outcome 1)

1. New secondary schools established 2. Existing schools upgraded 3. Main infrastructure features (libraries, books, science laboratories, laboratory

consumables, computer laboratories, and so on) available at school 4. Vocational education introduced in secondary schools 5. Additional teachers recruited

Objective/Outcome 2: Improved equitable access to secondary education

Outcome Indicators 1. Gender parity index (GPI) in enrollment


1. Number of girls per 100 boys completing grade 10 2. Share of Scheduled Castes in secondary completers 3. Share of Scheduled Tribe in secondary completers 4. GER for students from disadvantaged social categories


1. Girls’ hostels established in educationally backward blocks 2. New schools established in underserved areas 3. Teachers trained on gender sensitization

18 For the purpose of reporting to the World Bank Management, all PDO-level indicators and selected intermediate results indicators are included in the ISRs.


Page 39 of 55

Objective/Outcome 3: Enhanced quality of secondary education

Outcome Indicators 1. New system of National Assessment of Students’ Achievement developed and used to

inform the RMSA’s Quality Improvement Policies 2. Gross graduation rate (in schools covered by RMSA)


1. Proportion of schools with all subject specific teachers available in required number 2. Proportion of teachers with professional qualification 3. Proportion of schools with specific teachers available to teach for five core subjects 4. Proportion of SDMCs received capacity/management training 5. All staff positions at state, district, and sub-district levels filled 6. Number of innovative activities approved 7. 25% of the completed innovation component projects which meet at least 75% of their

agreed targets


1. Relevant learning materials provided to schools 2. Curriculum framework 2005 adopted in states 3. National Program on School Standards and Evaluation developed 4. Qualified teachers appointed 5. Teaches trained through in-service professional training 6. Head teachers/principals trained for school leadership 7. Local institutions strengthened 8. National Assessment Survey for grade 10 conducted 9. Staff positions at state, district, and sub-district levels filled 10. Innovation activities approved and implemented 11. School Management and Development Committees (SMDCs) trained and established 12. Official database for key school-, teacher-, and student-related indicators improved


Page 40 of 55

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION

A. TASK TEAM MEMBERS

Name Role

Preparation

Supervision/ICR

Sangeeta Dey Task Team Leader(s)

Satyanarayan Panda Procurement Specialist(s)

Papia Bhatachaarji Financial Management Specialist

Rocio Mariela Malpica Valera Counsel

Sangeeta Kumari Team Member

Neha Pravash Kumar Mishra Environmental Safeguards Specialist

Tanusree Talukdar Team Member

Namrata Raman Tognatta Team Member

Kanchan Rajeevsingh Parmar Social Safeguards Specialist

B. STAFF TIME AND COST

Stage of Project Cycle Staff Time and Cost

No. of staff weeks US$ (including travel and consultant costs)

Preparation

FY10 48.120 204,150.42

FY11 130.328 323,787.37

FY12 36.232 139,985.87

FY13 1.259 5,435.19

FY14 0 0.00

Total 215.94 673,358.85

Supervision/ICR


Page 41 of 55

FY13 15.317 59,496.78

FY14 13.796 56,336.69

FY15 26.860 99,478.09

FY16 51.045 185,580.81

FY17 41.795 115,610.70

FY18 9.051 37,379.38

Total 157.86 553,882.45


Page 42 of 55

ANNEX 3. PROJECT COST BY COMPONENT

Components Amount at Approval

(US$, millions)

Actual at Project Closing (US$,

millions)

Percentage of Approval (%)

RMSA Program 500.00 475.87 95.1

Total 500.00 475.87 95.1

Note: The project was completed with full disbursement. There was some loss in U.S. dollars due to exchange rate fluctuation.


Page 43 of 55

ANNEX 4. EFFICIENCY ANALYSIS

1. The project achieved its objectives efficiently within the original time frame with full disbursement. The implementation and design of the project were well thought out, and a number of noteworthy efficiency gains took place under the project: (a) the project used resources in an efficient manner for the certain level of outcomes, despite lower counterpart funds than those estimated at project preparation, (b) the drop-out rate for secondary education decreased from 17.9 percent in 2013-2014 to 17.0 percent in 2016-2017. This is an evidence of increased efficiency of internal system and reduced wastage of public and private resources, (c) streamlined administration procedures with use of information technology has resulted in increased program efficiency, (d) an online project monitoring system (PMS) has been developed and used for monitoring progress in the states, districts, and secondary schools. This online PMS enhanced efficiency in the management and implementation of the RMSA in the states/UTs, (e) better fiduciary procedures that improved planning, budgeting and timely release of funds have improved implementation efficiency, (f) the project leveraged the expertise of national (for example, NUEPA and NCERT) and state-level institutions to provide technical assistance on aspects such as teacher training, school leadership, and school standards, (g) despite not implementing the Innovation Fund initiative, some states have developed and implemented innovative interventions. For instance, satellite technology was used for beaming lectures/lessons of subject experts remotely to classrooms, especially in schools without mathematics and science teachers; and information and communication technology was used for teacher training, and (h) the MHRD supported states/UTs to share good practices on innovative interventions that have had successful outcomes at the state level. Both qualitative and quantitative aspects of efficiency gains were observed. Thus, the efficiency rating is Substantial.

2. Economic analysis. A cost-benefit analysis conducted at the time of ICR shows a high economic rate of return to the project. Data realized at the end of the project are used to compute the stream of costs and benefits generated, yielding an economic internal rate of return (EIRR). The data realized at the end of the project are compared to the counterfactual (the estimated data in the absence of the project). In addition to this base case, a low case is estimated which assumes greater than trend improvements in the counterfactual, thereby resulting in a more conservative estimate of project returns. The EIRRs in the base and low case are 16.1 percent and 9.7 percent, respectively.

3. In addition to economic returns, the societal returns to investing in secondary education are considerable. There is no direct work or data on the externalities from investing in secondary education in India, but the literature suggests benefits to society and the economy accrue through a range of channels, including a more productive work environment, scientific innovation with widespread general benefits, better-functioning democratic institutions, careful use of natural resources, a healthier population, and more effective (and meaningful) collective action.

Table 4.1. Benefits

Priced Unpriced

Increase in graduate earnings due to

• Increased labor force participation

• Better pay

• Improved job satisfaction and quality of life from better jobs and employment conditions

Increase in tax revenue as incomes rise • General improvement in skill level of labor market


Page 44 of 55

entrants and corresponding productivity gains.

• Improvement in areas of social development due to a higher caliber workforce.

• Greater economic and social equity as education levels improve. Social indicators improve as education levels rise.

Cost savings and improved efficiencies due to

• Better data management and the use of data to inform policy decisions

• Better procurement and FM

• Better information flows

• Better overall management of the secondary education sector by the MHRD and state governments

Reduced costs as students complete secondary school faster

4. The PDO was “to assist the Recipient in achieving increased and more equitable access to good quality secondary education by supporting its ongoing program for secondary education.” Project activities have been assumed to yield economic returns primarily through an increased number of secondary completers entering the labor force and an increased wage premium for secondary education completers. Data used come from the national census, the national sample survey, administrative data on education, and the national budget.

5. The base and low cases were built on the following assumptions presented in table 4.2. Based on these assumptions, the net present value (NPV) of net cash flows from the RMSA are estimated to be US$3.5 billion in the base case and US$0.6 billion in the low case. The EIRRs in the base and low case are 16.1 percent and 9.7 percent, respectively.

Table 4.2. Assumptions

Variables Base Case Values Low Case Values

Secondary GER19

60.5% in 2011, growing at 3 percentage points annually

60.5% in 2011, growing at 2.9 percentage points annually

Secondary completion rate20



Wage premium for secondary completers over primary completers21

An increase of INR 7,500 per year over the 2011 value of INR 35,000 per year assuming 80% of secondary graduates are employed

An increase of INR 6,250 per year over the 2011 value of INR 35,000 per year assuming 80% of secondary graduates are employed

Recurring costs (public)22

90% of total public costs (see table on secondary education expenditure)


19 Source: Unified District Information System for Education (U-DISE). 20 Source: U-DISE. 21 Source: National Sample Survey 68th Round (2011). 22 Source: MHRD: Analysis of Budgeted Expenditure on Education 2010-2013.


Page 45 of 55

Investment costs (public)23



Private costs24 INR 6,788 per year INR 6,788 per year

Discount rate (%)25 8% 8%

6. Financial analysis. The GoI’s total expenditure on the RMSA was INR 194,650 million over the project period. The fiscal implications of the project were minor, with the project accounting for less than 0.25 percent of total GoI expenditure in all years. The RMSA, however, accounted for a significant proportion of expenditure on secondary education, reaching a high of 39 percent in 2015. This reflects the project’s broad reach, covering all government and government-aided secondary schools and reaching 35 million beneficiaries.

Table 4.3. Central Government Expenditure on Secondary Education (INR, millions)

2011 2012 2013 2014 2015 2016

Total government budget

13,187,190 14,308,250 15,904,340 16,811,580 17,853,910 16,167,110

Secondary education 88,950 92,950 100,250 99,920 91,840 n.a.

Secondary education as a percentage total of government budget

0.67 0.65 0.63 0.59 0.51 n.a.

RMSA 24,240 31,730 31,230 34,800 35,650 37,000

RMSA as a percentage of total government budget

0.18 0.22 0.20 0.21 0.20 0.23

RMSA as a percentage of secondary education expenditure

27 34 31 35 39 n.a.

Source: Revised GoI budget estimates.

23 Source: MHRD: Analysis of Budgeted Expenditure on Education 2010-2013. 24 Source: National Sample Survey 71st Round (2014), deflated to 2011 prices. 25 Source: Base lending rate, Reserve Bank of India.


Page 46 of 55

ANNEX 5. BORROWER COMMENTS


Page 47 of 55

ANNEX 6. BORROWER’S COMPLETION REPORT26

26 It should be noted that this is the borrower’s completion report of the RMSA Program.


Page 48 of 55


Page 49 of 55


Page 50 of 55


Page 51 of 55


Page 52 of 55


Page 53 of 55


Page 54 of 55


Page 55 of 55

ANNEX 7. SUPPORTING DOCUMENTS

A. World Bank Project Documents

World Bank. 2009. Country Strategy for India for the Period FY2009–2012.

World Bank. 2013. Country Partnership Strategy for India for the Period FY2013–2017.

World Bank. 2012. Project Appraisal Document.

Aide Memoires for the 1st–9th Joint Review Missions.

Financing Agreement (Credit No. 5088-IN).

Implementation Status and Results Reports (ISRs) No. 1–12.

Letter of Amendment to Financing Agreement for the Project.

Project Restructuring Paper (Report No. RES12866) dated June 5, 2014.

B. Government’s Documents

MHRD (Ministry of Human Resource Development).2017a. A Report on Implementation of RMSA Program.

———.2017b. Completion Report of Secondary Education Project (RMSA Program).

Planning Commission. 2008. India 11th Five-Year Plan 2007–2012.

Planning Commission. 2013. India 12th Five-Year Plan 2012–2017.

Implementation Completion and Results Report (ICR)...

Documents

Transcript of Implementation Completion and Results Report (ICR)...