ANNEX 7: DETAILED DESCRIPTION OF NATIONAL...

Best practice example of economic performance criteriaAnnexes 7 & 8 of Brazil: Second Northeast Basic Education Project (1993) SAR # 11298-BR

1 of 24

ANNEX 7: DETAILED DESCRIPTION OF NATIONAL COMPONENT ANDPERFORMANCE INDICATORS

National Component— Total Costs US$11.64 million

1. The objective of the project's national component is to help foster a new federalrole in basic education by supporting a program aimed at strengthening the capacity ofthe Ministry of Education (MEC), through SEF and its related entities (namely. CPS,FNDE, INEP, and FAE), to carry out key oversight and support functions in basiceducation, particularly (i) systematic evaluation of school systems, includingstandardized testing of student achievement; (ii) systematic policy analysis, includingdissemination of effective educational approaches, technologies, and innovations; (iii)management of educational information and statistics; (iv) monitoring and supervision offederally-funded projects and program, including technical assistance to state educationsecretariats; and (v) reform of the national textbook and school lunch programs, togetherwith implementation of a new system for book provision to be piloted in the Northeaststates covered by the project. The Bank will monitor progress towards the achievementof these broad objectives yearly through the set of performance indicators described inAnnex A.

2. The national component will consist of four sub-components: (A) educationalevaluation, (B) policy analysis, (C) education management, and (D) reform of thetextbook and school lunch programs. These sub-components will support the Ministry’shuman and institutional development by financing studies, technical assistance and otherconsultant services, equipment (mostly micro-computers and related hardware as well assoftware), dissemination activities, project supervision, and incremental operating costs(excluding staff salaries).

A. Education Evaluation (US$5.66 million)

3. The objective of this sub-component is to build up the national capacity forcarrying out systematic evaluations of student achievement, school effectiveness andsystem productivity; analyzing, interpreting and disseminating findings; and translatingthe resulting knowledge into policies, strategies and actions aimed at improving publicbasic education in Brazil. Towards this end, the project will support: (i) human resourcesdevelopment; (ii) institutional development of selected agencies, and (iii) strengtheningof the national student assessment system (SAEB). The program of human resourcedevelopment will operate through a special program of grants to be administered byCAPES, according to yearly operational plans to be formulated by CAPES and submittedto SEF for approval. The program of institutional development will be by SEF, and INEPwill be in charge of the support to SAEB.

4. To support human resource development in education assessment and testing, theproject will finance scholarships and fellowships for higher-level training in overseas andnational research institutes, universities, and state-of-the-art testing centers. Grants for


2 of 24

formal graduate studies as well as non-degree courses will be provided on a competitivebasis to talented Brazilians of various academic backgrounds interested in a career ineducation evaluation in Brazil, regardless of their pre-graduate studies employmentstatus. Eligible applicants will include staff (including teachers) in universities, researchinstitutes, and government agencies responsible for education. The operational plan forthis component will establish the criteria for the competitive selection of the applicants.

5. The project will support institutional development through a program of grantsaimed at establishing regional centers that specialize in the technical requirements ofeducation evaluation. These centers would have the capacity to develop as well as absorband adapt know how and technology applied to measurement, assessment and testing ineducation. To support the development of an operational research and developmentcapacity in educational evaluation, financing will be available for program development,research, staff exchanges, organization of and attendance at seminars and workshops,purchase of equipment (including computers and software) and library materials,publication of bulletins and periodicals, hiring of consultant services to carry out specifictasks or studies, internships, and study tours. To foster technological transfers, the projectwill finance technical cooperation between international and national academic and non-academic institutions.

6. Grants will be awarded biannually by CAPES on a competitive basis to selectedpublic and private domestic institutions that respond to a public call for proposals bysubmitting multi-year institutional development plans. The selection will be carried outby a high-level advisory committee formed by representatives of SEF, INEP and CAPES;two representatives from the Brazilian scientific community; and one international expertin education evaluation. The selected institutions will be expected to participate activelyin the improvement and consolidation of SAEB by (i) cooperating in research anddevelopment of tests, other survey instruments and computational resources, (ii) helpingstrengthen SAEB management; (iii) supporting SAEB data analysis; and (iv) suggestingmethods for dissemination of SAEB findings and results.

7. While the project pursues its broader objective of developing human andinstitutional capacity in measurement, assessment and testing in education, it will alsopromote. the improvement and consolidation of a recently developed national system forevaluating basic education, namely the Sistema de Avaliacao da Educacao Basica(SAEB), which utilizes sample-based methodology to evaluate inputs, processes andoutcomes of school systems. The project will support the systematic administration ofSAEB tests and questionnaires to students, teachers and administrators in public schoolsystems in all states of Brazil. In addition to achievement testing, SAEB will gather dataon school inputs, costs and management. The resulting database would allow forcomparisons of student achievement by geographical unit and type of school, as well asby quantity and quality of school inputs and processes.

8. The project will finance data collection, analysis, reporting and feedback to


3 of 24

school systems. Pre- and post-project comparisons would be made to assess the impact ofthe project. Also, the project will provide equipment, mostly computers and relatedhardware and software, to improve SAEB's routines for collecting and processing data,manipulating files and data banks, producing reports, and providing feedback to schoolsystems. Consultant services win also be hired to install and maintain software, packages,as well as to develop, install, test and maintain application programs and integratedcomputerized systems. The project will also finance technical for the team responsible foroperating SAEB.

9. Local and foreign consultants will be contracted on short as well as long termbases to carry out the research and development required to refine the SAEBmethodology, improve and validate instruments, and advise on technical matters of testapplication, such as sample size and definition, logistics, and security requirements. Theproject will support the development and refinement of criterion-referenced tests andtheir administration to a sample of students selected from the population of all second,fifth, and eight graders in federal, state, and municipal schools. Samples would bedesigned to cover the different types of school and educational modalities.

10. In principle, SAEB will be executed by INEP, with input from SEF, CPS andstate education secretariats, according to a detailed five-year research, development andoperation plan acceptable to the Bank. The plan will, inter alia, (i) ensure that the testingand other data gathering instruments will be carried out according to sound scientificmethodology, (ii) list interim and final outcomes, (iii) spell out the implementationschedule, and (iv) provide profiles of key staff. To verify that INEP develops the properinstitutional setting for SAEB, SEF will appoint and maintain a high- level technicalcommittee formed by two MEC representatives, two international experts in educationassessment, and two renowned Brazilian educators, all with qualifications and experienceacceptable to the Bank. The task of this high-level technical committee is to carry outperiodic assessments and technical reviews of SAEB procedures, methods, and outcomesin order to produce technical recommendations for its development, including, inter alia,the adequacy of its (i) institutional placement, (ii) staffing profile, (iv) operationalprocesses and procedures, (vi) methodology, and (v) instruments. On the basis of theserecommendations, SEF may select a new institution to house SAEB, acceptable to theBank.

11. Although most of the work will be contracted out to regional centers receivinginstitutional development support, INEP will establish a full time in-house team formedby regular staff as well as short- and long-term consultants. Three rounds of SAEBadministration are expected to be carried out during project execution: the first one in1993, using methodology and instruments already available; of the other two, one in 1995and another in 1997, using methodology and instruments accepted by the technicalcommittee. Seminars and debate forums for different constituencies and interest groupswill be organized to disseminate the findings of SAEB, promote its legitimacy as a publicpolicy instrument, and foster the creation of an “evaluation culture" within the educationcommunity of Brazil.


4 of 24

B. Policy Analysis (US$1.63 million)

12. The objective of this sub-component is to help SEF and its related agenciesdevelop the institutional capacity for systematic policy formulation, analysis and review.The project will finance applied research on basic education. Researchers at universitiesand other research centers will be eligible for this funding, which will support pre-investment and evaluation studies, operational research, and policy papers. The projectwill also finance dissemination of findings and recommendations to practitioners as wellas policy makers, through activities such as publications, workshops, seminars andconferences. At a minimum, the project will finance five conferences (one a year), wherethe studies executed that year will be presented and commented on, and abstracts will bepublished in annals, prefaced by executive summaries written by the high-level advisorycommittee.

13. Annual priorities for funding will be determined by SEF based on therecommendations of a high-level advisory committee appointed and maintained by SEF.This committee will include three government representatives (probably from SEF, MUand CPS), one international expert in education research, and two renowned Brazilianeducators, all with qualifications and experience satisfactory to the Bank. This committeewill meet periodically to (i) define annual analytic agendas and issue calls for proposals(once a year, probably in March, to define the agenda for the following year), (ii) reviewand select study proposals, and, upon completion of the studies, (iii) make comments onthe executive reports to be prepared by SEF in order to provide MEC authorities withspecific recommendations about how study findings can be integrated into policy making.In addition to the studies themselves, the project will finance travel expenses andhonoraria for the committee members and logistical support for its meetings.

14. The project will also support the establishment of a system for cataloging,evaluating and disseminating innovations and standards in basic education. Consultantservices for setting up and systematically updating data bases, as well as for carrying outevaluation studies and dissemination activities will be financed by the project. Amongother things, the project will commission a survey to compare desired and actual patternsof utilization of resources (financial, human, and physical) in public and private schoolsystems. The objective would be to establish and validate minimum standards for publicschool systems in terms of the quantity as well as the quality of inputs, processes, andoutcomes. Once operational, this data bank could provide services and information tostates and municipalities interested in adopting innovative approaches aimed atimproving the quality of basic schooling.


5 of 24

C. Education Planning and Management (US$3.4 million)

15. The objective of this sub-component is to help SEF and its related agenciesdevelop the institutional capacity for systematic planning and management through aprogram of technical assistance, training and purchase of equipment. The subcomponentwill support four lines of action; (i) management review; (ii) data management; (iii) stafftraining; and (iv) the project support units (UNESP and UESCN). The project willfinance consultant services to carry out management reviews aimed at assessing theadequacy of the processes, procedures and tools employed by SEF staff and managers intheir routine and strategic activities and, as a consequence, to develop integrated datamanagement systems required for proper implementation of the recommendations madeby the reviews. The project will finance consulting services, revision. of the content ofcurrent administrative records to include all necessary data and exclude irrelevantinformation, review of methods and instruments employed for data collection andprocessing, and manipulation of files and databases, data collection, improvements in theperiodicity, coverage, and quality of reports and publications, reviews of hardware andsoftware specifications, networking options, and operational requirements, and purchaseof equipment, mostly microcomputers and related hardware to house the integrated datamanagement systems.

16. Training opportunities for staff. The objective of this subcomponent is to providehigh- and middle-level managers with the concepts, skills and tools appropriate forpracticing the knowledge-based decision making required for improving the quality andefficiency of the services provided by SEF to its constituencies. The project will providethe staff of SEF and its entities with three types of professional growth opportunities: (i)higher-level nondegree courses in systems analysis, files management, informationsystems, policy analysis, institutional research and development, project analysis,educational finances, project management, institutional culture, textbook publication andprovision, teacher training, information-based decision , and so forth; (ii) hands-onexperience through internships, workshops and study tours to familiarize staff withinnovations, best practices and technological, advances in effective school systemsthroughout the world that are relevant to the Brazilian context; and (iii) upgradingopportunities through attendance at national and international seminars and conferences.These experiences will be offered to talented and motivated staff in various educationalbranches of MEC (namely SEF, CPS, INEP, and FNDE) and in the education secretariatsof the states covered by the project.

17. As part of this subcomponent, SEF will maintain a unit (Unidade Encarregada doSuporte Componente Nacional, or UESCN) to coordinate all activities under the nationalcomponent as well as the national project support unit (Unidade Nacional Encarregadado Suporte do Project, UNESP), which will liaise with all participating agencies,including state and municipal education secretariats, the UESCN, and other federalgovernment agencies (such as DTN, SEAIN, and the Central Bank). The principalfunctions of the UESCN and UNESP are to assure that project components are executedaccording to the agreed-upon terms and timetable. Similar to the project units in the stateeducation secretariats, these units will maintain computerized system acceptable to the


6 of 24

Bank for monitoring procurement, disbursements, and project activities. This MIS systemwill be modeled after the one currently being used in the Second Northeast Basic HealthProject.

18. UNESP and UESCN will receive the human, financial and material resources forefficient and timely programming, accounting and monitoring of project components.The project will finance (i) development, implementation and operation of acomputerized account and monitoring system for the project; (ii) equipment, consultantservices. development and operation costs; (iii) technical assistance to help statesimplement their investment and management reform plans in all areas covered by theproject, such as teacher training, educational planning and management, teachingmaterials, financial management, facilities planning and management and computerizedinformation systems; (iv) supervisory field work to assure timely execution of the project,including periodic trips to the states to inspect public works, distribution of textbooks andother didactic to public schools, and usage of these materials by the teachers, as well as tomonitor the adherence of each state to its management efficiency targets; and (v)incremental opposed costs, including communications, publications, travel expenses andhonoraria for advisory committee members and others, supplies, logistical support formeetings, through an administrative fund. In addition, workshops will be organized totrain staff assigned to project support units in state education secretariats and SEF insubjects relevant to project implementation: (a) planning and programming techniquesand tools; (b) bidding and other procurement methods; (c) accounting procedures andtools; (d) public works supervision, (e) disbursement practices and requirements; and (f)project management.

D. Reform of the Textbook and School Lunch (US$0.95 million)

19. The project will finance a major study of alternative financing mechanisms for thenational textbook program and a study of the cost-effectiveness of the centralized schoollunch program. These studies will, inter alia, identify the major problems with theseprograms and propose solutions, and analyze the costs, benefits, and operationalfeasibility of each alternative.

20. This subcomponent also includes financing for the program of technicalassistance (already underway) to improve textbook provision in the Northeast statesselected for the NE II and III projects. This program aims guaranteeing that all firstthrough fourth grade students in the state and municipal systems are provided with aminimum package of books. The project will finance (i) technical assistance forsupporting development, implementation and operation of the new system for bookprovision, including the design and implementation of monitoring and evaluationsystems; (ii) specialized training in book specification, procurement, printing, production,quality control and distribution. for the staff of the PNLD and the state educationsecretariats; and (iii) consultant services, computer hardware and software to support thedevelopment of a computerized programming and monitoring system for the PNLD.


7 of 24

21. Responsibility for executing this national component will be vested in theUESCN, which will liaise with CAPES, INEP, CPS, FAE, and FNDE to make all thenecessary institutional arrangements to assure that the planned activities are executedaccording to the agreed-upon terms and timetable.

22. As a condition of Board presentation, MEC presented a draft operational plan,acceptable to the Bank, which included initial timetables and cost estimates for activitiesto be carried out during the first year of project implementation, as well as a list ofactivities planned for subsequent years.


8 of 24

Draft as Oct. 6, 1992NEBE--II, NATIONAL COMPONENT

Performance Indicators Form (to be filled every year in December by UNESP)

Subcomponent: Education Evaluation 1992 1994 1995 1996 1997RE: Administration of SAEB1. HLTC appointed/maintained by MEC2. HLTC completed bi-annual review of SAEB According to the HLTC report, SAEB's..3. ...institutional placement is appropriate4. ...staffing profile is satisfactory5. ...processes and procedures are satisfactory6. ...methodology and instruments are satisfactory7. ...products and outcomes are satisfactory8. Last year's results are published9. This year's administration was completed10. At least one dissemination activity was organizedRE: Institutional Development11. At least 3 bi-annual institutional grants awarded/maintained12. At least 1 technology transfer contract signed/maintainedRE: Human Resource Development13. At least 10 scholarships for GS in EE awarded/maintained14. At least 5 grants for overseas internships in EE awarded15. At least 3 fellowships for short courses in EE provided16. Directory of graduates and trainees is published

Subcomponent: Policy Analysis 1993 1994 1995 1996 1997RE: Fund for Research17. AG appointed/maintained by MEC18. Annual research agenda defined by AG19. AG policy recommendations was issued20. Studies-policy making linkage summary is publishedRE: Innovations and Standards Data Bank21. Computerized data bank is operational22. Standards for resource use in school systems are published23. At least, six innovations are fully catalogued24. Bi-annual innovations catalog was publishedSubcomponent: Education Management 1993 1994 1995 1996 1997RE: Management reviews and systems25. At least one management review is completed26. Last year's educational statistics published by June of this year27. At least 25 staff attended training courses of 40 hours or more28. At least one overseas study tour for 15 staff was organizedRE: National project support unit (UNESP)29. Computerized system is operational30. Quarterly supervision reports submitted to the Bank31. At least 6 three-day supervision trips per project state were made32. At least 6 person-days of TA per project state were providedRE: Subcomponent: Reform of textbook and school lunch program33. Comprehensive management review of FAE was completed34. MEC is implementing reforms recommended by management review35. This year's books arrived in all schools by March of this year.

ACTUAL SCOREMINIMUM REQUIRED SCORE 25 26 28 30 33

MAXIMUM EXPECTED SCORE 30 33 33 33 34VALID CODES ARE AS FOLLOWS: 0 if answer is NO (i.e., no or partial compliance) 1 if answer is YES (i.e., total compliance)LEGEND: AG Annual Analytic Agenda EE Education Evaluation GS Graduate Studies HLTC High-Level Technical Committee TA Technical Assistance


9 of 24

ANNEX 8: PERFORMANCE INDICATORS FOR STATE PROJECTCOMPONENTS: EDUCATION MANAGEMENT,

TEACHER TRAINING, EDUCATIONAL MATERIALS

1. Two types of performance indicators will be used to monitor the implementationof the education management component, teacher training component, and theeducational materials component in each states. These are:

• key accomplishment indicators; and• outcome indicators.

A. Introduction

2. The first type of indicator is the key accomplishment indicator. This type ofindicator will be used to monitor critical accomplishments of each of the majorcomponents of the project permitting the Bank, MEC, and the states to establish broadtargets for project implementation. The key accomplishment indicators to be used, theirbaseline values, and the annual values target values for these indicators were agreed atnegotiations. The second type of indicator will measure the specific outcomes of each ofthese three components. These outcome indicators, which correspond to the “activities”specified under each subcomponent, will be specified in the operational plans for thethree components of each state.

B. Key Accomplishment Performance Indicators

3. The four groups of "key accomplishment" indicators presented below have beenselected to represent milestones in the 'critical path' for successful project implementationThey concern:

(a) Student flow in state and municipal primary school systems.(b) Project Implementation Checklists (PICs) for critical actions under the

three state components- education management, teacher training, and didactic materials.

(c) Annual school-level information database reports.(d) Annual project goods and services delivery report and discrepancy

analysis.

a Student Flow (Reference: Table A10-a, attached). The overall goal forSecond Northeast Basic Education Project (NEBE II) to "increase studentlearning, reduce repetition and dropout, and increase graduation rates in the four states”. The first key accomplishment performance indicator isdesigned to monitor progress on this overall goal. The emphasis here willbe on repetition, dropout, and graduation rates because , unfortunately,direct indicators of student learning, by state and sector within state, maynot be available until after the first few years of project implementation.


10 of 24

Without baseline measures, it is difficult to set performance targets.Consequently, the first set of key accomplishment performance indicatorsmeasures the flow of primary school pupils across the grades within thestate schools and also within municipal schools. The actual indicatorsmeasure the proportion of students bunched up in first grade, and theproportion of primary pupils who graduate each year from the primaryeducation system. The baseline measure is taken from CPS reports for 1989. Annual targets are estimated as the minimal significantimprovements on these measures, based on a 10% annual reduction in therepetition rate, and a 5% reduction in the dropout rate. These indicatorsand targets are presented in Table A10-a. The data for this indicator

shouldbe derived from MEC statistics, and the UNESP will be responsible forcalculating these indicators and completing Table A10-a.

b Project Implementation Checklists for the Education Management Component (Reference: Tables A10-bl to A10-b5, attached). These projectimplementation checklists (PICs) will serve as the primary means oftracking the overall execution of the education management component.Each of the first four checklists (PIC-1 through PIC-4) is designed to becompleted annually by each state's Project Support Unit (UEESP). Theproject leader in each state is to be held accountable for carrying out thistask. PIC-5 evaluated the UEESP itself, and the UNESP project leader isresponsible for completing this checklist on the operational effectivenessand efficiency of the UEESP. The objective of these checklists is to assurethat key accomplishments and actions of the component are monitored.Each item on a checklist may need additional explanation, in particular, tospecify precisely what evidence will be required to score that item. TheUEESP is responsible for preparing this additional documentation, asnecessary. Each item has a potential score, and the UEESP can give marksup to and including the target score, depending on the evidence available.In addition to being accountable for the completion of the annual checkliston time, the project leader is also accountable for the particular marksgiven each item on the checklist. The bottom row on each of thesechecklists summarizes the expected implementation for the project foreach year of project execution.

c Annual school-level information database reports (Reference: Table A10-c, PIC-6, attached). One of the main objectives of the state program of theNEBE II project is to improve management efficiency in each state’seducation system through improvements in the use of information,particularly on schools, the primary instrument for delivering educationalservices. Specifically, planners and administrators need up-to-date andaccurate information on staff and material resources available in eachschool, both in the municipal and state systems. The collection andtabulation of school-level information from each school will serve this


11 of 24

purpose1. Simultaneously, the school report will allow project monitors inthe UEESP to track the receipt of the various goods and services supportedby the project2. The performance indicator for this activity is the annualcompletion of survey forms, one for the municipal sector, and one for thestate sector, summarizing the information from the school-levelinformation database (SLID). To ensure the integrity of these data, thesetwo summary forms will be backed up by one summary form for eachmunicipality. Table A10-c provides the forms for which these data shouldbe reported, overall, by sector, and by municipality. These municipalsummary sheets will in turn be supported by computer printouts from thedatabase reporting the same information by school. The data will include acount of the number of classrooms the school contains, the total number ofstaff employed at the school, the number of staff directly engaged inclassroom teaching, the number of teachers who received inservicetraining the previous year, the number of school directors and assistantdirectors who received training the previous year, the number ofPortuguese textbooks and the number of mathematics textbooks availableand the number received last year for first, for second, for third, and forfourth grade children, the number of reading books previously in stock andthe number the school received last year, and the number of didacticmaterials kits in stock and the number the school received last year. Inaddition to this database, the UEESP will submit a summary of thisdatabase. Table A10-c is the performance indicator instrument that thisinformation will be provided.

d Annual report on delivery of projects and services. (Reference: TableA10-d, PIC-7, attached). These reports are designed to correspond to theschool-level information sheets. In this case, however, the data themselveswill be supplied by UEESP's Project Procurement and Delivery OrdersDatabase, in contrast to the previous case in which the reports will be builtup from the school-level information database. Table A10-d provides theformat for which these data should be reported, overall, by sector, and bymunicipality. UEESP will also provide, along with these sheets, adiscrepancy analysis, (based partially on the third column of this table)comparing the receipt of goods and services by the schools with thedelivery orders themselves.

1 Tables A10-c.1 through A10-c.4 represent instruments that can be used to collect this school level information.2 The subsequent indicator (reference: Table A10-d) tracks this same information from the site of distribution, and estimates a discrepancy analysis.


12 of 24

Outcome Indicators (Reference Table A10-e).

4. The purpose of outcome indicators is to monitor the achievement of the activitiesspecified under each subcomponent in (a) the education management component, (b) theteacher training component, and (c) the education component. Underlying theseindicators is the structure of the project design itself, which will be detailed in theoperational plans for these components, prepared by each state and finalized prior toproject disbursement for that state. Each component has a specific objective or set ofobjectives. Under each component, subcomponents were selected to achieve that purpose.These subcomponents may differ from one another in terms of their focus, orientation,and structure. Nonetheless, taken together, they constitute the necessary and sufficientconditions to achieve the component’s purpose. Component activities are the buildingblocks of the subcomponents. It is the activities that the project team actually implement.The component “teacher training" for example, whose purpose is described in the SAR, iscomposed of three, separate subcomponents: short-term teacher training; leadershiptraining; and school-based training. Each of these subcomponents is implementedthrough discrete “activities”. Under the third subcomponent, "school-based training"activities might include “sensitizing the school team to the high rates of repetition;” or“training school personnel to use teamwork” Activities are accomplished through theprocurement and application of any of four types of inputs (items to be financed):training, technical assistance, materials and equipment, and dissemination of information.This design structure is depicted in Figure A-1.

5. Output indicators are definitionally associated with level #(3), the discreteactivities to be carried out under each subcomponent. For example, under short-termteacher training, key accomplishment indicators would include, inter alia, the of numberof training curriculum guides produced, the number of trainers trained for each type ofprogram, the number of teachers or other staff trained in each type of program, thenumber of teachers or other staff passing an objective test measuring their acquisition ofthe knowledge and skills taught during the training program, etc. The rationale for outputindicators is that, as a first step, we need mechanisms to assure that each activityidentified under the component is accomplished as planned. Output indicators, however,are not designed to ascertain whether the objective of the component or thesubcomponent is achieved. To accomplish this, we will employ system indicators tocomplement the output indicators.


13 of 24

- Training- T.A.- Equipment- Other

Activity 1A(i)


Activity 1A(ii)

Subcomponent 1A


Activity 1B(i)


Activity 1B(ii)

Subcomponent 1B

Component 11.

2.

3.

4.

LEVEL

Figure A-1


14 of 24

Table A10-a. NEBE-II, OVERALL PROJECT GOALS: STUDENT FLOWTargets for Student Flow(to be completed by MEC)

States: Ceara __ Maranhaor__ Pernambuco__ Sergipe__Expected Targets/Actual Values (in %)

Baseline (measured in May of subsequent year)Distribution of Students Across Grade Levels (Year) 1991 1992 1993 1994 1995 1996 1997

Municipal grade 1-4 pupils enrolled in grade 1(%)? 30.5 (1989)State grade 1-4 pupils enrolled in grade 1 (%)? 44.9 (1989)Municipal grade 1-8 pupils enrolled in grade 1 (%)? 39.3 (1969)State grade 1-8 pupils enrolled in grade 1 (%)? 18.6 (1989)Municipal grade 1-4 pupils completing grade 4 (%)? 6.1 (1988)State grade 1-4 pupils completing grade 4 (%)? 11.9 (1988)Municipal grade 1-8 pupils enrolled in grade 8 (%)? 1.2 (1988)State grade 1-8 pupils enrolled in grade 8 (%)? 3.8 (1988)Municipal students enrolled in grade 1State students enrolled in grade 1Municipal students enrolled in grade 2State students enrolled in grade 2Municipal students enrolled in grade 3State students enrolled in grade 3Municipal students enrolled in grade 4State students enrolled in grade 4Municipal students enrolled in grade 5State students enrolled in grade 5Municipal students enrolled in grade 6State students enrolled in grade 6Municipal students enrolled in grade 7State students enrolled in grade 7Municipal students enrolled in grade 8State students enrolled in grade 8


15 of 24

Table A10-b1. NEBE--II STATE PROGRAM - KEY ACCOMPLISHMENT INDICATORSProject Implementation Checklist: PIC-1(Agent Responsible for Monitoring UESP)

States: Ceara __ Maranhaor__ Pernambuco__ Sergipe__Component: Education Management ComponentSubcomponent: Educational and Financial Planning

Expected Targets/Actual Score (measured inMay of subsequent year) (yes=2, almost=1,

no=0)

Category Indicator Baseline 1992 1993 1994 1995 1996 1997Studies A. Fundamental quality school study completed and updated

annually. Study of recommended characteristics and costs of afund. (min.) quality school for state and municipal systems,covering instructional hours, staffing norms, educational and

0 0/ 0/ 2/ 2/ 2/ 2/

other material standards, etc. Basic infrastructure criteria will be treated separately.

Training B. Specific training targets (including staff named to trainingprograms) for next year identified.

0 2/ 2/ 2/ 2/ 2/ 2/

C. Training targets for the previous year achieved. 0 0/ 2/ 2/ 2/ 2/ 2/TechnicalAssistance

D. Terms of reference (TOR) for technical assistance for thenext year specified.

0 2/ 2/ 2/ 2/ 2/ 2/

E. Objectives/products of TA, identified in TOR from previousyear specified.

0 0/ 2/ 2/ 2/ 2/ 2/

Chief Products F. Annual report submitted to UNSP and Bank summarizingchanges in educational and financial planning as a result ofreforms.

0 0/ 2/ 2/ 2/ 2/ 2/

G. Annual review of previous year's system conditions andperformance completed and delivered to secretary of education.

0 0/ 0/ 2/ 2/ 2/ 2/

H. Annual forecast of next years enrollment in state andmunicipal system completed and submitted to secretary ofeducation.

0 0/ 0/ 2/ 2/ 2/ 2/

I. Annual production for "same service" budget (the projectedcost of providing the same level of services next year as is beingprovided the current year, accounting for enrollment, salarychanges for staff, price changes, etc.).

0 0/ 0/ 2/ 2/ 2/ 2/

J. Annual production of plan to upgrade sub-quality schools tofundamental quality schools (specifying which schools will beupgraded, how they will be upgraded, and the cost of theseactions), and summarizing accomplishments to date on theseactions.

0 0/ 0/ 2/ 2/ 2/ 2/

K. Annual production for report of school investment planning,conducted jointly with municipal secretaries.

0 0/ 0/ 2/ 2/ 2/ 2/

L. Annual production of maintenance plan, for schools,delegates, and headquarters, indicating accomplishments andplans for next year.

0 0/ 2/ 2/ 2/ 2/ 2/

Total Score (expected target score/actual score) 0 4/ 12/ 22/ 24/ 24/ 24/


16 of 24



Expected Targets/Actual Score (measured inMay of subsequent year) (yes=2, almost=1, no=0)

Category Indicator Baseline 1992 1993 1994 1995 1996 1997

Studies A. Operational strategy to link personnel database with payrolldatabase.

0 0/ 2/

B. Survey of personnel practice and compensation by task (jobtitle).

0 0/ 0/ 2/

C. Recommendations to improve practices in compensation andincentives.

0 0/ 0/ 2/

Training D. Specific training targets (including names) for next yearidentified.

0 2/ 2/ 2/ 2/ 2/ 2/

E. Training targets for previous year achieved. 0 0/ 2/ 2/ 2/ 2/ 2/Technical Assistance F. Terms of reference (TOR) for technical assistance for next

year specified.0 2/ 2/ 2/ 2/ 2/ 2/

G. Objectives/products of TA, identified in TOR fromprevious year achieved.

0 0/ 2/ 2/ 2/ 2/ 2/

Chief Products H. Annual report submitted to UNSP and Bank summarizingchangs in human resources management as a result of reforms

0 0/ 2/ 2/ 2/ 2/ 2/

I. Data forms and computer database defined to include, interalia: -employee ID number -personal dtga (including years until retirement) -job title (primary school teacher, etc) & code -contract ID number -contracted hours per day and per week

0 0/ 0/ 2/

-total contracts held by this employee in system -remuneration and other types of compensation -total monthly paid salary history, all categories -total monthly retirement history -current/previous assignments (location, position)

-tenure in current assignment -qualification (academic attainment, licence, etc) -inservice training history -immediate supervisorJ. Complete up-to-date (as of December 31 of previous year)records on all SE employees in database

0 0/ 0/ 0/ 2/ 2/ 2/

K. Complted report summarizing database indicating, interalia, number of employees and their compensation, by job titleand code, by delegacin, by various part-time and full-timecategories.

0 0/ 0/ 0/ 0/ 2/

L. Submission to secretary of education of report analyzingdatabase and recommending strategies to reallocate teachinghour contract assignments.

0 0/ 0/ 0/ 0/ 0/ 2/



17 of 24



Expected Targets/Actual Score(measured in May of subsequent year)

(yes=2, almost=1, no=0)


TechnicalAssistance

A. Terms of Reference (TOR) for technical assistance fornext year specified.

0 2/ 2/ 2/ 2/ 2/ 2/

B. Objectives/products of TA, identified in TOR fromprevious year achieved.

0 0/ 2/ 2/ 2/ 2/ 2/

ChiefProducts

C. Develop complete school profile informationquestionnaire, based as much as possible on existingschool reports, CPS surveys, etc.

0 0/ 2/

D. Complete plan for basic monitoring system toroutinely conduct state/municipal school survey, obtaininformation for questionnaire, complete survey, andupdate school-level information database.

0 0/ 2/

E. Establish computerized database to collect andtabulate school-level information from questionnaire.

0 0/ 0/ 2/

F. Complete annual report summarizing school databaseand comparing results, by municipality, to FundamentalQuality Standards (FQL).

0 0/ 0/ 0/ 2/ 2/ 2/

G. Develop, publish, and disseminate report card foreach school comparing each school to FQL standards, andto average state schools and municipal schools in samemunicipality, to best performing municipalities, and toworst performing municipalities.

0 0/ 0/ 0/ 2/ 2/ 2/

H. Update school questionnaire to match changes in FQLstandards.

0 0/ 0/ 2/ 2/ 2/ 2/



18 of 24






Training A. Specific training targets (including names) for nextyear identified.

0 2/ 2/ 2/ 2/ 2/ 2/

B. Training targets for previous year achieved. 0 0/ 2/ 2/ 2/ 2/ 2/TechnicalAssistance

C. Terms of reference (TOR) for technical assistancefor next year specified.

0 2/ 2/ 2/ 2/ 2/ 2/

D. Objectives/products of TA, identified in TORfrom previous year achieved.

0 0/ 2/ 2/ 2/ 2/ 2/

ChiefProduction

E. Request for ICB Proposals on computer hardwaretendered.

0 0/ 2/

F. Request for ICB Proposals on computer softwaretendered.

0 0/ 2/

G. Request for LCB Proposals on installation andannual maintenance and troubleshooting tendered.

0 0/ 2/

H. All three contracts awarded. 0 0/ 0/ 2/I. All products and services delivered. 0 0/ 0/ 2/J. Integrated Education Management InformationSystem installed and operational for all key secretariatdepartments or units, including payroll, finance,planning, school monitoring (the School LevelInformation Database, or SLID), educational

0 0/ 0/ 2/

materials, and civil material works.K. All computer equipment operational at 90%. 0 0/ 0/ 2/ 2/ 2/



19 of 24





Category Indicator Baseline 1992 1993 1994 1995 1996 1997Chief Products A. Operational Plans: Annual operational plan (OP) for Education

Management Component completed, including detailed costs forsubsequent year, and detailed Outcome Indicators for monitoring theimplementation of each activity identified in the OP.

Draft 2/ 2/ 2/ 2/ 2/ 2/

B. Operational Plans: Annual operational Plan for TeacherTraining Component completed, including detailed costs forsubsequent year, and detailed outcome performance indicators formonitoring the implementation of each activity identified in the OP.

Draft 2/ 2/ 2/ 2/ 2/ 2/

C. OPs: Annual (OP) for the didactic materials subcomponentunder the Education Materials Component completed, includingdetailed costs for subsequent year, and detailed outcomeperformance indicators for monitoring the

Draft 2/ 2/ 2/ 2/ 2/ 2/

implementation of each identified in the OP.D. Key Accomplishment Indicators: Completed ProjectImplementation Checklists (PIC-1, PIC-2, PIC-3, PIC-4, PIC-6,PIC-7), updated by May following the year being monitored,submitted to UNSP by May 30 of current year.

0 0/ 2/ 2/ 2/ 2/ 2/

E. Key Accomplishment Indicators: Completed School LevelInformation Database (SLID) Report and Project Procurement andDelivery Orders Database (PPDOD) Report, both updated by Mayfollowing the year being monitored, submitted to UNSP by

0 0/ 2/ 2/ 2/ 2/ 2/

May 30 of current year.F. Database Development and Operation: The Project Procurementand Delivery Orders Database (PPDOD) up-to-date and operational.

0 0/ 2/ 2/ 2/ 2/ 2/

G. Outcome Indicators: Completed report on the OutcomeIndicators, indicating implementation progress on each activityunder each subcomponent, as specified in the OPs, updatedquarterly, summarized for entire year, and submitted to UNSP

0 0/ 2/ 2/ 2/ 2/ 2/

by May 30 of the current year.Total Score (expected target score/actual score) 3 Drafts 6/ 14/ 14/ 14/ 14/ 14/


20 of 24

Table A10-c. NEBE--II STATE PROGRAM - KEY ACCOMPLISHMENT INDICATORSProject Implementation Checklist: PIC-6

Annual School-Level Information Database Report(Agent responsible for monitoring UNSP)

Source of Data: School-level Information Database (SLID)

Summary of database for year:___ Date of Summary:___ Name of Computer file Supporting this Report: ______States: Ceara __ Maranhaor__ Pernambuco__ Sergipe__Summer report for an entire sector? Yes__ No__If yes, which sector:___ Municipal sector summary:___ State sector summary:___If no, name of municipality:__________________Was a complete printout of the school-level database for this municipality submitted to UESP? Yes__ No__

Resources (materials or staff) Number of Materials/Staffavailable at start

of year

Number of newMaterials/Staffarrived/trained

during school year

Number of Materials/Staffavailable at time of

school survey

Date ofschoolsurvey

Textbooks: Portuguese Grade 1 Portuguese Grade 2 Portuguese Grade 3 Portuguese Grade 4 Mathematics Grade 1 Mathematics Grade 2 Mathematics Grade 3 Mathematics Grade 4 Natural Science (all types) Grade 1 Natural Science Grade 2 Natural Science Grade 3 Natural Science Grade 4 Social Studies (all types) Grade 1 Social Studies Grade 2 Social Studies Grade 3 Social Studies Grade 4Classroom reading books (not textbooks)Didactic Materials (Kits)1.2.3.4.5.6.7.8.Number of directors received trainng this year:Number of assistant directors received training this year:Number of classroom teachers received training this year:

1st grade teachers:2nd grade teachers:3rd grade teachers:4th grade teachers:

Number of usable classroomsReconstruction, renovation, construction of school facilities: initiated Yes:__ No:__ completed Yes:__ No:__


21 of 24

Table A10-c.1. PROFIL DA ESCOLA

Informacoes provenientes da propria escola

Ao nivel de estudantes Pre-escolar Serie 1 Serie 2 Serie 3 Serie 4 Serie 5

Traditional Alfabetizacao

Numero de ahanos-meninos

Numero de ahanos-meninos

- idade: numero de alunos que tem mais que: 7 anos 8 anos 9 anos 10 anos 11 anos 12 anos

- numero de repetentes

Horne de classe por dia por este serie

Numero de livros textos: Portugues

O ano que estee livros textos do Portugues chegaram

Numero de livres textos: Matematica 1/

O amo que estes livros textos de Matematica chegaram

Numero de livros textos de Ciencios 2/

O ano que estes livres textos de Ciencios chegaram

Numero de livros textos de Estudos Socias 3/

O ano que estos livros textos de Estudes Socias chegaram

1. Conta todos es livros de matematica juntos, incluindo, por examplo, geometria, algebra, etc

2. Conta todos es livros de ciencios juntos, incluindo, por examplo, fisica, biologia, etc.

3. Conta todos es livros de estudos socias juntos, incluindo, por examplo, historia, geografia, etc.


22 of 24

Table A10-c.2. PROFIL DA ESCOLA (pagina 2)

A Informacoes provenientes da propria escola2. Ao nivel da escolaPerguntas sobre Salas de Aula

Quantos salas de aula na escola?Quantos sales da aula tem quadros-negros em bem condicione?Quantos salas de aula nao podem ser usados durante uma chuva?Quantos precisam de reforma grandes consertos?Quantos receberem as reformas grandes este ano?O que foi a data que estes reformas acabaram (dia, mes, ano) / /199_

Perguntas sobre livros de leituraQuantos livros de leitura existem na escola para os alunos das 1" a 2" series?Quantos desses livros chegaram este ano?O que e a data que estes livros chegaram (dia, mes, ano)c / /199_Quantos livros de leitura existem na escola para os alunos das 3" a 4" series?Quantos desses livros chegaram este ano?O que foi a data que estes livros chegaram (dia, mes, ano) / /199_Quantos livros de leitura existem na escola para os alunos das 5" a 8" series?Quantos desses livros chegaram este ano?Donde se guardem a maioria desses livros? - na sala da diretora (sim, nao)?

- nas salas de aula (sim, nao)? - na biblioteca (sim, nao)? - na dispensa (sim, nao)? - outro lugar: especifique:

Perguntas sobre materias didaticosA escola recebem materias didaticos (nao incluindo livros) este ano?O que foi a data que estes livros chegaram (dia, mes, ano)?Donde se guardem a maioria dessas materias?

- na sala da directora (sim, nao)? - nas salas de aula (sim, nao)? - na biblioteca (sim, nao)? - outro lugar: especifique:

Perguntas sobre a escolaA escola precisa de reforma grandes consertos?A escola recebeu as reformas grandes este ano?


23 of 24

Table A10-c.3. INFORMACEES VINDO DA ESCOLA PROPRIA

Servidores Nome Nameadoa qualseries

Anosde

idade

Numero deanos nestaprofissao

Experiencanestaescola

Mais alte nivelde educaco

obtide

O ultimo anorecebin

trenamento

Director: 1.2.

Director-Asst: 3.4.

Professores: 5.6.7.8.9.10.11.12.13.14.15.16.17.18.19.20.

Otros servidores Titule (ocupacao) Numero Iloras per dia OtrosServidores

Titule(ocupacao)

Numero

1. 6.2. 7.3. 8.4. 9.5. 10.


24 of 24

Table A10-4. NEBE--II STATE PROGRAM - KEY ACCOMPLISHMENT INDICATORSProject Implementation Checklist: PIC-7

Annual Project Goods and Services Delivery Report(Agent responsible for monitoring UESP)

Source of Data (for column 1, below): Project Procurement and Delivery Orders Database (PPDOD) at UESP

Summary of database for year:___ Date of Summary:___ Name of Computer file Supporting this Report: ______States: Ceara __ Maranhaor__ Pernambuco__ Sergipe__Summary report for an entire sector? Yes__ No__If yes, which sector:___ Municipal sector summary:___ State sector summary:___If no, name of municipality:__________________Was a complete printout of the school-level database for this municipality submitted to UESP? Yes__ No__Resources (materials or staff) Column 1

Number ofMaterials/Staff to be

Delivered/Trained(sources: OperationalPlan from previous

year)

Column 2Number of

Materials/ StaffDelivered/Trained

during Year(sources: PPDOD)

Column 3Number of

Materials/Staff thatSchool ReportedHaving Received(sources: SLID)

Difference betweenColumn 2 andColumn 3 (for

discrepency report)

Textbooks: Portuguese Grade 1 Portuguese Grade 2 Portuguese Grade 3 Portuguese Grade 4 Mathematics Grade 1 Mathematics Grade 2 Mathematics Grade 3 Mathematics Grade 4 Natural Science (all types) Grade 1 Natural Science Grade 2 Natural Science Grade 3 Natural Science Grade 4 Social Studies (all types) Grade 1 Social Studies Grade 2 Social Studies Grade 3 Social Studies Grade 4Classroom reading books (not textbooks)Didactic Materials (Kits)1.2.3.4.5.6.7.8.Number of staff employed by schoolNumber of staff directly engaged in classroom teachingNumber of directors received training this year: Number of assistant directors received training this year:Number of classroom teachers received training this year:

1st grade teachers:2nd grade teachers:3rd grade teachers:4th grade teachers:

Number of usable classroomsReconstruction, renovation, construction of school facilities: initiated Yes:__ No:__

completed Yes:__ No:__

ANNEX 7: DETAILED DESCRIPTION OF NATIONAL...

Documents

Transcript of ANNEX 7: DETAILED DESCRIPTION OF NATIONAL...