Towards better literacy achievements - UNESCO · To Mariano Palamidessi, Paula Scalliter and...

Towards better literacy achievements in the 2nd

grade of primary schools

http://www.iipe-buenosaires.org.ar

http://www.iipe-buenosaires.org.ar/




To the Director of the IIEP-UNESCO, Mr. Khalil Mahshi, and to the Head of IIPE-Buenos Aires, Ms. Margarita Poggi, who enabled a mechanism to support my participation in the

specialization phase of Advanced Training Programme (ATP).

To my supervisors, Muriel Poisson and Mariano Palamidessi, for the time spent in the attentive revisions and valuable guidance, and also for their personal support, which made me move

forward and complete this memoir.

To the Training Unit of the IIEP Paris and the team of professionals highly committed with education, especially Gabriele Göttelmann-Duret, Yasmin Haq, Patricia Dias da Graça,

Véronique Quénehen, Ludivine Gargam, Natalie Frederic and Jimena Pereyra.

To Mariano Palamidessi, Paula Scalliter and Cecilia Mata, from the IIEP-Buenos Aires, who offered all their support and guidance during the Regional Course in 2009, and more recently

helped me with the preparation of my participation in this second phase in Paris.

To Mauricio Holanda Maia, Marcia Campos, Lucidalva Pereira and Carmilva Flores, from the Ceará State Secretariat for Basic Education, for their availability and support, especially for the

interviews phase.

To my colleagues from the ATP 2011/2012, and those who were with me in the Regional Course in 2009, who taught me new ways of seeing and reading the world, and helped me to become more aware of the wonderful differences and similarities that characterize people around the world and the importance of education for all in our countries. To you all I offer my respect and affection. To the Board of the House of Representatives, who authorized my absence from my labour activities, for its commitment with the enhancement of the skills and competences of its employees as a form of strengthening the legislative branch in Brazil.

To my colleagues from the Legislative Consultancy of the Brazilian House of Representatives who were generous to encourage my participation in the ATP while they were overburdened in my absence. Finally, I would have never been able to have this professional and personal enriching experience without the support and love of my dear family, especially my husband Daniel, a companion in all dreams, and my son Gustavo and daughter Beatriz, who once again showed

how wonderful children they are.




AL-CE Legislative Assembly of Ceará

ANEB National Basic Education Assessment

ANRESC National Assessment of Educational Achievement

APRECE Association of Ceará‟s Municipalities and Mayors

CCEAE Ceará State Committee for Eradication of Illiteracy at School

CTT Classical Test Theory

CEC/CD Education Committee of House of Representatives

CNE National Education Council

COPEM Coordinating Body for Cooperation with the Municipalities

CREDES Regional Coordinating Bodies for Education Development

EGRA Early Grade Reading Assessment

FUNDEF Fund for Maintenance and Development of Primary Education

FUNDEB Fund for Maintenance and Development of Basic Education

HDI-M Human Development Index-Municipal

ICMS Value-Added Tax

IDE-Alfa Literacy Performance Index

INEP National Institute for Educational Studies and Research

IRT Item Response Theory

MEC Ministry of Education

NPE National Plan of Education

NRCOM Regional Centre for Cooperation with the Municipalities

PAIC Literacy Program at the Right Age

PCN National Curricular Parameters

PNLD National Program of Textbooks

SAEB National Evaluation System for Basic Education

SARESP São Paulo Education System of Learning Assessment

SEDUC Ceará State Secretariat for Basic Education

SME Municipal Education Secretariat

SPAECE Ceará‟s Permanent Evaluation System for Basic Education

SPAECE-Alfa Literacy Assessment System of Ceará

UFC Federal University of Ceará

UNDIME/CE National Union of Municipal Education Officer - Ceará

UNDP United Nations Development Programme in Brazil

UNESCO United Nations Educational Scientific and Cultural Organization

UNICEF The United Nations Children‟s Fund

WCEFA World Conference on Education for All




This paper describes and analyses the Programa Alfabetização na Idade Certa, (PAIC)

Literacy Program at the Right Age, which has been conducted by the Ceará State Secretariat

for Basic Education since 2007, aimed at better literacy achievements in the 2nd grade of

primary schools. The paper is focused on the program‟s design and implementation and its

highlights: a) the facts that influenced the decision-making process to formulate PAIC and

framed its design; b) the main features of the collaborative action structured by the State of

Ceará and its municipalities to implement the program; c) the role of the two kinds of external

learning assessments implemented under PAIC.

The program is based on two main messages - focus on learning and accountability for results.

It has managed to turn these messages into well-defined targets and goals and effectively

communicated them to all stakeholders. The state of Ceará and its municipalities forged a

model of intergovernmental cooperation that distinguishes itself from past experiences by

establishing a new institutional arrangement; creating a systemic approach and a

comprehensive set of actions to all municipalities, regardless of political bias; and improving

technical capacities in SME management and monitoring, among other aspects.

The experience of PAIC underlines that external and large scale learning assessments may

serve a variety of purposes if they are properly designed and have findings communicated to

the relevant audience. In Ceará, the findings have been disseminated in a way that system

and school managers are able to grasp and to use them to improve management practices

and pedagogical routines. The dissemination of a “culture of monitoring and evaluation” in the

education sector has been reinforced by financial incentives to the local government (by tax

sharing) and to schools (by awards and financial assistance).

The paper concludes with some lessons learned from this experience that might be useful for

other Brazilian states and municipalities which face the same challenge of fighting illiteracy at

primary schools.




1.1 Socio-economic and educational setting in Ceará1

Ceará, one of the 27 Brazilian states, is 148,825.6 km² and corresponds to almost 10% of the

Northeast Region. The population was estimated at 8.4 million in 2010 and is spread over 184

municipalities. Apart from the capital Fortaleza, other major cities are Juazeiro, Crato and

Sobral2. Most of Ceará‟s territory (93%) is in the semi-arid region, with a predominantly

tropical climate. Given the aridity, droughts are periodic and sometimes the population,

especially in the area of caatinga, faces problems of water scarcity.

Like most of the country, the state went through an accelerated process of urbanization and

paid a high price for that, facing many problems to provide adequate public services. In 1950,

the urbanization rate in Ceará was 25%, reaching 65.4% in 1991 and 77% in 2010, still under

the 84% of Brazil.

From the 1960‟s to the 1980‟s, prolonged droughts and low growth led large numbers to

migrate to the Southeast Region which concentrated the industrialization of the economy. This

migration flow is a highlight in the Brazilian demography dynamics. Besides the phenomenon

of droughts, agriculture with low productivity, huge income disparities and large land

concentration also contributed to intense migration. However from the 1990s on economic

crises and transformation of labor markets reversed this trend and many north easterners

made their way back.

Economy

Since 2000s, a process of economic growth has been taking place in much of the Northeast.

The first cycle of economic development was heavily based on tax incentives to attract large

companies. In the last decade, Ceará chose different strategies by investing in activities that

have helped to create a new perspective of economic development, such as: i) better

organization of the tourism sector, which gives rise to more employment opportunities and

income generation, not only in the coast, but also in the back lands of the state; ii)

implementation of new economic arrangements that seek to maximize local productive

vocations; iii) activities related to the exportation sector both with the production of goods and

provision of docking facilities, such as the port of Pecém.

Currently, the economy of Ceará has developed a good profile with respect to the distribution

of productive activities, with predominance of the service sector (69.3%) compared to the

1 All data in this section was retrieved on February,10 2012 from Instituto Brasileiro de Geografia e

Estatística (IBGE) and Instituto de Pesquisa e Estratégia Econômica do Ceará (IPECE), available on www.ibge.gov.br and www.ipece.ce.gov.br/categoria5/ceara-em-numeros, respectively. 2 There are at least two interesting historical facts about Ceará: it was the first state to abolish slavery in

Brazil, four years before the national act determining the end of slavery in 1888; and it also pioneered in the first democratic election in 1985, after twenty years of military dictatorship, electing a woman from the left-wing worker‟s party to be mayor of the capital.

http://www.ibge.gov.br/

http://www.ipece.ce.gov.br/categoria5/ceara-em-numeros




agricultural sector (7.1%) and the industrial sector (23.6%). In 2008, the State held the 12th

GDP of the country, but it is noteworthy that its relative share in the national GDP was only

2%, with the Brazilian economic activity still concentrated in the south and southeast (relative

shares of São Paulo 33.9%, Rio de Janeiro 11.2% and Minas Gerais 9.1%).

In line with the national growth wave, the state goes through a period of economic prosperity.

The Gross Domestic Product, at current values, increased 32.9% from 2002-2007. The budget

revenues in the same period grew by 88.4% and the expenditure by 84.2%. GDP per capita

grew on average 3.6% per year in the period 2000-2008, above the national average of 3.2%.

Nevertheless, it is still a poor state considering the GDP per capita in Brazil. In 2008, GDP per

capita of Ceará was R$3,681 less than half of R$8,277, the national average.

Social Indicators

The improvement in some social indicators is quite significant. The Human Development

Index-Municipal (HDI-M), produced by the United Nations Development Programme in Brazil

(UNDP), shows that the country has evolved from 0.696 to 0.766 between 1991 and 2000.

Ceará has evolved in a higher range from 0.593 to 0.700 in the same period.

The infant mortality rate3 in the state was 54.2% in 2000 and reached 27.6% in 2009. In turn,

the illiteracy rate among those aged 15 or older went down from 31% in 1996 to 18.6% in

2009. Despite these improvements, regional disparities still persist. The infant mortality rate in

Ceará is more than twice of that identified in Rio Grande do Sul (12.7%) in 2009, and the

illiteracy rate is over five times of that recorded in the Federal District in 2009 (3.5%).

Educational Setting4

a) Access and efficiency indicators

The state has made significant progress in its access indicators, especially in the last decade.

In 2000, the gross enrolment rate in pre-school and secondary was 48.7% and 55.9%,

respectively. In 2009, both were around 80%. In primary, gross enrolment rate went down from

123.2% to 107.6% in the same period. Ceará presents enrolment rates in pre-school above the

regional and national rates; however, it faces problems related to quality5. In primary school,

gross and net enrolment rates are quite aligned with the national rates. In secondary, net

enrolment rate is substantially lower than the gross rate. (Table 1)

3 Mortality rate at age one per thousand live births.

4 See appendix I for information on Brazilian Education System – Structure and Organization.

5 For additional information see: Maria Malta Campos and Jodete Füllgraf. “A qualidade da educação

infantil brasileira: alguns resultados de pesquisa”. In Cadernos de Pesquisa, v.36, n.127, p.87-128, jan-abr. 2006. Available on: http://www.fcc.org.br/pesquisa/eixostematicos/educacaoinfantil/pdf/aqualidadedaeducacaoinfantilbrasileira.pdf Fundação Carlos Chagas. “Educação Infantil no Brasil: avaliação qualitativa e quantitativa”. 2010. Available on: http://www.fcc.org.br/pesquisa/eixostematicos/educacaoinfantil/pdf/sumario_executivo.pdf




Table 1 - Enrolment rates in basic education-2009

Level

Gross

Rate

Pre-

school

Net Rate

Pre-school

Gross

Rate

Primary

Net Rate

Primary

Gross Rate

Secondary

Net Rate

Secondary

Ceará 80.0% 68.9% 107.6% 93.5% 79.2% 49.6%

Northeast 67.3% 59.5% 110,.% 89.4% 77.4% 39.2%

Brazil 62.2% 50.6% 105.7% 91.1% 83.0% 50.9%

Source: PNAD/IBGE; Todos pela Educação.

The state deals with a history of high repetition rates and over aged students. In Ceará, over

one third of the students enrolled in secondary are older (at least two years more) than the

recommended age, but in the Northeast Region the rate is even higher. In the period 1995-

2005, after the strong expansion of the education system, some efficiency indicators got

worse. (See appendix II).

Nevertheless, the state has managed to progress in recent years. For example: in the second

cycle of primary education6, over aged students rate went down from 73.7 to 31% between

2001 and 2010. In 2005, in the first cycle of primary, 79% of students had been approved, 15%

had failed and 6% had not completed the school year. In 2009, the rates improved

significantly: 87%, 10% and 2%, respectively.

b) Learning outcomes

The states in the Northeast Region, and Ceará is no exception, have performed learning

outcomes below the national average since the Sistema de Avaliação da Educação Básica,

SAEB (National Evaluation System for Basic Education) was implemented in 1995.

In 2003, the Instituto Nacional de Estudos e Pesquisas Educacionais, INEP (National Institute

for Educational Studies and Research) released a study on the performance of students in 4 th

grade, stating that 22.2% of them had performed very poorly in Portuguese. Those students

were considered illiterate or poorly literate, unable to respond to the simpler items in the test.

The results were bad throughout the country, but significantly worse in less developed regions,

North and Northeast. In the Northeast, the total number of students placed in the critical and

very critical stages of performance was around 70% (Table 2). (INEP, 2003)

6 Currently, the enrolment of all children aged 6-14 in primary education is mandatory. A constitutional

amendment passed in 2009 extends this requirement for all the population aged 4-17 years from 2016 on. In Brazil, primary education is divided in two cycles: 1st/5th grades and 6th/9th grades.




Table 2 - Percentage of 4th graders according to the competences developed in Portuguese – 2001

Levels Brazil

Regions

Northeast North Southeast South Midwest

Very Critical 22.2 22.6 33.4 15.8 13.5 20.5

Critical 36.8 44.9 41.8 30.8 35.7 39.2

Intermediate 36.2 31 22.9 45.2 45.8 36.6

Adequate 4.4 1.4 1.8 7.4 4.8 3.3

Advanced 0.4 0.1 0.1 0.8 0.3 0.4

TOTAL 100.0 100.0 100.0 100.0 100.1 100.0

Source: (INEP, 2003)

According to the scales of competences defined by SAEB, the minimum scores to be achieved

by students in grades assessed by the system are:

5th grade of Primary – 200 points in Portuguese and 225 in Math;

9th grade of Primary – 275 points in Portuguese and 300 in Math;

3rd grade of Secondary – 300 points in Portuguese and 350 in Math.

Based on these learning expectations, it can be said that the whole system – state, regional

and national levels perform far below what is considered minimum achievements, but still

some progress have been made in recent years. Table 3 shows that Ceará has got better

results in terms of average proficiency in Portuguese in the first cycle of primary education. It

raised the measure in 23 points between 2005 and 2009, almost the double compared with

national and regional levels. There is a relevant and positive change in the second cycle too.

The progress is lower in secondary education.

Table 3 – Average scoring in Portuguese according to SAEB – 2005-20097

Levels

Primary 4th/5th grade Primary 8th/9th grade Secondary 3rd grade

2005 2009 range 2005 2009 range 2005 2009 range

Ceará 157.0 180.1 23.1 217.9 238.0 20.1 257.8 263.1 5.3

Northeast 155.2 167.3 12.1 219.5 232.4 12.9 246.2 258.4 12.2

Brazil 172.3 184.3 12.0 231.8 244.0 12.2 257.6 268.8 11.2

Source: Todos pela Educação; INEP/MEC

7 Brazilian education system undergoes a transition to compulsory basic education from eight to nine

years. This explains why the data is presented to 4th/5th grades and 8th/9th grades.




Ceará has been especially successful in improving literacy achievements in the 2nd grade of

primary education. Since 2007 the State has implemented the Programa Alfabetização na

Idade Certa, PAIC (Literacy Program at the Right Age), with pedagogical and managerial

actions focused on better literacy achievements in the two first years of schooling.

According to the Sistema Permanente de Avaliação da Educação Básica do Ceará –

Alfabetização, SPAECE-Alfa (Literacy Assessment System of Ceará), in 2007 only 26% of the

municipalities had average scores corresponding to the first two levels (sufficient and

recommended) of the scale used to assess proficiency in reading for second graders. In 2010

this percentage came to 99%.

Table 4 shows the evolution of second graders performance in terms of average proficiency in

reading after the program‟s adoption which reached its target in 20108. In this period, the

percentage of students considered illiterate dropped from 33 to 7% and the percentage of

students grouped in the recommended stage increased from 30 to 55%.

Table 4 - PAIC results: average proficiency and percentage of students in each level of the scale of

reading proficiency 2007-2010

Year Average

Proficiency Illiterate

Incomplete

Literacy Intermediate Sufficient Recommended

2007 118.9 32.8 14.6 12.7 9.9 29.9

2008 127.7 20.0 16.2 17.3 15.7 30.8

2009 142.5 14.4 14.0 15.7 14.5 41.5

2010 162.7 6.9 9.4 13.0 15.8 54.9

Source: SEDUC/SPAECE-Alfa

In short, Ceará used to be depicted as a poor state, with low learning outcomes and disparities

in the last decades, as well as the Brazilian Northeast. More recently, it has been going

through a period of economic development and of improvement of some social indicators. It is

expected that some structural policies such as education may pave the way for future

sustainability of this process. As seen above, the latest results from SAEB and SPAECE-Alfa

have shown that the state has been successful in its efforts to raise learning outcomes in

language for students in primary schools.

1.2 The topic and its rationale

This study will focus on the design and implementation of the Literacy Program at the Right

Age (PAIC) that has been conducted by the Secretaria de Educação Básica do Ceará, SEDUC

(Ceará State Secretariat for Basic Education). The study will cover the period from 2007 to

8 Ceará state education system has established the recommended proficiency over 150 points.




2010 when the program was successful in raising learning achievements in reading for 2nd

graders.

The changes that occurred in Ceará are highly interesting for the whole country. Brazil has

managed to enroll 97% of school-age children (from 6 to 14 years old) in primary education,

but it is not surprising that after eight or nine years of schooling a young boy or girl still

performs reading and writing at insufficient levels. This is one of the most serious quality

problems because the learning gaps begin in the early years of schooling, deepen with time

and tend to become unmanageable towards the end of basic education. It is a great loss to

children, particularly the poor ones.

This poor learning performance is especially serious for the poorest states. By failing to ensure

effective early learning, school also fails in its role of contributing to reduce inequalities. This

program is particularly interesting because Ceará is located in the Northeast of Brazil. The

region is historically marked by higher levels of poverty, low economic development and

educational indicators are frequently below national average, as seen in the previous section.

Brazil is a federation with three government levels that has faced difficulties in implementing

collaborative actions to induce changes in quality and equity education issues. Furthermore,

there are difficulties at all government levels to use the data generated by the system of

learning assessment to guide system management, policy options and decision making.

In Ceará, the governments of state and municipality levels are working in a collaborative action

for implementing PAIC and have introduced a set of initiatives to address the problem of

illiteracy at school, among them specific learning assessment for literacy outcomes as a tool to

guide planning and monitoring overall program results. This study is based on the premise that

intergovernmental collaborative action and external assessment are the main aspects of PAIC

and that the context where the decision-making process to implement the program took place

forged this collaborative action and framed the program design.

A number of national academic studies have tackled primary education quality issues. Most of

them are focused on the evolution of learning outcomes at the end of the primary cycles (5 th

and 9th grades) based on national data. Little effort has been dedicated to this specific issue

(better literacy outcomes in early child literacy and programs designed to achieve them), and

even less attention has been driven to how these programs have been designed and

implemented.

Currently, there is a bill on setting up a new National Plan of Education (NPE) at the Brazilian

National Congress that includes a goal on children‟s literacy. Given the momentum, it is an

opportunity for new learning on the subject. The new plan is pointing to the need for

government action towards a more effective literacy process in the first years of primary

education.

Therefore, the aim is to contribute to the design and implementation of other institutional

interventions in states and municipalities where children have also shown low levels of reading

and writing.

A better and deeper understanding of this program can also be useful to the National

Congress in fulfilling its responsibilities for following up and evaluating the NPE‟s




implementation and for supporting the accomplishment of its goals by approving laws and

regulations that can induce it.

1.3 Problematic

Considering that child‟s literacy at appropriate age will gain even more prominence when the

new NPE is approved, and in order to assist the states and municipalities with some

challenges they may face in fulfilling the goal, this study hopes to shed light on how the State

of Ceará has managed to turn literacy achievements into a priority for the municipal education

systems during the period 2007-2010. What are the main features of the collaborative action

and the external learning assessment put in place by the state? In the light of this main guiding

question, there are selected critical issues that will be described and analyzed in this study.

They are presented in the following questions:

1. What are the main policies and practices related to child‟s literacy in Brazil? What is the

legal framework for intergovernmental collaborative action and what kind of cooperation is

developed in this field? How child‟s literacy has become the focus of learning assessments?

2. What was the context for the emergence of PAIC in Ceará and what are the relevant actors

in this process? Why and how was a collaborative action between State and its

Municipalities structured to address the problem?

3. What is the role of external evaluation in the design of PAIC?

4. What lessons can be drawn from the experience implemented in Ceará?

1.4 Objectives of the study

In order to respond to the questions posed above, the study has identified the following

objectives:

- To identify key concepts, relevant issues and policies related to literacy in Brazil. To

understand the legal framework for intergovernmental collaborative action in the

Brazilian federalism. To identify the main characteristics of learning assessments and

to analyze how they were linked to literacy achievement.

- To describe relevant facts and the role of main stakeholders involved in the events that

preceded the implementation of PAIC and to analyze how they have influenced the

design of the program. To describe PAIC main features and to identify the framework

for the collaborative action that was established.

- To describe and analyze the design and the uses that external assessment has taken

in PAIC‟s implementation.

- To identify lessons that can be drawn from the experience implemented in Ceará.

1.5 Methodology Aspects

The study utilizes descriptive and explanatory design and is based on primary and secondary

sources of information. In order to meet the objectives stipulated by the study, different types

of data were collected and analyzed. A brief literature review on child‟s literacy in Brazil and

the legal framework related to intergovernmental collaborative action will help to place the

program in the Brazilian context and hopefully make it easier to understand its conception and




relevance. The literature review also tries to deal with the characteristics and expansion of

learning assessments since this is an important dimension of PAIC.

Data on the context of PAIC‟s emergence, design and implementation was obtained from

institutional documents review, relevant reports and records available on the Internet. The

statistical data is gathered on PAIC‟s webpage. National statistics on learning outcomes used

as a means of comparison are obtained from the INEP webpage.

Besides that, data was also collected through six interviews, conducted on the 14 th, 15th and

16th May 20129. At state level, the interviews were with two key agents, both responsible for

managing the program in SEDUC: the Coordinator of Coordenadoria de Cooperação com os

Municípios, COPEM (Coordinating Body for Cooperation with the Municipalities) and the

leader of the Management of State Programs Cell. At municipal level, they were with two

Municipal Secretaries of Education and two PAIC City Managers. The interviews intended to

collect information on the structure established, constraints and challenges that program

managers have faced. They should also complete, refine or correct the findings derived from

the document analysis.

Two municipalities (São Luiz do Curu and Caucaia) were chosen based on the different scale

of its education system (one is considered small and the other large by Ceará patterns). The

other selection criterion was the system performance through SPAECE-Alfa results in 2007-

2010. One managed to take a great leap (more than 100 points in average proficiency) and the

other has not advanced so much (less than 10 points). The point, of course, is not to explain

the difference between the cities, but just to add a different perspective to the issues

addressed by this study.

This seemed to be the best approach considering the access constraints to data, time and

changes and relocations the author faced to write this paper.

1.6 Scope and limitations of the study

Attention will be drawn to literacy achievement in public schools of Ceará, so the private sector

will not be covered. Curriculum contents and pedagogical methods for literacy are discussed

very briefly, with the sole intention of contextualizing choices (or lack of them) in public

policies. Due to space constraints, adult illiteracy is not treated in this study.

Moreover, the discussion on learning assessment does not intend to focus on the diverse

influences on pupil achievement, but rather on how the results generated by the system can be

articulated with governmental actions aimed at a better quality of education and at ensuring

children‟s learning.

Finally, regarding the experiences of intergovernmental cooperation, only the major actions

taken by the Union were included in the study. They are the most frequent model due to more

availability of funding and trained personnel at federal level.

9 The author also attended a meeting held in at CREDE in Itapipoca City on 15

th May,2012, with PAIC

city managers and municipal secretaries from that zone, when SPAECE-Alfa results from 2011 as well as other operational topics were presented and discussed.




1.7 Structure of the paper

This initial chapter presents a general perspective of socio-economic and educational setting in

Ceará and introduces PAIC and its main achievements from 2007-2010. The objectives of the

study, the methodology applied to carry it out and its limitations are also presented.

Chapter two presents the debate on concepts of partial and full literacy that have influenced

practices and policies in Brazil. It also describes the role of intergovernmental collaborative

action established by the Constitution for the education sector as a means of balancing

autonomy and interdependence, and thus, of reducing disparities among federation entities.

Lastly, it tries to identify the main issues in learning assessments, its growth in the developing

countries and how they were linked to child‟s literacy.

Chapter three is dedicated to the facts and the role of main stakeholders involved in the events

that preceded PAIC implementation and to analyze how they influence the design of the

program. This section also describes PAIC‟s main features and analyses the framework for the

collaborative action that was established between the state government of Ceará and its

municipalities.

Chapter four explores the two kinds of external learning assessments design and uses

implemented by Ceará under PAIC. It analyses the rationale for their implementation and

describes their core aspects in order to understand how they became relevant instruments in

fighting illiteracy at Ceará‟s public schools. Finally, chapter five brings some lessons drawn

from PAIC‟s experience that can be useful for the three government levels in Brazil.




Introduction

This chapter presents some key concepts, relevant issues and national policies related to

child‟s literacy in Brazil, as well as the role of government collaborative action in this scenario.

The chapter also tries to identify the main issues of large-scale learning assessment, how they

have been disseminated in developing countries and recently linked to child‟s literacy.

2.1 Literacy policies and practices in Brazil

2.1.1 Literacy in the first cycle of primary school and academic failure

As presented in chapter one, Learning outcomes in the Brazilian education system haven‟t met

the adequate government standards. Low levels of student performance in Portuguese

assessed by SAEB are often associated to a lack of effective literacy in the early years of

primary school. (Oliveira, n.d.; MEC, 1997; Soares, s.d.)

In the analyzed literature, effective literacy in early years (1st ,2nd and 3rd grades) of primary

education is correlated with the development of the necessary skills for students to read and

write autonomously. (Oliveira, n.d.; Soares, n.d.; CD, 2003)

In Brazil, there is no national curriculum10, but the Conselho Nacional de Educação, CNE

(National Education Council) is responsible for releasing guidelines on the organization of the

systems and on the approach to the areas of common core. These guidelines are far from

being specific about competences and skills to be developed in each grade of primary

education. Thus, with regard to child‟s literacy, some states such as Ceará and Minas Gerais

have opted to develop a reference matrix of competences and skills to be pursued in the early

years of schooling in order to provide more specific guidance to their teachers.

It is difficult to address literacy policies and practices in Brazil without at least mentioning the

controversies over the competencies to be developed in the literacy process; as well as the

methodological debate. Although the scope of this study does not allow for further discussion

of these issues, they are relevant to be cited because apparently they were able to curb the

government action for fighting failure in early schooling.

When it comes to what kind of skills children must perform and when (at which grade) there is

no consensus. Partial and full literacy11 are frequently assumed as distinct concepts. While

partial literacy is more identified with decoding skills (related to alphabet knowledge, letter

names and sounds, concepts about print, phonemic awareness), full literacy is more

connected with early language skills (with emphasis on developing early readers‟ vocabulary

and text comprehension). Priority placed in one or another dimension gives rise to a lot of

10

Federal act nº 9.394 of 1996 sets up some areas to be included in the curriculum as a common core. There is also a diversified part to be defined by each education system. 11

Free translation adopted by the author to distinguish two different Portuguese words for literacy: alfabetização and letramento.




debate. The concern is that these latter skills are the ones to really indicate long-term

difference in individual literacy proficiency, while the first ones must be interpreted with caution

because they indicate a relative development in this field. On the other hand, some argue this

process is misunderstood by teachers leading them to fail in developing basic skills in child‟s

literacy.

What seems to be important in terms of public policies is that neither legislation nor national

guidelines have been able to clearly define education objectives for the first three years of

schooling. By doing so, it would help teachers to understand what skills should be pursued and

which criteria should be used to evaluate students, and determine to what degree they

succeeded at each stage.

The Parâmetros Curriculares Nacionais, PCN (National Curricular Parameters), released by

the Ministry of Education (MEC) in 1997, suggest that the practices widely disseminated

among Brazilian teachers to address literacy in two stages of development must be overcome.

Moreover, according to this document, the analysis of most recent theories on literacy would

recommend a review of traditional practices adopted in primary schools such as the use of

cartilhas12, which should be replaced with a wide variety of texts. They should be used as a

basic unit in the teaching and learning process. This strategy would replace the quite

widespread literacy practice of instruction using an incremental sequence of contents: syllables

(or letters), words, sentences and texts.

This approach is in line with the “whole language13” paradigm, a literacy theory which

emphasizes that the literacy process should comprise a holistic model. It emphasizes that the

acquisition of reading and writing should not be seen as a cumulative process. It is usually

associated with a constructivist approach and it states that the language should be presented

as a complete meaning unit to early readers rather than as small chunks. Therefore, whole

language is often contrasted with phonics-based methods of instruction which emphasize

decoding and spelling skills.

In Brazil, it has drawn criticism by those who plead "back to basics" pedagogy because whole

language is based on a limited body of scientific research. Oliveira (2004) says that the

paradigm became hegemonic in undergraduate teachers training courses and in public policies

and it has done a great disservice to education by diverting Brazilian teachers from literacy

methods which had been proved more effective.

This is obviously a very synthetic summary of these ideas and polemics14. The debate on

these issues is endless. The trend of school ineffectiveness identified by SAEB since the

1990‟s and the methodological/ideological debate led the Comissão de Educação da Câmara

dos Deputados, CEC/CD (The Education Committee of the House of Representatives) in 2003

to organize a work group on child‟s literacy in Brazil. The main concern was with the future

12

Cartilha is a kind of Spelling Book to teach children how to read by following a particular letter and syllable method, which was used by most schools in Brazil, but are not so popular nowadays. 13

See the ideas of Ken Goodman and Noah Chomsky. 14

See Ferreiro, E. 1992. Com todas as letras; Oliveira, J.B.A. 2004. Alfabetização de crianças e adultos: novos parâmetros; Kramer, S. (ed.). 1986. Alfabetização: dilemas da prática; Soares, M. 1985. As muitas facetas da alfabetização; CEC/CD. 2003. Relatório final do grupo de trabalho sobre alfabetização infantil: os novos caminhos.

http://en.wikipedia.org/wiki/Constructivism_(learning_theory)

http://en.wikipedia.org/wiki/Phonics

http://en.wikipedia.org/wiki/Back_to_basics_(education)

http://en.wikipedia.org/wiki/Pedagogy




performance and retention of these children within the education system and the need to break

the cycle of illiteracy in the country15.

This work group concluded that there is indeed a wide variety of interpretations among the

education systems in states and municipalities16 about what it means to be literate and at what

point (grade) of schooling it should be achieved. This fact stems from the lack of more precise

definitions released by the national level. Moreover, the uncertainties about performance

expectations make proper evaluation a difficult task for teachers. The work group concluded

that the scenario of vague definitions is also affected by poor training of teachers. According to

the study, specialists who select the material to be used in training courses for teachers

working in initial grades tend to prefer books related to theories and ignore the publications

geared to literacy methods and techniques. (CEC/CD, 2003)

The debate on the literacy concept and methods is far from being exclusive to Brazilian

education. According to UNESCO (2006), academics from a wide-range of disciplines such as

psychology, economics, linguistics, sociology, anthropology, philosophy and history have

engaged in this ongoing debate. UNESCO states that:

The most common understanding of literacy is that it is a set of tangible skills –

particularly the cognitive skills of reading and writing – that are independent of the

context in which they are acquired and the background of the person who acquires them.

Scholars continue to disagree on the best way to acquire literacy, with some advocating

the „phonetic‟ approach and others „reading for meaning‟, resulting in what has

sometimes been called the „reading wars‟. (UNESCO, 2006, p.149)

2.1.2 Collaborative action: framework and federal initiatives aimed at improving learning in the

first years of primary school

The Constitution defines that each federation body is responsible for the offer of priority

educational levels17, but it determines that the responsibility for primary education should be

shared between the states and the municipalities. It also provides autonomy for each

government level as regards the organization and management of their education systems.

However, they must operate in a collaborative way to ensure compulsory education for all, in a

way that balances the principles of autonomy and interdependence, and thus, reduce possible

inequalities. Unfortunately, there is no concept or features associated with the legal

requirement of collaborative action.

15

In 2009, the illiteracy rate among people aged 15 and over was about 10%, representing 14 million people. The Paulo Montenegro Institute measures the levels of functional literacy of the population aged 15 to 64 years, all over the country. The Indicator of Functional Literacy (INAF) generated by the Institute tries to reflect the skills of reading, writing and numeracy of this group. In 2009, according to INAF‟s sample, 52% of Brazilians who have studied up to 4

th grade reached only a rudimentary level of

literacy. This means they could only find explicit information in short and familiar texts (like a small ad or letter). UNESCO adopted the concept of functional literacy in 1978: „A person is functionally literate when he/she can engage in all those activities in which literacy is required for effective functioning of his group and community and also for enabling him to continue to use reading, writing and calculation for his own and the community‟s development‟. (UNESCO, 2006, p.154) 16

In Brazil there are 5.561 municipalities. 17

The Union is responsible for the funding and the management of the federal educational system, mainly higher education institutions; states must provide secondary education and municipalities are responsible for early childhood education.




As Abrucio (2010) says, this dualism in Brazilian federalism deeply affects educational public

policies. The institutional configuration in the education sector adopted by the Constitution has

caused acting gaps and intergovernmental tensions. Most municipalities claim that the

decentralization of educational responsibilities was not accompanied by the corresponding

financial resources and that the collaborative principle was rarely put in place to help building

the new model envisioned.

The decentralization was deepened, from 1996 onwards, with the setting up of the Fundo de

Manutenção e Desenvolvimento do Ensino Fundamental, FUNDEF (Fund for the Maintenance

and Development of Primary Education), which sought to balance public funding distribution

based on enrolments. However, municipalities were unprepared to take over their new tasks.

Most had and still have very low technical and administrative capacity.

Much of the aid to make up for institutional weakness in Brazilian municipalities has originated

in the federal government, which has a redistributive and supplementary role to ensure equal

opportunities for students and a minimum standard of quality. According to the Constitution,

this should be put in place through technical and financial support. As a consequence, the

collaborative action is generally associated with these two lines of action.

The transfer of financial and technical resources from Union to states and municipalities is the

most frequent model of cooperation due to more availability of funding and trained personnel at

federal level. Despite that, the collaborative principle around those two lines of action covers

both vertical and horizontal models of cooperation within the Brazilian federation as shown in

Figure 1.

Figure 1 – Types and levels of collaborative action in education within Brazilian federalism

Source: Author

Actions taken by the federal government

In order to fulfill the role of national coordinator of the education sector and responsible for

releasing general guidelines, the federal government has promoted some important changes in

the legislation related to the issue addressed by this study18. Since 2005, the Brazilian

education system has undergone a transition due to the adoption of Act no. 11,114 of 2005

which establishes compulsory primary school at age six and Act no. 11,274 of 2006 which

establishes a nine-year primary education.

In 2007, the National Congress approved the extension of FUNDEF, an enrolment based

formula funding mechanism, to consider the students of early childhood education (and also

18

There are also other objectives which will not be detailed in this study.

Collaborative Action

Technical

Resources

Financial Resources

Horizontal: Between states Between municipalities

Vertical: Union-State /Municipality

State-Municipalities




secondary school) through the creation of the Fundo de Manutenção e Desenvolvimento da

Educação Básica, FUNDEB (Fund for the Maintenance and Development of Basic

Education19). In 2009, constitutional amendment no. 59 extended the compulsory and free

education to all population aged 4 to 17 years. It should happen gradually and be completed

by 2016.

In practice, this set of changes allowed, in a first stage, the enrolment of 6-year old children in

public schools and the reorganization of the literacy cycle. Then, with FUNDEB‟s granted

funding, it was possible to expand access to preschool for children aged 4 and 5. The target is

to assist the children who live in low-income households since this is already a reality for

wealthier families. In summary, the new legislation aims to ensure earlier access to the

education system and thus to provide greater opportunities for learning, as well as reduce

disparities. Behind it there is the expectation that more years at school will result in more and

better learning.

Besides these changes in the legal framework, the Ministry of Education has implemented

national programs towards better literacy achievements in primary schools. The actions are

currently focused on three major areas: in-service teacher training, textbooks and assessment

through Pro-Letramento (Pro-Literacy); Programa Nacional do Livro Didático, PNLD (National

Program of Textbooks); and, Provinha Brasil, respectively20.

Pro-Literacy

It is aimed at improving Portuguese and Mathematics teaching in lower grades of primary

school. It offers a distance course with some face sessions and it is based on a tutorial model

operated by trainers21. Its implementation depends on states and municipalities adherence to

the program. Silva and Cafiero (2010) evaluated the initiative and considered that the program

has positive aspects such as more practical approach, but faces high levels of dropouts.

Textbooks

Since 2007 the PNLD has been revised to meet the needs of new legislation and the

enrolment of children aged six in primary schools. The previous single volume of literacy

textbook has been replaced by a collection of two volumes, which considers that the literacy

process should be consolidated at the end of the second grade. Here again there is a lot of

debate on which skills should be focused on these materials22. Other disagreement in the field

of literacy policies can be easily identified by contrasting the PNLD collection and the guidance

issued by CNE. The Council states that the literacy cycle should cover the first three years of

schooling23.

19

Act no. 11,494 of 2007 20

The federal government also implements the National Programme for Restructuring Schools for Early Childhood (ProInfância) which supports new schools for children up to five years (building and equipments). 21

The word used for trainers in Portuguese is multiplicadores, to imply the idea they are responsible for „multiplying‟ knowledge. 22

See Oliveira, s.d.; Silva and Cafiero, 2010 23

See CNE Res. CEB nº 7 of 2010




Provinha Brasil

The Provinha Brasil is a diagnostic test designed to assess reading in students enrolled in the

2nd grade of primary schools. Since 2008 the Ministry of Education develops the instrument

and makes it available on the Internet. School systems are autonomous to decide whether to

use it or not. The test can be applied by the teacher or by an external agent at the discretion of

each manager. The analysis, use and dissemination of results are also defined by each

system.

2.2 The growth of learning assessments in developing countries

In the educational reforms of the 1990s, learning assessment became a key concept in

developing countries. Several have put in place national systems of assessment, generally in

the context of broader processes of changes such as devolution of responsibilities, curriculum

reform, training for teachers without proper qualifications etc. These national systems of

assessment are a systematic, regular and external instrument to measure student learning

achievement. They are frequently identified in the literature as large-scale learning

assessments.

Ferrer (2003) points out that those assessment trends started in the 1990s covered other

modes of action, like institutional evaluation, evaluation of teachers‟ professional performance

and impact assessment of educational policies and programs. Nevertheless, the evaluation of

educational outcomes measured by student learning achievement became the most prominent

mode and faced great development over the following decade. Here, we will be concentrated

only on this latter mode24.

Assessment may refer to “any procedure or activity that is designed to collect information

about the knowledge, attitudes, or skills of a learner or group of learners” (Kellaghan &

Greaney, 2001 p.19). Thus, learning assessment is an integral part of the educational process.

In the past, it was just an internal school issue. The adoption of learning assessments carried

out by agents external to the school as a performance appraisal of national education systems

is associated with a number of factors, with different emphases depending on the author

analysis (Kellaghan and Greaney, 2001; Lockheed, 1996; Pigozzi, 2008; Saus, 2003;

Kellaghan, 2008).

In 1990, the World Conference on Education for All (WCEFA) that took place in Jomtiem

emphasized the countries‟ commitment in assuring learning acquisition rather than exclusively

in expanding enrolments. The WCEFA framework for action states clearly that basic education

is more than an end in itself and should provide foundation for lifelong learning and human

development. Regarding these needs, two activities were identified as relevant: i) defining

acceptable levels of learning acquisition for education systems, and ii) improving and applying

24

A different approach for external learning assessment is the national exams, which intend to assure that a certain level of knowledge was acquired to issue diplomas or to guarantee the access to higher levels of education. It will not be covered by this paper.




systems to assess learning achievement. This fact is pointed out as one of the reasons for the

growth of national assessment systems in developing countries.

Since then, the reforms have proposed the use of assessment to improve educational quality

and have three main characteristics: focus on outcomes; the assessment is conducted by an

agency outside the school; and the expectation is that the assessment will act as a lever of

change. In this sense, learning achievements are assumed as a proxy for evaluating the

quality25 of education offered by a national system.

The systems used to measure inputs, such as textbooks and facilities, as a proxy for quality.

The shift in focus from inputs to learning outcomes in national evaluations is also seen as an

attempt to ascribe some technical rationality to the education sector. Concepts such as

management by results, targets, use of indicators and efficiency (the ratio of inputs to outputs)

were introduced to public sector management as part of lessons learned in private enterprises

and began to influence educational policies.

Moreover, the quantitative expansion of the education system in terms of enrolments and the

rising costs associated with it would reinforce the growing demand for more accountability on

outcomes. In recent years, accountability has achieved an increasing importance in public

administration in many countries.

Finally, concerns related to the economic development of countries and linked to the

globalization of markets, the needs of increasing levels of economic competitiveness and the

so-called new competences that will be demanded for future workers have also played a role

in this process.

Postlethwaite and Kellaghan (2008) explain that the information obtained in a national

assessment can supplement information on inputs and on educational processes. Together,

they provide policy-makers and managers with evidence about their education system‟s

achievements, operational constraints, and the problems it may be experiencing. Since it is

difficult to plan without such information, national assessments can be considered as an

essential component of the professional administration of any education system.

That is why learning assessments have become the subject of much attention for both

industrialized and developing countries. They also got interested in working in international

initiatives such as the Program for International Student Assessment (PISA), Southern and

Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ) and Latin American

Laboratory for Assessment of the Quality of Education (LLECE)26.

SACMEQ was created as a „hands-on‟ capacity-building programme in the skills of

assessment research. It selected grade 6 as the target population. SACMEQ differs from other

studies in that it takes a great deal of trouble to discover the major policy concerns and the

research questions its ministries wish to answer.

25

Quality in education is an issue on the policy agenda of nearly every country and it has of course different and complex meanings. One of EFA goals is “improving all aspects of the quality of education and ensuring excellence of all so that recognized and measurable learning outcomes are achieved by all, especially in literacy, numeracy and essential life skills”. Quality is seen by UNESCO as a determinant element of enrolment, retention and achievement and its expanded vision comprehends characteristics of learners, processes, contents and systems. (UNESCO, 2005, p.2) 26

See Postlethwaite (2004a).




PISA opted for an age group (15-year-olds), regardless of the stage they were at in the school

system. It aims to provide policy-relevant information on the cumulative yield of education

systems, measured in terms of performance of students in applying knowledge and skills they

have acquired.

LLECE forms the network of the National Educational Assessment Systems of 16 countries in

the region. It is co-ordinated by the UNESCO Regional Office for Latin America and the

Caribbean (OREALC).

According to Postlethwaite (2004a, p. 10) of more interest than the relative positions of the

countries and regions in achievement standard has been the development of a more

accessible and meaningful approach for displaying results reporting student performance in

terms of a hierarchy of skills levels. “This is a more useful approach than the rank order of

countries because it can inform the Ministry of Education about what percentage of pupils in

each country can do what.” The international studies have been conducted with a very high

level of technical expertise, therefore they also can help countries to acquire technical

background to undertake good national studies.

The main steps involved in the initial planning for a national assessment are: a) decide on the

purposes of the assessment; b) construct the test; c) decide on educational indicators; d)

define the target population; e) ensure that a good sampling frame exists; f) decide one where

„over-sampling‟ is required and then draw the sample; g) try test items, questionnaires, and

procedures; h) plan the data collection and train data collectors. There is a sequence of

events: field work, data preparation, data analysis and reporting results. (Postlethwaite and

Kellaghan, 2008)

In short, when it comes to learning assessment, three questions are pointed out as the most

relevant ones: What is the purpose of it? What and how to evaluate? How might the

information derived from learning assessments be used?

2.2.1 What are the purposes of a learning assessment?

Generally speaking, the main purposes of a learning assessment are to judge the

effectiveness of an education system or to check the adequacy of the performance of a school

or even of a student. However, learning assessments can have a wide variety of purposes,

which sometimes are blurred and are used to provide information for different levels. The most

common ones are:

to inform policies by providing information that can guide decision-making process or

allocation of funds;

to monitor achievement trends over time;

to raise or to maintain standards;

to establish a basis for holding managers and schools accountable for student‟s

performance;

to promote changes in instruction;

to identify disparities between subgroups (by gender, language, regions etc);

to balance control in a decentralized education system;

to ascertain the degree to which children have acquired what is established in the

national curriculum or guidelines.




to identify correlates of learning (the causes and effects of learning in classroom);

to diagnose individual learning needs.

The purposes should be expressed in the research questions to be answered. Officials and

researchers should agree about this aspect from the beginning so that utmost care will be

taken to ensure that the study will collect the data required. As said by Postlethwaite and

Kellaghan (2008, p.18): "This may sound like common sense, but many national assessments

have been undertaken in which ministry of education officials were not involved in stating the

research questions, and many others have collected large amounts of data that were not

focused on the concerns of officials”.

2.2.2 What, when and how should it be assessed?

This question is intrinsically related to the information needs generated by the purposes of an

evaluation. By and large the purpose of the assessment determines its overall format or

design.

The assessment is summative when it occurs after a certain period of time such as a school

year or a cycle of education. It intends to summarize learning up to that point or to verify if the

system has met the educational objectives set. Another approach is the formative assessment.

The findings of this type of evaluation are supposed to be used to improve the instruction and

to ensure that the objectives are more likely to be accomplished.

The decision on evaluating in a census or sample basis also depends on the objective

pursued. A sample-based assessment cannot identify problems in a particular school.

Therefore some national assessments are applied in a census-based. On the other hand, if the

goal is to monitor learning outcomes for a general picture of system performance, then a

sample-based evaluation will be adequate and more cost-effective. As stated by Postlethwaite

and Kellaghan (2008) “when all students in an education system are tested (…) the costs can

be very high and therefore such „census-based‟ approaches may not be affordable in poorer

countries”27.

In national or international learning assessments, the comparability of data collected is a major

concern, so tests are standardized as well as administration procedures and scoring. Utmost

care is taken to ensure the credibility and accuracy of the findings, so measures of validity and

reliability are taken. Validity has to do with consistency and accuracy. Reliability is usually

established by using pre-tests, but there are other equivalent methods.

What to evaluate and when to evaluate are also relevant issues to guide instrument design.

Considering the difficulty of obtaining consensus on what competences and skills should be

acquired in the schooling process, most of the learning assessments focuses on basic

cognitive skills, such as literacy and numeracy. Therefore, the fields mostly assessed are

language and math.

27

Postlethwaite (2004b, p.87) explains that the more numerous the differences among schools, the more schools are needed in the sample. In poor countries, the differences among schools tend to be greater and therefore it is those countries that need more schools in the sample.




Given the general objective of getting information on the performance of an educational

system through learning achievements of its students, many assessments focus on the main

school exit points or on the end of cycles depending on the organization of each education

system.

The idea of measuring quality by learning outcomes often implies establishing standards28.

Standards are mainly presented as a measure of what is considered adequate for students to

have acquired at particular levels of schooling. Setting standards, as said by Wagner (2011),

involves the exercise of choice and the application of values.

Most large scale assessments utilize a statistical technique called Item Response Theory

(IRT). This approach allows covering a number of competences/skills in the instruments, but

each student is tested in only a few of them. It involves scaling of student responses and

allows comparability of results over time as some items are repeated. The analysis of data

requires more sophisticated skills from the personnel involved in the assessment. IRT

contrasts to the traditional approach of Classical Test Theory (CTT) with regard to raw scores

or percentage of items correctly answered by students.

According to Postlethwaite and Kellaghan (2008), some countries experienced to test the

teachers of the students in national assessments. This was undertaken in the Vietnam survey

in 2001 and also in the international surveys of SACMEQ. Testing teachers can be used to

identify gaps in their knowledge about school subject matter – which, in turn, can help to point

to areas that can be addressed in in-service teacher training.

Finally, large-scale assessments often make use of questionnaires to collect data related to

the socioeconomic background of students, school conditions and teachers, among others.

They are seen as complementary data to the analysis of learning achievements. This data

allow more meaningful comparisons since there is evidence in the literature that student's

academic performance is mostly explained by background and socioeconomic status29.

Postlethwaite (2004a, p.14) explains that great efforts have been made in international

assessments to identify the factors in and out of school that could have influenced

achievement. It is this kind of information that is needed by countries. In nearly all systems of

education the home background of pupils was important, but this is an issue to be carefully

analyzed. The author summarizes some conclusions from the studies conducted by the

International Association for the Evaluation of Educational Achievement (IEA) in the last 30

years. One of the points highlighted: “although the effects of home background variables were

similar across subject areas, the effects of the learning conditions in the schools differed

between subject areas, and in some subject areas were equivalent to or greater in the size of

their influence than the effects of the home”.

28

Standard-based assessments (or criterion-referenced) are, according to Wagner, “those that allow scores to be judged against some level of expected result. Norm-referenced assessments are those that provide a level of skill that is matched against the learners‟ relative position among peers taking the same test”. (Wagner, 2011, p.30) 29

The Coleman Report, commissioned by the Department of Education of the United States in the 1960s, is the most widely cited study about school effects.




2.2.3 How might the information derived from an assessment be used?

Here the main aspect is how the information generated is used. In theory, the more

appropriate the assessment model is to its purposes, the greater the chances that it addresses

the needs are, and therefore, the more likely their results are used. However, the use made of

large-scale assessments is also related to the analysis produced and the communication

procedures adopted to inform and disseminate findings. Wagner states that “learning

assessments are only as good as the uses that are made of them”. (Wagner, 2011, p.14)

Most widely used procedures of data dissemination include the preparation of reports with

variable coverage (international, regional, national, by sector) and aimed at different audiences

(education authorities, researchers, media, school managers, teachers, and families). Here

again, appropriate formats and language features are very relevant and should consider the

audience they are targeting. A wider dissemination strategy is designed when an important

social and media impact is intended.

There are also cases in which data is not disclosed. This approach takes into consideration

that the best use to be given to some assessment results is to offer the institution and its

professionals the opportunity to compare and reflect on their own performance. At the other

extreme, there are results that are published as league tables, ranking systems or schools.

This approach has received many criticisms because it tends to encourage comparisons and

interpretations out of context and based only on ranking position. This type of data

presentation has also been accused of producing competition between schools. Nevertheless,

as Kellaghan and Greaney (2001) have highlighted, it seems to arouse more interest in the

media. This interest is useful to put education in the spotlight and to generate mobilization

around issues of quality.

Postlethwaite (2004b, p.89) claims that “there is general agreement that research results have

a „drip effect‟ on public (not the general public, but the public most interested in the results)

opinion and that over time the results affect how planners think about education”. In some

cases direct actions are taken, in others the effects take a generation to induce improvements

in the schools. The author affirms that one important feature of repeat surveys is that it is

possible to check if action has been taken.

One of the drawbacks of external assessments already identified is the teaching-to-the test. It

tends to be more present in systems where schools are placed under great pressure to

improve their results. Regarding the use of assessments for funding, some negative aspects

have been identified, such as hierarchies among schools or rejection of teachers to be

deployed to institutions with poor results. It is noteworthy that they are strongly linked to the

reality of the United States, where such initiatives are more common.

Performance pay programs

The United States have adopted some experiences with the so-called bonus programs. The

government pays bonus for the best results, considering either school-wide performance or

teacher‟s group performance separately. This use for external learning assessments is very

controversial in the education sector.




In 2011, New York City Department of Education announced that its Schoolwide Performance

Bonus Program would be discontinued30. The program lasted for three school years, and its

broad objective was to improve student performance through school-based financial

incentives. A study commissioned by the city “found that the program did not, by itself, improve

student achievement, perhaps in part because conditions needed to motivate staff were not

achieved.” (Marsh, 2011, p.14)

The evaluation concluded that the program had no effects on teacher reported attitudes,

perceptions and behaviors. It was hypothesized by the researchers that lack of results might

be due to the limited motivational power of bonus – resulting from school staff viewing the

award as reward or recognition rather than an incentive.

Another hypothesis is that New York schools faced a lot of pressure to increase the grades of

their students, under the threat of severe penalties, such as closing the institution and in this

environment the bonus, which could reach $ 1,500 per teacher after taxes, but could still not

motivate them. On the other hand, Goodman (2011) found some positive findings and

suggested that a heavy reliance on this concept may hamper the effectiveness of it:

(…) what this experiment makes frustratingly clear for merit pay proponents is that the

structure of the payment scheme can make a large difference. For merit pay to improve

student outcomes, teachers must face strong incentives to improve their performance.

Our study indicates that school-wide bonus programs may be able to provide those

incentives in schools with relatively small teaching staffs. They may also be appropriate

for schools characterized by a high degree of staff cohesion, in which teachers work

collaboratively to improve student learning and it is difficult to isolate the performance of

a single teacher. (Goodman, 2011, p.71)

New York City Program inspired a similar initiative in the São Paulo State. The program

awarded a bonus to teachers and non-teaching staff based on the students‟ performance. The

school should meet targets designed by the State Secretariat of Education based on the Índice

de Desenvolvimento da Educação do Estado de São Paulo, IDESP (Index of Education

Development of the São Paulo State). The index is generated from the performance of

students in the Sistema de Avaliação de Rendimento Escolar do Estado de São Paulo,

SARESP (São Paulo Education System of Learning Assessment). In addition, teachers must

attend work at least two thirds of school calendar. Unlike New York, the initiative faced

problems with unions from the beginning, since it was seen as a means of taking the

responsibility on education problems out of public authorities and putting them on the teacher‟s

back. In 2011, facing negative variation in learning outcomes identified by SARESP, the state

decided to promote some changes in the proposal.

Old demands and new trends in learning assessments

Several authors have indicated that the use of large-scale assessments is now the weakest

part of assessment systems in developing countries, with little effort made to understand the

30

See “New York City Abandons Teacher Bonus Program”, published by The New York Times in 17 October 2011. Retrieved from: http://www.nytimes.com/2011/07/18/education/18rand.html in 12 April 2012.

http://www.nytimes.com/2011/07/18/education/18rand.html




reasons for that. They agree that more efforts have been made in structuring models and

collecting data than in analyzing it and in communicating it properly to stakeholders. (Wagner,

2011; Ferrer, 2003; Iaies, 2003; Kellaghan and Greaney, 2001)

The poor use is due to a variety of factors: lack of, or poor analysis, insufficient dissemination

or exclusive use of mass media to do so, low ownership of the findings by local actors, long

delay between field work and dissemination of results which hampers the use of reports as a

planning tool.

In reviewing the evaluations implemented in the 1990s in Latin America, many authors

recognized their importance to increase public awareness and to inform political debate,

especially regarding inequalities within the education systems. Broadly, they agree that

information obtained from national assessments has been used in the discussions about public

policies, but stress their low impact on school practices. Even ministry teams responsible for

training and curriculum areas rarely use them as input for their work.

This is a critical issue for the region because, as argued by Iaies, no significant variations in

the results have been observed in the period assessments have been conducted (Iaies et al,

2003).

According to this review, the teams responsible for building learning assessment systems in

Latin America were clear about the demand of organizing devices, but have little guidance

regarding their purposes or the impact that was intended to generate by their results.

The advances generated since the first experiences were implemented seem to have

been built with autonomy over purposes. There are few innovations in instruments.

Those already produced have to do with statistical or technical-pedagogical adjustments.

The innovative element appeared in some international devices that incorporated new

models and contents to the instruments. (Iaies, 2003, p.29-30) (in translation)

The review suggests that it would be interesting to reflect upon the current assessment

strategies in order to embed the advances made and to reorient these tools. But it

recommends that these changes should be consistent with policy objectives defined by the

governments.

Indeed, different approaches of assessing learning in developing countries have been tried

recently. One of them is the Early Grade Reading Assessment (EGRA) which focuses on early

readers and considers local contexts rather than national or international comparability. It tries

to deal with policy constraints such as ethnic diversity or linguistic variation that are sacrificed

in large-scale assessments to ensure comparability of results. It also pays close attention to

individual differences in learning.

Wagner (2011) states that EGRA is part of a new trend to put the focus of assessment design

on less developed countries context and needs. Its concept is based on the paradigm of

smaller, cheaper and faster assessments. Its methodology is simple and allows wide

„shareability‟ which reduces costs. The timeliness of analysis is more suitable to local

decisions.

Because the test is applied by teachers, EGRA cannot be considered an external learning

assessment. Nevertheless, it is interesting to note that its emergence meets one of the most




urgent needs of education systems in developing countries: to identify reading problems much

earlier than the national or international tests taken by the countries.

Another trend is the dissemination of experiences with large-scale assessments through lower

levels of government. This trend has been possible due to the advances in technology, which

hugely improved the capacity for processing data, making data collection and analysis easier

and faster. The dissemination of technical expertise on learning assessment and training of

teams also played an important role in this process.

2.2.4 National experience in learning assessments

Brazil started implementing the SAEB31 in 1990, but it was only after 1995 that the Item

Response Theory was adopted, opening the comparability of results over time. SAEB is an

external large-scale learning assessment system. It aims to provide information about the

quality of education and to inform the design, redesign and monitoring of educational policies

at national, state and municipal levels.

Currently SAEB comprises two complementary learning assessments. The first, called

Avaliação Nacional da Educação Básica, ANEB (National Basic Education Assessment), is

based on a sample of students from public and private schools, in rural and urban areas.

Groups must have a minimum of ten students enrolled in each of the 5 th and 9th grades of

primary school and also in the 3rd year of secondary. Polled results are provided by national,

regional and state level.

The second, called Avaliação Nacional do Rendimento Escolar, ANRESC (National

Assessment of Educational Achievement), implies a similar format to ANEB‟s. The difference

is that it is implemented, at a census basis, in some 5 th and 9th grades in public school groups.

Each school must have a minimum of 20 students enrolled in each grade assessed. Results

are provided by national, state, municipal and school level. Both tests include Portuguese and

Mathematics, focusing respectively on reading and problem solving, and are held every two

years.

ANRESC is well known as Prova Brasil and is a very recent initiative. It has been implemented

since 2005. It was designed to meet the demands of public authorities, educators, researchers

and society in general for information about the education offered in each municipality and

school. The objective, as said by MEC, is to assist local governments in the allocation of

technical and financial resources, as well as the school community in setting goals and

implementing actions aimed at improving the quality of education.

Long before Prova Brasil had been created, several state governments32 adopted their own

learning assessments to complement or to address deficiencies related to SAEB. They are

often used by state governments as a mechanism to meet the legal requirement necessary to

evaluate their educational systems.33. Legislation requires evaluation of educational institutions

under state responsibility, there is no specific instruction as regards students‟ achievements.

31

To access data or to get more information about SAEB see www.inep.gov.br 32

Rio Grande do Sul, Ceará, São Paulo and Minas Gerais, among others. 33

See article 10 of Act nº 9.394 of 1996 which establishes guidelines and objectives for national education.

http://www.inep.gov.br/




More recently, some large-scale learning assessments are also being held at the municipal

level, especially in state capitals such as Recife or cities with greater economic development

as Campinas, which can afford the financial costs and have trained personnel to implement

them. Municipalities do not face the legal requirement to perform evaluations in their education

system.

The need for identifying reading problems in earlier stages and the goal of all children being

literate by the age of eight, as stated by the NPE, have introduced new demands for learning

assessments in Brazil. Besides Provinha Brasil, provided by MEC to states and municipalities

and already mentioned in the previous section, the NGO Todos pela Educação (All for

Education) conducted the first large-scale learning assessment in 2011 (Prova ABC) at a

sample basis to diagnose 2nd graders‟ learning in writing and reading. However, in Brazil, there

is still no systematic assessment on this topic at a national scale. There are experiences

conducted at state level, such as those occurred in Ceará and Minas Gerais.

Conclusion

Learning assessments were widely spread out in developing countries due to a broad

combination of factors which have had many purposes. In general, they have been conducted

at national levels through large-scale models. Recently, their drawbacks and more specific

needs – to monitor literacy achievements in the first years of schooling – have lead to some

innovations.

In Brazil, the issue of literacy outcomes does not concern only to learning assessment

innovations. There is a wide variety of interpretations among the education systems in states

and municipalities about what it means to be literate and at what point (grade) of schooling it

should be achieved. This fact stems from the lack of more precise definitions, so better

guidelines should be the focus of federal government action. Although the responsibility for

literacy outcomes is posed mostly on municipalities, the more fragile part of the federation in

terms of financial and institutional resources, the path is also linked to more intense and

effective forms of intergovernmental collaboration.




Introduction

This chapter describes relevant facts that influenced the decision-making process to formulate

and implement PAIC and the main features of the collaborative action structured by the State

of Ceará and its municipalities. It analyses some political and technical aspects that

contributed to identify the issue the program intends to address and that underlies PAIC

design. The program is understood as an output of this process and it is described regarding

the actions carried out and the responsibilities of each government level.

3.1 The Committee for Eradication of Illiteracy at School (2004-2006)

3.1.1 How and why was it set up?

In 25 March 2004, the Assembléia Legislativa do Ceará, AL-CE (Legislative Assembly of

Ceará) set up the Comitê Cearense para a Eliminação do Analfabetismo Escolar, CCEAE

(The Ceará State Committee for Eradication of Illiteracy at School). This initiative was a

response to the impact of the 2001 SAEB results, which were released in 2003.

It is important to highlight that the year 2001 marks the lowest inflection point reached by the

learning outcomes of the Brazilian educational system. After increasing the number of students

enrolled in primary schools in the 1990s, the system indicated that it had not managed to keep

quality levels34.

Originally, the idea of investigating low learning levels in Ceará‟s education system was

proposed by the then State Representative, Cid Ferreira Gomes, as a Parliamentary

Commission of Inquiry35, but it was transformed into a Committee led by the state legislature,

with plural participation of stakeholders from the three government levels and an international

34

Neri and Carvalho associate this drop in students‟ mean performance with other public policies, such as the setting-up of Programa Bolsa-Escola* and the adoption of the automatic grade progress policy, which stimulated students at critical socio-economic conditions, with very poor school performance to remain in school. (Nery & Carvalho, 2002, apud Todos pela Educação, 2007, p.13). *Bolsa-Escola was a cash transfer program to poor families that guaranteed their children (maximum three) minimum school attendance. Later, it was redesigned and named Programa Bolsa Família (Direct Cash Transfer Program). 35

It is a largely used tool in the Brazilian Parliament in order to investigate and include relevant topics in the political agenda.




institution, during its tenure36: União Nacional dos Dirigentes Municipais de Educação,

UNDIME/CE (National Union of Municipal Education Officers), INEP, SEDUC and The United

Nations Children‟s Fund – (UNICEF)37.

Over an 18-month period of work, the Committee‟s agenda was divided into two main lines of

action: i) to carry out a set of diagnostic studies to provide more accurate data on the actual

literacy level of Ceará children, considering that „information on children illiteracy and on the

quality of education is scattered or from samples, insufficient to mobilize forces towards

more strategic actions and to influence resulting policies in the field‟; and ii) to mobilize

government and society and mainly public municipal power to acknowledge and face the

problem. (AL-CE/CCEAE, 2006, p. 26, emphasis added)

3.1.2 Diagnosis of illiteracy in Ceará schools

By means of technical and financial support from both federal and state governments, the

Committee put five universities38 in charge of carrying out three studies39:

i. Diagnosis of 2nd graders literacy levels (reading and writing) in public schools40;

ii. Identification of ways and conditions of organizing school work and teaching work in

child literacy.

iii. Identification of both initial and continuing teacher training mechanisms for child literacy

education41.

Some of the main findings were:

a) Assessment of the students‟ literacy levels as regards three dimensions – text production,

reading and text comprehension – indicated that two thirds of the students were not literate at

least in two of the dimensions assessed.

b) The school‟s textbooks and the children‟s literature books were insufficient and of poor

quality. Little or inadequate use was made of the existing material.

c) School calendar and lessons timetable were not being followed; besides, the actual lesson

time was being wasted with irrelevant activities for children‟s learning.

d) There was low student school attendance and high teacher absenteeism.

36

The Committee also had the systematic collaboration of other stakeholders: main public and private universities, state and municipal councils of education, bodies representing mayors and the economic sector, and also non-governmental entities linked to children‟s rights defense. 37

UNICEF assists several Brazilian municipalities technically in the development of programs that lead to fulfilling the right to learn. 38

Universidade Federal do Ceará (UFC), Universidade Estadual do Ceará (UECE), Universidade Estadual Vale do Acaraú (UVA), Universidade Regional do Cariri (URCA) and Universidade de Fortaleza (UNIFOR). 39

The Committee considered that the academic performance of students under literacy process is a result of both intra and extra-school factors; thus it was also useful to investigate the supply conditions and the ways of organizing school work. 40

The study involved 8 thousand 2nd graders from public schools in 48 Municipalities. 41

The core objective of this study was to assess knowledge about child literacy mastered by students and teachers in Pedagogy, Child Education and School Teaching Courses, in all public teacher education institutions in the State of Ceará.




e) Teacher profile had changed regarding training; most of them had a degree, but they were

still poorly qualified for the literacy teaching task.

f) Most teachers sought explanation for the phenomenon of illiteracy at school in outside-

school-related factors.

3.1.3 The importance of the Committee’s work

As the final report itself highlights, the CCEAE research findings were already well known and

the school organization problems were not new. Thus, what was the relevance of the

Committee for the elaboration of a program, such as PAIC, to fight illiteracy at school?

Roughly speaking, it can be said that the Committee‟s work represented a source of change

for the formulation process of PAIC. One of the great achievements could be the capacity to

translate insights and indirect indicators (produced by SAEB) into evidence-based diagnosis

which was disseminated and quite discussed with key stakeholders within the State. They

were fully involved in the Committee and helped to carry out the work from the beginning. At

the end of the process, they were all quite aware of the diagnosis and what goals should be

pursued to address the problem of illiteracy at school.

In the CCEAE final report, some key issues in the policy-formulation process are clearly stated,

such as the vision and goal setting, priorities and some strategies to reach them. All the

speeches made by these stakeholders, included in the final report, stress two basic elements

that framed PAIC design: i) focus on learning (effective literacy in the first years of primary

education) as a sine qua non condition to fulfill the right to education; and ii) accountability for

results. Generally speaking, these key actors presented political will to tackle the problem.

Apart from involving representatives of the three levels of government during its administration,

the Committee was successful in the task of mobilizing and including the illiteracy at school

issue in the agenda of municipalities, civil society and the media. Several events and public

hearings were held in the state capital city of Fortaleza and in small towns of the State, which

turned into discussion forums on the issue.

The process of mobilization and sensitization of stakeholders (providers and beneficiaries) to

the need for a change was made both within the Committee‟s activities and by media

coverage. The final report offers a collection of news published in mainstream State media,

particularly with respect to research findings.

The activities undertaken by the Committee may also have played an important role in

generating legitimacy for its findings and a sense of ownership by all groups when PAIC

started to be implemented.

The final recommendations made by CCEAE highlighted the importance of focus on learning,

but also suggested in-service teacher training, municipal management strengthening, and

better monitoring of learning results at school and system level. These recommendations, as

will be seen, were broadly incorporated by PAIC.

Finally, the local legislative power leadership on this process was unusual, reversing the sense

of political power and setting up of agendas that normally stem from the Executive Branch, at




all levels of the Brazilian federation. It is worth highlighting this fact because it indicates new

directions for other legislative forums and collaborates with the consolidation of the democratic

institutions in the country.

3.2 Sobral’s experience in fighting illiteracy at school (2001-2004)

The Committee organized by the Ceará state parliament and later the team in charge of PAIC

benefited from a previous experience of fighting illiteracy at school that had already taken

place in the same state. The city of Sobral42 had implemented from 2001 onwards an

educational policy that prioritized literacy of all students in the first cycle of primary school.

Similarly to the process that will occur within state scope, the decision of implementing actions

addressing effective children literacy stems from a diagnostic assessment with students who

finished the 2nd grade. The results showed extremely low reading skills. In Sobral, the

diagnostic was used to mobilize and sensitize the educational community to the issue.

(Coelho, 2006)

3.2.1 Lessons learned from Sobral’s experience

The core aspects of Sobral‟s experience are presented in the PAIC set-up, such as: focus on

learning in the early school years, defining clear and well-known targets43, accountability for

results and specific external assessment design to guide and monitor the changing process44.

Carried out twice a year, the external assessment provided indicators of both reading and

writing development levels of 6 and 7 year olds enrolled in the first two primary school years.

The results gathered in the first semester allowed schools to adopt new strategies to meet their

learning targets. At the end of the second semester, the second assessment found out the

learning outcomes and enabled adjustments for the following years. Apparently, one of the

most interesting lessons learned from Sobral and incorporated in PAIC was the use of the

external assessment results as a managerial instrument by the Secretaria Municipal de

Educação, SME (Municipal Education Secretariat) and by the schools. Learning outcomes

were scrutinized and were used to guide planning and subsequent adjustments in actions

implemented by these teams.

42

Sobral is one of the 184 municipalities in Ceará and it is located in the backlands of the state, in the Northeastern semi-arid region. According to the 2010 census, its population totals 190 thousand people, being the 5th more populated town in the state. It is an industrial hub and one of the biggest economies of Ceará. Its GDP was R$1.9 million in 2009, when its per capita GDP of R$10,769.79 was higher than Ceará‟s (R$7,686.62) and the Northeast region‟s (R$8,167.75). Data source: Instituto Brasileiro de Geografia e Estatística, IBGE (Brazilian Institute of Geography and Statistics), “Gross Domestic Product by Municipalities 2005-2009”. Date of Access: 15.02.2012 http://www.ibge.gov.br/home/estatistica/economia/pibmunicipios/2005_2009/ 43

During the 2001-2004 tenure, seven targets were defined for Sobral‟s educational system. Two of them were prioritized: i) literacy of all 6 and 7-year-old children; and ii) under a compensation basis, literacy of all students from the 2nd to the 6th grades who cannot read. 44

Sobral‟s experience was registered by INEP in: Vencendo o desafio da aprendizagem nas séries iniciais: a experiência de Sobral/CE. Série Projeto Boas Práticas na Educação nº1. Brasília: INEP, 2005.

http://www.ibge.gov.br/home/estatistica/economia/pibmunicipios/2005_2009/




Another relevant point gained is the search for accountability for students‟ learning results.

External assessment allowed tracking the competences developed by students individually.

Each group and each teacher could be correlated with their respective results. This is a crucial

issue in the Brazilian educational system – also pointed out by the Committee – because,

generally, students‟ low learning is attributed to only outside school related factors45. INEP‟s

report informs that:

Interviews with school principals and coordinators certify that vague phrases such as

“most can read”, or “some improved”, which little contributed to checking learning

success, are not part of the school‟s discourse anymore. The teacher started to know

every student by their names, as well as each one‟s advances and difficulties, therefore

getting more prepared to deal with their main difficulties. Principals and coordinators also

started to learn the difficulties each teacher had, what enabled them to design new

training strategies and guidance. Thus, external assessment enabled the consolidation of

a systematic culture of teacher‟s assessment . (INEP, 2005, p. 44)

One more lesson learned and incorporated in PAIC was the design of a set of comprehensive

actions to fight illiteracy at school, shaping it as a public policy rather than as an isolated

program: expanding access to pre-school, improving school supervision, strengthening

technical support to teachers, complying with school calendar and timetables, and availability

of children‟s books and learning games, among others.

In order to achieve the desired changes in the literacy teaching practice and face teachers‟

poor qualification, well structured teaching materials were introduced, describing contents and

step-by-step daily activities to be developed with students. Teachers attended specific training

sessions to use these materials and also attended meetings with the support team from the

Secretariat to discuss learning targets.

As regards the reorganization of Sobral‟s education system, the actions were on creating rural

school clusters and on selecting school principals through public exams and technical profile

analysis, aiming at appreciating professional merit rather than political nominations for the

occupation of these positions. These were ideas that had a great impact for a positive change

and are also incorporated in PAIC.

Finally, Sobral‟s experience resorted to mechanisms of financially rewarding schools and

teachers that met the learning targets, creating the “Prêmio Escola Alfabetizadora” (Literacy

School Award), and a bonus, equivalent to 30% of their monthly basis salaries, to the literacy-

promoter teachers who met the learning targets for their groups. This measure also resulted in

unwanted effects, such as forced school transfer of students with low performance. As a

lesson learned, this school award strategy is later redesigned by the State.

In 2004, when CCEAE unveiled the outcomes of its data collection on the literacy level of

students in Ceará, Sobral students‟ performance proved to be well above the one achieved by

45

In 2011, the NGO Todos pela Educação (All for Education) analysed the questionnaires completed by teachers for Prova Brasil, the exam that assesses 5th and 9th graders‟ performance and collects information from principals and teachers in order to identify the main causes of learning problems, according to these professionals‟ view. The analysis of that data indicated that teachers believe low learning levels are mainly due to outside school related factors, such as lack of family assistance and support (88.1%), students‟ lack of interest and effort (88.1%), and socio-economic levels (79.5%). The analysis results are available at http://www.todospelaeducacao.org.br .

http://www.todospelaeducacao.org.br/




the State (Table 5). There was also a reduction in school dropout rate of students enrolled in

the first cycle of primary school. Between 2001 and 2004 it fell from 9.6% to 0.7%. In

summary, the experience seemed to be quite successful to inspire and legitimate an action to

be taken within the scope of the whole State46.

Table 5 - Percentage of students who finished 2nd grade of primary school with adequate performance

according to CCEAE assessment standards, by dimension assessed, Ceará and Sobral, 2004.

Level Text Writing Text Comprehension Reading

Ceará 42% 15% 34%

Sobral 80% 31% 59%

Source: Data retrieved from INEP, 2005 and AL-CE, 2005. Note: In reading, the adequate performance

refers to students who read above the speed of 50 words per minute. The two other dimensions refer,

respectively, to those who managed to produce a text and to understand the issue required.

3.2.2 Policy priorities and technical expertise transfer from the city of Sobral to Ceará’s

Government

Government decisions are not made by a monolithic state based on rational choice, but rather

are negotiated by various leaders who sit on top of the organizations involved in that particular

decision-making process. Each leader is compelled by his own conception of the problem as

well as by the imperatives of his organization and his own personal goals. (Haddad, 1995)

In PAIC‟s case, it should be registered that there were links among some core actors and this

could somehow have influenced the agenda and, consequently, policy options. CCEAE‟s

president, representative Ivo Gomes, who led the liaising process among several entities, was

Secretary of Education in Sobral. PAIC‟s piloting stage was implemented during the mandate

of the then Ceará‟s governor, Lúcio Alcântara, but the program gained more scope from the

moment Cid Gomes, the new governor and Sobral‟s former mayor, was first elected and took

up office for the 2007-2010 mandate, and recently re-elected for another 4-year mandate

(2011-2014)47. Additionally, the managers who were placed in SEDUC in 2007 were the same

leaders of the changes that occurred in Sobral.

Accurate diagnoses of issues and mobilization around them are not enough to induce

consistent changes in the education sector. Setting priorities and simultaneously making it

clear that all efforts should be put in place to implement them depend on the commitment of

political leaders and on the technical capacity of the managers in charge. Apparently PAIC

may be benefited from a combination of political leadership and technical expertise, both

considered vital elements for the formulation and implementation of effective policies.

46

In 2007, the same year of PAIC‟s expansion to 100% of Ceará‟s municipalities, the Ministry of Education created the Índice de Desenvolvimento da Educação, IDEB (Education Development Index), an indicator aimed at measuring the advances in the quality of public education combining learning outcomes with flow data. The target set to all federative entities is to reach value 6 in year 2021. IDEB registered an index of 4.5 for the first years of primary school in Sobral, in 2005. The national and state mean was, respectively, 3.8 and 3.2. In 2009, IDEB‟s results were: Sobral (6.6), Ceará (4.1), Brazil (4.4) 47

Ivo Gomes and Cid Gomes are affiliated to the Partido Socialista Brasileiro, PSB (Brazilian Socialist Party).




3.3 PAIC and government collaborative action in its implementation

“Once a problem within the educational system is recognized then a solution is frequently

forced upon the system. This is especially likely to occur when there is a public debate

about a problem”. (Haddad, 1995, p.31)

The mobilization generated by the Committee and the media48 repercussion of the children‟s

low literacy level diagnosis led to a consensus and a commitment to action. However,

transforming consensus and commitment into action would be a complex task as learnt from

CCEAE‟s and Sobral‟s experience. They proved that fighting illiteracy at school would demand

acting in several spheres, mainly in the management of municipal education systems, and not

only in classrooms. Even if the State took over the matter as a priority in its agenda, it would

be necessary that municipalities acted as the changing agents.

The first stage of the Program (2005-2006), not by chance, was headed by the Associação de

Municípios e Prefeitos do Ceará, APRECE (Association of Ceará‟s Municipalities and Mayors)

and by UNICEF, with the support of the state government. Timing in this case was a crucial

element because it was necessary to raise mayors‟ awareness (the ones elected in 2004 for

the 2005-2008 term of office) towards “their commitment for action” as regards the setting up

of the program within their mandate. Between 2005 and 2006, PAIC was set up in 56

municipalities. Although it had not been institutionally defined as such, in practice and

experience it worked as a pilot, especially regarding external assessment mechanism.

In 2007, the new governor of Ceará, Cid Gomes, decided to turn the program into a state

public policy, which implied a more systemic approach to fight the problem of illiteracy at

school and to design a model for specific cooperation with the municipalities that complied with

the priority taken. In the following pages, it is described how PAIC was structured and why it

presents innovative features49.

3.3.1 Institutional arrangements for PAIC implementation

Institutionalizing the proposal was done in two ways: i) The State Act nº 14,026 / 2007, which

institutes PAIC; and ii) a Memorandum of Understanding among 100% of the town halls. The

Memorandum called the mayors to take on responsibility for the issue as a priority in their

mandate and committed to act under seven lines of actions: relocation of financial resources to

all programs in the area; review of career plan, prioritizing the literacy teacher position;

encouragement of teacher‟s commitment with child learning by means of training; technical

criteria setting for selecting school principals; setting up municipal systems for learning

assessment; and adoption of policies for encouraging reading and writing.

As regards its structure, PAIC has a managing committee with consulting and supervisory

functions, composed of institutions that had been involved in CCEAE, which have accumulated

48

The final CCEAE report brings the set of articles published in the media. During 2004, there was intensive coverage of the Committee‟s work, totalling 42 articles published. 49

In the same year, the state of Minas Gerais also started a program dedicated to help 3rd graders develop better literacy skills. Nevertheless, it was designed for schools under the state system responsibility.




experience in mobilizing and giving technical support to the municipalities50. Possibly this

networking allows a wider follow-up of the program with different approaches and

contributions, depending on the view of each institution involved.

Thus, a specific organizational arrangement was set up: SEDUC created the Coordenadoria

de Cooperação com os Municípios, COPEM (Coordinating Body for Cooperation with the

Municipalities)51, with its own personnel and budget. Each of the 20 Coordenadorias Regionais

de Desenvolvimento da Educação, CREDES (Regional Coordinating Bodies for Education

Development), the decentralized branches of SEDUC, created a Núcleo Regional de

Cooperação com os Municípios, NRCOM (Regional Centre for Cooperation with the

Municipalities) to mirror COPEM‟s structure and to operate at local levels. This structure is

responsible for giving technical support to the municipalities in the setting up of PAIC, but it

also manages other state government actions, such as school transport.

At COPEM, around 30 people run the program grouped by the acting axes and count on ad

hoc advice from Ceará universities‟ experts, whose work is paid by the state. Each Secretaria

Municipal de Educação, SME (Municipal Secretariat for Education) also organized its own

team of around three people, depending on the size of the education system, for the setting up

of PAIC. The State gives a financial bonus to the members of PAIC‟s municipal teams. This

bonus policy has caused some dissatisfaction among SME personnel not involved in PAIC, but

SEDUC has seen it as a strategic measure to signal for literacy priority, team organization and

for encouraging engagement. Figure 2 summarizes the management structure of PAIC at the

state, regional and municipal levels, identifying the main responsible for the actions and the

agency/staff the manager reports to within the administrative structure.

Figure 2 – Synthesis of PAIC Management Structure

Source: Author

50

Participating Institutions: Associação para o Desenvolvimento dos Municípios do Estado do Ceará, APDM-CE (Association for the Development of Ceará State‟s Municipalities), experienced in managing programs for competent reader training; Aprece and Undime, already mentioned in the text, are municipal entities; Fórum de Educação Infantil (Preschool Education Forum), a civil society entity which works to promote preschool education and has a vast experience in the field; Secretaria de Cultura do Estado do Ceará (Ceará State Secretariat for Culture), whose role is to strengthen the links among reading, writing and culture; and Unicef, also already mentioned, which implements the program “Selo Unicef Município Aprovado” (Unicef Passing Stamp for Municipalities) in Brazil. 51

See appendix III.




3.3.2 PAIC Design

Clear target and a comprehensive set of actions

The core target is literacy of 100% of the students at the end of the 2nd grade of primary

school, but PAIC is organized around five acting axes: Literacy, Municipal Management,

Preschool Education, Competent Reader Training, and External Assessment. Each one of

these axes encompasses a specific set of goals52, for instance full compliance with the 200-

day academic calendar in each municipality, universal access to preschool and availability of

reading corners in all 1st and 2nd grade classrooms. The State led the municipalities to commit

themselves to a set of actions considered relevant to ensure the literacy of a child at the “right

age”.

Municipal policies should incorporate these goals and have predetermined indicators to

monitor its implementation. Thus, for universal access to preschool, for instance, the indicator

will be the enrolment rate of 4-5 year-old children. Each municipality is responsible for

preparing a work plan, including targets and projections for four years, timetable and the

responsible parties involved. It should consider political support, baseline indicators and the

availability of financial resources to be allocated for the execution of each goal.

Improving municipal management and inducing policies at local level

The poor technical capacity of SME and the plurality of action fronts started by PAIC led

SEDUC to help with the process of improving municipal planning and management. A

thorough operation took place in order to guide each town to carry out a diagnosis of its

educational system, identifying action priorities, and defining targets and strategies related to

PAIC.

In this process, the state does not transfer financial resources to municipalities but gives

technical assistance. It also provides statistical data, planning tools and information systems

and suggests actions and indicators to be monitored. For example, it offers guidance to create

rural school clusters, to establish a selecting process for school principals through public

exams and technical profile analysis, or to define criteria to review career plan and teacher‟s

pay.

This is an extremely interesting process if it is compared to the traditional model of cooperation

between state and municipalities, which used to operate mainly through isolated programs and

transfer of financial resources. SEDUC decided to support the capacity building of SME in

diagnoses and planning to improve the effectiveness of municipal policies and thereby to be

able to meet the targets set by PAIC.

According to the information collected, the municipalities had initial difficulties in taking in the

idea that diagnosis is an essential step before setting priorities and planning actions. They

were not used to dealing with large quantities of data to be systematized and considered it a

waste of time for the manager and his team. Currently, these processes have been

incorporated into the dynamic functioning of the secretariats.

52

See Appendix IV for a detailed list of objectives and targets for each acting axis.




This does not mean that all the recommended processes were implemented. Sometimes the

creation of clusters is necessary - about 70% of the 6,524 primary public schools are in rural

areas – but runs into the opposition of communities. The appointment of principals based on

political nominations rather than professional merit is still the rule. The state finds it difficult to

discontinue this traditional practice in municipal policy. These issues have effects on the

program‟s implementation. Municipal managers said that multi-grade classrooms of rural

schools hinder the adoption of PAIC teaching materials. Furthermore, they say that some

principals little committed with results are, in many cases, a major cause of school slow

progress towards its learning outcomes, despite all efforts.

SEDUC designed other strategy of cooperation to ensure school transport in rural areas and to

expand education access to early childhood/preschool. Both involve financial requirement hard

to be taken only by municipalities. In these cases, for each vehicle purchased by the cities,

another is state funded, up to five units. In addition, each school built by the municipality to

assist children up to 5 years old adds another one with state funds. This is, of course, a way of

giving financial aid and inducing policies simultaneously.

Capacity building to manage the cycle of public policy – from diagnosis and planning to

monitoring and evaluation – can contribute to the sustainability of PAIC, and additionally may

help to turn the local leadership in education into actual professionals in the field. First

evidence of this can be inferred from the reported cases where the PAIC city manager has

been appointed to the position of municipal secretary of education.

At school level: improving management and offering in-service training

At school level, there are two main lines of action. The first is the provision of in-service

training for teachers working in the first two years of primary education. This training is linked

to the use of structured teaching materials. This material provides the contents and step-by-

step daily activities to be developed with students. The option for this type of teaching material

is due to the low theoretical and methodological teachers‟ background identified by CCEAE,

which have resulted in ineffective teaching practices in the classroom.

In Brazil, these kinds of materials face some criticism claiming that they are incompatible with

the autonomy and creativity expected from a teacher. It seems that the state decided to avoid

the controversy over the emphasis given (or not) to partial or full literacy53. The option was to

face the well-known weaknesses in initial teacher education54 by offering materials that

enable them to work and obtain better learning outcomes. On the other hand, schools still

receive textbooks from PNLD (they are to be used as supplementary material), which seems to

be a waste of public resources.

SEDUC states that textbooks selected by PNLD are incompatible with PAIC objectives and

that the kits Ceará purchased for 2nd grades includes teacher manuals, student textbooks,

classroom pedagogical materials and 80 hours of in-service training. Teachers can choose

53

According to SEDUC, PAIC materials consider full and partial literacy as interrelated processes. The approach combines the development of decoding skills with the constant interaction with different types of text and activities oriented to facilitate the construction of comprehension skills in reading activities. (SEDUC, 2011, p.86) 54

See Gatti, B. A. and Barreto, E.S. 2009. “Professores do Brasil: impasses e desafios”. UNESCO: Brasília.




from four different publishers, who will carry out monthly training sessions in each city55. For 1st

grade teachers, one specific teaching material was developed by a team of consultants from

Ceará. They attend training sessions conducted by PAIC‟s local team who receives guidance

from the consultants that had produced the material. In priority municipalities - with lower levels

of literacy proficiency - training is carried out directly by the consultants. Training is provided by

the state, but it is each city‟s responsibility to enable teachers‟ attendance (providing transport,

food and appointment of substitute teachers when it occurs in school days).

The selection, acquisition and distribution of the materials were bureaucratic and took long

time. In the first year, they were made available only after the beginning of the school year

because each municipality had to collect them in the capital. Later, an agreement with

publishers allowed them to be sent to CREDES, which streamlined the distribution process.

Another ongoing challenge is to ensure teacher attendance to training sessions. SEDUC

proposes these sessions are held on Saturdays, but this suggestion is not always welcome by

teachers.

The lessons from Sobral and the Committee grounded the decision that offering material and

training was not enough. It was essential to guide the attention and efforts of schools and SME

teams towards the relevance of monitoring some indicators: attendance rate of students,

teachers and principals; number of titles read by students; number of school days complied;

use (or not) of PAIC materials; attendance rate of teachers in training sessions and reading

status of each student, among others. SEDUC made available an online information system56

to gather and to process data collected from schools and SME. Bimonthly, reports are

analyzed at different levels of aggregation and by different agents: school managers, PAIC city

manager, PAIC regional teams (CREDES) and finally by COPEM team.

These strategies were designed to ensure “focus on student‟s learning” and to disseminate a

“culture of monitoring”. Both state and municipal agents who were interviewed said that the

merged processes of in-service training, pedagogical support and ongoing monitoring are

critical to the success of PAIC. According to the interviewees, cities that carry them out more

carefully are those which progress more.

The most important thing we have is the consistent support from the state, maintained

over the years. Training and monitoring actions are done every month. It is a strong

support, but much depends on the city to play its role. Here we do not let teachers miss

the training sessions. We seek to provide all the necessary logistics to facilitate such

participation. (…) When you are involved in this program you have to look at all parts of

the educational process. I am fully aware of what improved in my city, what is getting

better and what can improve in the future. (Municipal Agent)

Promoting education changes does not mean giving money to the municipalities. What

changes education is teacher training, provision of materials, follow-up of what is

happening in the classroom, and focusing on students‟ learning. The State provides

training, assistance in monitoring, but the municipality should play its role of ensuring

teacher‟s attendance and assuring the use of the structured material. If teachers are not

using the material in classroom, what are they doing instead? Are students learning? (…)

The more successful cities are, the greater their commitment is. The municipalities that

55

PAIC city manager and coordinators from schools also join these meetings. 56

Sistema de Acompanhamento das Ações do PAIC, SAAP (System for Monitoring PAIC Actions).




have participated in the training, which have kept their best teachers in the 1st and 2nd

grades, already have a good number of literacy-promoter teachers. They went forward.

They were concerned about placing teachers in the right places. Deployment is no longer

just a bureaucratic task. (State Agent)

Training and systematic monitoring are essential. Every month, our regional teams meet

with municipal teams to carry out follow–up work. And we (COPEM) meet monthly with

regional teams and bimonthly with municipal teams in order to monitor and to detect

problems and the need for visits. The municipal team never feels unattended, never feels

alone in the process. A network was built. There is a sense of collective responsibility.

(State Agent)

The training gives them (the teachers) the tools, but pedagogical support strengthens the

commitment. They become more cooperative when they are understood. (…) And then

the monitoring gives us information on how things run. (Municipal Agent)

There are other actions undertaken by the state: acquisition of children‟s literature books to

start “reading corners” for 1st and 2nd grade groups; production and distribution of children's

literature books based on local themes, aimed at linking regional culture and education;

meetings among schools and SME teams to exchange best practices and provision of training

for staff and preschool teachers. This latest action will be a great step forward in improving

preschool quality, a shortcoming already identified by research in Ceará supply57.

As a final stage, an external assessment was created to evaluate PAIC‟s achievements in

terms of literacy outcomes. Learning assessment gained its own dimension within this

educational policy and was designed as a means of refining municipal and school managers‟

accountability. Considering its relevance, this dimension will be further analyzed in the next

chapter.

In short, the target was set, several actions have been carried out and monitored and, at the

end of the process, the state evaluated how much the system progressed. Based on the

documents and on the interviews, it can be said that the state was successful in

communicating the main goal of PAIC and the need for acting and for monitoring progress in

different areas to reach it. Obviously the process does not always run smoothly.

It is expected that local teams use the monitoring reports to quickly identify which schools are

not doing well. In general, coordinators are the first to be contacted to understand the reasons.

Then, the schools are visited, and school managers and teachers receive specific guidelines

57

The CCEAE report highlights that 76% of the children who participated in the research for assessing literacy levels affirmed they had attended preschool, an index considered high by the researchers for the Ceará context and certainly meaningful for the national context (AL-CE/CCEAE p. 51). Coelho, in reporting Sobral‟s experience, states that “6-year-old children, assisted at preschool, finished the academic year without performing the first acts of reading and writing autonomously (...)” (Coelho, 2006, p. 158). The obvious inference is that if the early years of the first cycle of primary school are not successful in teaching a child to become literate, preschool education is not fulfilling its role of triggering the process either. This interpretation seems to be consistent with the studies of Maria Malta (2009), evidencing that higher accessibility to preschool education in the Northeast Region had frequently been associated to low quality education services. The issue becomes especially serious now with universal access to 4 and 5-year-old children until year 2016, as stated in the Constitutional Amendment 59/2009, and deserves attention for future studies. Preschool access has been considered by recent studies a relevant factor for success in subsequent school grades (Felício et. al., 2009; Calderini & Souza, 2009).




based on the problems identified. Considering the territorial extension – with many schools in

rural areas - the accomplishment of this assistance in situ depends on the availability of

transport. This is a permanent issue for PAIC city teams who depend on the support of

municipal leaders in SME and town halls. They depend on it to attend meetings at CREDES

and in the capital (COPEM) too. Logistical aspects are supposed to be tackled locally, however

if it is not possible due to technical or political reasons, CREDES and COPEM should be

involved in order to keep the prioritization of literacy in the agenda of the municipality.

The state agents who were interviewed consider that most municipalities have indeed

prioritized literacy and have been able to successfully implement the set of actions proposed.

Nevertheless, larger education systems face more difficulties to give pedagogical support in

situ and to monitor in a continuous way. It is also in the larger cities, such as the capital and

some CREDES headquarters, where there is a greater challenge of building a view of

partnership and non-competition among government levels.

We have 220 schools and 60 thousand students. Many are in rural areas. We have

multigraded classrooms. It is difficult to organize the logistics to make visits. We have to

group people responsible for different actions. When they go, sometimes they spend two

or three days. In rural schools everything is more difficult because the professionals are

less qualified. People do not want to work there. (Municipal Agent)

There is a historical duality between state and capital teams. It has to be with a

competition between the technical teams that are very similar and sometimes change

positions. It is a territorial issue. We got some improvements, but still have a long way.

The state schools were always the best, people from the state secretariat felt better than

the others... At CREDES the same thing happened. The regional coordinator was called

delegate in the past. There was a relationship of dependency and subordination between

the regional and the municipal teams. This is more evident in the largest cities. There,

the struggle is more evident. They used to think that everything that comes from SEDUC,

comes from the top down. But we have begun to improve this relationship. We are

improving little by little. (State Agent)

The work done in the first two grades have given rise to some discomfort among teachers

working with other groups. Another problem identified was that some municipalities were able

to get good results in SPAECE-Alfa, but 5th graders‟ learning outcomes remain below the

expected level. Upon the promotion, the 2nd grader dropped in the "real" condition of supply

and quality of municipal systems, said one municipal agent. In 2010, SEDUC considered that

the first step was taken in terms of literacy achievements and that it was time to meet the

demands to expand the program to 3rd, 4th and 5th grades. From 2011, it began to implement

PAIC +5.

Table 6 summarizes the role of the state and its municipalities in the development of PAIC

main lines of actions discussed in the last sections: organizational structure, SME

management, school management and in-service training/structured material.




Table 6 - Synthesis of PAIC main lines of actions and level of responsibilities

Topic State Municipality

organizational

structure

pays bonus to PAIC local teams identifies and selects PAIC teams

SME management makes technical assistance and

planning tools available

makes diagnosis, identifies priorities

and prepares work plan

school management suggests indicators to monitor and

make information system available;

aids in report analyses and in

shortcomings in terms of technical

and political aspects

collects indicators; analyses

reports; visits schools and helps in

solving the shortcomings identified

by the monitoring; logistics support

in-service training +

structured material

identifies options, selects, in

consultation with the cities, and

purchases the materials associated

with training sessions and other

pedagogical materials

Provides logistics to ensure

teachers‟ attendance to training

sessions and monitors the use of

materials in classrooms.

Source: Author

3.3.3 Inter-government collaborative action: what has changed?

It is important to note that cooperation between the state and its municipalities is, according to

Vieira (2010), a remarkable feature of the educational policy in Ceará, which began in the

1970s and deepened during the 1990s and 2000s, under the recent educational legislation58.

The long process towards the decentralization of education supply in the first cycle of primary

education occurred in Ceará in an intensive and dramatic manner in terms of impact on the

quality, as well as in most Northeast Region. Nevertheless, the staff interviewed reported that

the previous cooperation was more focused on the transfer of financial resources, assignment

of state personnel to work at local institutions, funding for building schools, and on keeping

school transportation. Actions aimed at improving learning outcomes and management at SME

and school level were rather feeble.

What did we have? The state used to have a good relationship with the municipal

entities. By 2006, SEDUC had a very small unit to deal with municipalities. The core of

this work was financial cooperation, personnel, cooperation for construction of schools or

for school transportation. Practically, actions oriented to support the quality of teaching

were rare. (State Agent)

In the past, the state government was much more focused on the problems of its own school

system. With PAIC, it has started to be a partner and co-responsible for improving quality in

municipal education. Moreover, the state has taken over the role of enhancer of municipalities‟

competences, a task that none of them would be able to lead.

58

The 1996 Act nº 9,394, which establishes the guidelines for national education, and the 2001 Act nº 10,172, which approves the National Plan for Education. They are norms aimed at guiding the organization and planning of educational policies in Brazil.




It was easy to get adherence of the municipalities to PAIC because of the committee's

work and the mobilization around the issue. What we realized was that the proposal was

well received by the municipalities. It was as if they were expecting the state to take over

this leadership. The pact signed by the mayors just formalized this option. (State Agent)

Another point is the systemic approach, from the induction of more effective and relevant

municipal policies to the organization of teachers' pedagogical routines. One of those

interviewed, who has worked for 35 years in the Ceará‟s educational system, stated that it is

the first time that state government is able to "get inside" the classrooms of municipal schools.

“At most, we arrived at the municipal office.” The idea of a systemic approach also appears in

the speech of municipal staff and leaders when they say that the “same opportunities” are

offered by the state to all municipalities, regardless of political bias. They add, however, that

prioritizing the issue and making the best use of the resources made available depend on local

leaders. All of them agreed that previously the intergovernmental collaborative action relied

heavily on demands of each city or political relations between mayors and the state secretary.

As we could see, the actions foreseen in PAIC to be properly implemented depend on the

prioritization of the issue on the agendas of the municipal secretary of education and the

mayor. There is consensus on this point. Rather than rely solely on the memorandum of

understanding, signed by the mayors when joining PAIC, regional and state teams work

together to demand this support whenever local teams or indicators show the need. In

addition, the state uses the external assessment as a means of reinforcing this priority, as will

be seen in Chapter 4.

These measures are associated with others, such as institutionalizing the program in state

legislation, to try to avoid the administrative discontinuity which commonly occurs after mayors‟

elections. The disruption of educational policies and programs is a widespread feature of the

education sector in Brazil. Much less present in recent years in state and federal governments

and still very frequent in municipalities where the processes are less institutionalized and

social control is weaker. PAIC successfully went through a first change in town halls in 2008

and will face another one this year 2012. The news is not only this accomplishment, but the

concern in structuring the continuity.

Lastly, the institutional arrangement that was set is also a unique point regarding the small

organizational unit that previously was in charge of articulating the two government levels. This

is another important signal to the municipalities about the priority given by the state to child´s

literacy. Moreover, this institutional arrangement will enable the continuous monitoring

planned in PAIC‟s design.

It can be said that PAIC‟s design and implementation is an attempt to respond to a set of

issues faced by most Brazilian states and municipalities: i) the difficulties resulting from the

dualism of Brazilian federalism in the educational field, in which one sought to define priorities,

but at the same time to generate shared competences that forced an interdependence among

the entities and reinforced the idea of a federation; ii) the process of turning the educational

sector into a complex realm, in which the solution to a problem demands several actions in

various fields which are interconnected and influence each other; and iii) the need to face the

issue of municipality dependence and frailty to occupy its own scope of action granted by

decentralization and effectively take responsibility over public education policy.




When taking on the coordination of this intergovernmental cooperation mechanism, Ceará

government clearly tries to induce the municipality to take over tasks that it considers relevant

for the development of the state as a whole, mainly to grant better learning levels in the

subsequent grades – these ones under state responsibility – but also with a view on sustaining

the socio-economic development that benefited this traditionally poor region more recently.

In order to achieve that, the state makes tools, systems and human resources available to

enable task execution within the municipal scope. Few Brazilian states took on the task of

liaising and coordinating educational policies along with its municipalities whereas this mission

is stated in the Constitution. This role used to be played mainly by MEC. This is good news

and if it is well applied, it can lead to better intergovernmental balance and may reduce

educational disparities.

Conclusion

In Ceará, the mobilization of different stakeholders, together with strong political and technical

leadership, was able to introduce the issue of illiteracy at school in the public policy agenda. In

the design and implementation of PAIC, the state government has taken a central role with

respect to: a) definition of focus; b) effective communication of priorities and alignment of

policies to achieve them; c) coordination of processes and actions in terms of management

and training, tasks that few municipalities have technical or financial conditions to accomplish

by themselves. The cooperation established also seeks to induce in the municipalities the

same priority that the state government has given to literacy outcomes by: a) creating a new

institutional arrangement; b) fostering more accountability through monitoring and evaluation;

c) implementing a systemic approach and a comprehensive set of actions to all municipalities,

regardless of political bias.




Introduction

This chapter explores the two kinds of external learning assessments implemented by Ceará

under PAIC. It begins with its rationale and describes its approach within the program. Then,

their main aspects are analyzed in order to understand how they became relevant instruments

for diagnosis and pedagogical interventions at school level and monitoring progress, promoting

mobilization (increasing public awareness) and accountability at system level. It also provides

a brief view on how the learning outcomes assessed are used for funding the municipal

systems and reward schools.

4.1 Rationale for external assessment on literacy achievement

As highlighted in Chapter 2, the institution of SAEB in the 1990s and later the addition of Prova

Brasil framed a national system for assessment in order to address the needs for describing

students‟ learning achievements, identifying shortcomings and informing public policies

throughout the country. In fact, it has been fulfilling these purposes over the years;

nevertheless, the centralization of this process results in at least two drawbacks: i) there is no

ownership over its results since most of the process takes place away from the municipalities;

ii) there is a long delay between fieldwork and communication/dissemination of reports.

Gradually, as learning outcomes got more attention from political actors and attracted

headlines in the media, the idea spread out was that since those evaluations were designed to

meet national system needs, they would not meet the needs of states and municipalities,

especially to reorient local policies and programs and to have impact on teacher practices and

school management. The reduction of costs and dissemination of technical knowledge in this

field are also aspects to be considered59.

This has led some states to develop their own assessment system, which in general seeks to

be complementary to the national one. Ceará is one of the pioneers in this process; it has

implemented the Ceará‟s Permanent Evaluation System for Basic Education (SPAECE) since

1992. Thus, when the state decided to implement PAIC and to give great relevance to external

assessment making it one of the structuring axes of the program, it already had a sort of

tradition and knowhow established at state level.

As seen in the previous chapter, PAIC is focused on learning achievement and on assigning

and raising accountability over it. In order to achieve this, it tries to promote changes in the

management of municipal systems of education, but also in teacher practices at classroom

level. Still in the pilot phase, the initial strategy of implementation was to build an external

assessment model which should set forth the reality at municipal and school levels and also for

each group and each student. The idea behind it is that evaluation findings should be truly

59

Expenditures on learning assessments are around 10% of PAIC total budget.




understood by managers, principals, coordinators and teachers, so that they can know

strengths and weaknesses of students and produce tailor-made interventions. It is obvious that

program designers see external assessment as a lever for change.

Under PAIC, two annual, census-based external assessments are conducted . The first one is

called PAIC Test and is carried out early in the school year with 2nd graders. It has a diagnostic

role and their results are not published. It is coordinated by SEDUC, but implemented

separately by each municipality of Ceará. The other is conducted by SEDUC only and it is a

branch of the state assessment system (SPAECE) which was expanded to incorporate the

assessment of literacy in public schools. SPAECE-Alfa, as it is known, is carried out at the end

of the school year with second graders and should provide information on the literacy process

as well.

If focus is on literacy outcomes, a serious problem arises: what should be considered an

adequate level of literacy for children in the first years of schooling? When is the time to test

them? As presented in chapter two, these are relevant questions still in the center of many

studies, however, they are not to be addressed in this paper. The important point here is that

Ceará State and its municipalities were able to reach a consensus and to set a standard about

this issue: children at the end of second grade must perform some competences in reading.

This is a very specific path in Ceará public policy since the CNE (National Council for

Education) indicates that the first three years of basic education constitute a literacy cycle,

which should be completed at eight years old (see CNE, 2010). Complying with this guideline,

the PNE states that the national target is all children become literate at age eight. Hence,

some Brazilian states, such as Rio Grande do Sul, Mato Grosso, Minas Gerais and Espírito

Santo, carried out tests in the 3rd grade.

Ceará does not ignore the guideline for a three-year literacy cycle in primary, but made a

choice of having children literate at age seven and designed its external assessment to be

adequate for second graders. Once again it seems that the context of PAIC formulation and

design helped to shape that consensus and to give it legitimacy. The Committee promoted

open sessions to discuss these concepts and the experience of Sobral is focused on literacy at

the end of the first two years of schooling too.

4.2 PAIC Test - external learning assessment as a tool for diagnosis, guidance to

pedagogical interventions and for monitoring of learning achievements.

PAIC Test resembles a hybrid assessment such as presented by Wagner (2011) in two

aspects: i) it focuses on local needs; and ii) there is a shorter timeline through the phases of

data collection, analysis and communication of reports. However, there are also differences.

The characteristic of potential "shareability" is less evident, since it was designed to cater for

the state context. It does not present the informal characterization described by Wagner for

hybrid assessment either.

The state established a thorough assessment protocol to be applied by municipalities so as to

guarantee the credibility of the assessment and reliability of its results. The fact that it is

applied to all students in a standardized pattern and carried out by an agent outside the school

approximates it to a large-scale model. So, it could be said that it is actually in between these

two models, but SEDUC identifies it as a large-scale assessment.




4.2.1 Design and implementation

PAIC Test aims to improve the quality of the literacy process providing a diagnosis of learning

achievement at disaggregated levels – city, school, group and student. The idea is to provide

information on early reading and writing skills of each student in time for the school to correct

paths and redesign pedagogical strategies for those who present deficiencies, as well as to

gradually respond in the direction of the target of all pupils‟ literacy at the end of 2nd grade.

Thus, the assessment has a formative intention and it is seen as an instrument for learning by

the government. Results are not to be published.

In order to plan this new model of assessment, SEDUC established partnership with

Universidade Federal do Ceará - UFC (Federal University of Ceará), so as to organize a

reference matrix of literacy competences and skills60 which should subsidize the design of the

assessment instrument (See Table 7). This initiative also responded to the initial obstacle that

there was no clarity in what the benchmarks should be for skills in literacy. UFC is responsible

for the three stages: developing the test items, carrying out their statistical analysis and

conducting pre-tests.

Table 7 – Reference matrix of competences and skills for children literacy (synthesis)

Domains Competences Descriptors

Alphabet acquisition

Recognize letters, differentiating them from other

graphic signs. D1 and D2

Recognize graphic conventions. D3, D4 and D5

Decode words. D6, D7, D8, D9, D10,

D11, D12, D13

Reading procedures

Find information. D14 and D15

Make inferences and/or meaning. D16

Support implications Indentify genre, function and target in a text

range. D17 and D18

Source: Manual for Reading Findings of PAIC Test.

The evaluation instrument is simple, with 20 multiple choice questions and 4 activities that

assess writing and reading skills outlined by the 18 descriptors of the reference matrix61. The

60

In 2007, UFC received technical assistance of Centro de Alfabetização, Leitura e Escrita,CEALE (Center for Literacy, Reading and Writing) , a body of the Faculdade de Educação da Universidade Federal de Minas Gerais, FE-UFMG (Federal University of Minas Gerais School of Education). 61

In 2009, PAIC Test consisted of 20 multiple choice items with a single correct alternative, which covered the acquisition of reading. There were 13 items that assessed the most basic skills, which were read aloud by the person in charge. The remaining seven items contained the comprehension skills of sentences and texts, and therefore, students were asked to read alone. Skills related to the writing system (it involves some skills for identification and recognition of the Portuguese code) were evaluated by other specific four activities: write his/her own name, some words, one sentence and a small text. (Manual de Leitura dos Resultados da Provinha PAIC, 2009, p.21)




results are computed by percentages of correct and wrong answers and do not refer to a

proficiency scale as adopted by SPAECE-Alfa and SAEB. The descriptors span from the most

basic skills such as identification of letters among other symbols (D1) to more advanced skills

such as inferring information in texts (D18).

PAIC local teams are responsible for conducting the tests in their cities, correcting them and

analyzing data, under state supervision and collaboration of experts from UFC. They were

trained to do that. Moreover, the evaluation routines were documented in a video and there is

a protocol that works as a roadmap with criteria for correction and detailed guidance for all

steps. SEDUC provides an on line information system, named SISPAIC, where the local team

processes the data collected. Reports are available immediately. The CREDES (the regional

branches of SEDUC) and COPEM are in charge of checking data consistency and doing

aggregated levels of analysis.

According to the local actors that were interviewed, this process runs smoothly. COPEM staff

confirms that the assessment axis is one of the best mastered by municipalities, although in

the past as in the present there are operational problems such as limited availability or lack of

paper, printers and computers. Due to this fact, SEDUC decided to provide one computer to

each SME to be used idealy in external assessment activities, but the agreement implies that

each level should fund its activities. It is important to highlight that local assessment teams

have been nurtured by SEDUC in view of an explicit target of PAIC.

The assessment goes smoothly. The whole team is mobilized to complete the process

quickly. When necessary, we have someone from outside SME. Last year, we called a

pedagogical coordinator with a literacy-promoter teacher profile to assist us in the test

scoring phase. (Municipal Agent)

If there is one thing that municipalities have seized is the assessment process. To put it

into the hands of municipalities was not just a matter of methodology, but a decision to

make them able to develop monitoring and evaluation. Municipalities have all kind of

difficulties: to make copies, to meet deadlines, to have people available, but they are

making continuous progress. Currently, the assessment goes without great problems. A

culture of evaluation is already part of their work. (State Agent)

4.2.2 Communication and uses of data

If PAIC Test is designed to monitor learning achievement and to impact directly on SME,

schools and teachers, then it is necessary to understand how the data collected is

communicated and used by them. That is how PAIC manages to close or at least minimize the

gap between what the assessment tools make available in terms of findings and how they are

used by stakeholders. As seen in chapter two, this issue has received very little attention in

external learning assessments.

This study did not intend to discuss whether or not there were effective changes in the

instructional practices of teachers based on PAIC Test findings. Nevertheless, this is an

interesting field that should be targeted for further investigation. The approach here is to

understand the procedures adopted in order to promote a better analysis and ownership of the

data generated. In PAIC‟s context, this trial of disseminating a better understanding of

numbers and graphs generated by SISPAIC can be divided into two phases.




The first phase is in charge of local actors. Findings are analyzed by SME teams and

discussed with the Secretary of Education and principals, as well as coordinators and

teachers. Pedagogical meetings are held to shed new light on and guide teachers‟ practice,

when there are insufficient results. The reports of each school, group and student performance

are the main material for these meetings and workshops. The focus is in the awareness of

general performance and problematic findings, but also in guiding pedagogical interventions if

specific deficiencies are identified.

SEDUC developed a user-friendly manual for guiding data analysis on PAIC‟s Tests at local

level. So it brings practical examples such as the meaning of the average of correct answers

for each item test or even what it reveals about the pupils‟ competence in accordance with the

ability of writing their own name. The manual also suggests exercises to be practiced with local

data and a set of questions to prepare its own diagnosis of learning achievement.

There was an intense training process for local teams. The 20 CREDES organized workshops

with SME managers and their teams to work on this analysis, and also visited each town to

assist them individually. Over time, both SME and school teams have become more

acquainted with the indicators, the process has become more dynamic and quick and the

teachers have started to improve their own strategies for the daily monitoring of their students‟

learning. In a parallel second phase, CREDES and COPEM also analyze the data and carry

out meetings with publishers responsible for in-service training to discuss over the topic. The

purpose is to enhance activities towards the skills that are less developed in students‟ groups,

according to SEDUC.

This is a very interesting process in Brazil and in the Northeast Region context, since one of

the criticisms made on the national assessment is the low ownership generated by its results in

local institutions. Other relevant point is that Ceará government does not expect that teachers

will be the only ones accountable for pupils‟ performance. All the main local education

managers and leaders are seen as responsible parties and should support teachers if changes

are needed. Moreover, SEDUC also plays its role in this field. Specific deficiencies in reading

or writing that arise in a systematic way and very poorly performing schools are the focus of

more technical assistance and more frequent visits organized by state and regional PAIC

teams.

More recently, as part of the capacity building strategy for shaping assessment competences

in municipalities, SEDUC has invited teachers from primary schools to take part in workshops

so as to prepare test items to assess skills and competences related to literacy. This action is

justified by SEDUC as a way to develop teachers‟ better understanding of PAIC Test and also

to induce better use of its findings. By performing these tasks it is expected that they will be

more aware of learning objectives and more capable of monitoring and assessing pupils in

class. The best items produced by them in these workshops are in fact incorporated into the

test.

Kellaghan and Greaney (2001) say that efforts to involve teachers in learning assessments

have been more frequent, and some try to involve teachers in the developing assessment

instruments and/or in scoring student responses as a strategy to impact their practices. They

acknowledge that these strategies can be useful if the proposed actions to address problems

in an education system require the dissemination of new knowledge and skills or changes in

teaching. In any case, “problems associated with changing the behavior of teachers should be

recognized” (Kellaghan and Greaney, 2001, p. 61).




Clearly, there is a good point in helping teachers to develop good assessment practices and

hopefully influencing their teaching through more effective strategies. However, considering

that one of the drawbacks of external learning assessments is the malpractice of teaching-to-

the-test, care must be taken to promote teacher‟s autonomy and to keep his/her focus broad

enough to encompass the development of all skills the CNE established to the initial phase of

schooling in Brazil:

School experiences include all the aspects of school environment: those comprising the

explicit part of the curriculum and those that also implicitly contribute to the acquisition of

socially relevant knowledge. Values, attitudes and guidance are conveyed not only by

formal knowledge, but also by means of routine, rituals, social rules, festivities, time and

organization of the educational space, learning materials and breaks, and finally by

means of all experiences offered by the school (CNE, 2010, Res. 7 of 2010, art. 9º, § 2º,

p.3). (in translation)

As the municipalities have met the challenge of implementing PAIC Test for 2nd grade, the

initiative is now spread out at the request of SME. It has been expanded gradually to the other

grades of the first cycle of primary school. In 2009, the assessment was expanded to 3rd, 4th

and 5th graders in Portuguese. In 2010, it reached literacy level of first graders and included

Math for 3rd to 5th graders.

4.3 SPAECE-Alfa - external learning assessment as a tool for monitoring achievements,

accountability and mobilization.

4.3.1 Design and implementation

By the end of each school year, Ceará carries out SPAECE-Alfa62, an external and census-

based assessment in reading for all public schools with 2nd grade groups. It provides a picture

of system performance on literacy skills after the two first years of schooling, therefore it is a

summative assessment. It considers the needs for monitoring, informing policy-makers and

raising standards. Since it provides data from state/regional/municipal levels to more

disaggregated ones (per institution, per group and per student), it is used as a feedback

parameter for the efforts made in a given school year.

Additionally, there are also other important roles this learning assessment plays in Ceará.

Perhaps, they are just as important as those mentioned above since they improve

accountability and mobilize educational actors to go on with the efforts and actions within

PAIC, as we will see ahead.

SPAECE-Alfa is designed in a more, say, „orthodox pattern‟ for large-scale assessments.

Instead of using classical test theory (CTT) like PAIC Test does, this assessment uses a more

sophisticated instrument involving the scaling of student responses and is based on item

response theory (IRT). As an “orthodox” large-scale assessment, SPAECE-Alfa uses an

62

The whole SPAECE system also comprehends learning assessment tests in a census-based approach in Portuguese and Math for 5

th and 9

th graders of primary school and 1st, 2nd and 3rd graders

of secondary school. The tests are carried out every two years, interspersed with the national assessment ones.




instrument that enables comparisons of results over the years. It focuses on municipal system

performance rather than on individual student's achievement. It is carried out for the state

government only; municipalities do not take part in data collection, processing and analysis.

Some items used by the assessment in 2009 were included in appendix V.

SPAECE-Alfa uses a continuum proficiency scale which goes from 0 to 200 points and it is

aligned with SAEB scale which goes from zero to 350 points for primary grades63. This

alignment enables comparisons between the results of state assessment and the ones

obtained from SAEB. There are two ways of interpreting learning outcomes using the scale:

the first one is linking them with levels of proficiency, and the other one is making the

connection with performance standards. From their position in the scale, a municipality or a

school knows which skills students have already developed.

4.3.2 Communication and uses

Data communication is carefully prepared. One of the features used by SEDUC to

communicate external assessment findings to all stakeholders is very simple, but with great

visual impact. Since 2007, after each evaluation round, the state map is painted in accordance

with the average performance cities have reached. Colors fade a gradient from dark green

(recommended standard) to white (illiterate) (see table 8).

Besides being a simple and quite objective tool to inform stakeholders, this measure of

comparison also seems to play a role in mobilization aspects, apparently helping to push the

process of change that the state government has planned. There are many references coming

from politicians, SME leaders and school managing teams in reports, speeches, newspaper

articles and blogs, resorting to ideas like "painting our town green" or highlighting that the city

already got its "green seal”.

The data dissemination process is very wide. Meetings are organized to inform municipal

managers, PAIC local teams, principals, teachers and mass media. SME and schools receive

a user-friendly report with SPAECE-Alfa findings about their students. The state produces

detailed reports which seek to guide managers of municipalities and schools to understand

them, to optimize the use of data available and to rethink strategies designed by SME and

schools for a more effective teaching-and-learning process.

Reports come with instructions on how to read the results. They describe the meanings of the

present average proficiency and the evolution over the years, percentage of pup ils‟

participation in the evaluation, students in each level of proficiency, general correct answers

and the individual percentage for each descriptor. They can get more details per student in the

information system64.

Both the most simplified information such as the total percentage of success in tests and the

more sophisticated one such as the results according to the standards of performance levels

are available by state, CREDES and municipalities. They enable comparisons among the

different management levels. The government does not produce ranking lists, but the press

does.

63

The scale is divided into intervals of 25 points. 64

See http://www.spaece.caedufjf.net/spaece-inst/

http://www.spaece.caedufjf.net/spaece-inst/




Table 8 - Scale for interpretation of SPAECE-Alfa

Performance Standards

No literate Literacy Incomplete Intermediate Sufficient Recommended

Proficiency

Levels

0-75 points 75-100 points 100-1250 points 125 a 150 points Over 150 points

How to

interpret

(summary)

Indicates

that the

student

has neither

learnt the

writing

system

code yet,

nor

developed

basic skills

for

reading.

Beginning of

learning process and

systematization of

basic literacy skills.

Indicates that

the student

acquires

autonomy in

reading syllable

words in (or

through)

various patterns

and begins to

understand

simple

sentences.

The student can

find information

in about 50

word- long texts;

the student can

also identify the

topic and

purpose of a

short narrative

text.

Early

development of

skills for

proficient

reading.

color in the

map

white red orange light green dark green

Source: Manual for Reading Findings of SPAECE-Alfa, 2009.

The use of league tables by the media has played an important role because it constrains

those mayors of cities with the worst outcomes, puts them under pressure and has helped to

keep the issue as a priority on the municipal policy agenda, according to the interviews. Cases

of teachers who are reluctant to be placed in schools with low results in the assessment have

already been identified by the municipalities. Nevertheless, they do not consider it as an issue.

One must admit that league tables are a way to forge accountability, but the media is only

interested in headlines. As highlighted in Brunner (2003), a social phenomenon to be treated

as news by the media needs to be pulled from the net of events in which it is produced.

Context factors are not relevant but only the results produced. This is the reason why this

approach has received criticism, as we can see in chapter two. It tends to encourage

comparisons and interpretations based only on ranking position, reducing the complexity of

academic achievement to a table of winners and losers. Maybe the most undesirable effect

could be schools focusing on what is best for their league table position and not on what is

best for their students.

The combination of the two assessments, PAIC Test and SPAECE-Alfa, allow a mini-

longitudinal look over the early literacy skills of students. They are seen as complementary by

the state. Indeed, the simplicity of PAIC Test produces rapid results that enable changes in

instruction and more effective literacy strategies. Teachers and students can benefit from their

formative role. In turn, the comparability and greater methodological rigor applied in SPAECE-




Alfa supply the means for monitoring the progress of cities over the years and for adopting

financial incentives which require extreme reliability to work well.

The state has set the level of difficulty in SPAECE-Alfa according to the improvements in

proficiency levels. The progress already achieved prevents the use of the same matrix of

Provinha Brazil, considered too easy for the students of Ceará.

Table 9 summarizes the main characteristics of the two learning assessments in terms of

theory used for designing the instrument, the approach adopted and its main purpose. It also

brings information on who carries out the assessment, the scope and the period of evaluation.

Table 9 - Synthesis of main characteristics of PAIC learning assessments

Assessment

Theory used for

instrument

design

Approach Main

purpose

Who

carries out When Basis

PAIC Test CCT formative diagnosis municipality

1st semester

of school

year

census

SPAECE-

Alfa IRT summative

monitoring ;

accountability state

end of school

year census

Source: Author. Note: Both instruments are based on the matrix of skills for children literacy in Ceará

4.3.3 Financial incentive – ICMS Sharing

Apart from being used for increasing public awareness on education issues and accountability

purposes, SPAECE-Alfa is also used for resource allocation and school rewarding. They

constitute financial incentives, intentionally designed to lead the system in the “right” direction.

In December 2007, the state government of Ceará revised the sharing of the Imposto sobre

Comércio de Mercadorias e Serviços, ICMS (a Value-Added Tax – VAT - equivalent), through

State Act nº 14.023. ICMS is the most important tax in the set of state revenues and,

consequently, a major source of revenue for municipalities65. A quarter of total ICMS is

distributed by the state to city governments through the so called Quota Share.

This share has two parts, the first one is calculated according to the city participation in the tax

generation66 (75%), and the second one (25%) is calculated based on three indexes

(education, health and quality of the environment)67. For the education sector there is the Index

of Education Quality, a mathematical equation that measures educational progress through

learning outcomes in the large scale assessments for students in grade 5 and grade 2 with a

greater weight on the latter, as shown in Figure 3.

65

The Federation of Industries of the State of Rio de Janeiro conducted a study on the fiscal management of municipalities. The study pointed to the great dependence of municipalities on transfers of federal and state funds. The vast majority of municipalities (83%) generated less than 20% of its revenue in 2010. See http://www.firjan.org.br/IFGF/ifgf_release.html 66

Valor Adicional Fiscal, VAF, (Fiscal Value Added). 67

Before the amendment, the smallest share was distributed considering population criteria and percentage of education spending over municipal revenues. There was also a percentage evenly distributed to all municipalities.

http://www.firjan.org.br/IFGF/ifgf_release.html




Figure 3 - ICMS Quota Share

Source: Author (adapted from Cartilha Cota Parte do ICMS)

The state government has established some measures to adjust the average scores of

municipalities. The index considers the extent of student participation in the test. It seeks to

curb unintended behaviors in the system, such as the willful exclusion of children with low

levels of proficiency, to raise the average scores. Cities are also penalized if they present a

high level of inequality in the performance of their students. Thus, the change promoted in

ICMS sharing has made the students participation in learning assessments an issue of great

relevance to municipal finances.

The engagement would be different without the ICMS and the School Award. The

assessment works as a lever. It's a chain. The school wants to win the award, teachers

naturally want to see their work recognized, the SME leaves the meetings convinced that

the results need to be improved to make a good impression, the mayor wants to receive

the funds. And his support is a critical point. With his support anything can happen (or

not). (Municipal Agent)

This decision seems to be crucial to make mayors more aware of children‟s illiteracy problems

and to induce practical actions at local level. ICMS Act certainly did not influence the mayors‟

decision of implementing PAIC, since it was taken after all cities started the program

implementation. Nevertheless, it is interesting to note that the participation of students in

SPAECE-Alfa has indeed increased, reaching nearly 100% in 2010. (Figure 4)

Figure 4 - Evolution of Students Participation Rates in SPAECE-Alfa from 2007 to 2010

Source: Author. Data from SPAECE-Alfa Reports

A) 18% based on education results: A.1) 12% based on 2

nd graders literacy evaluation;

A.2) 6% based on the quality index education of 5

th graders.

B) 5% based on health results. C) 2% based on quality of environment.




It is not possible to make considerations on potential equity problems derived from this

measure. More comprehensive data would be necessary to give an idea about municipalities‟

reality. However, it is interesting to notice that cities that got better and faster results at the

beginning of PAIC are the ones to receive less resources recently because the sharing

considers the index variation between the last two years. As stated by SEDUC (2011), some

small towns were able to triple its revenue from ICMS, while part of the cities with the largest

population had shrunk due to a slower index growth. Some mayors became quite dissatisfied

with this change. This is good news since small towns in Ceará are generally the poorest ones

without significant economic activity. And paradoxically, as seen previously in chapter three,

larger education systems face much more difficulties to fully implement PAIC strategies such

as in situ assistance.

In Brazil, the state of Minas Gerais adopted the most well known case of redistribution of ICMS

from the richest municipalities to the poorest ones, the so-called Robin Hood Act. However,

there are substantial differences from the Ceará initiative. The Robin Hood Act (Acts no.

13,803 of 2000 and no. 18,030 of 2009) considers a wide set of variables: ranging from

population size to cultural activities, from health to environment. In regard to education, it does

not consider learning outcomes, but aspects related to access. The priority given to the area is

also different as can be inferred from the weight dedicated to it in the formula.

4.3.4 Financial incentive – School Award

The state government also established the Prêmio Escola Nota Dez, (Mark Ten School

Award). The objective is to reward schools and to strengthen teacher efforts regarding the

children‟s literacy process. The state Act no. 14,371 of 2009 determines two conditions for the

awards: i) up to 150 schools can be awarded if they score between 8.5 and 10 on the Índice de

Desempenho Escolar – Alfabetização, IDE-Alfa (Literacy Performance Index); and ii) the

school must have a minimum of 20 students enrolled in the 2nd grade. The latter aims to induce

school clusters in rural areas, which is one of PAIC‟s targets.

The index is based on the results of SPAECE-Alfa. In the award design, the state has adopted

the same measure considered in the tax sharing. The score of each school is adjusted by the

percentage of participation of the students assessed in relation to the ones enrolled. It intends

to avoid the exclusion of children with low levels of proficiency in the tests. Furthermore, the

amount of money the school will receive takes into account the number of students assessed.

This is an attempt to have a real picture of school work and requires professionals to pay

attention to all students regardless of their strengths or weaknesses. On the other hand, it is

advisable to be aware of adopting non-explicit selection mechanisms in future enrollments.

The most interesting part is that the same number of schools (150) with lowest ratings in IDE-

Alfa also gets money from the state as a financial assistance to implement an action plan so as

to improve performance. The amount represents only half of what the „best‟ ones get per

student assessed. In both cases (awarded schools and schools supported), part of the prize

depends on increasing (or maintaining for the top ones) performance in the following year.

Another condition is that each of the top schools should „adopt‟ and give pedagogical support

to one at the bottom of the ranking during a school-year. The strategy intends to promote

cooperation among schools to disseminate best practices and to inspire more schools to

improve their learning results.




The results of this strategy are too inconsistent to be interpreted without a study of school

contexts and constraints. In 2009 56% of the schools awarded fail to maintain its index score

and in 2010 this percentage dropped to 24%. On the other hand, among the schools

supported, 72% achieved the target, at minimum score 5 in IDE-Alfa, in 2009. In 2010, this

percentage reached 90%. SEDUC says that it is hard for the best schools to keep the same

high results from one year to another. Another piece of data that seems interesting and could

be further investigated by the state is related to reducing disparities. The lower limit of IDE-

Alfa, where schools at the end of the ranking are placed, evolved from the interval (1.6 - 3.4) in

2009 to (3.5 - 5.7) in 2011, while the IDE-Alfa of top schools ranged from the interval (8.5 -

10.0) to 10.0 in the same years.

Figure 5 synthesizes the structure of the two financial incentives adopted, based on the

SPAECE-Alfa results. Tax sharing with the municipalities is based on IDE. Another index (IDE-

Alfa) is used to select schools that will be awarded or given financial aid. These schools must

be also involved in giving or receiving pedagogical support and must present an action plan to

get part of the financial resources. Both initiatives are regulated by state laws.

Figure 5 - Financial Incentives based on SPAECE-Alfa

Source: Author

Although this initiative has specific features, as noted above, proposals like this are very close

to the so-called bonus programs adopted in some parts of the United States. Bonus-pay

programs, considering either school-wide performance or teacher‟s group performance

separately, are a controversial issue in the education sector. The experiences of New York and

São Paulo, as it was seen in chapter 2, were not successful. Actually, it seems that there are

more uncertainties than certainties in this field.

In Ceará‟s case, it should be considered that some aspects can reinforce it positively. The

bonus initiative was implemented as a means of rewarding and recognizing efforts. It is also

relevant to consider that the context where the idea is implemented apparently has not been

considered as a way to push responsibility to the weakest link in the education system, the

teacher. All in all, it has motivated schools agents towards better learning outcomes so far, as

can be seen from documents and the interviews.

Moreover, the proposal is included in a structured set of actions that seek to improve learning

outcomes, and provide effective support for schools to perform them. Therefore, to add one

more aspect to Goodman‟s hypothesis, it is not only “the structure of the payment scheme”

that may make a difference in terms of effectiveness, but also the policy structure in which the

proposal is inserted. Anyway, the experience is very recent and should be carefully evaluated

by the state in the future.

Spaece Alfa and Financial

Incentives

Schools

Local Govern-

ment

Level

Tax Sharing

Awards and Financial Aid

Instruments

Index of Education

Quality (IDE)

IDE-Alfa / Pedagogical Support / Plan

Criteria Legal

Framework State Act nº

14.023 of 2007

State Act nº 14.371 of 2009




In 2011, the legislation related to the financial incentives was revised to be adjusted to the

goals of PAIC +5. The award now includes over 150 schools with better learning outcomes in

the 5th grade. The ICMS sharing now has a better balance between the results of 2nd and 5th

grades. Both regulations included changes in favour of the education systems that reach the

highest percentage of students in adequate levels of proficiency. “We want to drive the who le

system to universal learning. Now many cities present adequate averages, but have only 50%

of the students at the recommended level”, explained the coordinator of COPEM. This is good

news because Ceará shows that is able to adapt PAIC to the current stage, when the

education system has responded to the interventions planned. The state is concerned about

keeping pushing it to improve quality and to reduce disparities.

As reported by COPEM‟s coordinator, a survey conducted by the state indicated that 40% of

municipalities have already set up their own assessment systems. Many implement bonus

programs based on performance in these assessments. Municipalities do not want to wait for

the results of SPAECE-Alfa. In fact, one secretary of education interviewed stated his (and the

mayor‟s) intention of starting such initiative as a way of recognizing merit. Moreover, it is

common to have simulated testing in the cities, which aims to familiarize children with the

instrument used in SPAECE-Alfa.

Many cities have implemented the 14th salary, 15th salary for teachers and they cannot

(or do not want) to wait for the state assessment. Then they establish other

assessments. Many municipalities make their own assessments now. This is not a good

thing, it is too much, but weighing up everything, I prefer their concern for ceasing

illiteracy among their students. I prefer they cease to find it normal. In many places

children aren‟t prepared for the objective test, because they are not used to it. They do

not know what is to set the correct alternative. We do not agree with this practice, but I

understand it. Let me emphasize that I do not blame them for wanting the child to get

acquainted with the instrument. (State Agent)

Whereas the students‟ achievement is "cash value" for the municipalities, it seems that they do

not want to run the risk of receiving less money because children do not understand well the

multiple-choice test or for any other reason. States and municipalities should be aware of the

risk of overtesting practices. They are not welcome in any stage, but are especially

inappropriate in a very early stage of schooling.

Raising expectations

The state of Ceará seems to have put into practice by the book lessons about the role of

external assessment in education. The government targeted the effective literacy of all 2nd

graders, set the standards to measure it and has been monitoring students‟ performance by

way of external assessment since then. It has sought to disseminate a „culture of evaluation‟,

to communicate better the results and to make effective use of findings. These lessons are

easy to be learnt, but not so simple to be implemented in a complex federation like Brazil.

What seems to be an important step forward in this process is that both municipal

leaders/managers and school personnel raised their expectations over children‟s learning

outcomes. This has special meaning in an education system that got used to great proportions

of students with low levels of achievement as SAEB has showed since 1990‟s. This is also a




good path to start motivating all school personnel in performing better their activities, and to

make teachers more aware of students‟ needs. In this sense, the intended ownership over

external assessment findings may be interpreted as a natural consequence of it.

Another interesting point is that the external assessment is put in a central role for forging

accountability and prioritization, but it is also aligned with a set of actions as presented in the

last chapter: instructional materials, teacher training, capacity building, pedagogical support,

etc. This perspective reinforces the idea that measurement can be an important source of

change, but it is not enough. Between pros and cons, learning assessment has been used in

Ceará as a management tool to improve the quality of education. One more aspect that should

be highlighted is the action of giving further support to schools that have presented low levels

of learning achievement in the assessments. This is a relevant strategy if one considers that

one of the main objectives of intergovernmental cooperation is to reduce disparities.

On the other hand, returning to the by the book guidelines and considering the financial

incentives that were put into practice, it might be interesting to pay more attention and even to

commission studies on overtesting to ensure that the program remains on track.

Conclusion

Regarding the learning assessment within PAIC, three points stand out:

a) The state government has indeed used its results as a management tool to monitor the

education system, measuring progress and reorienting actions.

b) In order to induce this practice at local level, SME and school teams should be guided to

grasp and to use evaluation results to improve management practices and pedagogical

routines. This process is unlikely to occur spontaneously given the institutional frailties in

Brazil, and especially in the Northeast region. If the focus set by educational policy is clear and

well communicated, this process tends to be much more effective.

c) The dissemination of a “culture of monitoring and evaluation” in the education sector was

reinforced by financial incentives that seek to lead prioritization and mobilization. This

combination has proven to be virtuous and has resulted in greater accountability and political

support, which are critical issues for PAIC. It is worth noting that the system has been put

under pressure for results and should be aware of over testing practices in the long run.




This brief chapter tries to summarize some lessons learned from Ceará regarding the design

and implementation of PAIC, a recent experience of fighting illiteracy at schools.

Context analysis

The state government took into account relevant facts while formulating its policy towards

better literacy achievements at the right age, which have enhanced the success of its

implementation:

a) The mobilization generated by the Committee headed by the State Assembly represented a

source of change and made it easier to get the adherence of all mayors from 184 cities to the

proposal. It also provided the necessary legitimacy to some complex decisions, such as setting

the target of all children literate at age seven and promoting changes in ICMS sharing.

b) The state learned it from an initiative already implemented in one of its municipalities, the

city of Sobral, and perfected it, in a rare case of bottom-up policy in Brazilian federalism.

Unfortunately, most of the state legislatures perform a very weak role in setting agendas and in

discussing and contributing to public policies. The Ceará experience shows how these

institutions can have a more vigorous role in this field.

Intergovernmental collaborative action

PAIC is based on two main messages: focus on learning and accountability for results. The

program has managed to turn these messages into well-defined targets and goals and

effectively communicated them to all stakeholders and also has managed to align policies to

achieve them.

In the design and implementation of PAIC, the state government has taken a leadership with

respect to the actions implemented. The State has chosen to tackle the most common issues

for the cooperation between states and municipalities in the education sector rather than get

around them:

a) The issue of municipality dependence and frailty to occupy its own scope of action granted

by decentralization and effectively take responsibility over public education policy.

b) The difficulties resulting from the dualism of Brazilian federalism in the education sector, in

which one sought to define priorities, but at the same time to generate shared competences

that forced interdependence among the entities. It caused gaps and tensions among

government levels.




c) The growing complexity of education sector, in which the solution to a problem demands

several actions in various fields which are interconnected and influence each other.

The state of Ceará and its municipalities are overcoming these challenges through a model of

intergovernmental cooperation that distinguishes itself from past experiences, with emphasis

on:

a) creating a new institutional arrangement;

b) implementing a systemic approach and comprehensive set of actions to all municipalities,

regardless of political bias;

c) fostering more accountability through monitoring and evaluation;

d) improving technical capacities in SME management (in terms of diagnosis, planning,

monitoring and evaluation);

e) improving monitoring of relevant indicators at school level concerning the objective of having

greater levels of literacy in 2nd graders;

f) supply of in-service training focused on what happens in the classroom and on the

shortcomings of teacher‟s qualification.

Few municipalities would have technical or financial conditions to lead this process of changes.

Under the coordination and ongoing support of the state government, most of the

municipalities got motivated to implement PAIC actions and achieved great progress, as

shown by the results of the program.

And yet the game is not won. The challenge of keeping the municipal education systems

focused on student‟s learning remains. There are great difficulties in changing certain

management practices, such as the appointment of principals by political nomination. In spite

of all the information disseminated on the principal's role in the quality of education, there is

resistance in abolishing it from the system. Either in Ceará or in any other state, simple

aspects such as the full compliance with the 200-day academic calendar should not be taken

for granted. That is why monitoring is so important. Even if the in-service training and the

meetings are considered important, there is the constant issue of assuring transport for

teachers and coordinators. The issue can be even more difficult to deal with in large states

such as Amazonas and Pará or in states with lots of municipalities such as Bahia and Minas

Gerais.

External learning assessment

The experience of PAIC underlines that external and large scale learning assessments can

indeed serve to a variety of purposes: from monitoring systems‟ performance to fostering

greater levels of accountability and awareness of student´s achievements at school; from

diagnosis to funding and mobilization.

Nevertheless, to meet these objectives they should be properly designed and should have

findings communicated to the relevant audience. Ceará has been successful in these tasks: i)




it has tailored PAIC assessments according to their needs, goals, availability of human

resources and local experience; ii) it has also disseminated its results in a way that system

and school managers are able to grasp and to use them to improve management practices

and pedagogical routines. Through this strategy it overcame one of the great drawbacks of

learning assessments generated so far, the low ownership over the data by local staff.

Since the focus set by the educational policy is clear and well communicated, this process has

run smoothly.

The experience of Ceará indicates that political support from mayors and municipal education

secretariats is needed if one expects to keep literacy as a priority in local agendas in the long

run. Therefore, the dissemination of a “culture of monitoring and evaluation” in the education

sector has been reinforced by financial incentives. Tax sharing and the school awards have a

role in PAIC results as have led to more prioritization and mobilization around the issue. It is

worth noting that the system has been put under pressure for being more accountable for

learning results, but it has been the focus of a set of support actions to move forward into the

planned direction.

In Ceará, the challenge of keeping PAIC actions as a priority to the municipalities is

permanent, but deepened significantly in government elections. The threat of administrative

discontinuity is always lurking around the cities of Brazil. That is why it is so essential to

institutionalize the actions as much as possible and to get local staff ready to do their

functions. On the other hand, the adoption of financial incentives is always controversial. It

seems that these measures can be adopted with minor restrictions when there is great

legitimacy involved in the decision.

Whereas the students‟ achievement is "cash value" for the municipalities or local performance

pay initiatives start to be implemented, states and municipalities should be aware of a risk of

overtesting practices. They are not welcome in any stage, but are especially inappropriate in a

very early stage of schooling.

The results from external learning assessment have been used not only to monitor the

program, but to guide new actions when the system responded to the first interventions

planned. Nowadays the state is concerned about keeping pushing it to improve quality and to

reduce disparities in the next years. The more recent changes in tax sharing seek to ensure

that higher percentages of students reach the appropriate stage of literacy and to reduce the

relevance of average proficiencies. Clearly, there will also be new and greater challenges to

achieve better learning results at the end of the first cycle of primary schools as proposed by

PAIC+5. This is either because the approach is more complex (for instance, the inclusion of

new disciplines such as math) or because the expenditures on in-service training, monitoring

and logistics among others will increase.

Finally, Ceará shows that the assessment can be used as a tool to raise expectations about

student‟s learning, which is definitely an achievement in Brazil where education systems

became very used to low levels of performance in the last two decades.




Abrucio, F. L. 2010. “A dinâmica federativa da educação brasileira: diagnóstico e propostas de

aperfeiçoamento”. In R. P. Oliveira and Wagner Santana (ed.) Educação e federalismo

no Brasil: combater as desigualdades, garantir a diversidade. Brasília: UNESCO -

Representação Brasil. pp. 39-67

AL-CE/CCEAE (Assembléia Legislativa do Estado do Ceará/Comitê Cearense para a

Eliminação do Analfabetismo Escolar). 2006. Final Report. Fortaleza: AL-CE.

Bruner, J. J. 2003. “Límites de la lectura periodística de resultados educacionales”. In G. Iaes

et al Evaluar las evaluaciones. Una mirada política acerca de las evaluaciones de la

calidad educativa. Buenos Aires: UNESCO/Instituto Internacional de Planeamiento de la

Educación. pp. 67-80

Calderini, R. and Souza, A.P., 2009. Pré-escola no Brasil: seu impacto na qualidade da

educação fundamental. Article presented in XXXVII Encontro Nacional de Economia

(ANPEC). Retrieved in 29 February 2012, in

http://www.anpec.org.br/encontro_2009.htm#trabalhos

CEC/CD (Comissão de Educação da Câmara dos Deputados). 2003. Relatório final do grupo

de trabalho – alfabetização infantil: novos caminhos. Brasília: Câmara dos Deputados.

Coelho, I. C. de A. 2006. “Sistema Municipal para Avaliação da Alfabetização de Crianças”. In

AL-CE/CCEAE (Assembleia Legislativa do Estado do Ceará/Comitê Cearense para a

Eliminação do Analfabetismo Escolar). 2006. Final Report. p.156-163. Fortaleza: AL-CE.

Felicio, F. and others. 2009. The effects of early child education of literacy scores using data

from a new Brazilian assessment tool. Article presented in XXXVII Encontro Nacional de

Economia (ANPEC). Retrieved in 29 February 2012, in


Ferrer, A. T. 2003. “Qué pretendemos evaluar, qué evaluamos y qué conclusiones podemos

extraer de la evaluación?” In G. Iaes et al Evaluar las evaluaciones. Una mirada política

acerca de las evaluaciones de la calidad educativa. Buenos Aires: UNESCO/Instituto

Internacional de Planeamiento de la Educación. P.111-132.

Goodman, S. and Turner, L. 2011. Does whole-school performance pay improve student

learning? In Education Next. Spring. pp. 66-71. Retrieved from: www.educationnext.org

in April 12, 2012.

Haddad, W. D., 1995. Education policy-planning process: an applied framework.

Fundamentals of Educational Planning nº 51. Paris: UNESCO/International Institute for

Educational Planning.

Iaies, G. 2003. “Evaluar las evaluaciones”. In G. Iaes et al Evaluar las evaluaciones. Una

mirada política acerca de las evaluaciones de la calidad educativa. Buenos Aires:

UNESCO/Instituto Internacional de Planeamiento de la Educación. pp. 15-35

http://www.anpec.org.br/encontro2009/inscricao.on/arquivos/000-2faec48de6ce147ca713e5bc1a22e59f.docx

http://www.anpec.org.br/encontro2009/inscricao.on/arquivos/000-2faec48de6ce147ca713e5bc1a22e59f.docx



http://www.educationnext.org/




Iaies, G. et al 2003. Evaluar las evaluaciones: una mirada política acerca de las evaluaciones

de la calidad educativa. Buenos Aires: UNESCO/Instituto Internacional de Planeamiento

de la Educación.

INEP (Instituto Nacional de Estudos e Pesquisas Educacionais). 2003. Qualidade da

Educação: uma nova leitura do desempenho dos estudantes da 4ª série do ensino

fundamental. Brasília: MEC/INEP.

_________. 2005. Vencendo o desafio da aprendizagem nas séries iniciais: a experiência de

Sobral/CE. Série Projeto Boas Práticas na Educação nº1. Brasília: INEP.

IPECE (Instituto de Pesquisa e Estratégia Econômica do Ceará). 2010. Ceará em Números

2010. Retrieved in 12.02.2012, in www.ipece.ce.gov.br/

Kellaghan, T. 2008. “Qué mecanismos de monitoreo se pueden utilizar para efectuar estudios

internacionales comparados y estudios nacionales?” In Kenneth N. Ross & Ilona J.

Genevois (ed.). Estudios internacionales sobre la calidad de la educación: La

planificación de su diseño y la gestión de su impacto. Paris: UNESCO/ Instituto

Internacional de Planeamiento de la Educación. (pp. 54-69)

Kellaghan, T. and Greaney, V. 2001. Using assessment to improve the quality of education.

Fundamentals of Educational Planning nº 71. Paris: UNESCO/International Institute for


Lockheed, M. E. 1996. “International Context for Assessments”. In Paud Murphy et al

(ed.).National assessments: testing the system. Washington D.C.: Economic

Development Institute of the World Bank (pp. 9-19).

Marsh, J. and others (2011). A Big Apple for Educators - New York City's Experiment with

Schoolwide Performance Bonuses: Final Evaluation Report. New York: Rand

Corporation and Vanderbilt University.

Ministério da Educação, Brasil (MEC). 1997. Parâmetros Curriculares Nacionais: Língua

Portuguesa. Brasília: MEC

Oliveira, J.B.A. n.d. Alfabetização, fracasso escolar e políticas públicas. Instituto Alfa e Beto.

April 11, 2012 retrieved from:

http://www.alfaebeto.org.br/Arquivos/Documentos/Artigos/Cap%C3%ADtulo%207.%20J

o%C3%A3o%20Batista%20Ara%C3%BAjo%20e%20Oliveira.pdf

Oliveira, J.B.A. 2004. Alfabetização de crianças e adultos: novos parâmetros. Belo Horizonte:

Alfa Educativa.

Pigozzi, M.J. 2008. “Que és la „calidad de la educación‟?(Desde la perspectiva de la

UNESCO)”. In Kenneth N. Ross & Ilona J. Genevois (ed.). Estudios internacionales

sobre la calidad de la educación: La planificación de su diseño y la gestión de su

impacto. Paris: UNESCO/ Instituto Internacional de Planeamiento de la Educación. (pp.

41-53)

http://www.ipece.ce.gov.br/

http://www.alfaebeto.org.br/Arquivos/Documentos/Artigos/Cap%C3%ADtulo%207.%20Jo%C3%A3o%20Batista%20Ara%C3%BAjo%20e%20Oliveira.pdf

http://www.alfaebeto.org.br/Arquivos/Documentos/Artigos/Cap%C3%ADtulo%207.%20Jo%C3%A3o%20Batista%20Ara%C3%BAjo%20e%20Oliveira.pdf




Postlethwaite, T. N. 2004(a). What do international assessment studies tell us about the quality

of school systems? Background paper prepared for the Education for All Global

Monitoring Report 2005, The Quality Imperative. Paris: UNESCO.

Postlethwaite, T. N. 2004(b). Monitoring educational achievement. Fundamentals of

educational planning n. 81. Paris: UNESCO/International Institute for Educational

Planning.

Postlethwaite, T.N. and Kellaghan, T. 2008. National assessments of educational

achievement. Education Policy Series n. 9. Paris: UNESCO/International Institute for


Saus, J. B. 2003. “Encuentros y desencuentros con los procesos de evaluación de la calidad

educativa en América Latina”. In G. Iaes et al Evaluar las evaluaciones: una mirada

política acerca de las evaluaciones de la calidad educativa. Buenos Aires:

UNESCO/Instituto Internacional de Planeamiento de la Educación. (pp.37-65)

SEDUC (Secretaria de Educação do Estado do Ceará). 2011. Regime de Colaboração para a

garantia do direito à aprendizagem: o Programa Alfabetização na Idade Certa (PAIC) no

Ceará. Fortaleza: Governo do Estado do Ceará. Mimeo.

Silva, C. S. R. and Cafiero, D. 2010. “Alfabetização e letramento de crianças de seis anos: as

políticas para a implantação do ensino fundamental de nove anos”. In InterMeio. Revista

do Programa de Pós-Graduação em Educação, v.16, n.32, p.27-48, jul./dez. 2010.

Campo Grande: Universidade Federal do Mato Grosso do Sul.

Soares, M. n.d. Por uma alfabetização até os oito anos de idade. Todos pela Educação. April

2, 2012 retrieved from http://www.todospelaeducacao.org.br/biblioteca/1387/por-uma-

alfabetizacao-ate-os-oito-anos-de-idade/

Todos pela Educação. 2007. Nota Técnica Preliminar Metodologia para a Obtenção das

Metas Finais e Parciais. Comissão Técnica do Compromisso Todos pela Educação. São

Paulo, 10 de setembro de 2007.

UNESCO (United Nations Educational Scientific and Cultural Organization). 2005. Education

for all: the quality imperative. Global Monitoring Report. Paris: UNESCO.

_______. 2006. Education for all: literacy for life. Global Monitoring Report. Paris: UNESCO.

Vieira, Sofia L. 2010. “Educação básica no Ceará: construindo um pacto colaborativo”. In R. P.

Oliveira and Wagner Santana (ed.) Educação e federalismo no Brasil: combater as

desigualdades, garantir a diversidade. Brasília: UNESCO - Representação Brasil. pp

271-286

Wagner, D. 2011. Smaller, quicker and cheaper: improving learning assessments for

developing countries. Paris: UNESCO/International Institute for Educational Planning.

http://www.todospelaeducacao.org.br/biblioteca/1387/por-uma-alfabetizacao-ate-os-oito-anos-de-idade/

http://www.todospelaeducacao.org.br/biblioteca/1387/por-uma-alfabetizacao-ate-os-oito-anos-de-idade/




Act no 9.394 of 1996: establishes the basis and guidelines of national education.

Federal Constitution

CNE (National Education Council). 2010. Regulation no 7 of 14th December 2010:

establishes national curriculum guidelines of the 9-year primary education.

Decree no. 29,896 of 16/09/2009: regulates the Act no. 14,371/2009.

Decree no. 30,796 of 29/12/2011: makes amendments to Decree no. 29,881/2009 in

order to improve the calculations of the Municipal Index of

Educational Quality.

Decree no. 30,797, of 29/12/2011: regulates Act no. 15.052, of 06/12/2011.

State Act no. 14,026, of 17/12/2007: institutes PAIC – The Literacy Program at the

Right Age. State Act no. 14,023 of 2007: distributes ICMS resources, according to the

external assessment results.

State Act no. 14,371, of 19/06/2009: institutes the Mark Ten Award.

Act no. 14,580, of 21/12/2009: make amendments to Act 14,371/2009.

Act no. 14,949, of 27/06/2011: make amendments to Act 14,371/2009, including the

award to the 5th grade of primary school.

Act no. 15,052, of 06/12/2011: intitutes the Mark Ten Award to the Best 2nd and 5th

grade schools.

Boletim do Sistema de Avaliação SPAECE – Volume I

Boletim do Sistema de Avaliação SPAECE – Língua Portuguesa – 5º ano – Volume II

Boletim de Resultados da Escola – SPAECE-Alfa 2010 – Língua Portuguesa – 2º ano do Ensino Fundamental – Volume III

Boletim de Resultados da Escola – SPAECE-Alfa 2009 – Língua Portuguesa – 2º Ano do Ensino Fundamental – Volume III

Boletim de Resultados Gerais – SPAECE-Alfa 2009 – Língua Portuguesa – 2º Ano do Ensino Fundamental – Volume II

Cartilha Cota Parte do ICMS – Governo do Estado do Ceará e IPECE

Manual de Leitura dos Resultados – Provinha PAIC 2º Ano – 2009 – Eixo Avaliação Externa (SEDUC e Universidade Federal do Ceará)

Manual de Orientação para o Acompanhamento das Ações do PAIC – Regional – 2009

Manual de Orientação para o Acompanhamento das Ações do PAIC – Município – 2009

Matriz de Referência da Alfabetização

Relatório Final do CCEAE

Relatório Geral de Resultados do Estado do Ceará – Atividade Avaliativa PAIC-Alfa 1º ano – 2010.

Relatório Geral de Resultados do Estado do Ceará – Provinha PAIC 3º Ano e Provinha PAIC 4º e 5º Anos - 2010

Oficina de elaboração de itens – Material de apoio didático – 2009

Orientações Curriculares para a Educação Infantil – Secretaria de Educação do Estado do Ceará – 2011




Orientações para elaboração do plano de aplicação dos recursos referentes às escolas premiadas e apoiadas, para o exercício financeiro de 2011,segundo a Lei nº. 14.371, de 19 de junho de 2009.

Notas taquigráficas de audiência pública realizada, em 08/06/2011, pela Comissão

Especial do Plano Nacional de Educação (Projeto de Lei nº 8035 de 2010) na Câmara

dos Deputados, com a presença da Secretária de Educação do Ceará, Izolda Coelho,

e o governador do Estado do Ceará, Cid Gomes.

Notas taquigráficas de audiência pública, em 07/04/2010, realizada pela Comissão de

Educação, Cultura e Desporto do Senado Federal, com a presença da Secretária de

Educação do Ceará, Izolda Coelho.


http://www.idadecerta.seduc.ce.gov.br/

http://www.inep.gov.br

http://www.ibge.gov.br


http://www.idadecerta.seduc.ce.gov.br/

http://www.inep.gov.br/




I) Brazilian Education System – Structure and Organization

A) Correspondence between the International Standard Classification of Education (ISCED 97)

and Brazilian educational levels.

ISCED 97 levels Brazilian educational levels

ISCED 0

(Pre-primary level of education)

Crèche: 0 up to 3 years old

Pre-school: age 4 and 5

ISCED 1

(Primary level of education)

Ensino Fundamental, Fundamental (or Primary)

Education

(1st to 5

th grade)

(age 6 to 10 years)

ISCED 2

(Lower secondary level of education)

Ensino Fundamental, Fundamental (or Primary)

Education

(6th to 9

th grade)

(age 11 to 14 years)

ISCED 3

(Upper secondary level of education)

Ensino Médio, Secondary Education

(1st to 3

rd grade)

(age 15 to 17 years)

ISCED 4

(Post-secondary non-tertiary)

Not applicable to Brazil

ISCED 5

(First stage of tertiary education)

Tertiary education Type B (Technological Higher

Education)

Tertiary education Type A

(undergraduate, master‟s degree and professional

master‟s degree courses)

Source: Based on information available in “Education at a Glance 2011 OCDE Country Note - Brazil”.

Retrieved in July 21, 2012 from: http://www.oecd.org/dataoecd/44/63/48657313.pdf

B) All members of the Brazilian federation held education responsibilities. The Union is

responsible for the funding and the management of the federal educational system, mainly

higher education institutions; states must provide secondary education and early childhood

education (crèche and pre-school) is in charge of municipalities. States and Municipalities

share responsibilities over primary education.

http://www.oecd.org/dataoecd/44/63/48657313.pdf




II) Efficiency indicators for primary and secondary levels in Ceará, Northeast Region and Brazil

Levels/Rates Primary - 1º/5º Primary - 6º/9º Secondary

Overaged Rate*

2001 2010 2001 2010 2001 2010

Ceará 46,5% 21,0% 73,7% 31,0% 60,2% 34,7%

Northeast 59,8% 26,6% 66,6% 40,4% 67,4% 46,6%

Brazil 39,4% 18,5% 45,7% 29,6% 50,0% 34,5%

Drop out Rate

1995 2005 1995 2005 1995 2005

Ceará 6,8% 6,9% 11,8% 18,5% 1,4% 2,6%

Northeast 12% 11,4% 14,1% 15,0% 3,5% 3,3%

Brazil 9% 7,4% 13,1% 14,1% 2,8% 3,4%

Promotion Rate

1995 2005 1995 2005 1995 2005

Ceará 73,5% 76,6% 73,5% 64,1% 89,2% 82,3%

Northeast 63,4% 68,7% 64,5% 66,3% 82,6% 80,5%

Brazil 71,9% 78,3% 69,0% 68,5% 83,7% 81,8%

Repetition Rate

1995 2005 1995 2005 1995 2005

Ceará 19,7% 16,5% 14,7% 17,4% 9,4% 15,1%

Northeast 24,6% 19,9% 21,4% 18,7% 13,9% 16,2%

Brazil 19,1% 14,3% 17,9% 17,4% 13,5% 14,8%

Note: *Expresses % of students at least two years more than the recommended age.

Source: Todos pela Educação/INEP/MEC




III) Institutional arrangement for COPEM in Ceará State Secretariat for Basic Education

(SEDUC)




IV) Objectives and targets of PAIC’s five acting axes

Management Axis

Objective: to promote institutional strengthening of municipal teaching systems by providing

technical assistance for the setting up of a management model focused on learning results.

Targets:

1. Raising IDEB to 6.0 in the early years of Primary Education.

2. Assistance for 100% of children in the 6 – 14 age group.

3. Reduction of the dropout rate to 0% in the first cycle of Primary Education.

4. Reduction of the dropout rate to 0% in the second cycle of Primary Education.

5. Correction of school flow.

6. Creation of school clusters.

7. Compliance of 100% of the 200 days of the school calendar.

8. Strengthening of school autonomy.

9. Policy making on assistance to municipal schools.

10. Setting up technical criteria for the selection process of school managers prioritizing

merit.

Literacy Axis

Objective: to provide pedagogical and technical assistance to municipalities in setting up

didactic literacy proposals which guarantee effective children‟s literacy.

Targets:

1. Literacy of 100% of children up to the end of the 2nd grade of Primary Education.

2. Literacy of pupils who have not achieved full literacy between the 3rd and 9th grades of

Primary Education.

Preschool Education Axis

Objectives: To contribute to improve quality in the service offered by Preschool Institutions and

to the implementation of both pedagogical proposals and in-service training of Preschool

teachers.

Targets:

1. Broaden the offer of preschool places to children aged 0 to 3;

2. Have all children between 4 and 5 years old enrolled in preschool.

Children Literature and Competent Reader Training Axis

Objective: To fulfill both children‟s and teachers‟ right to human development, to cultural

instruction and to social inclusion through access to literature, promoting acquisition,

distribution and effective use of book collections, as well as permanent in-service training,

focusing on the importance of Children Literature during the full literacy process.




Targets:

1. 100% of children in Ceará‟s municipalities with access to quality book collections and

having read a minimum of 5 (five) children‟s books per year, from the 2008 school

calendar onwards.

2. All educators working in preschool and in the early years of Primary school have

attended a Continuing and Permanent Training Program of at least 120 hours per year,

from the 2008 school calendar onwards.

3. All teachers have participated in the Effective Children Literature Workshops and have

acquired the Effective Children Literature Kit.

4. Creation of a reading collection in the classrooms, with a minimum of two books per

children, from the 2008 school calendar onwards.

5. All pupils mastering reading fluency and comprehension, checked by regular

assessment.

Assessment Axis

Objectives: i) To disseminate an educational assessment culture in Ceará‟s municipalities, so

that they have a team in the Municipal Education Secretariat. ii) To diagnose reading, writing

and text comprehension learning status of pupils in the early grades of all municipal schools,

communicating assessment results by municipality, school, class and student. iii) To provide

subsidies so that the Education Secretariats develop a management focused on students‟

learning and implement policies to improve the quality of the teaching offered in the early

grades.

Targets:

1. Carrying out diagnostic assessments with children enrolled in the early grades of

primary school (2nd to 5th grades) in all Ceará‟s municipalities at least once a year.

2. Availability of an information system to input external assessment data collected by the

municipalities.

3. Publishing assessment results to directors, supervisors and teachers, aiming to

improve the quality of education in the early years of primary education.

4. Ownership of the assessment results by the Municipal Education Secretariats through

the various teams in charge of management, assessment, pedagogical management,

preschool education and children‟s literature, in order to suggest actions that aim to

improve pupils‟ learning process.

5. Use of external assessment results by the municipalities for local educational policy

making.

6. Setting up of an external assessment team in the Municipal Education Secretariat.




V) Items from SPAECE-Alfa 2009

A) Item P020170A9 - skill assessed: letter identification

Tick the box containing the two letters I am going to say appear:

The letters are: D Z.

% Scores: A = 1.2% B = 1.4% C = 93.5%

D = 2.0% Blank and null = 1.9%

B) Item P020194A9 – skill assessed: writing direction identification

Tick the box containing the last word of this verse:

How can a living fish live out of cold water?

% Scores:

A = 4.7%

B = 4.5%

C = 77.1%

D = 12.2%

Blank and null = 1.5%

G S

O C

D Z

T J

HOW

COLD

WATER

FISH




C) Item AL0163MG – skill assessed: different letter spelling recognition.

Read the words: BARATA - TOMADA

Tick the box containing the words you have just read.

% Scores: A = 8.2% B = 4.9% C = 81.2%

D = 2.9% Blank and null = 2.8%

D) Item D15N3 – skill assessed: sentence comprehension.

Tick the sign indicating two products for sale.

balada - topada

banana - tocada

barata - tomada

batata - torrada

ROAST CHICKEN

FOR SALE

FARM CHICKEN

FOR SALE

CHICKEN AND EGGS

FOR SALE

FREE RANGE CHICKEN

FOR SALE




% Scores: A = 10.2% B = 10.2% C = 69.2%

D = 7.5% Blank and null = 2.9% E) Item PA2.0032 – skill assessed: explicit information finding in texts.

Read the text:

Tick the box that shows what Goldilocks found.

% Scores: A = 72.9%

B = 11.2% C = 9.2% D = 4.6% Blank and null = 2.1%

Source: Boletim de Resultados da Escola – SPAECE-Alfa 2009 Volume III – Alfabetização 2º

ano do ensino fundamental.

Note: Free translation by the author.

Once upon a time there were three bears living in a cottage in the woods. Papa Bear, Mama Bear and Baby Bear. They loved oatmeal. In the same forest also lived a girl named Goldilocks because of her long blond hair, which she loved to comb. She loved to find new places and one day she found the cottage of three

bears.

THE COTTAGE OF THREE BEARS

THE WOODS OF THREE BEARS

OATMEAL

BABY BEAR

Towards better literacy achievements - UNESCO · To Mariano Palamidessi, Paula Scalliter and...

Documents

Transcript of Towards better literacy achievements - UNESCO · To Mariano Palamidessi, Paula Scalliter and...