Click here to load reader

  • date post

  • Category


  • view

  • download


Embed Size (px)


Policy Frameworks

Policy Analysis and Decision Makingwith Emphasis on Chronic Non-communicable DiseasesCristina Puentes-MarkidesLesson 4: Instruments of Public Accountability and Policy Change1datePublic accountability (Functions of monitoring and evaluation)Policy changeAgenda settingAdvocacyPolicy learningKey Points2CPM-20122dateGenerally, public health policies are evaluated through health programs and plans.National/local health plans need to include the corresponding programs (with the appropriate goals, objectives, measures and targets) that address the priority NCDs.The products of evaluation are useful only when they inform and guide decision making, and when they complement monitoring activities. Monitoring and evaluation activities within established performance frameworks of production and reporting contribute to public accountability We can all participate in modifying policies through various means.Key messages3CPM-2012NCDs are addressed through a variety of policies: broad health policies, public health and health promotion policies, health services policies, issue-specific policies (dealing with specific diseases or health risks), etc.

3dateAccountability is defined as an obligation or willingness to accept responsibility or to account for one's actions. (Merriam Webster Dictionary)Therefore, being accountable means that one is required to explain actions or decisions to others. What is accountability?4CPM-20124dateMonitoring and Evaluation5dateMonitoring and evaluation are no longer simple measurement exercises.Increasing requirements for modern societies: transparency and accountability of governments, demands by better informed and organized citizens.Media and communication/information technologies have contributed to increase the publics expectations, channel their opinions and demands.Increasing Importance of M & E6CPM-20126dateTimely and reliable monitoring and evaluation can contribute to the identification of areas of improvement and thus to better quality and results. Reporting that informs decision-making and management activities improves performance. Accountability and compliance are also enhanced (demonstrates whether the duties and responsibilities have been conducted according to standards & procedures established by legislation, regulatory agencies and/or professional associations).M & E can explain why the results and outcomes of a particular policy or program are what they are, why they differ between groups or localities. Benefits of monitoring and evaluation7Is the policy change effected having the impact it was meant to have?Have there been any unintended consequences of the policy which detract from its overall effectiveness?Could the policy be improved further?Were the costs (financial, political, etc.) spent on the change worth the resulting benefits?What lessons were learned in effecting the change that could be used to improve the effectiveness of future policy advocacy efforts?If you are interested in the topic, I suggest you download the following document: U.S. Department of Health and Human Services Centers for Disease Control and Prevention. Office of the Director, Office of Strategy and Innovation. Introduction to program evaluation for public health programs: A self-study guide. Atlanta, GA: Centers for Disease Control and Prevention, 2011. that registers changes in the key variables after implementation of a particular policy or program; documents whether there were changes due to the policy or program. Requires the identification of variables, targets and indicators to measure changes, and the means to assess whether the process is free of bias (e.g. related to the design, the process, influence of those who support or those that are against that particular policy/program, etc.)Monitoring8CPM-20128dateEvaluation is concerned with establishing the value premises necessary to produce information about the performance of policies. evaluation refers to the production of information about the value or worth of policy outcomes. (Dunn, W. 1994. Public Policy Analysis. 2nd Ed).The purpose of evaluation is to measure and explain the effects of a program against the goals it set out to accomplish, providing information for later decisions about the program Evaluation9Program evaluation is a subset of policy studies, and a such, of great importance.9There are many definitions of evaluation:evaluation is an activity that involves the systematic application of theory, methods and tools from the social sciences to identify and assess the processes and impacts of public policies and programs. It can be retrospective or prospective, secret or public, by government organizations or by others who are not associated to the government. Pollitt C., 1993the systematic collection of information about the activities, characteristics, and outcomes of programs to make judgments about the program, improve program effectiveness, and/or inform decisions about future programming Patton, M. Q.1996."Evaluation is the systematic assessment of the operation and/or the outcomes of a program or policy, compared to a set of explicit or implicit standards, as a means of contributing to the improvement of the program or policy." Weiss, C. 1998.conceive, obtain and provide information which decision makers in their various forms (voters, opinion-leaders, stakeholders, policy-makers, social planners or administrators) can use to make decisions about the future of specified programs or policies. Norris (1990)Evaluation research is the systematic application of social research procedures in assessing the conceptualization and design, implementation, and utility of social intervention programs. Rossi and Freeman( 1982)dateRecognize the moral debate and that everyone has the right to their legitimate opinions.Recognize that there is power, lack of power and empowerment.Develop a genuine dialogue between organizations and users.Promote openness to criticism, complaints, questions.Enable and promote that groups give their opinion about the program/policy.Good Practices of Evaluation (Everitt, 1996)10CPM-2012Democratic and honest processes of evaluation are essential to ensure good and fair practices.10dateEvaluation, as the application of social sciences to human problems is paved with decisions. (Weiss).Policy denotes relationships of power and influence and some type of resource allocation by those individuals/entities that have authority over those resources.Some stakeholders and relevant actors have particular interests about what type of evaluative data/information they are interested in and want to be public.Yet, evaluation is also political11See Weiss, C. H. Where politics and evaluation research meet. In: D. J. Palumbo. Ed. The politics of program evaluation. Newbury Park: Sage. 1987.11date12Benefits and Possible Constraints of EvaluationBenefitsConstraintsInforms about policy performance, and the goods and services received by he beneficiaries.Inadequate levels of human and financial resources assigned to evaluation activities. Contributes to clarify underlying values in the selection of alternatives, goals and objectives.Personnel with evaluation skills may not be available.Results can be useful inputs for future policy analyses or for the design of new programs. The objectives and targets for certain programs are difficult to measure.The actual change in behavior and obtained results reflect the policy impact. Measuring the impact of certain interventions may result to be challenging and costly.The use of evaluation results in decision-making is limited.Evaluation is integrated into every phase of the health policy development cycle to guide decision making and improve effectiveness. Evaluation is of limited use if it only provides feedback after the fact. For optimal use, evaluation is therefore integrated into the quality management processes which support the health policy development cycle. Assessing the Effects of Public Health Policies and Actions. Evaluation Strategic Framework. Federal Office of Public Health, Switzerland. December 2005. Another excellent resource is GAO. Designing Evaluations. 2012 Revision. Government Accounting Office, GAO-12-208G.

12date13Major Types of EvaluationQuestions that the evaluation type seeks to answerFormative Evaluation: Aims at continuous program improvement; collects different types of information about program activities outputs client satisfaction obstacles to implementation. May include needs assessments, evaluability assessments, implementation evaluation & process evaluations. Asks what aspects of our situation most shaped our ability to do the work we set out to do in our community? What did out program accomplish in our community.Summative Evaluation: Seeks to learn whether the policy program or intervention actually works focus on demonstrated program outcomes and impacts only. Monitoring is a concurrent activity. Sometimes they are also called "impact" evaluations but can also include outcome evaluations, CEA & CBA secondary and meta-analysis. What is out assessment of what resulted from our work in the community? What have we learned about doing this kind of work in a community like ours?Impact Evaluation: A form of outcome evaluation. It focuses on the program outcomes and impacts, assesses the net effect of a program by comparing program outcomes with an estimate of what would have happened in the absence of the program. IE is employed when external factors are known to influence the programs outcomes in order to isolate the programs contribution to achievement of its objectives. Seeks to answer whether the program worked or did it have its intended effects? If so who was helped and what activities or characteristics of the program created the impact? Did the program have any un