CMR5603 07 Ebrahim 118. Files/CMR5603_07... · by philanthropic foundations over three decades...

What Impact?A FRAMEWORK FOR MEASURING THESCALE AND SCOPE OF SOCIALPERFORMANCE

Alnoor EbrahimV. Kasturi Rangan

Organizations with social missions, such as nonprofits and social enterprises, are under growing pressure todemonstrate their impacts on pressing societal problems such as global poverty. This article draws on severalcases to build a performance assessment framework premised on an organization’s operational mission, scale,and scope. Not all organizations should measure their long-term impact, defined as lasting changes in the livesof people and their societies. Rather, some organizations would be better off measuring shorter-term outputsor individual outcomes. Funders such as foundations and impact investors are better positioned to measuresystemic impacts. (Keywords: Performance Measurement, Scale, Scope, Mission, Nonprofit, Social Enterprise)

The world of nonprofit organizations, philanthropy, and social enter-prise has been preoccupied with two powerful mantras in recentyears. Since the early 1990s, the refrain of “accountability” has beenascendant, with funders, taxpayers, concerned citizens, and clientsdemanding that nonprofits be more transparent about their fundraising andspending, how they are governed, and what they have achieved with the resour-ces entrusted to them.1 A more recent manifestation of this discourse has centeredon the mantra of “impact,” or demonstrating results in addressing complex socialproblems such as poverty and inequality.2 This attention to impact, following onthe heels of accountability, is mainly driven by funders who want to knowwhether their funds are making a difference or might be better spent elsewhere.It is also driven by an increasing professionalization of the sector, which has ledto the emergence of common administrative norms including the use of creden-tialed experts such as auditors and evaluators.3

In this article, we identify key developments around impact measurementand their implications for performance management in social sector organizations.By social sector, we refer to organizations driven primarily by a social purpose,

The authors are grateful to the editor and three reviewers for their insightful suggestions, and for thesupport of the Division of Research and Faculty Development at Harvard Business School.

118 UNIVERSITY OF CALIFORNIA, BERKELEY VOL. 56, NO. 3 SPRING 2014 CMR.BERKELEY.EDU

such as nonprofit or nongovernmental organiza-tions and social enterprises. We offer a frameworkfor determining which kinds of measures areappropriate, as driven by the mission and goalsof the organization. Our normative argument isthat it is not feasible, or even desirable, for allorganizations to develop metrics at all levels of aresults chain, from immediate outputs to long-

term societal impacts. The more important challenge is one of alignment: design-ing metrics and measurement systems to support the achievement of well-definedmission objectives. Measurement efforts extending beyond this frame are a dis-traction from the operational work of the organization. Often, it is the funder,who sits at a higher level in the social sector ecosystem, that will have a broaderand more integrative perspective on how the work of several implementingorganizations fits together to advance systemic goals. Such impact is better mea-sured at the funder level, leaving the individual organizations to do what theydo best—focus on their more specific missions.

The Current State of Social Performance Measurement

Much of the literature on the topic of performance in the social sector isunder-theorized and in need of conceptual framing. As a result, the bulk of thematerial we cite below is not from academic journals; instead, it is taken directlyfrom practice—from influential think tanks, government agencies, funding organ-izations, and prominent consulting firms that are shaping the practice of socialperformance. The academic literature in nonprofit studies, philanthropy, andmanagement lags behind in providing theoretical and analytical insights to thisburgeoning field (see Appendix 1 for a summary of the diverse approaches tosocial performance measurement). Second, because of the vastness and diversityof the social sector, we focus our discussion on organizations working broadly inthe field of human services, public benefit, and poverty alleviation. In 2010, therewere roughly 300,000 such organizations in the United States alone, accountingfor roughly $500 billion in revenues. We leave out a discussion of organizationsprimarily engaged in advocacy (such as human rights and environmental policyorganizations) or the arts and culture (such as museums, symphonies, and dancecompanies), where we believe the measurement of performance is even morecomplex and nuanced.

The most widely advocated set of approaches to social performance mea-surement involve an assessment of impacts or results, which are broadly labeledas “impact evaluation” and “outcome measurement.” Such assessments are oftenconducted after program implementation is complete and are driven primarily byfunders such as foundations and governments: Has the program reduced poverty?Has it improved health outcomes? Has it placed trainees in permanent jobs, andthereby increased their lifetime earnings potential? They involve a range of meth-odologies drawing on benefit-cost analyses, summative evaluation, and experi-mental methods such as randomized control trials.

Alnoor Ebrahim is an Associate Professorin the Social Enterprise Initiative at HarvardBusiness School.

V. Kasturi Rangan is Co-chair of the SocialEnterprise Initiative at Harvard BusinessSchool, where he is the Malcolm P.McNair Professor of Marketing.

What Impact? A Framework for Measuring the Scale and Scope of Social Performance

CALIFORNIA MANAGEMENT REVIEW VOL. 56, NO. 3 SPRING 2014 CMR.BERKELEY.EDU 119

The term “impact” has become part of the everyday lexicon of social sectorfunders in recent years, with frequent references to “high-impact nonprofits” or“impact philanthropy” and “impact on steroids.”4 But the term impact has notbeen consistently defined. An established literature in international developmentand evaluation often uses the term to refer to “significant or lasting changes inpeople’s lives, brought about by a given action or series of actions.”5 Morerecently, impact has also come to be associated with results that target the “rootcauses” of a social problem.6 Others use impact more narrowly to refer to anorganization’s specific and measurable role in affecting a social result (attribution)requiring a counterfactual for assessment.7 In this article, we distinguish betweenoutcomes and impacts, with the former referring to lasting changes in the lives ofindividuals and the latter to lasting results achieved at a community or societallevel. Attribution remains a challenge regardless of whether one is assessing out-comes or impacts.

Many frameworks for measuring social performance employ a “resultschain” or “logic model,” which has its roots in the evaluation of programs andprojects, and was originally developed for the United States Agency for Interna-tional Development (USAID) in the late 1960s.8 Table 1 shows the key compo-nents of the basic logic model—inputs, activities, outputs, outcomes, andimpacts—and includes examples of the types of measures typically included undereach step. Through their adoption and dissemination by program evaluation spe-cialists, logic models have emerged as a primary means through which social sec-tor organizations identify impacts and other performance metrics.

Funding organizations—from philanthropic foundations and governmentalagencies to impact investors—increasingly expect the organizations they supportto measure their outcomes and impacts. However, the evidence on whether out-come measurement has led to improved performance is mixed. A study of thirtyleading U.S. nonprofits found that measurement was useful to the organizationsfor improving outcomes, particularly when they: set measurable goals linked tomission (rather than trying to measure mission directly); kept measures simpleand easy to communicate; and selected measures that created a culture ofaccountability and common purpose in the organization, thus helping to alignthe work of disparate units and chapters.9 The United Way of America, whichwas one of the first national agencies to ask members of its network to distinguishbetween outputs and outcomes, showed gains in effectiveness among 391 agen-cies that it surveyed in 2000. It reported that an overwhelming proportion of itspartner organizations found outcome measurement useful for communicatingresults and identifying effective practices (84-88%), as well as for helping toimprove service delivery of programs (76%). However, the results were not allpositive: a significant number of agencies reported that implementing outcomemeasurement has led to a focus on measurable outcomes at the expense of otherimportant results (46%), has overloaded the organization’s record-keeping capac-ity (55%), and that there has remained uncertainty regarding how to make pro-gram changes based on identified strengths and weaknesses (42%).10

Foundations, too, have a mixed record of using impact assessments andevaluation in their decision making. In an analysis of evaluation methods used



TABLE1.

LogicMod

el

Inpu

ts→

§funds

§equipm

entandsupp

lies

§know

ledgeandtechnical

expertise

Activities

→

§basic

needsdelivery,

such

asfood

andshelter

§servicedelivery,such

asjobtraining

and

counseling

§infrastructureconstructio

n,such

astransportatio

n

Outpu

ts→

Results:immediate

§peop

lefed,ho

used,o

rtreated

§peop

letrainedor

educated

§roadsbuiltandgood

stransportedto

market

Outco

mes

→

Results:m

edium-andlong-term

§improved

quality

oflife,

health,educatio

nal

attainment,etc.

§increasedincomes

(measuredforindividuals)

Impa

cts

Results:effectson

root

causes,

sustainedsignificantchange

§sustaineddrop

inpo

verty

(orob

esity,illiteracy,etc.)

§improvem

entsin

human

developm

entindicators

(measuredin

term

sof

communities,p

opulations,o

recosystems)



by philanthropic foundations over three decades since the early 1970s, nonprofithistorian Peter Dobkin Hall has argued that such evaluations lack rigor, and thatkey decision makers are often indifferent to the findings.11 Others have suggestedthat philanthropic giving is often motivated by the expressive interests of donorsand not necessarily by evidence of what works and what doesn’t.12 Many founda-tions continue to struggle with how to integrate a range of measurement approachesinto their decision making. A 2007 survey of emerging approaches to evaluationin philanthropy pointed to a “shift from the use of evaluation to measure the impactof past grants and toward a more timely and pragmatic process of gathering forward-looking information that will enable both grantors and grantees to make ongoingimprovements in their work.”13 The report found this tension to be fairly commonin foundations, motivated on one hand by a need to identify the long-term outcomesand impacts of past projects, while also seeking more flexible and timely sources ofdata to help with present performance.

In recent years, however, there has been considerable progress in develop-ing measurement and evaluation methods with numerous approaches beingdeveloped by prominent consulting firms (see Appendix 1), as well as a moredeliberate emphasis on measurement by foundations and impact investors. Theseinclude not only established philanthropic institutions, such as the William andFlora Hewlett Foundation, the Edna McConnell Clark Foundation, and the Rock-efeller Foundation, but also a number of relatively young entrants to the fundingfield—The Bill and Melinda Gates Foundation, Robin Hood Foundation, and Acu-men Fund, to name just a few—that have advocated for an explicit emphasis onmeasurement and the development of standardized metrics.14

At the same time, a chorus of skeptical voices, particularly from practitioners,has suggested that while impact and outcome measurement appears to be “a goodtool to help funders see what bang they’re getting for their buck,” it runs the riskof being counterproductive in the long run, both by drawing precious resources awayfrom services and by putting too much emphasis on outcomes for which the causallinks are unclear, thus reflecting more of an obsession with institutional expectationsof accountability to funders than an interest in actually finding ways of improvingservices and results.15 Other practice-focused studies have also concluded that inthe end, a project’s success had less to do with whether measurement systems weredeveloped and more to do with “whether the organization was able to create aculture that valued the process of self-evaluation.”16

In short, this practitioner-based literature suggests that outcome andimpact measurement are undertaken for multiple purposes, including evaluatingeffectiveness, satisfying external accountability expectations, and guiding organi-zations in improving their actions. However, for all the intended benefits, theiradoption does not necessarily improve outcomes, impacts, or decision making.

What to Measure? A Performance Framework for OperatingOrganizations

Conventional wisdom in the social sector suggests that one should measureresults as far down the logic chain as possible, to outcomes and societal impacts.



This expectation is based on a normative view that organizations working on socialproblems, especially if they seek public support, should be able to demonstrateresults in solving societal problems. Yet it is worth considering whether, and towhat degree, such measurement makes sense for all social sector organizations.

For instance, many international NGOs—such as the Red Cross and Doc-tors Without Borders—are engaged in emergency relief work. Measuring thework of such organizations is conceptually fairly straightforward: count the time-liness and delivery of emergency supplies such as tents, food, water, and medicalsupplies, as well as the numbers of people reached. Emergency relief is thus typi-cally measured in terms of activities and outputs. While it is a complex activity,requiring highly sophisticated coordination and logistics management capabilities,it is focused on meeting immediate survival needs rather than long-term develop-ment outcomes or impacts. The links between inputs, activities, and outputs fol-low logically: the organization plans its requirements of supplies and staff(inputs) and the logistics for delivering those supplies (activities) in order to pro-vide relief to the people most affected by the emergency (outputs). When theeffort is well planned and executed, the program will be able to orchestrate activ-ities that lead to measurable outputs.

Outcome measurement, on the other hand, requires answers to a morecomplex causal question: Are the activities and outputs leading to sustainedimprovements in the lives of affected people? Outcome measurement is less com-mon and more difficult to do, given that organizations have the most control overtheir immediate activities and outputs, whereas outcomes are often moderated byevents beyond their organizational boundaries. For example the emergency relieforganization that has done excellent work during and after a natural disastermight still fall short on outcomes of rehabilitating and resettling those displacedfrom their homes and livelihoods, especially if those outcomes depend onextended coordination with local governments, businesses, and other NGOs. Con-necting outcomes to societal impacts, such as a sustained drop in poverty in theregion, is even more complex due to the number of additional factors at play—involving the larger political, social, cultural, and economic systems—that arebeyond the control of any one entity. In short, outputs don’t necessarily translateto outcomes, and outcomes don’t necessarily translate to impact.

This much is clear: every organization should at least measure and reporton its activities and outputs, as these results are largely within its control. How-ever, when should it step forward into the domain of outcomes and impacts?Two organizations that have confronted this problem in different parts of theworld are Aravind Eye Hospital in southern India and the Harlem Children’s Zonein New York City.17

In 2012 alone, Aravind Eye Hospital performed over 340,000 surgeries,most of them for cataract surgeries in one province in India; and it screened overten times that number of people. Its outputs are remarkable: providing vision cor-rection to over 3 million individuals since its founding in 1979. However, Aravindrealizes that output alone, judged by the number of patients screened or treated,is not an adequate indicator of its performance. Rather it needs to know theoutcomes of its intervention. How many people who received treatment were



satisfactorily cured? In other words without a high quality intervention, its out-puts do not convert to outcomes. Aravind engineered a superb operational processby which the surgery was performed at the highest quality standards. The rate ofcomplications year after year were less than half that of hospitals in the UnitedKingdom.18 With such a high quality of service, Aravind was able to establish atight linkage between outputs and outcomes. We may never know whether thoseoutcomes resulted in poverty reduction (impact), but it is logically reasonable toassume that those with recovered eyesight would be better able to fend for them-selves. Even given the tight linkage in Aravind’s operations between outputs andoutcomes, the organization assumes but does not measure impact—that individu-als with recovered eyesight from cataract treatment will be able to lead productivelives once again and thereby contribute to society. While this assumption seemsreasonable, the organization has cautiously stayed away from making that leapand seeking to take credit for impacts.

In contrast, there is Harlem’s Children Zone (HCZ) in New York. WhileAravind is highly focused in its scope of activities, conducting vision screeningand cataract surgeries, HCZ offers a broad scope of activities aimed at developinga child’s capabilities through various stages of educational development. The orga-nization runs pre-natal programs for parents; early childhood and after-schoolprograms for children; charter schools, vocational training, and college prepara-tion for youth; health programs to help families address asthma and obesity;and even programs for adults that aim to create a supportive and caring commu-nity for children. Unlike Aravind which focuses on a singular intervention, HCZattempts to provide a “pipeline” of interventions “from cradle to college to com-munity” that collectively enhance the chances of a child in Harlem making it fromschool to the workplace and on to self-sufficiency and responsible citizenry. HCZ’s“theory of change” is that educational support has to be continuous from pre-school through high school, and it has to be supplemented by extracurricularand community support to solidify the young person’s all-round development.19

In order to provide this pipeline of comprehensive interventions, HCZ hasconcentrated its activities in a narrow geographical region of nearly 100 cityblocks of Harlem, under the assumption that it will be better able to control thechild’s overall environment. In 1998 before HCZ scaled its programs across Har-lem, 60% of the children lived in poverty and only 20% of children in elementaryschools were able to read at grade level. In 2011, the sixth graders in its two maincharter schools had shown significant improvements: approximately 80% were ator above grade level in statewide math exams, and between 48-67% (dependingon the school) were at or above grade level in English. Moreover, 95% of seniorsin public schools who attended HCZ after-school programs were accepted into col-lege.20 The grade level metrics are primarily output measures, while collegeacceptance may be considered an outcome measure. The time horizon for theseinterventions is long (5 to 19 years), and the organization is undertaking longitu-dinal studies to better assess its results. Even then, drawing a causal link betweenHCZ’s interventions and longer-term outcomes such as lifetime incomes of itsgraduates, and impacts such as a decline in poverty in Harlem, remains compli-cated due to numerous social and economic factors that HCZ cannot control.



A key distinction between Aravind and HCZ is how they make the leapfrom outputs to outcomes. Aravind can measure outcomes because of the tightcausal linkage between its outputs (corrective surgery) and outcomes (qualityvision), which hinges on the quality of its surgery. HCZ can measure outcomesbecause it is able to vertically integrate a comprehensive set of interventions (apipeline from cradle to college) that it controls in a tightly bounded geographicspace (100 city blocks). More generally, measuring outcomes is possible undertwo conditions that are uncommon in the social sector: when the causal linkbetween outputs and outcomes is well established, or when the range of the inte-grated interventions needed to achieve outcomes are within the control of theorganization.

Clarifying the Operating Mission

These preliminary observations lead us to the construction of a frameworkthat relates performance measurement to the mission of a social sector organiza-tion. A unique characteristic of such organizations is their drive towards missionfulfillment, which is their primary purpose. Implicit within an organization’s aspi-rational vision, which may be eliminating blindness or hunger or poverty, is amuch more modulated “operating mission” that focuses on the immediate workat hand.21 From an operational perspective, Aravind’s mission centers on provid-ing surgical correction to those suffering from cataract blindness, regardless oftheir ability to pay. Knowing that nearly 75% of the blindness in India is causedby cataract formation, starting with such a sharp focus helps the organizationmake rapid progress against its mission. HCZ’s operational mission is to ensureeducational outcomes for kids in Harlem. Even though the organization aims ata larger level to address poverty in Harlem, its day-to-day activities and programsconcentrate on improving the quality and context of children’s education.

Thus while a nonprofit or social enterprise may have an aspirational mis-sion for what the world should look like, in practice its work is best captured byits more pragmatic operational mission. This operational mission is not alwaysexplicitly articulated, but it can be observed in the work that the organizationattempts to do on a day-to-day basis. Figure 1 depicts this distinction. The opera-tional mission, and how to measure progress towards achieving it, can be furtherunderstood by examining the scale and scope of the organization’s work.

Scale and Scope

Implicit in every social mission is a statement of the scale of the problemthe organization intends to address and the nature of the intervention requiredto address it. While Aravind’s general mission makes reference to the populationof blind people in the world, its operational mission is much more narrowlyfocused on one provincial region, Tamil Nadu State with about 750,000 blind peo-ple, and the surrounding region with roughly another 2 million blind people.Nearly half of this group is poor and can’t afford to pay for treatment. In opera-tional terms, this is Aravind’s target client group against which performanceshould be judged. For HCZ the target area is narrower, the 100 city blocks cover-ing Harlem, and the nearly 20,000 children who attend Harlem’s public schools.



The scale of an organization’s operations can be expected to evolve withtime. As it gains reputation and funding, the organization will be attracted toexpand the reach of its operations. Consequently performance assessments mustbe based on the explicit target identified by the organization in its operational mis-sion. Aravind started off in one city, Madurai in south India, and then steadilyexpanded through outreach programs and more hospitals to blanket its homestate, eventually branching out to neighboring states. By 2012, it was performingnearly 350,000 surgeries in its focused region. Harlem Children’s Zone startedwith 24 blocks in central Harlem in 1997, steadily expanding to cover the entire100-block region within a decade, reaching over 9,400 children and 6,200 adultswithin this urban zone in 2011. Thus Aravind seeks to address the problem ofvision at a regional scale, while HCZ tackles the problem of educational attain-ment at a local scale. For these two organizations, it makes sense to measure per-formance against these regional and local operational targets, respectively.

The second dimension, scope, is a measure of the range of activitiesrequired to address the need identified in the operational mission. As notedabove, the scope of Aravind’s intervention is narrow, focused on providing highquality eye surgery. The organization can measure outputs (number of peoplescreened and treated), which are then easily and explicitly linked to outcomes(number of people with quality vision). Figure 2 illustrates this explicit link fromoutputs to outcomes in the scope of Aravind’s activities (the darker arrows). In thelast decade, Aravind has taken advantage of its vast outreach (screening 3 millionindividuals) to add a second line of treatment—fitting eyeglasses for vision correc-tion. This addition in scope is aligned with its mission: like cataract surgery, onceindividuals receive corrective lenses, their vision improves. Again, the causal linkbetween outputs (number of people receiving glasses) and outcomes (qualityvision) is explicit, provided that quality is maintained in the fitting of eyeglasses.The case of Aravind provides an example of a narrow scope of activities resultingin measurable outcomes. Do corrective surgeries or lenses lead to an improvedquality of life and reduced poverty for affected people (impact)? Although this

FIGURE 1. Operational Mission

OrganizationalMission

Operational Mission

Scale Scope

What range of activities?What reach of operations?Local, Regional, National, Global



link between outcomes and impact seems highly plausible, Aravind does notattempt to measure impacts, preferring to stick to what it does best: deliveringquality surgery and eyeglasses (outputs), which are tightly linked to the outcomeof improved vision. The link between outcomes and impacts is implicit in its work(hence the lighter arrow in Figure 2). Aravind’s performance can thus be assessedin terms of its effectiveness in producing quality vision at a regional scale.

HCZ’s performance can be similarly examined in terms of scale and scope (seeFigure 3). Its scale is local, centered on the zone inHarlemwhere it operates.However,its scope is very wide, made up of activities from early childhood education to collegepreparation. As children pass through its pipeline, HCZ hypothesizes that the combi-nation of outputs (such as educational experiences, test scores in math and English,conflict resolution, and parenting skills) will add up to measurable outcomes in theform of college acceptance and quality jobs (the darker arrows in Figure 3). Some ofthese links are substantiated through research, while others are hypothetical. HCZ isable to track outcomes because its services to children are vertically integrated all theway to job placement and college admission. But even if the outcomes are achieved(college and jobs), will they lead to impacts such as reduced poverty in Harlem? Thislink is implicit in the organization’s work (the lighter arrow in Figure 3), althoughthe longitudinal studies that it is undertaking may help to tighten the causal connec-tions and make them more explicit. In short, HCZ’s performance can be assessed interms of its effectiveness in producing educational outcomes for children at a localscale. The time horizon for these outcomes is 19 years.

In sum, the notion of scope captures the set of activities necessary foraddressing a social problem, while scale captures the target size of the problem.The problem itself is articulated in the organization’s operational mission. Clarityon all three components—operational mission, scale, and scope—is necessary inorder to know what to measure. Specifying the expected time horizon for results

FIGURE 2. Aravind’s Performance Vector

Global

Regional

Outputs Outcomes

Explicit link

Activit

iesArav

ind Implicit link

ScopeImpact

Sca

le

National

Local



is also important, particularly where outcomes and impacts are expected to bevisible only after many years.

Improving Performance: Expanding Scale and Scope

Our performance framework suggests that social sector organizationsshould primarily focus on delivering against their operational mission. Clearly,all organizations should be capable of measuring the outputs of their operations.However, only some will be able to go further to make credible and measurableclaims about outcomes. This is possible under two conditions:

§ the organization implements a narrow scope of activities where the causallink between outputs and outcomes is clearly established through evidence(e.g., Aravind’s eye surgeries lead to improved vision); or

§ the organization implements a broad scope of activities that is verticallyintegrated to increase control over outcomes (e.g., HCZ’s cradle to collegepipeline).

The time horizons for achieving outcomes can vary considerably. WhileAravind may be able to gather evidence of improve vision fairly soon after sur-gery, HCZ’s interventions require a decade or more to see their outcomes materi-alize. Only rarely will organizations be in a position to go even further by claiminglong-term sustained “impacts” on their communities and society. These two con-ditions further suggest how organizational performance can be improved: byexpanding operations, while retaining the current scope, in order to reach a largertarget population (increase scale); and/or by offering more services down the logicchain that will enable greater control over outcomes (increase scope). Whethersuch an expansion of scale or scope is warranted is a strategic question for theorganization’s leadership and key supporters.

FIGURE 3. Harlem’s Children Zone’s Performance Vector

Global

Regional

Outputs Outcomes

Activities

Harlem’s ChildrenZone

Implicit Link

ScopeImpact

Sca

le

National

Local



Scale in the social sector can be achieved not only through organizationalgrowth, but also via a myriad of other means, particularly through influencingpublic policy and coalition building, training others to replicate and adapt itsmodel, or even through the creation of new industries.22 For instance, nonprofitsserved as the initial pioneers in the microfinance industry, but the explosion ofthe industry was fueled by the creation of a supportive regulatory environmentand the entry of opportunistic private sector players.23

Expanding scope in the social sector, in order to increase control overoutcomes, is not limited to vertical integration within the organization. Analternate strategy is to partner with organizations that carry out complementarywork along the results chain. For instance, Aravind often gets asked to partici-pate in blindness prevention programs. At a superficial level, such an upstreamexpansion in scope might seem to meet the stated mission of the organization,but when examined more closely does not fit the capabilities of the organiza-tion. Prevention programs require educating and persuading the community,especially households, to alter the way they prepare and cook their food, suchas with locally available plants and vegetables with high content of Vitamin A.The target population is families with children, not those who already are blind.Moreover Aravind’s core operating model is centered on delivering high qualitysurgical intervention rather than on mass-media prevention campaigns. Toexpand scope, Aravind would be better off partnering with a social marketingorganization that could fill a distinct niche rather than integrating this newactivity into its existing organization.

Aravind has chosen to dramatically expand the scale of its operations butnot its scope. First, it has launched a training facility where hundreds of adminis-trators and healthcare workers from 43 countries come to learn the Aravind“management model.” They learn about its low-cost and high-quality patient caresystems, its outreach programs, and its hybrid funding model (roughly 35% of thepatients pay market prices and the rest of the 65% pay below market prices, withmany getting free treatment). Armed with this knowledge they go back to theirregions or countries to launch their own version of the Aravind model. In certainchosen locations, Aravind sends a team of doctors and nurses to actively engagewith the local sponsor to set up systems and ramp up the model. For exampleAravind’s start-up team helped Grameen Bank to launch an eye hospital inBangladesh. Such an approach scales the work of Aravind through virtual net-works, as depicted by the dashed arrow in Figure 4.

On the other hand, if Aravind sought to expand its scope in order to havegreater impact, it might partner “upstream” with organizations working on pre-vention and “downstream” with organizations that help its clients gain entry intothe workforce once their vision is restored. Notably, both options for improvingperformance—expanding scale or scope—can be achieved by collaborating withother organizations rather than by attempting to grow the organization. EvenHCZ, which has steadily expanded the scope of its activities through vertical inte-gration across the results chain, remains local in scale. Any expansion in scale to anational or regional level will likely require collaboration with other organizationsthat could replicate its model in other cities. The scaling challenge for HCZ lies in



identifying the most critical components of its results chain and how those mightbe reproduced elsewhere. More generally, when both scale and scope have to beaddressed, we expect that it will be especially difficult for organizations to achieveperformance single-handedly.

Performance and the Role of Funders

Performance measurement does not operate in a vacuum and is the subjectof much tension between operating organizations and their funders. Most organ-izations are typically resource-dependent on their funders, be they philanthropicfoundations, governments, donors, or investors. Much of the growing emphasison measuring performance and impact in the sector has come from institutionalfunders seeking to allocate their resources more effectively and strategically24

and in publicly legitimizing their allocation decisions.25 Performance measure-ment thus plays a critical and highly politicized role in mediating the relationshipbetween operating organizations and their funders, and in building capital mar-kets that reward performance.

How do funders assess their grantees or investees? As noted, funders havea mixed record of using performance assessments and evaluation for making deci-sions about grantees. How do funders assess their own performance? While manyoften demand performance measurement from their grantees, rarely are theyexplicit about how they measure their own performance. The most critical chal-lenges of performance measurement lie not at the level of operating organiza-tions, but among aggregators such as foundations, governments, and impactinvestors. It is at this level—where the funder is able to oversee hundreds of oper-ating organizations—that it is possible to measure societal impacts.

FIGURE 4. Expanding Scale and Scope

Global

Regional

Outputs Outcomes

Explicit Link

Activit

iesArav

ind Implicit Link

ScopeImpact

Sca

le

National

Local



What Funders Measure

We examined three funding organizations widely considered by their peersto be leaders and innovators of performance measurement: Acumen Fund, RobinHood Foundation, and Millennium Challenge Corporation (MCC).26 All threeprovide capital to operating nonprofits, social enterprises, or government agen-cies, either as investors or grant makers.

What do these organizations seek to measure? It turns out that they assessdifferent things: Acumen measures outputs, such as mosquito nets made and dis-tributed; Robin Hood focuses on long-term outcomes in the lives of individuals,such as gains in income; and MCC aims for both individual outcomes and broaderimpacts on society, such as reduction in poverty rates.

Consider Acumen Fund, a venture philanthropy firm with a portfolio ofover 75 investments in for-profit social enterprises in Africa and Asia. Its primarysocial metric is the number of lives reached in base-of-pyramid markets (an out-put measure). This means that when it invests in a company that manufacturesanti-malarial bed nets, Acumen will count the number of nets manufacturedand distributed. For an enterprise that builds toilet and shower facilities in urbanslums and business districts, it will tally up the number of times the toilets andshowers are used. However, why stop at these output measures, rather than out-comes such as reduction in malaria or improvements in health and environment?Acumen’s former chief information officer argues that doing so is complicated,expensive, and often impractical for early-stage enterprises.27 Instead, Acumen’sstrategy is to review the literature and consult experts to establish a link betweena specific output and impact (for instance, how bed net distribution leads tomalaria reduction) and to then count the outputs. This approach is similar to Ara-vind’s, where the emphasis is on measuring outputs and their related outcomes,while using existing research or evidence to make credible claims about the linkto community impact.

Robin Hood, on the other hand, is a grant-making foundation created byhedge fund managers with a penchant for hard numbers. With a mission to fightpoverty in New York City, the foundation puts each of its 200-plus grants to non-profits through a cost-benefit analysis every year. One of its primary metrics is theexpected increase in lifetime earnings of its clients. For grants focused on educa-tion, for example, its staff first identifies a set of results that can be immediatelyobserved—such as school attendance, standardized test scores, and high schoolgraduation. Then, they search for studies that link those measures to expectedlifetime earnings or quality of life. For instance, some longitudinal research sug-gests that a 10% increase in test scores is correlated with a 4% increase in highschool graduation rates, which in turn is associated with $6,500 in increasedincome per year. Robin Hood uses these figures with caution, employing themas placeholders for estimating benefits until better research comes along. Initiallythe foundation placed the burden of measurement on its grantees, but quicklyfound that few grantees had the capacity or resources to do so. As a result, theorganization has since built its own internal capacity for developing outcome met-rics, employing a full-time researcher and contracting with a research firm for fur-ther support.



The MCC’s approach is even more complex as it operates on a 20-year timehorizon. The U.S. government agency makes grants to emerging market countriesto reduce poverty through economic growth. Its extensive due diligence process—often two years long—first analyzes the barriers to economic growth in the country,and then identifies the sectors where the grants would most likely reduce poverty.For example, the MCC awarded $547 million to the government of Ghana to buildroads and ferries to get farm goods to market. To start, it estimated the number offarmers likely to benefit, and what those benefits would be: reduced cost and timeof getting goods to market, access to new markets, and opportunities for wageemployment. These data were used to anticipate an economic rate of return, withthe primary outcome metric being increases in farmer incomes, along with impactmetrics such as a reduction in regional poverty rates. Once a contract is signed, amonitoring and evaluation process kicks in to allow for mid-course correction.Benefits are expected to begin accruing only after five years, once the infrastruc-ture is built and operating. The MCC plans to eventually conduct long-termevaluations, but has not yet done so. Unlike Acumen and Robin Hood who haveinfluence on the process as it pertains to outputs and in some instances to out-comes, MCC’s big bet on impacts on poverty depends on how the Ghanaiangovernment and other complementary players build on its efforts to promoteeconomic growth.

Among these three funding organizations, two features of their measurementapproaches stand out. First, all three funders develop metrics in partnership with theoperating organizations, trying not to overburden them. Acumen focuses on metricsthat are useful to the entrepreneur in building the business. Robin Hood works withgrantees to identify key outputs, but then elaborates the links to outcomes itself. AndtheMCCworks with government agencies to build their capacities for implementingprojects and tracking progress, often sending its own staff and evaluators to providesupport. These kinds of partnerships are rare; more commonly funders expect oper-ating organizations tomeasure outcomes and impacts themselves, without providingthe funding or technical support to do so. Many expect accountability, but few arewilling to support the overhead costs necessary for enabling it.

Second, none of these three funding organizations typically measuresimpact. They hypothesize what the outcomes and impacts might be but only insome instances are they able to follow through by commissioning their ownresearch or multi-year evaluations. These are sophisticated funders and investorswho are much better positioned to measure long-term results than the front-lineorganizations that contend with funding shortages and operational challengesevery day. Surely measuring impact matters, but the experiences of these funderssuggest that there are severe constraints to doing so. Assessing impact requires alevel of research expertise, commitment to longitudinal study, and allocation ofresources that are typically beyond the capabilities of operating organizationsand sometimes even their funders.

Measuring the Performance of Funders

While funders such as foundations, impact investors, and governmental aidagencies seek to assess the performance of their grantees or investees, it is less



common for them to apply the same standards to measuring their own perfor-mance. Foundations that have large endowments face less pressure to do so, asthey are not resource-dependent on their external environments. However,why would it matter for funders to assess their own performance, given that theactual implementation is done by operating organizations? Because it is at thelevel of the funder that systemic impacts—long-term sustained changes in soci-ety—can best be observed. Impacts are rarely achieved by individual organizationsacting alone, but more often by collections of actors working towards a sharedgoal.28 Because they support and oversee hundreds of nonprofits and social enter-prises that typically act independently of one another, funders are specially posi-tioned to connect that work in order to assess impacts.

A foundation that supports efforts in health care, for example, is uniquelysituated to see how the work of all its grantees might link together as a portfolio—to connect the dots, or vertically integrate, among a series of outputs and outcomesin order to assess how they might lead to impacts. A central strategic challenge forfunders is to articulate their own causal pathway for achieving impacts and to partnerwith an integrated portfolio of social sector organizations to achieve those impacts.Figure 5 illustrates this argument. The solid lines represent the activities and resultsof operating organizations such as nonprofits and social enterprises, which can belinked together by the funder (integrated) in order to achieve impact. The dashedlines represent the links between outputs, outcomes, and impacts established bythe funder through its support of research that helps establish causal links.

It is unlikely that there is a single best way for funders to assess their ownperformance, or the collective performance of their grantees or investments. TheRobin Hood Foundation calculates the average benefit-cost ratio for all of itsinvestments. It estimates that for every dollar it spends on grants, the foundation

FIGURE 5. Systemic Impact: A Role for Funders

Global

Regional

Outputs Outcomes

Org. 1

ScopeImpact

Sca

le

NationalOrg. 4

Org. 2

Org. 3

Local



improves living standards for poor people by 18 dollars. This model is popular inthe impact investment community, which seeks to measure the social return onits investments. Bilateral government agencies that support thousands of anti-poverty projects around the world are also investigating such aggregation meth-ods, especially as they face pressure from taxpayers to demonstrate the resultsof international aid. A critique of this model is that it may atomize the social sec-tor, creating a market that rewards the performance of individual organizationswhile discouraging integrative or collective action.

Acumen Fund works on a different model in which it attempts to identifypromising organizations working on a social problem and helps them to scale. Itssuccess can thus be measured in terms of the growth and reach of the organiza-tions it funds. Acumen Fund invests in social enterprises that it believes can bescaled to reach over 1 million clients in base-of-pyramid markets. The organiza-tion provides not only “patient capital” over a period of several years in the formof debt or equity, but also offers management capacity building in order to help itsinvestees build businesses capable of scaling.

While Acumen Fund attempts to create impact through scale, the Millen-nium Challenge Corporation attempts to do so along the scope dimension byassembling a complementary collection of several public agencies, private sectorfirms, and nongovernmental organizations in the service of its mission to alleviatepoverty through economic growth. The Bill and Melinda Gates Foundation’sapproach to global health can also be characterized in these terms, as the organi-zation brings together key players needed to eliminate major infectious diseases.For instance, a $100 million joint effort by Merck Pharmaceuticals and The GatesFoundation in Botswana aims to set up a comprehensive healthcare screening,testing, consulting, and delivery system for HIV/AIDS through a number of localand international partner organizations in the country.29

Conclusions and Reasons for Optimism

The core of our framework for measuring social performance is relativelysimple: clarify the operational mission, specify the set of activities to address thatmission (scope), and identify the target size of the problem (scale). Yet, such mea-surement is rare in practice. Our approach provides a basis for assessing perfor-mance that is empirically rooted, rather than relying on unsubstantiated claimsand unrealistic expectations about performance. It is a general approach thatcan be applied to any organization, while still allowing organizations to adapttheir metrics of scale and scope to their specific contexts. All too often, social sec-tor organizations seek to measure, or take credit for, impacts that extend wellbeyond the scale and scope of what they actually do. As a result, they risk eitherexposing themselves to permanent failure, or being taken to task for impacts theycannot realistically achieve.

Despite the conceptual simplicity of our model, we recognize that carryingit out poses significant challenges for managers as well as scholars. There is anurgent need for better knowledge on the challenges of scale and scope in thesocial sector. As noted, scale in the social sector can be achieved not only through



organizational growth, but also via myriad other means, particularly throughinfluencing public policy, building collaborative networks, or the creation ofnew industries. However, even in terms of growth, the scaling rate is shockinglylow: of over 200,000 new nonprofits created between 1975 and 2008, only 201grew to revenues exceeding $50 million (excluding hospitals and universities).30

Much remains poorly understood. For each of the diverse means of achievingscale, what are the specific mechanisms and suitable industries? What are the mainimpediments to scale in terms of capital markets, capacity, and regulation—andhow can they be addressed?31

Yet, there are many reasons for optimism. Social sector leaders and theirfunders are increasingly embracing performance measurement as critical to help-ing them achieve their missions at scale. They are shifting from a focus on evalu-ating impact after implementing their interventions, to also using measurementduring program design and implementation in order to get real-time feedbackfor improving their work. This is true of large established organizations as wellas new entrants to the field. For example, international NGOs such as CARE,Oxfam, and ActionAid International are all building monitoring and learning sys-tems that can better enable mid-course correction. Large philanthropic founda-tions such as Gates, Hewlett, Rockefeller, and Edna McConnell Clark have allbuilt up strong evaluation departments as part of a deeper commitment to out-come-oriented philanthropy.32 At the same time, a new generation of socialentrepreneurs and impact investors are all demanding better data for informingdecision making; they are supported in this aim by infrastructure-builders likethe Global Impact Investing Network that are developing standardized metricsfor use by investors. In terms of performance measurement methodologies, therehas been a surge in the development of more participatory and integrative toolssuch as constituency feedback, most significant changes techniques, and develop-mental evaluation, as well as network-based approaches such as collective impactand outcome mapping (see Appendix 1)33 suited to settings of high complexityinvolving interactions across multiple organizations and sectors.

In other words, the social sector is in a period of vibrant innovation on per-formance measurement. Given this moment of ferment, it is useful to step backand gain some clarity regarding the unit of analysis. The social sector has had along history of program evaluation, mainly because funding has traditionally beendirected to programs rather than to the organizations that design and deliverthem. Organizations delivering multiple programs have often found that pro-grammatic success does not equate to success in fulfilling their overall organiza-tional missions. For that to happen, the various programs have to hold togetherin an integrated fashion. Our framework focuses squarely on the operationalmission of the organization, under which multiple programs can be strategicallycombined.

There is a unique opportunity for funders to integrate multiple levels ofanalysis —programmatic, organizational, and societal—in assessing and improv-ing performance in the social sector. This can be done in at least two ways. First,funders should allocate greater resources to building the management capacity ofnonprofits and social enterprises to enable them to better integrate their own



programs. Funders should view themselves as part of a syndicate to enable mis-sion success, even though a large part of their funding may be earmarked for pro-grams. The three examples cited—Acumen, Robin Hood Foundation, andMillennium Challenge Corporation—all devote resources to such organizationalcapacity building, but they are the exception rather than the rule. The generaltrend has been in the opposite direction: towards a reduction in capacity buildingand other overhead costs in favor of direct funding to programs where the resultsare easier to measure. However, the experience of venture capital, and increas-ingly of venture philanthropy, has shown that investments in human and systemscapacities are foundational to generating longer-term results. It is difficult for anoperating organization to think strategically when it is at the mercy of differentfunders with different programmatic goals.

Second, funders should turn their attention to their own performanceand impact, while easing off on their demands for operating organizations toprove their impacts. Foundations, governments, and impact investors areuniquely positioned—at a higher level in the organizational ecosystem thantheir grantees or investees—to see how the work of several operating organiza-tions fits together to achieve impacts that are greater than the sum of the parts.They are also better resourced than most nonprofits and social enterprises tocommission such research. Impacts on systemic societal problems are unlikelyto be achieved by organizations acting alone; it thus makes more sense for fun-ders rather than operating organizations to take on the challenge of measuringthose impacts. In other words, nonprofits and social enterprises that operate ina niche should measure their outputs, and sometimes their outcomes, withinthat niche. Funders that operate higher up in the ecosystem should measureimpacts at a societal level. While a handful of foundations (such as Gates, Hew-lett, and McConnell Clark) are already oriented in that direction, the vast major-ity of funders, both institutional and individual, are still in the business ofproviding support for programs yet expecting systemic benefits. Bridging thesemultiple gaps—between performance at the level of programs, organizations,and society—will require funders to think more strategically about the differentorganizations and programs they fund so that their collective investment canachieve systemic impacts, with each individual piece playing its role. As withany organization in the social sector, doing so will require clarity about theiroperational mission, scale, and scope.



APPENDIX

1.Metho

dsforAssessin

gSo

cialPerfo

rmance

(continuedon

next

page)

Metho

dKey

Adv

ance

sInno

vators

andUsers

(exa

mples)

Sam

pleResour

ces

LogicMode

ls

§Lo

gicMod

elsor

Fram

eworks,R

esults

BasedManagem

ent,

OutcomeManagem

ent

§Identifiesandlinks

thekey

results

expected

from

aninterventio

n:inputs,activities,

outputs,ou

tcom

esandimpacts.

§UnitedStates

(USA

ID)

§European

Com

mission(EC)

§W

.K.K

ellogg

Foundatio

n

§Inno

vatio

nNetwork

§Urban

Institute

§UnitedW

ayof

America

§http://usaidsite.carana.com/con

tent/logical-framew

ork-lf

§http://pd

f.usaid.gov/pdf_d

ocs/PN

ABN

963.pd

f

§http://ec.europ

a.eu/europ

eaid/m

ultim

edia/publications/

documents/to

ols/europeaid_

adm_p

cm_guidelines_2

004_

en.pdf

§http://www.wkkf.org/kno

wledge-center/resou

rces/200

6/02

/wk-kellogg-fo

undatio

n-logic-mod

el-develop

ment-

guide.aspx

§http://www.inno

net.org/client_d

ocs/File/logic_mod

el_

workboo

k.pd

f

§http://www.urban.org/publications/310

776.html

§http://www.you

runitedw

ay.org/outcome-measurements

Exp

ectedRetur

nan

dCost

Effe

ctiven

ess

§Measuressocialvalueof

investments

byquantifying

benefitsandcosts,thus

enablingcomparison

across

potential

investments.U

sedprimarily

byinvestorsto

optim

izesocialvalueper

unitof

investment,oftenreferred

toas

impact

investing

§So

cialRe

turn

onInvestment

(SRO

I)Network

§RE

DF–socialreturn

oninvestment

§Acumen

Fund

–bestavailable

charitableop

tion(BACO)

§Ro

binHoo

dFoundatio

n–

benefit-costratio

s(BCR)

§Mille

nnium

Challenge

Corpo

ratio

n–econ

omic

rate

ofreturn

(ERR

)

§http://www.thesroinetwork.org

§http://www.redf.org/learn-fro

m-redf/p

ublications/119

§http://acum

en.org/blog/news/the-metho

d-behind-our-

metrics/

§http://acum

en.org/wp-content/u

ploads/200

7/01

/BACO%

20Con

cept%20

Paper_01

.24.07

1.pd

f

§http://www.ro

binhoo

d.org/metrics

§http://www.mcc.gov/pages/activities/activity/econo

mic-

rates-of-return

Exp

erim

entalMetho

ds

§Random

ized

Con

trol

Trials(RCT)

§Assessestheimpact

ofan

interventio

n,andmeasuresthe

sizeof

that

impact,bycomparing

outcom

esin

grou

psthat

receivean

interventio

nto

thosethat

dono

t.

§Abd

ulLatif

JameelP

overty

ActionLab,MIT

§Inno

vatio

nsforPo

vertyAction

§http://www.povertyactio

nlab.org/m

etho

dology

§http://po

verty-actio

n.org/provenimpact



APPENDIX

1.InitialDataSo

urces

Metho

dKey

Adv

ance

sInno

vators

andUsers

(exa

mples)

Sam

pleResour

ces

Systematic

Rev

iews

§Sumsup

thebestavailableresearch

onan

issue

orquestio

nthrough

synthesizingresults

ofnumerou

sstudies.

§Cam

pbellC

ollabo

ratio

n

§InternationalInitiativeforImpact

Evaluatio

n(3ie)

§http://www.cam

pbellcollabo

ratio

n.org/

§www.3ieimpact.org

Participa

tory

and

Relationship-Based

Metho

ds

§Con

stituency

Feedback

andPerceptio

nRe

ports

§MostSignificant

Changes;

Story-BasedEvaluatio

n

§Participatory

RuralA

ppraisal(PR

A)and

variants

§Participatory

Poverty

Assessm

ent

§So

licits

theperceptio

nsof

constituentsabou

tperfo

rmance

(e.g.,

granteefeedback

onfunders,

community

feedback

onNGOs)

that

isthen

benchm

arkedagainstpeers.

§Identifiesimpactsandchanges

withou

ttheuseof

preset

indicators,

relyingon

field-levelstoriesof

change

andthesystem

aticselectionof

the

mostsignificantof

thesestoriesby

designatedpanels.

§Drawsou

tknow

ledgeof

localp

eople

inanalyzingtheirow

nrealities

asa

basis

forplanning

anti-po

verty

interventio

ns.O

utsid

ersserveas

facilitatorsandlisteners

rather

than

experts.

§AdaptsPR

Atechniques

topo

licyissues,

thus

includingthepo

orin

discussio

nson

policypriorities.

§CenterforEffectivePhilanthrop

y

§Keystone

Accou

ntability

§Yo

uthTruth

§M&ENew

s(RickDavies)

§Clear

Horizon

(Jess

Dart)

§Institute

ofDevelop

mentStudies

(Rob

ertChambers)

§InternationalInstitutefor

Environm

ent&Develop

ment

§U.N.Foo

dandAgriculture

Organization(FAO)

§W

orld

Bank

§http://www.effectivephilanthrop

y.org/index.php?page=

assessment-tools

§http://www.keyston

eaccou

ntability.org/abou

t/publicrepo

rt/

what

§http://www.you

thtruthsurvey.org/students

§http://www.mande.co.uk/docs/MSC

Guide.pdf

§http://clearhorizon

.com

.au/category/resou

rce-hub/most-

significant-change/

§http://www.ids.ac.u

k/publication/fro

m-pra-to

-pla-to-pluralism-

practice-and-theory

§http://www.iie

d.org/participatory-learning-action

§http://www.fao.org/do

crep/W

3241

E/w32

41e09.htm

§http://go.worldbank.org/Q

AASG

4TK8

0

(continuedfrom

previous

page,continuedon

next

page)



APPENDIX

1.InitialDataSo

urces

Metho

dKey

Adv

ance

sInno

vators

andUsers

(exa

mples)

Sam

pleResour

ces

IntegrativeApp

roac

hes

§StrategicLearning,

Planning,and

Evaluatio

n

§CollectiveImpact,

Com

munity

Change,

andCom

plex

System

s

§Linksperfo

rmance

measurement

tostrategy

usingmetho

dssuch

asform

ativeanddevelopm

ental

evaluatio

n,balanced

scorecards,

strategy

maps,dashbo

ards

and

relatedtools.

§Brings

together

organizatio

nsacross

sectorsto

solvesocial

prob

lemsby

buildingacommon

agenda

andshared

measuresof

success.

§Em

ergent

setof

ideasand

approaches

forunderstanding

complex,non

linear,andadaptive

system

sthat

cannot

beprecisely

predicted.

§Bridgespan

Group

§FSG

SocialImpact

Adviso

rs

§McKinsey&Com

pany

Social

Sector

Office

§Ascendant

Strategy

Managem

ent

Group

§Blue

Avocado

andCom

passPo

int

§ActionA

idInternational

§iScale(fo

rnetworks)

§GrassrootsBu

sinessFund

§FSG

(collectiveimpact)

§InternationalD

evelop

ment

Research

Centre

(outcomemapping)

§FullFram

eInitiative

§New

EnglandCom

plex

System

sInstitute

§http://www.brid

gespan.org/Publications-and-Too

ls/Perfo

rmance-M

easurement.aspx

§http://www.fsg.org/OurApp

roach/StrategicEvaluation.aspx

§http://lsi.mckinsey.com/

§http://www.ascendantsm

g.com/wiki/index.cfm

/Balanced_

Scorecard

§http://www.blueavocado

.org/con

tent/non

profit-

dashbo

ard-and-signal-light-boards

§http://www.actionaidusa.org/publications/alps-accountability-

learning-and-planning-system

§http://www.scalingimpact.net/files/IDRC

_Network_IPARL

_Paper_Final_0.pd

f

§http://www.gbfund.org/sites/default/files/iPAL%

20Fram

ework%

20Factsheet.pdf

§http://www.fsg.org/OurApp

roach/CollectiveImpact.aspx

§http://www.outcomem

apping.ca/

§http://fullframeinitiative.org/resources/evaluatio

n/

§http://www.necsi.edu/research/international/

(continuedfrom

previous

page)



Notes

1. See, for example, A. Ebrahim and E. Weisband, eds., Global Accountabilities: Participation, Plural-ism, and Public Ethics (Cambridge: Cambridge University Press, 2007); L. Jordan and P. VanTuijl, eds., NGO Accountability: Politics, Principles and Innovations (London and Sterling, VA:Earthscan, 2006); K.P. Kearns, Managing for Accountability: Preserving the Public Trust in NonprofitOrganizations (San Francisco, CA: Jossey-Bass, 1996).

2. See, for example, P. Brest and H. Harvey, Money Well Spent: A Strategic Plan for Smart Philan-thropy (New York, NY: Bloomberg Press, 2008); L.R. Crutchfield and H.M. Grant, Forces forGood: The Six Practices of High-Impact Nonprofits (San Francisco, CA: Jossey-Bass, 2008); MonitorInstitute, “Investing for Social and Environmental Impact: A Design for Catalyzing an Emerg-ing Industry,” Monitor Institute, 2009.

3. H. Hwang and W.W. Powell, “The Rationalization of Charity: The Influences of Professional-ism in the Nonprofit Sector,” Administrative Science Quarterly 54/2 (June 2009): 268-298;M. Power, The Audit Society: Rituals of Verification (Oxford: Oxford University Press, 1999).

4. P. Brest, H. Harvey, and K. Low, “Calculated Impact,” Stanford Social Innovation Review, 7/1(Winter 2009) 50-56; M. Marino, Leap of Reason: Managing to Outcomes in an Era of Scarcity(Washington, D.C.: Venture Philanthropy Partners, 2011); T.J. Tierney and J.L. Fleishman,Give Smart: Philanthropy that Gets Results (New York, NY: Public Affairs, 2011).

5. C. Roche, Impact Assessment for Development Agencies: Learning to Value Change (Oxford: OxfamGB, 1999), p. 21.

6. Crutchfield and Grant, op. cit., p. 24.7. H. Jones, “The ‘gold standard’ is not a silver bullet for evaluation,” London, Overseas Develop-

ment Institute, 2009.8. L. Bickman, “The Functions of Program Theory,” in L. Bickman, ed., Using Program Theory

in Evaluation, New Directions for Program Evaluation (San Francisco, CA: Jossey-Bass, 1987);P.J. Rogers, “Using Programme Theory to Evaluate Complicated and Complex Aspects of Inter-ventions,” Evaluation, 14/1 (January 2008): 29-48; C H. Weiss, Evaluation Research: Methods forAssessing Program Effectiveness (Englewood Cliffs, NJ: Prentice-Hall, 1972); J. Wholey, Evalua-tion: Promise and Performance (Washington, D.C.: Urban Institute Press, 1979).

9. J. C. Sawhill and D. Williamson, “Mission Impossible? Measuring Success in Nonprofit Organ-izations,” Nonprofit Management and Leadership 11/3 (Spring 2001): 371-386.

10. United Way of America, “Agency Experiences with Outcome Measurement: Survey Findings,”Report 0196, Alexandria, VA, 2000.

11. P.D. Hall, “A Solution is a Product in Search of a Problem: A History of Foundations and Eval-uation Research,” 2005, .

12. P. Frumkin, Strategic Giving: The Art and Science of Philanthropy (Chicago, IL: University ofChicago, 2006).

13. M. Kramer, R. Graves, J. Hirschhorn, and L. Fiske, “From Insight to Action: New Directions inFoundation Evaluation,” FSG Social Impact Advisors, 2007, p. 5.

14. “Interview—Paul Brest, Jed Emerson, Katherina Rosqueta, Brian Trelstad and MichaelWeinstein,” Alliance Magazine, April 1, 2009, ; B. Trelstad,“SimpleMeasures for Social Enterprise,” Innovations, 3/3 (2008);M.M.Weinstein (with C.E. Lamy),“Measuring Success: How Robin Hood Estimates the Impact of Grants,” New York, NY, RobinHood Foundation, 2009.

15. B. Glasrud, “The Muddle of Outcome Measurement: Be Careful How You Measure Programs,”Nonprofit World, 19/6 (November/December 2001): 35; S. Torjman, “Are Outcomes the BestOutcome?” Ottawa, Caledon Institute of Social Policy, 1999.

16. G. Hernández and M.G. Visher, “Creating a Culture of Inquiry: Changing Methods—andMinds—on the Use of Evaluation in Nonprofit Organizations,” San Francisco, CA, The JamesIrvine Foundation, 2001, p. 2.

17. The challenges of performance measurement, scale, and scope facing these two organizationsare elaborated in two cases: “Aravind Eye Hospital, Madurai, India: In Service for Sight,”Harvard Business School Case Number 593098, 2009; “Harlem Children’s Zone: DrivingPerformance with Measurement and Evaluation,” Harvard Business School Case Number303109, 2004.

18. T. Vickers, “Driving Down the Cost of High-Quality Care: Lessons from the Aravind Eye CareSystem,” interview with Dr. Srinivasan, Health International, 11 (2011): 23.

19. For resources on theory of change, see: ActKnowledge, “Theory of Change,” 2010, ; Andrea A. Anderson, “Theory of Change as a Tool for Strategic Plan-ning: A Report on Early Experiences,” New York, NY, Aspen Institute Roundtable on Community



Change, 2004, ; Key-stone Accountability for Social Change, “Developing a Theory of Change,” London, 2008,.

20. Harlem Children’s Zone, “2010-2011 Biennial Report,” New York, 2011, accessed August 22,2013, .

21. V.K. Rangan, “Lofty Missions, Down-to-Earth Plans,” Harvard Business Review, 82/3 (March2004): 112-119.

22. R.M. Kanter, “Even Bigger Change: A Framework for Getting Started at Changing the World,”Harvard Business School Background Note 305099, 2005; R.M. Kanter, “The Change Wheel:Elements of Systemic Change and How to Get Change Rolling,” Harvard Business SchoolBackground Note 312-083, 2011; P. Uvin, P. Jain, and L.D. Brown, “Think Large and ActSmall: Toward a New Paradigm for NGO Scaling Up,” Word Development, 28/8 (August2000): 1409; P. Uvin and D. Miller, “Paths to Scaling-up: Alternative Strategies for Local Non-governmental Organizations,” Human Organization, 55/3 (Fall 1996): 344.

23. World Microfinance Forum Geneva, “Is it fair to do business with the poor? Report on thedebate between Muhammad Yunus and Michael Chu,” Geneva, 2008.

24. P. Brest, “A Decade of Outcome-Oriented Philanthropy,” Stanford Social Innovation Review, 10/2(Spring 2012): 42-47; Frumkin, op. cit.

25. A. Ebrahim, NGOs and Organizational Change: Discourse, Reporting and Learning (Cambridge:Cambridge University Press, 2003); R. Eyben, “Uncovering the Politics of ‘Evidence’ and‘Results’: A Framing Paper for Development Practitioners,” April 2013, ; T. Wallace, L. Bornstein, and J. Chapman, The Aid Chain: Coercion and Commitmentin Development NGOs (Rugby, UK: Intermediate Technology Publications, 2006).

26. The measurement challenges facing these three organizations are elaborated in three cases devel-oped by the authors: A. Ebrahim and V.K. Rangan, “The Millennium Challenge Corporation andGhana,” Harvard Business School Case Number 310-025, 2010; A. Ebrahim and V.K. Rangan,“Acumen Fund: Measurement in Venture Philanthropy (A),” Harvard Business School CaseNumber 310-011, 2011; A. Ebrahim and C. Ross, “The Robin Hood Foundation,” Harvard Busi-ness School Case Number 310-031, 2010.

27. Trelstad, op. cit.28. S. Earl, F. Carden, and T. Smutylo, Outcome Mapping: Building Learning and Reflection into Devel-

opment; J. Kania and M. Kramer, “Collective Impact,” Stanford Social Innovation Review, 9/1(Winter 2011): 36-41.

29. V.K. Rangan, “African Comprehensive HIV/AIDS Partnerships (ACHAP): The Merck/GatesInitiative in Botswana,” Harvard Business School Case Number 505-057, 2007.

30. P. Kim and J. Bradach, “Why More Nonprofits are Getting Bigger,” Stanford Social InnovationReview, 10/2 (Spring 2012): 15-16.

31. R.S. Kaplan and A.S. Grossman, “The Emerging Capital Market for Nonprofits,” Harvard Busi-ness Review, 88/10 (October 2010): 110-118.

32. Brest (2012), op. cit.33. D. Bonbright, D. Campbell, and L. Nguyen, “The 21st Century Potential of Constituency Voice:

Opportunities for Reform in the United States Human Services Sector,” Alliance for Children &Families, United Neighborhood Centers of America, and Keystone Accountability, 2009;R. Davies and J. Dart, “The ‘Most Significant Change’ (MSC) Technique: A Guide to ItsUse,” (2005), ; Earl, Carden, and Smutylo, op. cit.;S. Khagram, C. Thomas, C. Lucero, and S. Mathes, “Evidence for Development Effectiveness,”Journal of Development Effectiveness, 1/3 (2009): 247-270; F. Twersky, P. Buchanan, andV. Threlfall, “Listening to Those Who Matter Most, the Beneficiaries,” Stanford Social InnovationReview, 11/2 (Spring 2013): 41-45; M.Q. Patton, Developmental Evaluation: Applying ComplexityConcepts to Enhance Innovation and Use (New York, NY: Guilford Press, 2011).

California Management Review, Vol. 56, No. 3, pp. 118–141. ISSN 0008-1256, eISSN 2162-8564. © 2014by The Regents of the University of California. All rights reserved. Request permission to photocopy orreproduce article content at the University of California Press's Rights and Permissions website athttp://www.ucpressjournals.com/reprintinfo.asp. DOI: 10.1525/cmr.2014.56.3.118.



/ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 266 /GrayImageMinResolutionPolicy /Warning /DownsampleGrayImages false /GrayImageDownsampleType /Average /GrayImageResolution 300 /GrayImageDepth 8 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /FlateEncode /AutoFilterGrayImages false /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 900 /MonoImageMinResolutionPolicy /Warning /DownsampleMonoImages false /MonoImageDownsampleType /Average /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck true /PDFX3Check false /PDFXCompliantPDFOnly true /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile (None) /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False

/CreateJDFFile false /Description

CMR5603 07 Ebrahim 118. Files/CMR5603_07... · by philanthropic foundations over three decades...

Documents

Transcript of CMR5603 07 Ebrahim 118. Files/CMR5603_07... · by philanthropic foundations over three decades...