Nonprofit Organizations and Social Impact: Social Return on Investment Analysis and

Evaluation Methods

April 22, 2010

PIA 2096: Capstone Seminar Graduate School of Public and International Affairs

University of Pittsburgh

Nonprofit Organizations and Social Impact: Social Return on Investment Analysis and Evaluation

Methods

April 22, 2010

Class Participants:

Jennifer Engel

Paul Shetler Fast Mpho Letima

Anne Mulgrave Katie Muller

Emily O’Keefe Meg Pinney Austin Price

Kathryn Vargas Anne Marie Toccket

Instructors: Professor Sabina Deitrick, Mr. Tracy Soska, and Ms. Kathryn Collins

PIA 2096: Capstone Seminar Graduate School of Public and International Affairs

University of Pittsburgh

1

Table of Contents Executive Summary…………………………………………………………………. 2 Introduction……………………………………………………………………………. 3 Methodology…………………………………………………………………………… 4 Case Studies……………………………………………………………………………. 7 Lessons Learned……………………………………………………………………… 16 Conclusions and Recommendations………………………………………… 17 Contact Information………………………………………………………………… 19

Acknowledgments: We would like to acknowledge the instructors for this course, Dr. Sabina Deitrick, Mr. Tracy Soska and Ms. Kathryn Collins. In addition, we thank Kristen Burns of the Grable Foundation, Adrienne Walnoha of Community Human Services, and Ehaab Abdou of the Brookings Institute for their time and the invaluable information they provided. We would also like to thank Dr. Dave Miller and the Public Administration Department at GSPIA, Dr. Lou Picard and the International Development Department of GSPIA, and Dr. Kevin Kearns and the Johnson Institute for Responsible Leadership for generous donations that made this project and presentation possible. Finally, we would also like to thank all of the organizations and staff who offered us their time and expertise. This project would not be possible without your assistance.

2

Executive Summary This report and presentation are the culmination of a Master’s capstone course at the University of Pittsburgh, Graduate School of Public and International Affairs. The 2-part capstone examined Social Return on Investment (SROI) as a method for better capturing the impact of nonprofit organizations on their beneficiaries and on their communities. Students were exposed to the methodology in the fall and were able to gain hands-on experience by performing the analysis in teams of two on an organization and project of their choice in the spring. The SROI framework is an expansion of the traditional business method of cost-benefit analysis and return on invest. It builds on business analysis methods by including economic, social, and environmental costs and benefits. It uses stakeholders, those who affect and are affected by a project, as its starting point and seeks to understand what they input into a program and how they ultimately benefit from a program, monetizing these costs and benefits to paint a more complete picture of impact. SROI began as an analysis tool for social enterprises but is evolving into a more flexible tool that can also be used for direct service delivery organizations, capacity building organizations and as both a management and fundraising tool. Ten students completed 6 SROI analyses on different types of organizations and projects. Two projects were focused on international trade, and one group of students was able to travel to Nicaragua, the site of their project, to gather data and enrich their analysis. One project examined a local microfinance program, utilizing existing data collected by the organization to tell their story in a new and illuminating way. The preventative healthcare SROI revealed the large amounts of data available and the value of understanding what data is important and what is less so in performing this analysis and in understanding impact. The analyst of another project was employed at that organization and found that she had the benefit of easy data access and understanding of the project, but also found impartial, outside perspectives to be invaluable. Finally, one group examined a capacity building organization that works with nonprofits to improve their ability to serve. This analysis showed how difficult it is to measure the impact of trainings and networks on organizations, but also the use of this analysis in revealing gaps in data. As a class, the students found that there are some common lessons for all organizations. To perform a SROI, an analyst must set clear boundaries for the analysis, set up data gathering tools during the planning stages of a project, take advantage of multiple perspectives, keep track of information, be transparent and utilize already existing data, though it may need to be transformed into a usable format. This analysis is flexible and can be used by many different types of organizations for a variety of reasons, but it is important to always be transparent, determine the important data to collect (and collect it), visit the sites where services are being delivered, utilize pre-existing research t help find monetized value for outcomes, be conservative, understand how the project seeks to reach its goals (and what those goals are) and maintain open communication throughout an organization. SROI is not a static analysis, but a dynamic and continual process. Much can be learned from the SROI process that is beneficial for an organization, for the funding community and for policy makers.

3

Introduction This report and presentation are the culmination of a Master’s Capstone course at the University of Pittsburgh, Graduate School of Public and International Affairs. The capstone, Nonprofit Organizations and Social Impact: Understanding Social Return on Investment and Evaluation Methods, exposed students to this methodology and allowed them to gain professional experience through a hands-on project with an organization and program of their choosing. Learning the methodology, students discussed key questions including: Can nonprofit organizations measure social impact? What are the methods organizations use to evaluate their programs? How can impact be measured and best presented to tell a complete story? Using these tools, students choose an organization and project on which to perform a social return on investment (SROI) analysis. In the fall semester, students were introduced to social impact analysis through a variety of readings and class activities. Working in teams of two, they chose the nonprofit organizations and programs that they would evaluate in the spring. During the spring semester, students designed and executed their SROI evaluations and met weekly to discuss the components and solicit feedback from their instructor and peers. They produced an SROI analysis for one program within their chosen organization and presented the lessons learned by the class throughout the process to the Pittsburgh nonprofit community. The class is comprised of students from the Graduate School of Public and International Affairs, the School of Social Work, and the School of Education. A total of six organizations were chosen by the ten students for analysis (see Table 1). TABLE 1: Capstone Participants, Organizations and Projects Names Organizations Program Jennifer Engel Sarah Heinz House Heinz House Camp Paul Shetler Fast and Emily O’Keefe

Family Services of Western Pennsylvania

Ways to Work

Anne Mulgrave and Mpho Letima

The Midwife Center for Birth and Women’s Health

With Women Fridays

Anne Marie Toccket and Katie Muller

Building New Hope Nicaraguan Coffee Import Project

Meg Pinney and Austin Price

Ten Thousand Villages Guatemalan Artisan Groups

Kathryn Vargas Mid-Atlantic Network for Youth and Family Service

Baltimore Homeless Youth Initiative

SROI is a flexible tool that can be used by a variety of types of organizations. It can be used as a management tool for an organization to understand and clearly lay out how they hope to impact society. Linking inputs to outcomes can help organizations better allocate resources to achieve maximum impact. The process of SROI can also help determine the important data to collect that can be used to improve management decisions.

4

SROI can be used for external use by and for funders to better tell an organizational and programmatic story. It helps to organize data and makes it easier for audiences to understand a programs impact. It can also help communicate to the wider community that a program is not a drain on tax dollars, but helps save society money by providing benefits to direct program participants, to federal and local governments, and to the broader community. An SROI evaluation is a resource intensive process. Before expending the time and money to conduct an SROI, an organization should consider the following questions:

1. How will you use the information? SROI is such a flexible tool that you must be careful how you use it. Decide in advance if you will use the results in marketing, to seek funding, or to make management decisions. An SROI is most beneficial if you plan to use the information produced throughout the process.

2. Is it possible to monetize the impacts that interest your organization? Every group discovered that impacts that seem easy to monetize actually are not. If an organization is focused on specific impacts, do the research on appropriate financial proxies before undertaking the more resource intensive parts of the SROI.

3. How much effort is required to gather the data? Will someone have to review all the

intake forms and client records to gather the data? Do you need to survey current and past clients? Make sure the organization is willing to commit the time and money to collecting the right data.

4. Can you approach the SROI with an open and critical mind? The SROI process often reveals that organization-wide assumptions about a program are incorrect. It is important to be able to let go of those assumptions and follow the data where it leaves. If the SROI is internal, employees not associated with the program should be involved with the SROI to provide a different perspective. In addition, one person must be accountable for collecting accurate information.

Methodology

The social return on investment (SROI) framework is an expansion of the traditional business method of cost-benefit analysis and return on investment (ROI). Cost benefit analysis is a widely used technique to predict the financial impacts and other business consequences of an action. To perform it, the benefits of an action are added up and its costs are subtracted. ROI is a more specific type of cost benefit analysis. It allows decision makers to evaluate investments by comparing the magnitude and timing of expected gains to the magnitude and time of its costs. It is depicted as a ration or percentage in which a ROI ratio greater than 0.00 (or percentage greater than 0%) indicates that the investment returns more than it costs.

5

SROI, originally developed by REDF (formerly The Roberts Enterprise Development Fund), builds upon the ROI method by including economic, social, and environmental costs and benefits. It is an outcomes measurement tool that is a capacity-building process and measurement framework. By incorporating economic, social, and environmental impacts for a range of stakeholders, it more accurately reflects the value that organizations are achieving. At the same time it allows organizations to evidence their claims and demonstrate that specific changes are attributable to their actions. Essentially, SROI encourages a new way of thinking about value and enables a mode of decision-making that is informed by the things that matter to people and communities. In this sense, SROI is not a one-time evaluation or analysis of a program that requires ongoing expert opinions, but a measurement system that should be embedded within an organization, that should inform strategic planning and that stresses that those delivering services are often best placed to engage with their stakeholders and respond to new information. SROI provides a way of expressing, in quantitative terms, the social, environmental and economic benefits of a particular organization or project that are often not reflected in traditional return on investment figures. By assigning a monetary value to the social and environmental benefits of a program, organizations can paint a more complete picture of their impact. When the full range of benefits is compared to the program’s inputs, a new, arguably more comprehensive, return is calculated.1

The calculation of SROI is:

Social Return on Investment = (Financial, social and environmental gains – Cost of investment) Cost of Investment Calculating Social Return on Investment The process for calculating the SROI figure requires additional steps from a traditional cost-benefit analysis. These steps are not entirely discrete and do not always happen in a linear fashion. 1.) Setting Boundaries: The first step is to define the scale and scope of the SROI analysis. Will

a specific project or an entire organization be the center of the study? What time period will be examined? What is the goal of the research? Who is the audience? What resources, specifically time and money, are available?

2.) Identifying Stakeholders: The next step is to determine and prioritize stakeholders (those

who affect or are affected by a project). Stakeholders are at the center of SROI analysis since the benefits they receive are ultimately used to determine the SROI ratio. It is essential to identify those who affect the project or are affected by the project early in the process so their voices come through in the final analysis.

1 J., Wachowicz, J. and Chun, S. Emerson, "Social Return on Investment: Exploring Aspects of Value Creation in the Nonprofit Sector," 1999.

6

3.) Collecting Data: The analyst determines the inputs, outputs and outcomes for each of the stakeholders and an indicator to measure if changes are occurring. Information for this stage is gathered by engaging stakeholders through interviews, surveys and other forms of primary research. 4.) Selecting Financial Proxies: The analyst determines values for non-financial inputs, outputs and outcomes. This is done by examining potential financial proxies and determining which most accurately reflects the social and environmental impacts of the project. Financial proxies should be reasonable and easily justifiable and should only be used when there is no direct monetary value for the outcome. 5.) Calculating Impact: Once the steps above are completed, the analyst calculates the impact of the program or project. To do this, a number of caveats are established. First, not all outcomes will be monetized. Given resource limitations, it is often not possible to monetize all outcomes. In addition, the analyst must determine the extent to which outcomes would have occurred in the absence of the program and how much benefit can be attributed to this program. This often requires secondary research from scholars, policy makers and practitioners. The total costs and benefits of the program and project are determined and entered into the formula above.

Key Terminology Stakeholder: Anyone who affects or is affected by the program

Input: Investments, monetary or otherwise, into the program

Output: Activities resulting from program

Outcome: The change resulting from the program’s activities

Indicator: Piece of information the helps determine whether change occurred

Proxy: The substitute value assigned to a non-monetary outcome

Deadweight: The extent to which outcomes would have occurred in the absence of the program

Attribution: The amount of the impact that directly results from the program

Impact: Outcomes, less deadweight and adjusted for attribution

7

Case Studies The next section summarizes the case studies conducted by the class: Building New Hope’s Nicaraguan coffee import project, Family Services of Western PA’s Ways to Work project, Sarah Heinz Houses’ Heinz House Camp, Ten Thousand Villages’ Guatemalan artisan project, the Mid-Atlantic Network for Youth and Family Services work with the Baltimore Homeless Youth Initiative and The Midwife Center for Birth and Women’s Health’s With Women Fridays project. These programs offer a variety of costs, benefits, and challenges. They demonstrate the range of data collection methods used in SROI as well as the diversity of nonprofit organizations that could benefit from using SROI. Building New Hope is a coffee import project that primarily benefits farmers in a small Nicaraguan village. The team of students traveled to Nicaragua in February of 2010 to collect data and create a better understanding of the impacts of this program, discovering the wide variety of activities resulting from this program. Their fieldwork enabled data collection that was unavailable from the local Pittsburgh office. In contrast, the group working with Family Services of Western PA’s Ways to Work program utilized existing data already collected by the organization to perform their analysis, creating a framework that would be helpful in management, fundraising, and could be easily replicated by the organization. Sarah Heinz Houses’ Heinz House Camp analysis was performed by a project manager of Sarah Heinz House, presenting a different perspective from the rest of to class by working from the inside rather than the outside. The projects chosen are indicative of the variety of nonprofit organizations that exist and also the ability of SROI analysis to be flexible and adapt to different contexts. Case Study 1: Building New Hope

Building New Hope (BNH) is a 501(c)(3) organization based dually in Pittsburgh, Pennsylvania and Granada, Nicaragua. Founded in 1992, it supports and operates grassroots development projects in Central America. In order to further its mission of providing “support and partnership to communities in development in Central America, particularly in El Salvador and Nicaragua," Building New Hope operates a wide range of programming, including an organic coffee import project, a veterinary clinic, alternative schools and literacy programs. This SROI focuses on Building New Hope’s coffee import project in 2008 and projections for 2009-2013. T Building New Hope provides support to the coffee cooperative El Porvenir (literally, “the future”) through the purchase of the farm’s organic, shade-grown coffee and through direct assistance for community investment. The organization has been paying above fair trade prices for the cooperative’s coffee since 2002. The coffee is roasted in Southwestern Pennsylvania and sold by coffee shops, fundraising groups and other vendors throughout the region and the country. After paying the farmers, the profits from the coffee project, along with support from private donations, goes toward development projects in El Porvenir and other parts of Nicaragua. In El Porvenir, Building New Hope has provided funds for a partial-road, scholarships for post-secondary students and a supplemental teaching salary. Coffee revenue also funds

8

approximately 50 percent of Building New Hope’s programming based in Granada, Nicaragua including the alternative schools and veterinary clinic. We traveled to El Porvenir in Nicaragua in February 2010, enabling data collection unavailable from studying from Pittsburgh. In doing so, we found that the project has an extensive range of activities creating a large number of stakeholders. Stakeholders in or near El Porvenir include the farmers and their families, seasonal coffee workers, and those, such as the teacher, students and scholarship recipients, who receive direct support through Building New Hope’s community development initiatives. Other organizations that work with El Porvenir, such as Jubilee House and Theirbucks, are also stakeholders. The community in which the farmers live on is also considered a stakeholder since the farmers have a large impact on the existing local ecosystems. Additionally, those who participate in or volunteer with the Granada-based projects have a stake in the coffee project, as those projects are funded in part by coffee revenue. Finally, there are stakeholders who are found in the United States: Building New Hope, the roasting company, coffee shops and other vendors, fundraising groups, volunteers and the Pittsburgh community as a whole. Our SROI shows that Building New Hope’s organic coffee import project is positively affecting social and economic outcomes for the project’s key stakeholders in Nicaragua and the United States. Based on data for 2008, for every one dollar invested in the project, $3.05 of benefit is generated. Driving the high return are the following outcomes:

• increased wages for the farming community, seasonal employees and youth, • increased and improved programming for Building New Hope’s Granada office, and • improved preservation and promotion of local ecosystems.

Looking ahead, the social return is projected to rise over the course of the next five years, peaking in 2013 at $8.43. As the coffee farmers continue to earn above fair trade wages, they are able to invest in their community and ensure the livelihoods of their families and the continued practice of their craft.

$1.00 Invested

$3.05 Benefits Realized

Figure 1: Future SROI Projections

9

Case Study 2: Ways to Work

Ways to Work (WTW) is a microfinance program of Family Services of Western Pennsylvania (a local nonprofit organization). The program is also a branch of a national organization (Ways to Work Inc.), which provides technical support, advocacy assistance, and serves as the lending institution for some WTW clients. WTW provides “high touch” microloans (of up to $6000) to low income families who would otherwise not have access to credit so that they can buy a used car, or repair a car they own. Applicants must attend financial literacy classes and demonstrate stable employment. WTW’s combination of services leads to improved independence and family welfare (see Figure 1). WTW operates in Allegheny, Armstrong, Beaver, Butler, Fayette, Greene, Washington, and Westmoreland counties. It funds up to 100 loans per year.

Figure 2: Theory of Change

In this study, the approach taken is to make the Social Return on Investment (SROI) methodology as useful as possible to FSWP and WTW. For this reason, the data used in the evaluation was restricted to information already collected by the agency and is readily available to people within the organization. The goal has been to create a useful, robust, conservative, and defensible evaluation framework that would be helpful in management and fundraising and could easily be replicated by the agency in future years at essentially no additional cost. With conservative assumptions, the monetized impact can be seen as the value of the loans on changes in employability, income, travel time, and financial literacy were monetized. Also important impacts of the program include financial independence and security, self-esteem, and greater ability to access mainstream finance that were not non-monetized due to a lack of time and resources. The outcome monetization calculations for the period 2007-2009 are:

High touch loan to low income family

Reliable transportation and increased financial literacy

Increased ability to work, study, and manage money

Increased income, ability to access financial institutions, and financial indendependence

Improved Independence & family welfare

10

0.27

0.53

0.92

0.000.100.200.300.400.500.600.700.800.901.00

2007 2008 2009

SRO

I Rat

io

Year

Table 2: Monetized Impacts

Outcomes n= Average annual effect per person

Years of impact Gross Impact

Net present value (3% discount rate)

Change in employability

150 $ 131 4 $ 78,199 $ 74,749

Change in income 150 $ 79 4 $ 17,172 $ 16,415

Change in travel time

150 $ 2155 4 $ 1,217,592

$ 1,163,888

Change in financial literacy

312 $ 208 1 $ 64,818 $ 61,896

The total net present value impact for three years of service is estimated at $1,317,012, while the total cost (social input) is $815,667. This yields a net benefit to society of $501,344, or a social return on investment of 0.61. This means that every dollar put into the program was repaid, plus yielded an additional 61 cents of social impact. It is also important to note that because of the conservative assumptions made and the many non-monetized impacts in this evaluation, these figures represent minimal estimates of the program’s social return. Additionally, the program is still young, and its SROI has been increasing rapidly during this period (see

Figure 3). This means that each year the social benefits of the program have been growing in proportion to the costs. This rise in social impact has continued despite the recession of 2008-2009, which has tended to reduce the employment and income (drivers of impact) for the people served by this program. This increase in the ratio of net benefits to costs despite the economic circumstances is driven by continually serving more clients, more efficiently.

$1.00 Invested

$0.61 Benefits Realized

Figure 3: Change in Yearly SROI

11

Case Study 3: Heinz House Camp Sarah Heinz House (SHH) is a Boys & Girls Club located on the North Side of Pittsburgh. The mission of Sarah Heinz House is to develop and strengthen the leadership, character, and life skills of especially those children who need us most by providing high quality recreational, social, educational, and mentoring programs which enable youth to reach their highest potential; to use our exemplary programs, facilities, and staff as a resource to the larger community. Today, the organization has close to 1,100 members and serves close to an additional 3,000 youth through outreach. The Heinz House camp has 4 10 day sessions each summer, 2 for boys and 2 for girls. There are typically 78 campers per session and each camper can stay for 2 sessions if they choose. There are many activities in which the campers participate each year including waterfront activities (swimming, boating, fishing), high and low ropes courses, sports (like soccer, archery, and softball), arts and crafts, and outdoor skills. The campers range in age from 7-14. From age 15-17, campers become Counselors in Training (CIT) and although technically still campers, they begin to prepare and train to eventually become camp counselors. There are 18 staff for each camp, 15 camp counselors (of which, many fill specialty roles as well such as First Aid Director, Waterfront Director, etc.), two cooks and a Camp Director. Heinz House Camp’s budget is just over $148,000, which includes utilities, staff, supplies, and everything else related to the camp’s operation. Approximately $58,000 is generated by camp fees and the Summer Food Service Program administered by the state. The remaining $90,000 comes from a variety of other sources, including grants and individual donations specifically earmarked for camp, the Heinz Endowments, and operating income from other programs. Out of the $148,000 budget for camp, $49,000 of that expense would exist even if the camp program did not exist (but SHH still owned the property). The goals of the Heinz House Camp are depicted in the programs’ theory of change to the right. Data was obtained from surveys were conducted with campers, parents, and seasonal staff. Survey results were tabulated and used to develop questions for focus groups and individual interviews with campers, parents, seasonal staff, volunteers, alumni of the camp, and full time staff. Addition research was conducted through journal searches and from Richard Louv’s book “Last Child in the Woods”.

InputsChildren

aged 7-17Natural setting

SHH InputsHighly trained

compassionate staff,

Volunteers (Program

and maintenance)

, Programs

Outcomes - childrenDecreased ADD Rates

Increased exposure to natureStronger friendships and support networks

More skilled teens and young adultsHealthier lifestyles

Children with increased skills (life and others)

12

The results of this research yielded the following set of outcomes and indicators that were then monetized:

• Reduction of rates of ADD • Skills learned from participation at camp • Childcare/other camp costs for children • Processing water costs • Scholarship opportunity for camp staff and CIT’s • Decrease in need for therapy due to stronger friendships and support networks • Number of volunteer hours (youth, program adults, and maintenance) • Training hours offered to staff and Counselors in Training (CIT’s)

Although Heinz House Camp has additional outcomes that are very important to its various stakeholders (such as becoming more independent for campers), those outcomes were not monetized. The results of this SROI are that for every dollar invested in Heinz House Camp, a $2.10 social benefit is seen. The $1 to $2.10 ratio is comprised of only those financial indicators listed above. The overall impact of Heinz House Camp would appear to be much greater, but the other outcomes are not able to be monetized in a SROI.

$1.00 Invested

$2.10 Benefits Realized

12

Case Study 4: Ten Thousand Villages

Ten Thousand Villages was founded in 1946 with a mission to provide income to poor artisans by selling their products in the U.S. market. It has grown to include more than 100 artisan groups around the world whose crafts line the shelves of more than 150 retail stores in the U.S. and Canada. The artisan groups that produce for Ten Thousand Villages receive both financial and creative support from the organization and are guaranteed a fair wage and sustainable income for their products. The payments they receive for their products are divided between artisan wages and the various social programs that the artisan groups sponsor. These programs vary by group and include assistance in education, health, job training, and nutrition. As a social enterprise, Ten Thousand Villages creates social value far beyond the wages the organization generates for poor artisans around the world. This report analyzes the social return on investment that Ten Thousand Villages generated working with three artisan groups in Guatemala in 2009: a snapshot of the much larger impact that the organization makes worldwide. The Artisan groups Ten Thousand Villages works with in Guatemala are predominately female and indigenous, a demographic traditionally marginalized by society and underserved by the public sector. Considering malnutrition rates in Guatemala are among the highest in the world and there are staggering economic and social inequalities between rural and urban dwellers, Ten Thousand Villages’ work in these communities creates substantial social and economic benefit. In this report, we attach a monetary value to the benefits that the artisans, their families, and their community receive by working with Ten Thousand Villages across four indicators: (1) increased income (2) increased education for their children (3) increased nutrition (4) increased government revenue. The result of Ten Thousand Villages operations in Guatemala is a social impact of 0.66. In addition to the monetized benefits, the report discusses a multitude of unquantifiable benefits delivered by the artisan group programs such as: increased gender equity, increased childhood nutrition, and improvements in social capital and political organization.

Ten Thousand Villages

establishes an artisan group relationship

TTV assists in product

development and provides 50% advance

payment

Artisans are paid a fair wage

based on production and

level of difficultyFinished

products are shipped to TTV warehouse and artisan groups

receive final payment

N. American shoppers purchase

artisan products at TTV retail shops and

online

TTV uses profits to

invest in new artisan

relationships

Figure 4: Theory of Change

$1.00 Invested

$0.66 Benefits Realized

13

Case Study 5: Mid-Atlantic Network for Youth and Families: Baltimore Homeless Youth Initiatives The Mid-Atlantic Network for Youth and Families (MANY) is an intermediary organization that provides capacity building consulting services to a membership network of service providing organizations for youth and families. In 2007, MANY was awarded a three year grant of $750,000 to work with the Baltimore Homeless Youth Initiative (BHYI) and provide capacity building services to a city-wide partnership of organizations to address the needs of Baltimore’s homeless youth, ages of 14-24. MANY’s primary role in this initiative was to provide technical assistance, trainings, and fund development to build organizational capacity and improve services for over 150 BHYI affiliated organizations. MANY, as a intermediary capacity building organization, has a great impact on the success and professional development of regional service providers and introduces a different type of organization that may benefit from SROI analysis. Figure 5: Theory of Change

There is a growing interest and need for effective intermediary organizations in the nonprofit sector. Intermediary organizations often serve as the go between for large funding sources and a collective body of service providers. Intermediaries also often have a strong membership network and provide their beneficiaries with affordable services to ensure that they are promoting sustainable development in the service sector. The primary stakeholders in this analysis are the participating service providers and government agencies, the funding body in this case the U.S. Department of Health and Human Services, and MANY. The Homeless Youth of Baltimore are also recognized as a key stakeholder however, due to the scope of this project, this population was not considered in the analysis.

Networking

•Create a network of youth serving nonprofits and government agencies that can support each other and work together to address the needs of homeless youth

Capacity Building

•Provide trainings, technical assistance, community engagement, access to funding sources and information sharing, to give these organizations the tools needed to adequately support homeless youth

Impact on Services

•Youth in Baltimore now have resources and services available throughout the city due to a network of capable organizations that have gained the skills necessary to provide sustainable and adequate services

14

Participant surveys were conducted as the primary form of data collection for this SROI. Of over 200 surveys sent, 47 surveys were completed. The survey’s main goal was to determine the specific capacity building tools gained from participation in the BHYI that beneficiary organizations identified as most valuable to their professional development. It was the purpose of this project to identify the key areas that MANY contributed to the growth of their participants, listed above. A survey of secondary data to determine possible monetary indicators reveals that the work of capacity building organizations remains vague and difficult to define among the nonprofit community. Social capital, networking, and the value of information sharing all have varied definitions, meanings, and uses. Without an agreed upon indication of these added value areas it was not feasible to determine justifiable monetization of these indicators. Additionally it is difficult to determine how many organizations utilized the funding resources that were made available and how many actually received additional funds. Long term data tracking would make this monetization possible. Since it was determined that these key indicators could not be monetized, the value added by MANY to these service organizations was its trainings. The total cost is for one year of the BHYI related trainings ($90,000). The total benefit was determined by the number and value of professional hours that were dedicated to participating in the BHYI activities. The survey helped to give an indication as to the appropriate value. The rationale for using this number is that this is the value that society places on these participants time. There was 38 hours of activity, attended by 137 participants. The calculation is done in a range, to account for the range of values that are assigned to the hours of participation in the trainings ($28-$45). This report is unable to determine the monetized equivalent of the areas of added social value, a piece that is critical to an accurate SROI in MANY’s programs. In order for SROI to work for capacity building intermediaries a number of things need to happen:

• The nonprofit sector needs to make a concerted effort to determine a common understanding of some of the broad concepts that often fall under the work of intermediaries. Two prime examples: capacity building and social capital.

• Intermediary organizations should collect long term data that tracks the progress of the beneficiary organizations. This would make it more feasible to attribute social impact specific to the intermediary interventions.

$1.00 Invested

$1.31-$1.61 Benefits Realized

Identified Areas of Added Value

1. Built social capital & network

2. Information sharing/ legal consulting

3. Knowledge of/access to additional funding sources

15

Case Study 6: With Women Fridays at The Midwife Center for Birth and Women’s Health With Women Fridays is a program of The Midwife Center for Birth and Women’s Health that provides easily accessible free or low-cost healthcare to low-income women. The population of this study is 150 low-to-moderate income With Women Fridays clients who were either uninsured or had public health insurance and received care at The Midwife Center for Birth and Women’s Health in 2009. The program is based on the relapse prevention model that strives to improve overall health through counseling on global lifestyle changes during easily accessible medical visits. Thus, the medical service provided during With Women Fridays is only one goal of the program. The overall goal is to encourage better health through holistic care that empowers women to become more proactive in their healthcare. We reviewed 1,200 of the 1,500 medical charts for 2007 through 2009 to locate client women with no insurance or public insurance. Information on 48 different variables was then gathered from the charts of uninsured and publicly insured midwife center clients. These variables included age, race, income, number of visits, type of visits and tests and treatment given. This data collection revealed that The Midwife Center for Birth and Women’s Health treated 150 With Women Fridays clients who either had public insurance or were uninsured in 2009. This represents 75% of With Women Fridays clients. Privately insured women account for the remaining 25% of With Women Fridays clients. A social return on investment analysis is only as good as its assumptions. While searching for appropriate dollar values to assign to social impacts, the decisions contained herein were deliberately conservative. The cost of health care in the United States is by most measures

16

exorbitant. Relying exclusively on cost-effectiveness analyses, which calculate the cost to save a quality year of life from a given procedure or test, results in a much higher SROI. Instead, cost-effectiveness analyses were only used to value to measures proven to save lives: blood pressure, breast cancer and cervical cancer screening. All other medical care impacts received a value based on cost savings, not increased life expectancy. Detailed calculations for each monetized impact are contained in Appendix A of the full report. The research established that the total cost of With Women Fridays is $137,615. Approximately 25% of With Women Fridays clients have private insurance. Thus, the cost of treating uninsured or publicly insured With Women Fridays clients is 75% of $137,615 or $103,211. The total monetized impact of With Women Fridays ranges from $470,485 to $498,045 yielding a SROI between 3.55 and 3.82. In other words, for every dollar invested in With Women Fridays, there is a social return on that investment of between $3.55 and $3.82.

Lessons Learned Through a diverse set of team SROI projects, the Capstone course discovered the following important lessons:

1. Set clear boundaries that match your time and resources to the scope of the project. Clearly defining the boundaries of the process is important because the process itself is fluid and changes over time. Being able to ground the research with distinct limits clarifies the process.

2. Set up a plan to gather the data before you begin an SROI analysis. Consider keeping track of participants over time to develop a longitudinal view of the SROI. It would also be helpful to set up a data gathering plan during the planning stages of a project to ensure the best results.

3. Gather multiple perspectives. It is easy to get so lost in the details of the process

that you lose sight of the end goal. Having other people involved in SROI to check your assumptions, financial proxies and thought processes substantially improves the quality of the end product.

4. Keep careful track of formulas, sources and assumptions because the process is so complex that it is easy to forget what you have done.

$1.00 Invested

$3.55-$3.82 Benefits Realized

17

5. Make assumptions and proxies clear and understandable. An SROI is only as good as its assumptions. When determining a financial proxy to use for an indicator, be transparent and use commonly agreed upon values as much as possible.

6. Be ready to transform data. The nonprofit organizations analyzed collected data,

but that data was often not in the best form to perform an SROI. Sometimes the data is in paper format, while other times it is not located in one place. It may take some time to properly organize data that an organization already possesses.

Conclusions and Recommendations As a class, we found that:

1. Data collection is both time intensive and vital to success. Much of the data needed for these projects was maintained by the organizations in hardcopy paper files. Other data was divided over different parts of the organization and even in different countries. However, the process cannot be of use if the data is not gathered and organized properly.

2. Site visits are both important for

understanding impact and are sometimes expensive. All projects found their site visits very beneficial in understanding the impact of the program. However, one of those site visits was to Nicaragua, proving that they can also significantly increase the cost of the SROI evaluation, yet allowed the project to be more complex and detailed.

3. Finding appropriate financial proxies is

complicated and often requires making tough decisions. The decisions of what outcomes to monetize and how to do it will have a large impact on the results of the evaluation. These decisions must be conservative or the SROI’s legitimacy will be vulnerable.

4. Conservative assumptions and proxies

increase the reliability of the report. It is understood that these programs have

Working with Nonprofits: SROI for Capacity Building Intermediaries 1. The activities of organizations working with nonprofits are difficult to monetize because concepts that are often used to describe their work such as "Capacity Building" and "Social Capital" do not have universally accepted definitions. Without a clear definition it is difficult to apply an adequate monetary value to measure success. 2. Organizations working with nonprofits have a broad impact, making it difficult to "draw the line" at how much impact can be attributed to them. Measuring attribution becomes critical because it becomes more difficult to assess how much of the impact is created by the organization working with the nonprofit and how much is driven by the direct service providers involved. 3. Organizations working with nonprofits need to keep long term data to best identify their impact. This should be a very careful and detailed evaluation of work over a number of years. Evaluations should go beyond the quality of services provided and also include follow up as to how service providers utilize the information or capacity building skills they gained from working with the intermediary organization.

18

benefits that are not monetized in an SROI and these benefits are sometimes very important. However, in order for the SROI to be accepted in a wider community, assumptions about the benefits, particularly the duration of benefits, should be conservative.

5. A theory of change improves SROI. It is important to clearly define how a program

plans to reach its goals in the short and long terms. A clear picture of the process helps to identify the outcomes that result from inputs and outputs, allowing for a SROI analysis to be conducted.

6. Organizational communication about SROI is important. Nonprofit organizations

are often understaffed and SROI may be perceived as more work for already overworked staff. However, by communicating SROI benefits to all staff, and by including them throughout the process, the SROI will improve the understanding of the benefits and costs of the program and help to better inform management and funding decisions.

Finally, an important question for an organization contemplating performing an SROI analysis is: Are you committed to learning from the process, not just the number? The SROI process reveals a tremendous amount of information about a program. The process itself is beneficial and focusing on the end number can limit learning from the process.

19

Contact Information

For more information regarding the individual case studies and to access the full reports, contact the students directly.

• Building New Hope: Anne Marie Toccket at (aut112@gmail.com) or Katie Muller at (Katie.j.muller@gmail.com)

• Ways to Work: Paul Shetler Fast at (paulsfast@gmail.com) or Emily O’Keefe at (eeokeefe@gmail.com)

• Heinz House Camp: Jennifer Engel at engel@sarahheinzhouse.com. If you are

interested in learning more about Heinz House Camp, please contact Bob Bechtold at Bechtold@sarahheinzhouse.com. If you are interested in learning more about Sarah Heinz House, please contact Jennifer or Bob or call 412-231-2377.

• Ten Thousand Villages: Meg Pinney (mrp44@pitt.edu) or Austin Price

(austinprice2@gmail.com).

• MANY: Kathryn Vargas (kcvargas22@gmail.com).

• With Women Fridays: Anne Mulgrave, JD at (anne.mulgrave@gmail.com) or 412-215-1464.

For general information, please contact one of the instructors:

• Kathryn Collins: kmc45@pitt.edu, Sabina Deitrick: sabinad@pitt.edu, Tracy Soska: tsssw@pitt.edu