Series 1- Social innovation, governance, impact, accountability

Measuring the impact of social innovation

Mireille Mathieu, Ph.D.

Professeure émérite de psychologie

Université de Montréal

1

Social innovation

Social innovation• Social innovation is a dynamic that, in response to

current clearly defined social needs, offers more appropriate and more sustainable solutions

• Social innovations taken up by an institution, an organization or a community offer measurable benefits for the collective, rather than simply for certain individuals. The impact of social innovation is transformative and systemic. By its inherent creativity, social innovation represents a break from what is.

(Réseau québécois de l’innovation sociale)

2

Social innovation

Social innovation

“Despite its undeniable foundation in research, social innovation is often, as we know, the unexpected outcome of a social tinkering that emerges from a context of need for change, for adaptation or for a more radical transformation. It calls for the involvement, conscious or not, but still essential, of a plurality of actors from civil society, from politics, public administration and research. “ (Louise Dandurand, 2013)

3

University and innovation

Innovation, social as well as technological, should be part of the DNA

of the University

4

University and social innovation

University has two main roles:• Produce social innovation

• Support the dissemination and implementation of sustainable social innovation

5

1. Produce social innovation

• Research, particularly, collaborative research, provides basic knowledge and tools for social innovation, for solving well defined problems

• Innovative training programs and innovative training methods should be the trademark of our universities

• Lifelong learning initiatives, for ex: University of the third age, MOOC (massive open online courses)

6

2. Support the dissemination and the implementation of social innovation

(Third mission)• Life long learning

• Knowledge transfer and innovation

• Social engagement

7

Why should University measure effects of social innovation ?

• “What gets measured gets valued”

• To improve program management

• To increase understanding of the « results »

• To achieve stronger communication of the value of its work to the people that matter (internal and external stake holders)

• To enhance attention to the social, economic and environmental value created by the institution

8

Accountability

• Inputs - resources invested in the activity

• Outputs – the direct and tangible products from the activity i.e. people trained, products sold,

• Outcomes – changes to people resulting from the activity i.e. a new job, increased income, improved stability in life

• Impact – Outcomes less an estimate of what would have happened anyway

9

AccountabilityWho? How? With whom? For whom?

• Descriptive and performance Indicators (a few examples)– Number of research grants and publications by

faculty and by graduate students – Number and description of innovative training

programs, number of students, number of graduates, data on employment etc.

– Data on knowledge and innovation transfer (number of products, number of target organizations, identity and number of external stake holders etc.)

– Nature, number and degree of satisfaction of external stakeholders involved as partners

10

AccountabilityWho? How? With whom? For whom?

• Development of many, many set of indicators over the last decade (see recent literature review by RQIS)

• The more important challenge lies in the choice of the relevant indicators at the very beginning of the project

• Even though these sets of indicators don’t allow for a satisfying evaluation of impact, they are the first step towards that goal. They at least provide a clear and reliable picture of inputs and outputs

11

AccountabilityWho? How? With whom? For whom?

There is an urgent need for better and comprehensive monitoring of inputs and

outputs by Universities.

It is as a necessary pre-requisite to evaluation of impact of social innovation

12

Evaluation of outcomes and impact of social innovation: a risky endeavor

By its very nature, innovation is:• Risky;• Unpredictable in terms of:

• which particular activity or intervention will work or prove useful;• who will benefit;• when benefits, if any, will occur;• under which particular set of circumstances an innovative approach would be applicable;

• whether the discovery and application will be asintended or of quite a different nature

13

Measuring impact: A risky endeavor

• “The indicators to measure the effects of social innovation are probably those that are less developed . Social and intangible impacts are particularly difficult to measure due to various factors. Note, among others, the specific nature of social innovations whose effects are less in terms of economic performance when compared to the technological innovations. The complex and dynamic nature of social innovation process also makes the task more difficult” (RQIS, 2015)

14

Measuring impact: A risky endeavor

• Social innovations are generally long term in nature, sometimes very long term

• Social innovation never occurs alone but always in a context of structured relationships, networks, infrastructures and in a wider social and economic context

• It is tempting to want a simple matrix of aggregate measures. But social change is messy. We can’t loose the context and non measurable insights

• Even with a clear picture of outcomes it is another thing entirely to tease out how the organization contributed.

15

A risky endeavor

• On average, good plans, people, and businesses succeedonly one in ten times.

• It should be considered a learning opportunity ratherthan a problem that as many as 80–90 percent ofinvestments or projects do not work out well, or evencollapse completely

• Evaluation of impact of social innovation is also verytime-consuming and very expensive

16

Innovation to Impact: Obama’s Social Innovation Fund

• Innovation to Impact: Obama’s Social Innovation Fund

• Evidence• All programs funded by SIF must be able to

demonstrate a preliminary level of effectiveness andthen take part in a rigorous evaluation to strengthentheir base of evidence and to document and assesswhether their approach works more effectively, costsless, or leads to better results for our country’scommunities. Too often, effective nonprofits lack theexpertise, resources, or infrastructure to conductmeaningful evaluation that demonstrates impact andcan take their programs from promising to proven. Infact, many grant programs do not adequately supportprogram assessments. However, SIF provides bothtechnical assistance and funding for requiredevaluations of all selected programs. These evaluationswill provide robust information about which programsand models generate strong results for individuals andcommunities and how they can be replicatedsuccessfully

17

Towards adequate evaluation of outcomes and impact

• Focus on careful analysis of case studies so as to learn from errors as well as from successes and put the emphasis on best practices

• Use Developmental evaluation both for monitoring the evolution of a given project and as an innovative accountability strategy for long term social innovation projects

18

Conditions for a successful developmental evaluation (1))

• The organization is trying to solve a complex problem with an adaptive solution

• The organization is open to testing new evaluation approaches

• The organization’s leadership is willing to take risks, be flexible and make necessary changes to the initiative

19

Conditions for a successful developmental evaluation (2)

• The organization’s values and culture support innovation, continuous learning and adapting to its environment

• The organization provides sufficient time, people , and financial resources for ongoing enquiry

• The organization is committed to open communications and ensuring that information is accessible and used internally and externally

20

Measuring impact:social return on investment

• A way to monetise the value of the social impact in financial terms

• Very often the result is expressed in avoided costs rather than in an increase of funds (ex: quality of life at work and depression)

• SROI is a way of thinking and we are not able to monetize everything

• SROI is just a number that is one part of the story but it is a powerful tool to convince stakeholders…

21

Self assessment of university leadership’s support of social innovation

• Another important step towards a culture of evaluation of outcomes and impact of social innovation

22

Self assessment of university leadership’s support of social innovation:

a few indicators• Mission statements and strategy documents

• Faculty promotion criteria (social innovation, knowledge transfer…)

• Effective monitoring and use of indicators of knowledge transfer and social innovation in internal and external reports

• Effective monitoring and use of indicators of social engagement in internal and external reports 23

Conclusion

• Universities should:

– Put more adequate resources in evaluation of social innovation

– Strongly support the development of new models for better evaluation of impact of social innovation

– Put more emphasis on their third mission: revitalize their social contract with society

24

Conclusion

• Universities should:

– Monitor more extensively their effective involvement in social innovation both in production and in support of implementation

– Monitor the treatment by promotion committees of knowledge transfer and innovation and take relevant measures and rewards to foster a culture of social innovation and of evaluation of social impact

25

Conclusion

The prize for getting it right will be considerable: more productive and valued universities engaged in the cultural and economic development of their host societies.

26

Knowledge transfer and innovation in Québec

• Services aux collectivités (services to the community) UQAM

• (CLT) Centres de liaison et de transfert -Centers for liaison and transfer

• (OLTIS) Organismes de liaison et de transfert de l’innovation sociale -Organizations of liaison and transfer of social innovation

27

Social innovation: the Québec reality

OLTIS and CLT act as catalysts of social innovation in support of the university researchers, of practice settings and of the community at large

28

CLT and OLTISCLT and OLTIS enhance the access of organizationsand communities to new knowledge and to theresults of research in the humanities and socialsciences generated by universities and publicresearch centers.

They also help the entire society benefit from theeffects of that knowledge on the quality of life ofindividuals and communities and on the productivityof organizations(Government of Quebec)

29

OLTIS and CLT• Networking between producers and users of research

and social innovation• Animation in practice settings• Analysis and dissemination of information• Development and use of strategic intelligence tools• Translation of research results into usable tools for implementation

of best practices• Supporting sustainable appropriation of best practices by

individuals and organizations in the community• Strong emphasis on evaluation of effects of social innovation

30

OLTIS supported by the Government of Québec

• CLIPP: Centre de liaison sur l’intervention et la prévention psychosociale

• CETREQ: Centre de transfert pour la réussite éducative su Québec (CTREQ)

• TIESS-OLT: Territoires innovants en économie sociale et solidaire Liaison et transfert

31

CLT supported by the Government of Québec

• CEFRIO: Centre facilitant la recherche et l’innovation dans les organisations l’aide des technologies de l’information et de la communication (TIC)

• CIRANO: Centre interuniversitaire de recherche en analyse des organisations

32

Réseau québécois en innovation sociale (RQIS)

The Québec Social Innovation Network (RQIS) is a space for knowledge and experience mobilization that helps make social innovation an essential lever for sustainable development of Quebec.

34

Réseau québécois en innovation sociale (RQIS)

• A network of the main actors of social innovation in Québec

• A very important and comprehensive literature review of indicators for evaluation of social innovation (2015)

• The production of a Guide for choosing indicators of social innovation (in progress)

• A laboratory for developing new methods for evaluation of impact of social innovation

35