My name is Rochelle Zorzi

And I am Yasser Ismail

And this little dude in the photo is how excited we want our clients to be when we talk to them about what our evaluations can do for them. If evaluation was cake, and that is our client, that’s the level of excitement we’d love to generate about evaluation.

And today, we are going to talk to you about how as evaluators, we can use storytelling as a way of demonstrating the impact that an evaluation makes.

But first we have an important question to ask you.

1

Rochelle: This is a blueberry chocolate cake I recently learned how to make. How many of you in this audience would be happy to try a piece if I invited you over for dinner?

2

Rochelle: Now, if Yasser also invited you over for dinner and offered these delicious looking ingredients as his dessert, how many of you would rather still have the piece of cake?

Most probably the show of hands will be for the cake/ Rochelle.

This is not surprising to me. I think it is reasonable to expect that when offered a set of ingredients as opposed to the cake, people will choose cake.

Evaluation as a field could perhaps benefit from more cake – let me tell you a story that might help you see where I am going.

One time Cathexis thought we had something to offer private sectorSet up meetings with # of private firms.

3

We went in and made lots of great presentations about WHAT we could do for them and told them about HOW we could do it for them and how great our team was and how excellent our methods.

But all our information was about what and how.We were telling them about the variety of cakes we could make them and showing them our ingredients and utensils and talking about the excellence of our team of chefs.

4

We could tell them about all of that, but we couldn’t tell them about how our past clients had saved money, or how past clients had been able to help more people with same amount of money – we couldn’t really tell them what difference evaluation would make for them!

We couldn’t even offer them photos of the cakes we’d made before let alone offer them a taste of how good our cake would be!!

For a while, a group of us has wanted to document the benefits of evaluation. We’ve wanted to understand: “What value do people get for their money? Do things actually improve as a result of evaluation?”

---

Not important only for evaluation consultants.

Important for evaluators as a fieldFed Gov—if we can’t demonstrate if it doesn’t make an impact/difference beyond an

5

accountability exercise, someone is going to look at the budget line and ask why on earth we are doing this…

**If we don’t understand which of our evaluations are making a positive impact/negative impact, we are not going to know how our work is affecting the programs we evaluate

we didn’t have SPECIFIC EXAMPLES that could illustrate (stories ARE specific examples, and they have the power to engage anyone and everyone)

5

When EvalPartners announced their Innovation Challenge to promote the use of evaluation, it was a perfect opportunity to correct this situation. We gathered an international editorial board to lead the project, and set ourselves the task of compiling and sharing a collection of stories about evaluations that have made a difference.

Copyright: <a href='http://www.123rf.com/profile_jezper'>jezper / 123RF Stock Photo</a>

6

We launched the International Call for Stories about “Evaluations that Make a Difference” in October.

7

We ended up getting 61 submissions

Talk about main criteria for selection. Each of these evaluations were selected as examples because they were deemed by the editorial board to have made a difference.

(We needed to explain to the evaluation community what we meant by “Evaluations that Make a Difference”.)It seemed to be a difficult concept to grasp. So, with the help of cartoonist Chris Lysy, we made a little video.

9

It seemed to be a difficult concept to grasp. So, with the help of cartoonist Chris Lysy, we made a little video.

http://www.youtube.com/watch?v=0Mg4vOop8PM

(Insert and play video)

10

(We needed to explain to the evaluation community what we meant by “Evaluations that Make a Difference”.)

To summarise:

It is not sufficient that the evaluation showed that the program being evaluated made a difference. We wanted examples of how use of the evaluation findings led to improvements in people’s lives.

Improvements in people’s lives = improvement in people’s lives; improvements in the environment; social betterment

11

12

Describe our end goals, what we want to achieve (inspirational) – may need a different picture

Image copyright: <a href='http://www.123rf.com/profile_veneratio'>veneratio / 123RF Stock Photo</a>

Yasser leads storytelling exercise

How did that go? Can you put your hand up if you feel like you have an example of an evaluation that made a difference?(Maybe get one or two to share)

Chances are that the examples you identified include some of the following characteristics:1. the evaluator and the evaluation user had a clearly communicated relationship from

the get-go2. the evaluation was well scoped (keeping end use in mind)3. The evaluator’s methods were rigorous and appropriate4. The evaluation ended up providing information that was actionable, and made an

impact on the program and ultimately it’s recipients

What you have also in-fact done, are remembered and listed all the amazing ingredients that went into making your evaluation-that-made-a-difference-cake.

I also have a question for the observers: what are some of the things that most caught your attention about the stories you heard?Chances are that the parts you found most engaging are the human stories inside it (from whichever perspective—might be evaluator trying something challenging, or program

15

recipient experiencing a positive change).

Are people surprised at some of the feedback? It’s interesting isn’t it, to hear it reflected back? What we\ve started to do is transform the evaluation report into a story.

15

Actually you’ve already moved a little bit further than that by virtue of participating in this exercise (the observer in your group probably has already started letting you know what they found most engaging)

So maybe, by virtue of indulging in this exercise what you have ended up with is actually THIS: a gooey unstructured mess of evaluation design, methods and impacts you thought it had. It looks delicious and has the potential to be an amazing cake but it’s not quite there yet.

You may not have the piece of cake yet, but You’ve moved past the ingredients, you’re on your way already even if things seem gooey and messy.

In fact, you are exactly where the Evaluation Stories project is right now. We’ve identified ten evaluations that we strongly suspect made a big impact; we’ve identified the ingredients that allowed these cakes to make such an impact, and we’ve gathered the ingredients to make the cake again, but all we’ve gotten so far is this delicious looking goeey mess.

16

In the end , of course, What we want of-course is THIS piece of cake that makes my knees a little week and by brows a little sweaty.

Now in all seriousness, if we move away from the cake metaphor for a second and hone in on proving evaluation impact to evaluation stakeholders including potential clients, what will make it most appealing is to illustrate the impact of the evaluation using humanly engaging stories.

Stories are universally understood, and, stories are formed out of example = which is the killer combination we want! We want a story that is engaging AND captures the impact of your evaluation work.

And I’d like to use the rest of this presentation to talk a little bit about a recipe for turning your evaluation into a story.

17

So how do we get from a to b to c?

How do we go from a technical report that describes an excellent evaluation to a humanly engaging story that demonstrates how and why the evaluation made a difference?

18

19

Tell your evaluation story to different people – quite a few different people.

Maybe a good place to begin is to tell people who know nothing about the evaluation you choose to tell them about…Much like the exercise you just completed, you can then ask them what was most engaging about what you told them.

Also tell your evaluation story to those who were involved in the evaluation; the evaluation user and very importantly, those whose lives were impacted by your evaluation. This is an opportunity to check in with those involved in the evaluation, to confirm that your

20

representation of the events match with theirs.

By the simple act of telling your evaluation story over and over again to different people, you will most likely become aware of what sticks. You will likely be able to hone in on the right angle from which to tell your evaluation story that is engaging AND captures the impact of your evaluation work.---Now, it is likely that most of us are not in the habit of tracking the impact of our work at the program recipient level - so it may be hard to access them after the fact. Consider building relationships proactively: throughout the evaluation, engage with and develop a relationship with both evaluation user and end user/ program recipients you get in touch with. Explain to them that you would like to follow up with them in x amount of time to see whether things improve or not.

Lay the foundation for collecting stories about the impact of your work.

20

The wonderful thing about stories is that they are universally engaging and there is a sense of magic and connection to them. They have a way of sticking to someone’s memory.

Chances are that by the time you have figured out possible angles from which to tell your evaluation story so that it demonstrates the impact of the evaluation, you’ve also started to get an idea about the aspects of the story that are most humanly engaging.

Pay particular attention to the details of what makes a story appealing to listen to: what sort of an environment does your story take place? Are there one or two people (characters) from whose perspective the story mostly makes sense? Is there a narrative structure that begs the listener of the story to stay glued to their seats (are their twists in the plot? Could there be foreshadowing of important events?) These are all storytelling devices that make them engaging. You all KNOW them; so use them. And if you don’t feel confident enough, the first time you try it talk to a writer or an actor and bounce your ideas off them.

21

Last but not least, be willing to take some risks AND have LOTS of fun along the way.

Remember - the story of your evaluation impact DOES NOT HAVE TO BE COMPREHENSIVE. As opposed to a technical report, a story is selective. It doesn’t pretend to tell the whole story. It doesn’t need to take a particular shape.

Remember a video can be a story, a podcast; a cartoon strip, an infographic, a short story, a dialogue, a little sculpture like the one the picture -- the formats in which stories are told are ENDLESS.

And they are ALL engaging to the audience.

By communicating about our work and the difference that it can make to those we are serving in the form of stories, maybe more people will begin to see how valuable this work is.

22

…chances are, more policy makers, program managers, program recipients and communities at large will become more like this guy– wanting to swim in evaluation cake.

23