How to Conquer Artificial Intelligence

Presenter

The title of this presentation sounds so militant. I’m definitely going to be presenting more of a strategy of “peaceful co-existence” with artificial intelligence. We will be conquering… but we’ll be using AI to help us do the conquering.

an insight agency focused on Strategic Marketing that works.

Presenter

Welcome, Aliza and I are both from Added Value. www.added-value.com

Aliza Pollack talks to humans about humans

Scott Porter talks to computers about humans

Dealing with Flux

Creating change Responding to change

Presenter

Why are we here at Planningness? I think we’re all grappling with change. I think business strategy has change at it’s core. We’re either trying to create it: drive growth, build awareness, change perceptions. Or we need to respond to it: changing trends in culture, increased competition, fractured distribution channels. I'd like each of you to take a moment an write down one change that you are either trying to effect in your business, or that you are trying to respond to. Later on, during the workshop section we'll come back to this, but for now, I just want you to write down one idea.

Success stories

Data � Macro trends � Data mining � Outliers

Human � Had an insight � Had an experience � Intuition

Presenter

How do people successfully find the right strategic response for change? There is a continuum of how people decide on the strategic action they are going to take… from the very human to the very data driven.Most success stories have a kernel to them like these items on the left… I had an insight. I had this experience that helped me realize something missing from the product experience or from the marketplace. I had this intuition that there must be a better way. But in our ever-more-data-rich world, we’re starting to hear stories like these items on the right. Our forecast of the macro trend showed that this was going to be a big new opportunity. We were able to mine the data and found some behavioral patterns that we could capitalize on. We’ve noticed that certain products are selling way above expectation for the support and distribution we have given them, and that led us to an opportunity. We’re going to want to be able to take advantage of both ways of finding the ways to effect the changes we’re looking for…but I figure that most of us are human and are pretty familiar with that side, so we're going to start by looking at the data side.

Predictive Analytics

Predictive model

Presenter

Here is a very simplified view of predictive analytics. This may be review for some of you, but let's start with a common base.To build predictive models we feed evidence from the past to these algorithms or special computer programs. The idea is to use the past data to construct some model of the relationships in the data, so that for new cases we can make predictions based on the relationships we saw in the past evidence.For example, we may look at people who have churned from our service in the past. We will have measured certain features of those people who have left and those that have stayed, and we use that to build a model.What we can then do is feed the model new cases… for example new customer records, and we get a prediction… for example how at-risk this customer is for leaving.The model is often a "black box"… we don't necessarily understand what is going on inside, but we can evaluate it's accuracy by comparing predictions to actual.

Past evidence

Did not hurry

hurried

Arrived on time

Arrived late

Presenter

Let’s play with a toy example. Let’s take as an example my punctuality to meetings. As part of the quantified office worker movement, I started using a special wristband to track data about my arrival times to meetings, and how fast I was walking to meetings… hurrying is above 120 steps a minute, not hurrying below. This is a graph of the data collected.Attribution: This example is based on an example in the CMU Causal and Statistical Reasoning Course which has been released under the Creative Commons Attribution-ShareAlike 3.0 Unported license (http://creativecommons.org/licenses/by-sa/3.0/).

Prediction

Predictive model

Will be late

Won’t be late

Presenter

Will I be late? We used this data to train a computer model. The computer model that will predict if I’ll be late by noticing if I hurry on the way to the meeting, and the model is right over 85% of the time.

Predictive modeling is not enough

New policy: Expectation:

Late 1 in 10

meetings

baseline:

Late less

often “Stop hurrying… hurrying is linked with being late”.

Presenter

However, prediction isn’t enough. Remember that we want to effect change. So I instituted a policy of not hurrying. Every time I walk above 120 steps per minute on the way to a meeting, the wristband gives me a small shock to remind me not to hurry. We try this experiment for two weeks.

Predictive modeling is not enough

New policy: Expectation:

Late 1 in 10

meetings

Late 5 in 10

meetings

baseline:

Late less

often “Stop hurrying… hurrying is linked with being late”.

result:

Presenter

However, instead of improving , the situation got much worse. Instead of being late to 1 in 10 meetings… I am late to half of my meetings. Something obviously went wrong.

What went wrong?

Presenter

Can we figure out what went wrong?We brainstormed as a group. In addition to the variables of hurrying and being on time, the group added the following:-Leaving on time-Distance to meeting-Scott's calculation of how much time was necessary to get to meeting (which everyone suspected was deficient)-Importance of the meeting-Earlier meetings running late-Time of day-Caffination level-Presence of food in the meeting

� Identifying what

might matter

� Tireless � Difficulty processing

complexity

� Tend to single solution

� No context

� Don’t recognize silly answers

Presenter

As this example we’ve just been through points out… computer models have some weaknesses in areas that humans can easily handle. Computers don't yet have the ability to easily generalize from a few variables to the missing context. And they don't realize when the answer is silly. Several of you started chuckling even before I revealed the answer because you could see that some important bits were missing.Humans have strengths in these exact areas. In just a couple of minutes, we were able to come up with a number of variables that probably matter in this case… and it seemed really intuitive for all of you.But as humans we also have some weaknesses. Even in this simple case, if you weren't carefully thinking it through, it wasn't obvious that the implications were backwards. And as we add more and more variables, we have even more difficulty understanding what the implications are, even if we've correctly identified all the variables that matter.We also tend to stop with a single solution. This is where computers have strengths… the computer is tireless, and will keep chugging through possibilities until I tell it to stop.The trick is figuring out how to work together.

Knowledge representation vs. prediction. Usable by both computers and humans

Presenter

This is where Artificial Intelligence methods come in, specifically something called Knowledge Representation. This is not just about the prediction, but about how machines can represent knowledge constructs using graphs or diagrams. A nice feature of these kind of diagrams is that they have been designed to be similar to some of the ways that we as humans might deconstruct or diagram relationships we see in the world. These diagrams are going to be the common language that we will use to help human and computer work together.Attribution: This example model based on the example in Judea Pearl's paper: "Bayesian Networks" UCLA Cognitive Systems Laboratory, Technical Report (R-246), Revision I, July 1997.

Easing human communications, too

X5

X4

X3 X2

X1

? answer Black box approach requires a game of telephone.

Graphical models facilitate discussion and collaboration.

Presenter

I talk a lot as if we’re collaborating with the AI, because I think it’s useful for us to remember that good ideas are going to be suggested by this back and forth between the human and computer. But I want to point out that there are human communication benefits as well. When the models are this sort of black-box that no one besides a highly trained data scientist can interface with, we introduce a game of telephone. There are things about the market that need to be understood to correctly set up the models, and all of that knowledge needs to be translated through the data scientist.Even when you have a really great data scientist, there is often an information loss here, because they don't know as much as the subject matter experts about the market or domain for this model. Using models that are built up with this AI approach using graphs to represent the model, it’s easier for the decision maker to interface a bit more directly with the models.This helps us make better models, because we're better able to get that context out of people's heads and onto "paper".

What are our roles?

Human: Brainstorm potential inputs

Computer: search possible models

trust

Attribute 1 favorability

Attribute 2

Other outcome

trust

Attribute 1 favorability

Attribute 2

Other outcome

Presenter

That brings us to an important point. How are we interfacing with the computer. What is our job? Although both human and computer are going to be creating similar diagrams, they really have different purposes. We humans, are only responsible for the thing we’re really good at. We just have to identify things that might matter… We don’t have to be positive everything we identify will matter, or know exactly how important each is, or be able to predict how all the items will interplay. Our job is to push to a wider view so when we move to the next stage, we know what data to collect for the computer.

Porter, Scott (AVLA1 cs)

The computer-generated diagrams, on the other hand, will represent likely connections, given the observed data. Some of the things we hypothesized may not actually play out the way we think, or may not turn out to be as important as we thought. The computer can help us sorting through these issues, as long as we feed it a complete enough set of data. Just to set expectations… today we’re only going to work on the human side of things. If we were spending the entire day, we could get into more of the back and forth. So we’re going to demonstrate how to push hypothesis generation wide enough to identify the data that computer algorithms need to have enough context to build better quantified models. Some of you are relieved that we’re sticking to that… but for those of you that are disappointed, don’t be. As we get into this, I think you’ll see that there is a lot of value in just pushing our ideas far enough that we could build a model.

What’s in the diagrams?

behavior perception

outcome

event intention

Variables (things we already measure or things we think we should start measuring)

Relationships between variables… arrows from causes to effects (guesses fine… don’t have to be sure about)

1 2

Presenter

Now some points of process… what are we doing again? We’re drawing diagrams that are our hypothesis about how the market or system works where we want to make a change. We’re going to be writing down variables… and unlike some other mind mapping or hypothesis techniques, we’re not going to be stickler about what these need to be. They can be basically anything that you currently measure or you’d like to measure… behaviors, perceptions, events, whatever. And if we think that something might affect something else on our diagram, we’ll draw an arrow from the cause to the effect. We don’t have to be sure of these connections… guesses are fine.

Don’t forget our goal

Identify what is under your control.

Identify what could change.

We can easily choose to change this.

You can’t control this, but you might suspect it could change.

Presenter

Although we won’t be marking them in any special way on the diagrams we produce today, it’s good to keep in mind variables that you have under your control, as well as things that are likely to change but aren’t directly controllable.A lot of times when we're brainstorming a change under our control, we tend to forget about the things not under our control.And when we're brainstorming a response to a change that wasn't under our control, we tend to forget about the variables under our control.Whichever type of problem we're tackling, we're going to want to include both types of variables in our diagrams.

Which is more actionable?

Why?

Presenter

There are a lot of ways that you can diagram the same problem. Something that’s going to be important is getting the right level of granularity as we diagram. Both of these diagrams are correct, but the process has been split out a bit differently. Can I get a volunteer to tell me which diagram is more actionable and why?[We brainstormed this together. People mentioned that understanding the mediating step of ground moisture allowed us to think of interventions we could take, such as sprinklers. We also talked about other interventions, such as fertilizer, that might have different side effects when the ground moisture came from rain versus sprinklers.] When we’re generating diagrams, we’re going to have some rules that will help us naturally break the diagram to the right level. Let’s take this example, and I’ll help demonstrate.

Hypothesis Generation

Step 1. Start with the outcome you hope to change.

Presenter

So as we generate hypotheses, we’re going to follow some rules to help us avoid forgetting important variables. Step 1 is to start with what’s most important. Think about your goals… what are you hoping to change. Let’s get that on the board first.



Step 2. For every hypothesized variable, add at least 2 causes.

Presenter

Step 2 is to add causes to each variable. The question I ask myself when doing this is… are there (at least) two different reasons why this can happen. For example, when doing this for a bank and brainstorming the outcome of people closing their accounts, I asked "Are there two different reasons why people might close their accounts"?The client said "Yes… when people move, and when people are unhappy with the service". Getting to distinct causes is extremely helpful.In this case, two distinct causes of the garden growing are sunlight and ground moisture.




Presenter

We're going to continue to add causes until we can't think of any more, or we run out of time.




Step 3. For every hypothesized variable, add at least 1 side effect.

$

Presenter

Step 3 is to add side effects. These aren't necessarily good or bad. But just think about the opposite direction. Like I mentioned, as humans we tend to stop with a single answer or story.Look at the variables on the board and see if you notice any additional effects.For example, in this case, running the sprinkler has another effect… it costs money.This might be important when we try to decide on the best course of action.




Step 3. For every hypothesized variable, add at least 1 side effect.

Step 4. Find at least 2 items that have a common cause. Add that cause.

$ repeat

Presenter

Step 4 is to add missing causes. This is generally the hardest step.Look at the variables on the board and see if there are commonalities that aren't already captured.For example, in this case, both amount of sunlight and amount of precipitation vary with a single variable… season.After you've done your best to add any missing causes, it's often good to go back through all the steps again to see and continue to push the diagram wider. Although today, we'll probably only have time to go through the steps a single time.

Can we draw what we think is going on…

Presenter

Now we're going to try this on examples from your own businesses.Remember earlier I had you write down something you wanted to change, or a change you were going to respond to? Now we're going to come back to those examples.1) We're going to divide into groups of 6-8 people.2) Each is going to share the idea they wrote down, and the group is going to pick one to focus on3) We're going to split into two subgroups of 3-4 people and go through the steps (dividing will help us "crowdsource" the mental models to make it more likely that we don't miss something).4) The two subgroups are going to share their ideas with each other and combine5) We're going to come back, report back and discuss

Porter, Scott (AVLA1 cs)

Outcomes chosen by the 4 groups-People churning from a fitness related tech product after ~6 months-Increasing viewership among men for a traditionally female skewing TV network-Increasing trial for an entertainment service that has high awareness but low trial-Improving agency retention of it's employees, who traditionally come, learn, and leave for other agencies after a short time.Questions asked of the group: What are some surprising items that came up as you conducted the hypothesis generation for your problem that you wouldn't have thought of without the structured approach?How might you measure some of these variables?Questions asked of Scott:Q. How can we learn more? A. Carnegie-Mellon has a free, undergraduate-level course in Causal and Statistical Reasoning which may give you a better understanding of the reasoning behind some of the rules we use here.Also, feel free to reach out to me on Twitter, LinkedIn, or at [email protected].

How to Conquer Artificial Intelligence

Business

Transcript of How to Conquer Artificial Intelligence