Download - Your Game Analytics Playbook

Transcript
Page 1: Your Game Analytics Playbook
Page 2: Your Game Analytics Playbook

Your Game Analytics PlaybookAllison Bilas

@allisonbilas / [email protected]

1. Introduction 1. Currently VP of Product @ GameAnalytics 2. Just moved to Denmark, loving hygge.

2. PopCap for last 3.5 years. 1. Hired when they didn’t know what to do with data. 2. Built game analytics team from scratch.

3. Learned a LOT about how to infuse data into the decision making process behind developing and operating games. 4. Hope to share some of my experience with you.

Page 3: Your Game Analytics Playbook

Agenda

‣ What is a game analytics playbook? ‣ Why is it important? ‣ Components of a “play” ‣ Examples: pre-season to end season

This presentation is about: !1. What a game analytics playbook is 2. Why it’s important to have one 3. I’ll go into the component of a game analytics “play” 4. And I’ll give you some concrete examples from the pre-season to the end season !Let’s get started…

Page 4: Your Game Analytics Playbook

What is a game analytics playbook?

1. A playbook comes from American football. 1. I’m not actually that into the sport. 2. But it’s a powerful tool.

2. I want you to imagine a football team on the field. The QB calls his team together. 1. Competitors position 2. Health of his team 3. Time left on the clock

2. Makes a call for a specific play > team takes action > success/fail 3. If the game isn’t over, they start again. !!

5. What allows the QB to make that decision is their playbook. 1. Put together at the beginning of season. 2. Anticipates every situation the team is likely to encounter. !

A game analytics playbook is your plan for using data to inform the development and operations of your game. !We had one at PopCap that was explicit, but it doesn’t have to be.

Page 5: Your Game Analytics Playbook

Information resolves uncertainty.

1. So, why is this important? 2. Assuming you’re here because you understand the power of data.

1. Playbook can be used to inform AEM and live ops. 2. In short, it helps to resolve uncertainty and speed up your decision making and development.

3. Caution that analytics doesn’t solve everything. Game still needs to be a great experience for your players.

Page 6: Your Game Analytics Playbook

Components of a Play

Context Question

1. Let’s get into the components of a “play”. 1. This is essentially the analysis process. 2. Most studios start with thinking about what to track, but there are a couple of steps that should happen before that…

2. Like the QB on the football field, first step is CONTEXT. 1. Where your game is at in its lifecycle. (We’ll go over these in the next sections.) 2. Competitive context and industry trends 3. Goals for your game in terms of KPIs

3. By assessing the context, you’ll start to come up with analysis QUESTIONS. 1. Typically focused on AEM, but also can be more technical in nature around things like bugs and frame rates. 2. Likely to have a LOT of questions. 3. Prioritize based on: measureable / actionable / most impact.

Page 7: Your Game Analytics Playbook

Components of a Play

Context Question Data / Tools Action

1. DATA AND ANALYSIS TOOLS 1. Here is where you need to start thinking about what to track in order to answer the questions you have… 2. In deciding what to track you should also consider the type of analysis you’re going to do, for example:

1. Segmentation, benchmarking, A/B testing and trend analysis. 2. Last component is ACTION. Here’s is where the playbook is most powerful.

1. If you’ve done everything right: CONTEXT > QUESTION > DATA/TOOLS 2. Deciding what action should be very easy. Typically implied in the questions you’re asking.

Page 8: Your Game Analytics Playbook

Pre-season (Dev)

Early Season (Beta)

Mid Season (Launch)

Late Season (Ongoing Ops)

Seasons

1. Next I’m going to give examples that follow the life cycle of your game from: 1. Pre-season or development 2. Early season or beta period 3. It’s not until the mid season that you’ve finally launched 4. And then the late season, or ongoing ops

Page 9: Your Game Analytics Playbook

Pre-Season (Development)

‣ Instrument tracking EARLY and TEST ‣ Be judicious in what you track ‣ Think through event hierarchies ‣ Consider multiple tracking systems

1. Pre-season is straight forward - SET UP TRACKING. 2. I recommend you start doing this very early.

1. This means you’ll track things initially that you don’t need in the end. This is okay. 2. It lets you start to refine the questions you have and ensure that you have the data needed to answer them. 3. Also gives you time to test. Every game I’ve every launched has found tracking bugs in the first day after launch.

3. Let’s you be smart about what you track. E.g. Game I was working with wanted to track every time a player turn on/off the sound. 4. Also in the early season, you should be thinking very closely about your event hierarchies.

1. These can dictate the type of analysis you can do, and it’s very nuanced. 2. It helps to start tracking and trying out analysis, and then making changes.

5. Multiple tracking systems — can add some complexity, but allows validation of data.

Page 10: Your Game Analytics Playbook

Early Season (Beta)

1. Early season is BETA period. 1. Select markets, and you have data! YAY! 2. Exciting period. Easy to loose focus.

2. Most important = look for unexpected behaviors in gameplay and monetization. 3. HIDDEN AGENDA

1. Hidden object game launched on FB a few years ago. Few hundred Australians playing it every day. 2. One player was spending a lot of money in the game. WHALE! 3. What was she spending money on > gotcha boxes of consumables. 4. How was she progressing in the game > noticed she was stuck on Level 18. 5. Needed a “finger print duster” to move forward. But that item wasn’t available until Level 20. BLESS HER!

Page 11: Your Game Analytics Playbook

Early Season (Beta)

1. Because it was the beta period, this wasn’t terrible. We were able to identify the weird behavior, find the problem and fix it. 2. In terms of analysis in the beta period:

1. Segmentation is very important. In addition to the few hundred Australians, we had thousands of Philipine installs coming and going in the game in order to pressure test.

2. Important to filter and look at them separately. 3. I also want to mention that A/B testing isn’t necessary. Lots of info just by observing.

Page 12: Your Game Analytics Playbook

Mid Season (Launch)

1. Finally at mid-season the game launches! This is when things start moving really quickly. 2. Analysis should focus on high level metrics > dive deeper when something isn’t working. 3. One example from my experience at PopCap was when I was working with PVZ2.

1. Immediately after launch, millions of installs. Playing for a few days and then leaving. Retention was horrible. Everything else was good! 2. Digging in to engagement… players spending less than 5 minutes per session in the game. BUT, time to complete a level took 15 to 20

minutes. 3. Players had to return 3 or 4 times to the game before they had a reward moment. Felt like a slog. 4. Team identified this and set about finding ways to shorten the game lengths.

Page 13: Your Game Analytics Playbook

Mid Season (Launch)

1. Quite honestly, too late! Millions came and left and it was hard to get them to come back. 2. In launch analysis, you’ll be looking at data all the time. HOURLY, DAILY. 3. Intense period lasts probably 1 month… THEN…

Page 14: Your Game Analytics Playbook

Late Season (Ongoing Ops)

1. At some point you need to slow down and go deeper with your analysis. This is your end game, but it should never actually end! 1. Focus on refinement and longer term game design. 2. Robust content pipeline and promotional program.

2. Start looking at trends in the data. 3. Bejeweled Blitz Rare Gems

1. Super interesting game. FB for 5yrs, mobile for 3yrs. 2. Success because of content and promotions. 3. Main content is Rare Gems, extra powerful boosts. 4. Had to launch a few of them look at trends in order to understand what made them successful.

1. First one was a big hit, just because it was first. 2. 2nd and 3rd were also good… 3. Wasn’t until 6th and 7th that we started to get it.

5. Rare Gem experience is made up of: 1. In game visuals and explosions 2. Last hurrah with explosions 3. Slot machine mechanic at the end

6. Most successful had ALL of them, but with trend analysis we could understand the tradeoffs between including each of these components or not.

7. This informed our development and let us make explicit decisions around the tradeoffs in the content design.

Page 15: Your Game Analytics Playbook

SummaryA game analytics playbook is your plan for using data to inform the decisions you make in operating your

game.

Context Question Data / Tools Action

1. I’ve told you what a playbook is and how it relates to building and operating you game. 2. Your next step is to consider where your game is at in its lifecycle, and start developing your plays. !3. I would love to talk more about this with you! I love thinking through this process and how to make it both accessible and actionable.

Page 16: Your Game Analytics Playbook

Allison Bilas @allisonbilas / [email protected]

THANK YOU!