Using Artificial Intelligence to Test the Candy Crush Saga Game

W12 Session 10/26/2016 1:30:00 PM

Using Artificial Intelligence to Test the Candy Crush Saga Game

Presented by:

Alexander Andelkovic

King/Midasplayer AB

Brought to you by:

350 Corporate Way, Suite 400, Orange Park, FL 32073 888---268---8770 ·· 904---278---0524 - [email protected] - http://www.starcanada.techwell.com/

mailto:[email protected]

http://www.starcanada.techwell.com/

Alexander Andelkovic King/Midasplayer AB The agile testing lead for Sweden-based King/Midasplayer AB, developer of the popular mobile game Candy Crush Saga, Alexander Andelkovic has worked on multiple complex test projects ranging from using session-based test management for quality assuring MED-Tech devices for life critical systems to establishing a world class approval process for Spotify apps used by Fortune 500 companies. Now he teams with developers in testing big data, business analytics, and game level regression testing using AI. Alex performs both system testing and exploratory testing with a focus on assisting teams with high-quality deliveries. He is a frequent speaker at international conferences including both STAREAST and STARWEST.

© King.com Ltd 2016

About us

King/Midasplayer AB • Founded 2003

• +200 Games

• 2000 employees

• 13 Studios(Stockholm, London, Barcelona, Seattle…)

• 409 million MAU(Q2 2016)

• Acquired by Activision/Blizzard(2016-02-23)

Page 2


Products

Franchise Games

Page 3


QA at King

King QA roles • ATL – Agile Testing Lead

• Ninja – Exploratory tester

• QRT – Quick Regression Team

• TAE – Test Automation Engineer

• DS – Data Scientist

• Dev – Frontend/Backend Developer

Page 4


QA Problem

More than 2000 levels in Candy

Page 9


Introduction

History of AI

Page 6


Introduction

So, what is a bot exactly?

Page 7


Introduction

Application that performs an automated task

Page 8


Introduction

AI-bot – why?

Page 9

• Level designers

• Quality Assistance (QA)

• Game-domain research


AI-bot

How would an AI-bot actually think(heuristic, simulation)?

Page 9


Heuristic

Heuristic is good, maintenance is bad!

Page 11

• Having a function that ranks moves – heuristic – could

help a lot

• Each simulation much richer

• Has to be generic

• Increasing complexity with more features

• Can we create the heuristic automatically?


MCTS

Simulation – Monte Carlo Tree Search

Page 12


MCTS

Page 13

Simulate example


MCTS

Page 14

Win Loss Win

Is this reliable?


MCTS

Page 15

36% 28% 19%

Repeat simulations – Monte-Carlo simulations


MCTS

Page 16

36% 28% 19%

Gradually we build a tree => Monte-Carlo Tree Search

40% 32%


MCTS

Search tree for level 13 in Candy

Page 17


MCTS

Available actions

Page 18


MCTS

Future states

Page 19


MCTS

Page 20

After 299 simulations


MCTS demo

Page 21

MCTS demo


MCTS - Impact

What do we have?

Page 22

• Generic method

• Works for all levels

• Works for other games


MCTS - summary

MCTS summary

Page 23

• Closer to human success rate

• Easy to tune with more simulation per

decision

• Time consuming, one level takes

~ 5-10 mins

• How can we improve?


NEAT

NEAT – NeuroEvolution of Augmented Topologies!

Page 24

• Artificial Neural Network(ANN)

• Reinforced learning(action, learn, evolve)

• Guided playout


NEAT

Play Candy Crush!

Page 25

4 1 0 0


NEAT

NEAT – Steps

Page 26

• Get training data - Predict score (hours)

• Create random bots

• Compete (hours)

• Create new bots

• Choose the best bot - ANN

• Play Candy choosing moves in playout with

ANN


NEAT

The jungle law in Candy

Page 27

Focus on

blockers!

Look at

jellies!

Special candies are tasty!


NEAT

Making babies

Page 28

Mom Dad


NEAT

Page 29

Newborn

Mom Dad


NEAT

Product of evolution

Page 30


NEAT demo

Page 31

NEAT training

• Red=Fittest Bot

• Yellow=Avg Bot

fitness

• Green=# Bot’s

• Blue=Fittest Bot

ANN nodes


NEAT demo

Page 32

NEAT training demo


Results

Guiding the playout helps

Page 33


NEAT

NEAT – Performance

Page 34

• Simulation time, game engine needs to replay

each game state

• Traversing time, game engine does not

support jump to state

• More GPU’s for bot training(5-6 hours > on

commit)


Content QA

Content QA – No bot

Page 35


Content QA

Content QA – With bot

Page 36


Benefits

Bot benefits

Page 37

• Level development: level difficulty, less

tweaks

• Quality assistance: crash testing,

performance testing, regression testing

• Data scientists: Game domain

knowledge, fun levels, game balancing


Challenges

Bot challenges

Page 38

• Ownership(Knowledge, Resources, Bot

team)

• Integration(Headless mode, C++/Lua/JS,

Hackday/Sprint, Bot Api)

• Maintenance(Bot Training, Infrastructure,

Extend)


Q&A

[email protected]

om

Page 39

Questions?


Thank you!

Page 40

Using Artificial Intelligence to Test the Candy Crush Saga Game

Software

Transcript of Using Artificial Intelligence to Test the Candy Crush Saga Game