Game Days: Crash-Test Your Application Danilo Poccia | Technical Evangelist danilop@amazon.com @danilop

GO BEYOND YOUR TEST ENVIRONMENT

PLAY IN A SIMULATED ENVIRONMENT

SIMULATION ENVIRONMENT

SIMULATION ENVIRONMENT FOR CRISIS SITUATIONS

GOOD WEATHER DOESN’T MAKE GOOD SAILORS

PREPARE FOR EXTREME SITUATIONS

THINGS ARE UNRELIABLE UNTIL THEY ARE TESTED

HOW TO PLAY ? GAME PREPARATION

SUPERVISORS & PLAYERS

MAKE A PLAN ROLE PLAY & SCOPE DEFINITION

SET A DATE (DEADLINE) MAKE IT FUN

CREATE THE SIMULATION ENVIRONMENT

GAMEDAY ENVIRONMENT

AWS ACCOUNT A

PRODUCTION ENVIRONMENT

AWS ACCOUNT B

CLOUDFORMATION TEMPLATE

USE AWS IAM CONTROL WHO CAN DO WHAT

DURING THE GAME

ACTIVATE AWS CLOUDTRAIL GAMEPLAY RECORDING & AUDITING

SIMULATE ACTIVITY

PRE-RECORDED NETWORK ACTIVITY

(THROTTLED) LOAD TEST USE SPOT INSTANCES + TOOLS

USING AMAZON EC2 TO SIMULATE

2.4 MILLION PLAYERS

COMMUNICATION CHANNELS IRC, SHARED DOCS, VIBER, …

PLAY ! THE FUN PART

SIMULATE FAILURES

TERMINATE RESOURCES CHANGE SECURITY GROUPS CHANGE IAM ROLES DISABLE IAM USER CHANGE /ETC/HOSTS FILE AMAZON RDS FAIL-OVER TEST

RECORD & ANNOTATE PLAYERS ACTIONS

IF YOU DON’T MEASURE IT, IT DIDN’T HAPPEN

AFTER PARTY CLOSING THE GAME

DEBRIEF WHERE THE MAGIC HAPPENS

HOW LONG DOES IT TAKE TO DETECT AN EVENT ?

AND TO RESOLVE AN ISSUE?

NEW HIRE ? GO TO THE SIMULATOR

VALIDATE ASSUMPTIONS PROVE YOUR ARCHITECTURE

KNOWLEDGE OF PROCEDURES

COMMUNICATION PROCEDURES

& CHAIN OF COMMAND

GOOD WEATHER DOESN’T MAKE GOOD SAILORS

TELL ME ABOUT YOUR GAMES!

@danilop danilop@amazon.com

Game Days: Crash-Test Your Application Danilo Poccia | Technical Evangelist danilop@amazon.com @danilop