Post on 20-May-2020
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Greg McConnel - Gaming Solutions Architect
June 19, 2017
Building Analytics Pipelines
for Games on AWS
Asking Questions - Breakaway
{‘event_id’ : ‘05b00439-6a07-4112-9c8d-165f1643e5d1’, ‘event_type’ : ‘player_death’, ‘event_timestamp’ : ‘2017-05-01T21:05:18.000Z’, ‘arena’ : ‘el_dorado’, ‘position_x’ : 507.12, ‘position_y’ : 551.61 }
Evolution of Data Analysis
Descriptive(Batch / Real-time)
Reports
Predictive
Forecasts
Prescriptive
Automation
Game Analytics on AWS
• Analytics Intro
• Analytics Pipeline with Case Studies
• Predictive and Prescriptive Analytics
Game Analytics on AWS
• Analytics Intro
• Analytics Pipeline with Case Studies
• Predictive and Prescriptive Analytics
The Analytics Pipeline
Ingest/
Collect
Consume/
visualizeStore
Process/
analyze
Telemetry
(events)
1 40 9
5
Metrics
(answers)
Analytics Pipeline for Breakaway:
Ingest/Collect
{ ‘event_id’ : ‘05b00439-6a07-4112-9c8d-165f1643e5d1’,‘event_type’ : ‘player_death’,‘event_timestamp’ : ‘2017-05-01T21:05:18.000Z’,‘arena’ : ‘el_dorado’,‘position_x’ : 507.12,‘position_y’ : 551.61 }
Amazon Kinesis makes it easy to work with
real-time streaming data
Amazon Kinesis
Streams
• For technical developers
• Collect and stream data
for ordered, replayable,
real-time processing
Amazon Kinesis
Firehose
• For all developers, data
scientists
• Easily load data into
Amazon S3, Amazon
Redshift, Amazon
Elasticsearch Service
Amazon Kinesis
Analytics
• For all developers, data
scientists
• Easily analyze data
streams using standard
SQL queries
The Analytics Pipeline
Ingest/
Collect
Consume/
visualizeStore
Process/
analyze
Telemetry
(events)
1 40 9
5
Metrics
(answers)
Athena - Query Data Directly from S3
• Serverless Interactive Query Service
• Query data in its raw format
• Text, CSV, TSV, JSON, weblogs, AWS service logs
• Or convert to an optimized form like ORC or
Parquet for the best performance and lowest
cost
• No ETL required
• Stream data directly from Amazon S3
https://aws.amazon.com/athena/
The Analytics Pipeline
Ingest/
Collect
Consume/
visualizeStore
Process/
analyze
Telemetry
(events)
1 40 9
5
Metrics
(answers)
Glue automates the undifferentiated heavy-lifting of ETL
Cataloging data sources
Identifying data formats and data types
Generating Extract, Transform, Load code
Executing ETL jobs; managing dependencies
Handling errors
Managing and scaling resources
Glue – Managed ETL Service
Sign up for the Preview!
https://aws.amazon.com/glue/
The Analytics Pipeline
Ingest/
Collect
Consume/
visualizeStore
Process/
analyze
Telemetry
(events)
1 40 9
5
Metrics
(answers)
CONSUME
Amazon QuickSight
Apps & Services
An
aly
sis
& v
isu
ali
zati
on
No
teb
oo
ks
IDE
AP
I
Consume/Visualize
Game Analytics on AWS
• Analytics Intro
• Analytics Pipeline with Case Studies
• Predictive and Prescriptive Analytics
Predictive & Prescriptive Analytics
Avoiding “One Size Fits All” – Personalized Experiences
Use Case: Reduce Player Churn
Other example use cases:
• Predict users with low probability to purchase in the app and send discount coupon
• Identify users with high probability to share the app and reach out to encourage them
• Recommend relevant content to users based on similar user’s behavior patterns
Amazon Machine Learning:
https://aws.amazon.com/machine-learning/
Predictive & Prescriptive Analytics Example
1Discover
https://aws.amazon.com/blogs/big-data/decreasing-game-
churn-how-upopa-used-ironsource-atom-and-amazon-ml-
to-engage-users/
2Predictor
Batch layer
Amazon
Kinesis
Data
stream
process
store
Amazon
Kinesis S3
Connector Amazon S3
Ap
plic
atio
ns
Amazon
Redshift
Amazon EMR
Presto
Hive
Pig
Spark answer
Speed layer
answer
Serving
layer
Amazon
ElastiCache
AmazonDynamoDB
Amazon
RDS
Amazon
ES
answer
Amazon
ML
KCL
AWS Lambda
Storm
Lambda
Architecture
Spark
Streaming on
Amazon EMR
http://d0.awsstatic.com/whitepapers/Big_Data_Analytics_Options_on_AWS.pdf
Demos you can try out
Centralized Logging:
https://aws.amazon.com/answers/logging/centralized-logging/
Streaming Analytics Pipeline: https://s3.amazonaws.com/solutions-
reference/streaming-analytics/latest/streaming-analytics-pipeline.pdf
Visualizing Traffic Using Kinesis Streams:
http://docs.aws.amazon.com/streams/latest/dev/kinesis-sample-
application.html
Kinesis Test Data Generator: https://github.com/awslabs/amazon-
kinesis-data-generator
Kinesis Analytics: https://aws.amazon.com/blogs/big-data/writing-sql-
on-streaming-data-with-amazon-kinesis-analytics-part-1/
2017:
Snowplow Analytics: https://www.youtube.com/watch?v=6Zmv5LgpjGY
2016:
Capital Games (EA): https://www.youtube.com/watch?v=bxfnWTiY7EM
Breakaway: https://aws.amazon.com/blogs/big-data/building-an-event-based-analytics-pipeline-for-amazon-game-studios-breakaway/
Telltale Games: https://www.youtube.com/watch?v=FYUwLaUz7TA
2015:
Glu Mobile: https://www.youtube.com/watch?v=ThLWrldseG4
Predictive Analytics: https://www.youtube.com/watch?v=PcWrSeJcb_4
2014 and earlier:
GREE: https://www.youtube.com/watch?v=Jhl2H7soW4Q
Z2: https://aws.amazon.com/solutions/case-studies/z2/
Supercell: https://aws.amazon.com/solutions/case-studies/supercell/
Rovio: https://aws.amazon.com/solutions/case-studies/rovio/
Customer Examples