F# at GameSys

F# at GameSys!(how we used F# to build games for millions)

Hi, my name is Yan Cui.

agenda

Domain Modelling!Infrastructure!Game Logic!Algorithms!

DSLs!OSS

1MILLION USERS

ACTIVEDAILY

250MILLION DAY

PERREQUEST

2MONTH

TBP E R

secops25,000

we are polyglot

why F#?

time to market

correctness

efficient

tame complexity

Collectables

Wager Size

Special SymbolAvg Wager Size

Web Server call

• Line Win!– X number of matching symbols on adjacent columns!– Positions have to be a ‘line’!– Wild symbols substitute for other symbols!

!• Scatter Win!– X number of matching symbols anywhere!– Triggers bonus game

What symbols should land?!Any special symbol wins?!

Did the player win anything?

What the player’s new average wager?

Should the player receive collectables?

type Symbol = Standard! of string!! ! ! | Wild

e.g.! Standard “hat”!! Standard “shoe”!! Standard “bonus”!! …

type Symbol = Standard! of string!! ! ! | Wild

i.e.! Wild

type Win = LineWin of int * Symbol * int!! | ScatterWin of Symbol * int

e.g.! LineWin (5, Standard “shoe”, 4)

type Win = LineWin of int * Symbol * int!! | ScatterWin of Symbol * int

e.g.! ScatterWin (Standard “bonus”, 3)

type LineNum = int!type Count! = int!!

type Win = LineWin of LineNum * Symbol * Count!! | ScatterWin of Symbol * Count

e.g.! LineWin (5, Standard “shoe”, 4)

e.g.! ScatterWin (Standard “bonus”, 3)

make invalid states unrepresentable

[<Measure>]!type Pence

e.g.! 42<Pence>!! 153<Pence>!! …

10<Meter> / 2<Second> ! = 5<Meter/Second>!10<Meter> * 2<Second> ! = 20<Meter Second> !10<Meter> + 10<Meter> ! = 20<Meter>!10<Meter> * 10! ! ! = 100<Meter>!10<Meter> * 10<Meter> ! = 100<Meter2>!10<Meter> + 2<Second> ! // error!10<Meter> + 2 ! ! ! // error

type Wager = int64<Pence>!type Multiplier = int!!

type Payout = Coins! of Wager!! ! | MultipliedCoins of Multiplier * Wager!! ! | Multi! of Payout list!! ! | BonusGame

type State = ! {!

AvgWager! : Wager!SpecialSymbol : Symbol!Collectables : Map<Collectable, Count>!

New avg wager Got a Collectable!

A pay line win!

lightweight syntax for types & hierarchies

great for domain modelling

make invalid states unrepresentable

better correctness

order of magnitude increase in productivity

player states are big

Stateless Server DatabaseClient

1:1 read-write ratio

ServerClient

Session 1

Session 2

ServerClient Database

Elastic Load Balancer

Auto scaling Group

Server A Server B

CloudFront

Stateful Server

The Actor Model

An actor is the fundamental unit of computation which embodies the 3 things!

• Processing!• Storage!• Communication!

that are essential to computation.!!

-Carl Hewitt** http://bit.ly/HoNHbG

The Actor Model

• Everything is an actor!• An actor has a mailbox!• When an actor receives a message it can:!– Create new actors!– Send messages to actors!– Designate how to handle the next message

Stateful Server

• Gatekeeper!– Manages the local list of active workers!– Spawns new workers!

!• Worker!– Manages the states for a player!– Optimistic locking!– Persist state after period of inactivity

Stateful Server

Game Server

Player A

Player B

S3Worker C

Worker B

GatekeeperR

Asynchronous

Stateful Server

Game Server

Worker C

Worker B

Gatekeeper

Worker Aok

sPlayer A

Player B

Asynchronous

Stateful Server

Game Server

S3Worker C

Worker B

Gatekeeper

Worker AR

Player A

Player B

Asynchronous

Stateful Server

Game Server

S3Worker C

Gatekeeper

Worker AR

Player A

Player B

Asynchronous

Stateful Server

Game Server

Worker C

Worker A

Gatekeeper

sPlayer A

Player B

Asynchronous

type Agent<‘T> = MailboxProcessor<‘T>!!!!

type Agent<‘T> = MailboxProcessor<‘T>!!type Message = ! | Get of AsyncReplyChannel<…>! | Put of State * Version * AsyncReplyChannel<…>

type Result<‘T> =! | Success! of ’T! | Failure! of Exception!!!!

type Result<‘T> =! | Success! of ’T! | Failure! of Exception!!type GetResult = Result<State * Version>!type PutResult = Result<unit>!

type Agent<‘T> = MailboxProcessor<‘T>!!type Message = ! | Get of AsyncReplyChannel<GetResult>! | Put of State * Version * AsyncReplyChannel<PutResult>

type Worker (playerId) =! let agent = Agent<Message>.Start(fun inbox ->! let state = getCurrentState playerId!! let rec workingState (state, version) = ! async { … }! and closedState () =! async { … }!! workingState (state, 1))

let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! | None -> ! do! persist state! return! closedState()! …! }!

non-blocking I/O

let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! | None -> ! do! persist state! return! closedState()! …! }!

let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! …! | Some(Get(reply)) ->! reply.Reply <| Success(state, version)! return! workingState(state, version)! …! }!

let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! …! | Some(Put(newState, v, reply)) when version = v ->! reply.Reply <| Success()! return! workingState(newState, version+1)! …! }

let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! …! | Some(Put(_, v, reply)) ->! reply.Reply <| Failure(VersionMismatch(version, v))! return! workingState(state, version)! }

let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! | None ! ! ! ! ! ! -> …! | Some(Get(reply)) ! ! ! ! -> …! | Some(Put(newState, v, reply)) when version = v -> …! | Some(Put(_, v, reply)) ! ! ! -> …! }

and closedState () = ! async { ! let! msg = inbox.Receive()! match msg with! | Get(reply) ->! reply.Reply <| Failure(WorkerStopped)! return! closedState()! | Put(_, _, reply) ->! reply.Reply <| Failure(WorkerStopped)! return! closedState()! }

5x efficient improvement

60% latency drop

no databases

90% cost saving

need to ensure server affinity

need to balance load

need to avoid hotspots

• Persist player state after short inactivity!• Move player to another server after persistence

need to gracefully scale down

Agents!• no locks!• async message passing!

Agents!• no locks!• async message passing!• self-contained!• easy to reason with

Agents!• low level!• exceptions kills agents!• primitive error monitoring!• no supervision!• no distribution

MS Orleans

Cricket

akka.Net

Async Workflow!• non-blocking I/O!• no callbacks

Pattern Matching!• clear & concise

Caught a Gnome

EXP Item Gold

Quest Progress

Caught a Gnome

Level Up

Quest Progress

EXP Item Gold

Caught a Gnome

Quest Complete

Level Up

Quest Progress

EXP Item Gold

Caught a Gnome

New Quest

Quest Complete

Quest Progress

EXP Item Gold

Caught a Gnome

Quest Complete

New Quest

Level Up

Quest Progress

New QuestAchievement

Progress

EXP Item Gold

Quest CompleteLevel Up

Caught a Gnome

Level Up

Quest Progress

EXP Item Gold

Caught a Gnome

Quest Complete

Progress

Achievement Complete

Level Up

Quest Progress

EXP Item Gold

Caught a Gnome

Quest Complete

Progress

Achievement Complete

100+ actions

triggered by different abstraction layers

non-functional requirements

message-broker pattern

Caught Gnome Trapping

Levelling

Quests

Achievements

Analytics

Partner Reporting

Levelling

Quests

Achievements

Analytics

Partner Reporting

Ignore

Process

Ignore

Levelling

Quests

Achievements

Analytics

Partner Reporting

EXPItemGold

Levelling

Quests

Achievements

Analytics

Partner Reporting

EXPItemGold

Levelling

Quests

Achievements

Analytics

Partner Reporting

EXPItemGold

Process

Ignore

Process

Ignore

Levelling

Quests

Achievements

Analytics

Partner Reporting

EXPItemGold

Level Up

Levelling

Quests

Achievements

Analytics

Partner Reporting

EXPItemGold

Level Up

need lots of facts

type Reward =! | GotExp! ! of Exp! | GotGold! ! of Gold! | GotItem! ! of Item * Count!!

type StateChange =! | LevelUp! ! of OldLevel * NewLevel! …

type Fact =! | StateChange! of StateChange! | Reward! ! of Reward! | Trapping! ! of Trapping! | Fishing! ! of Fishing! …

let process fact =! match fact with! | StateChange(stateChange)! -> …! | Reward(reward)! ! ! -> …! | Trapping(trapping)! ! -> …! …

C# interop

…!var fact = Fact.NewStateChange(!! StateChange.NewLevelUp(oldLvl, newLvl));!…

type IFact = interface end!!

type Reward =! | GotExp! ! of Exp! | GotGold! ! of Gold! | GotItem! ! of Item * Count! interface IFact

type IFact = interface end!!

type Reward =! | GotExp! ! of Exp! | GotGold! ! of Gold! | GotItem! ! of Item * Count! interface IFact

simple

flexible

location baitattraction rate

catch rate

auto-tuning trapping stats

Monsterstrength speed

intelligence

Trapstrength speed

technology

Monsterstrength speed

intelligence

Trapstrength speed

technology

Catch Rate %

trial-and-error

genetic algorithms

F#-powered DSLs.!Awesome!

wanna find correlations?

you can DIY it! !

why was payment service slow last night?

Pain Driven Development

Amazon!! .CloudWatch!! .Selector

github.com/fsprojects/Amazon.CloudWatch.Selector

Find metrics whose 5 min average exceeded

1 second during last 12 hours

cloudWatch.Select( unitIs “milliseconds” + average (>) 1000.0 @ last 12 hours |> intervalOf 5 minutes)

cloudWatch.Select(“ unitIs ‘milliseconds’ and average > 1000.0 duringLast 12 hours at intervalOf 5 minutes”)

did any cache nodes’ CPU spike

yesterday?

cloudWatch.Select( namespaceLike “elasticache” + nameLike “cpu” + max (>) 80.0 @ last 24 hours |> intervalOf 15 minutes)

regex support

Amazing

usable from anywhere you can run F# code

e.g. F# REPL, executable, ..

Internal DSL

useful for building tools e.g. CLI, …

External DSL

managed!key-value store

redundancy!9-9s guarantee

great performance

name your throughput

hash key + range key

hash_key_1

hash_key_2

hash_key_3

range_key_1

range_key_2

range_key_1

range_key_2

range_key_1

range_key_2

Hash Key Range Key

Local Secondary Index

Hash Key Range Key

Global Secondary Index

query:!! given a hash key!! filter on!! ! range key, or!! ! local secondary index

Hash Key Range Key

Global Secondary Index

Who are the TOP 3 players in “Starship X” with a score of at least 1000?

DynamoDB.SQL

github.com/fsprojects/DynamoDb.SQL

Disguise complexity

SELECT UserId, TopScore FROM GameScore WHERE GameTitle CONTAINS “Zelda” ORDER DESC!LIMIT !! 3 WITH (NoConsistentRead)

Execution

F# & FParsec*

*www.quanttec.com/fparsec

External DSL via

SELECT * FROM GameScore

Abstract Syntax Tree (AST)

FParsec

select GameTitle, UserId, TopScore from GameScores where GameTitle = “Starship X” and TopScore >= 1000 order desc limit 3 with (NoConsistentRead, Index(GameTitleIndex, true))

S3 Provider

github.com/fsprojects/S3Provider

F# type provider for Amazon S3

compile time validation

no code generation

a long time ago, in a language far far away from .Net - Erlang.

there was an idea to let you pattern match again binary

payloads and work with them at a bit level…

<<SourcePort:16, DestinationPort:16, ! SeqNumber:32,! AckNumber:32, DataOffset:4, ! Reserved:3, Flags:9, WindSize:16,! CheckSum:16, UrgentPointer:16,! Payload/binary>> = SomeBinary

BitSyntax

github.com/theburningmonk/bitsyntax

Working with Stream at bit level

do bitWriter stream {! do! BitWriter.WriteInt16(12345s) ! // source port! do! BitWriter.WriteInt16(12321s)! // destination port! do! BitWriter.WriteInt32(1) ! // sequence number! do! BitWriter.WriteInt32(1) ! // ack number! do! BitWriter.WriteInt32(2, numBits = 4) // data offset! do! BitWriter.WriteInt32(0, numBits = 3) // reserved! do! BitWriter.WriteBool(false) ! // NS! …! do! BitWriter.WriteInt16(42s)! ! // Checksum! do! BitWriter.WriteInt16(1208s)! // Urgent Pointer! do! BitWriter.WriteString("Hello World!") // payload!}

16 bits

32 bits

take ’first’ N bits

bitReader stream {! let! srcPort ! = BitReader.ReadInt32(numBits = 16)! let! destPort ! = BitReader.ReadInt32(numBits = 16)! let! seqNum ! = BitReader.ReadInt32(numBits = 32)! let! ackNum ! = BitReader.ReadInt32(numBits = 32)! let! dataOffset ! = BitReader.ReadInt32(numBits = 4)! let! _ ! = BitReader.ReadByte(numBits = 3)! …! let! payload ! = BitReader.Rest(UTF8.GetString)!! return srcPort, destPort, … payload!}

packs into 32 bit int

Byte1!Byte2!00000000!00000000

memory address

Byte1!Byte2!

bitReader stream {! let! srcPort ! = BitReader.ReadInt32(numBits = 16)! let! destPort ! = BitReader.ReadInt32(numBits = 16)! let! seqNum ! = BitReader.ReadInt32(numBits = 32)! let! ackNum ! = BitReader.ReadInt32(numBits = 32)! let! dataOffset ! = BitReader.ReadInt32(numBits = 4)! let! _ ! = BitReader.ReadByte(numBits = 3)! …! let! payload ! = BitReader.Rest(UTF8.GetString)!! return srcPort, destPort, … payload!}

packs into a byte

0 1 0 1 1 0 1 1

…!let! dataOffset = BitReader.ReadInt32(numBits = 4)!…

0 1 0 1 1 0 1 1

0 1 1 0 1 1 0 0

0 1 0 1 1 0 1 1

0 1 1 0 1 1 0 0

0 0 0 0 0 1 1 0

padding padding padding byte

0 0 0 0 0 1 1 0

1 0 0 1 0 0 0 1

1 0 0 0 1 0 1 1

1 0 0 1 0 0 0 1

1 0 0 0 1 0 1 1

1 1 0 1 1 0 1 0

0 1 0 0 1 0 0 0

Filbert

github.com/fsprojects/Filbert

BERT serializer and BERT-RPC client

@theburningmonktheburningmonk.comgithub.com/theburningmonk

F# at GameSys

Documents

Transcript of F# at GameSys

hristmas at G F

16.33 A At B 8 F

) at the EIC: (F Measuring F Some preliminary …Measuring F 2 n (F 2 d) at the EIC: Some preliminary thoughts EIC Collider Workshop: Electron-Nucleon Exclusive Reactions Rutgers University

AT ST F OUT1 OUT2

F&i training book at online

Gallery Park 2011 - Glenview · 1/11/2011 · Dan Gallery Memorial F F F F F F Designated Fishing Location F F F PARK RULES: Park is open at sunrise and closes at 11:00 pm. The water

39th These lAsh at...3. Let r 1 and r 2 be two circles intersecting at P and Q. The common tangent closer toP, of f 1 and f 2 touches f 1 at A and f 2 at B. The tangent of f1 at P

The F-15 Eagle at 40

Chilled Water Loop Optimization€¦ · Broida; Cooling Tower Approach Set Point WB 55 F WB 70 F Linear (WB 55 F) Linear (WB 70 F) For a given % load: AT = (AT 55WB - AT 70WB) * (70

Inclusive F 2 at low x and F L measurement at HERA

F. Jewell: At Evening Time

G&F The Task At Hand

F - At the Airport

f c e d Orient at at Ion 1

at Oregon State University F•llt•••

Bonita Lestina Performace Series F˚e˛ Admission F˚iday˝ at 8pm

LOOR LANS O RESIDENTIAL ROERTIES IN TE DEVELOMENT 40u … · 2019-11-28 · platform for flat g of tower 1a a.f. a.f. above at 39/f only alum. cladding at 3/f, 6/f, 8/f, 10/f, 12/f,

STRALIS AT/AD · 2019. 12. 14. · ZF 16 S 221 O.D. EuroTronic Automated 12 AS 2301 O.D. EuroTronic Automated 12 AS 2301 D.D. f f f f f f f f f f f f f f f f f f f f f f Allison MD

F f Quadrupole Pick-Ups at CERN & Fermilab A.Jansson FNAL.

Recess at 80 F