Post on 21-Jan-2018
stateful application
server
HiI’m Lukas
@Overbryd
Working@Wooga
stateful application
server
database based system
app
app
app
load db
clients
stateful system
app
app
app
client
app
app
app
dns load
clients
every server is equal
app
nginx jvm
redis ssd
state
jvm
{}
‣ SessionManager‣ Session‣ Resources‣ Items‣ Quests‣ Payments
...
the hard part
one place at a time
easy with one server,but how to scale out?
sharding!
“static sharding”
shard = facebook_id % shards
raise Invalid, “wrong shard” unless Shard.valid?(facebook_id)
concurrency
jvm
http threads
sessions
ticker thread
workerthreads
the hard part
serialized accessto a session
easy for low contentionlike a single session
class Session
attr_reader :lock
def initialize @lock = Mutex.new end
# ...end
session.lock.synchronize do session.fooend
harder for high contentionlike the SessionManager
class SessionManager
@current = ConcurrentHashMap.new
def self.get(id) raise WrongShard unless Shard.valid?(id) unless session = @current.get(id) new_session = create(id) unless session = @current.put_if_absent(id, new_session) session = new_session end end session end
end
t1
t2
{}S
S
t1
t2
{}S
S
t1
t2
{}S
S
session = @current.get(id)# => nil
session = @current.get(id)# => nil
S2
t1
t2
{}S
S
S2
t1
t2
{}S
S
S1
S2
S2
t1
t2
{}S
S
S1
S2
S2
S2
t1
t2
{}S
S
S1
S2
S2
S2
S2
t1
t2
{}S
S
S1
S2
S2
S2
S2
new_session = create(id)unless session = @current.put_if_absent(id, new_session) # => nil session = new_session # => #<Session:0x2>end
t1
new_session = create(id)unless session = @current.put_if_absent(id, new_session)# => #<Session:0x2>
t2
{}S
S
S1
S2
S2
S2
deployment
the hard part
one place at a time
handover
app
nginx jvm1
jvm2
S1
app
nginx jvm1
jvm2
S1
app
nginx jvm1
jvm2S1
app
nginx
jvm2S1
global state
weird problemwhen handling sharded state
even weirder whenevery machine should be equal
app
nginx jvm
redis ssd
global data is kept local
populates via UPD broadcast
not “a huge” problem if stale
last write winsself healing
class Broadcast
def self.spawn_listener Thread.new do socket = wait_until_socket_bind loop do if @stop_listener socket.close return end message, from = socket.recvfrom(1024) handle_message(message) end end end
end
appjvm redis appjvm redis
internal network
jvm
appjvm redis appjvm redis
internal network
jvm
appjvm redis appjvm redis
internal network
jvm