Maxim Zaks
Beyond JSON - An Introduction to FlatBuffers
Agenda
• About me
• The Why and How of Data Serialisation
• Deep dive into FlatBuffers
RESI News - Nachrichten, die Spaß machen
VS.
Why do we need data serialisation?
• Persisting State
• Machine to Machine Communication
• Representation of Configuration
How can we persist data on mobile?
• Custom binary representation
• Language provided binary serialisation
• Text based representation
• CSV, XML, JSON, YAML
• Embedded SQL or NoSQL DB
• Binary cross platform serialisation library
• FlatBuffers, Protocol Buffers, Cap’n Proto, SBE, Apache Thrift
Important criteria for persisting data
• Size on disk
• Speed of read & write / partial read / memory consumption
• Human readable and writable
• Support of OO language type system
• Data versioning / evolution / migration
VS.
JSON
• Size on disk 👍👎
• Speed of read & write / partial read / memory consumption 👎
• Human readable and writable 👍 👍
• Support of OO language type system 👍👎
• Data versioning / evolution / migration 👍👎
FlatBuffers
• Size on disk 👍👎
• Speed of read & write / partial read / memory consumption 👍 👍
• Human readable and writable 👍👎
• Support of OO language type system 👍 👍
• Data versioning / evolution / migration 👍 👍
JSON FlatBuffers
size on disk 👍👎 👍👎
read & write performance 👎 👍👍
human readable & writable 👍👍 👍👎
typed API 👍👎 👍👍
multi version support 👍👎 👍👍
But what about sending data?
Important criteria for sending data
• Size on disk wire
• Speed of read & write / partial read / memory consumption
• Human readable and writable
• Support of OO language type system
• Data versioning / evolution / migration
Enough theorylet’s do some practice
FlatBuffers IDL
./flatc -n person.fbs
FlatBuffers Schema Editor
FlatBuffers IDL
FlatBuffers IDL
FlatBuffers IDL
FlatBuffers IDL
What does it all mean in practice?
Size on disk / wire
Speed of read, writeand memory allocation
Partial Read
• over 3M cities
• ≈200 MB
• real time search
• in memory
• from file 10x slower
Human readable & writable is a manner of tooling
• ./flatc -b person.fbs person.json
• ./flatc -t person.fbs -- person.bin --raw-binary --strict-json
Support of OO Language type system
Data is backwards & forwards compatible
More information
• https://google.github.io/flatbuffers/
• https://github.com/mzaks/FlatBuffersSchemaEditor
• https://github.com/mzaks/FlatBuffersSwift
• https://twitter.com/iceX33
Questions?
Thank you!
Top Related