"Fast but not Loose" -- Typesafe client/server architecture at Gilt Groupe

Fast but Not Loose:Typesafe Clients in a

Distributed Service Architecture, a retrospective

Eric BowmanVP Architecture @ Gilt Groupe

@ebowmanebowman@gilt.com

#gotoaar #gilttech#gotocon

Monday, September 30, 13

•Scala

•Play

•PostgreSQL

•MongoDB

•Voldemort

•Kafka

•Aster Data

•Mahout

•Jersey

•SBT

•Docker

•Continuous Delivery

Microservices

http://upload.wikimedia.org/wikipedia/commons/1/16/Ruby_on_Rails-logo.pnghttp://wiki.postgresql.org/wiki/File:PostgreSQL_logo.3colors.svgMonday, September 30, 13

http://logonoid.com/images/thumbs/christian-louboutin-logo.jpgMonday, September 30, 13

http://data.iluxdb.com/data/christian-louboutin-daffodile-160mm-python-masai-1130127cm09_001.jpg?dd80c0Monday, September 30, 13

http://upload.wikimedia.org/wikipedia/commons/7/79/Operation_Upshot-Knothole_-_Badger_001.jpg

http://blog.verwilst.be/wp-content/uploads/2008/12/java.gif

user_service order_service payment_service product_service cart_service

pagegen

http://megmurph.com/wp-content/uploads/2013/03/success-1.jpg

Front End

Service X

Service Y

Service Z

Front End

Service X

Service Y

Service Z

Front End

Service X

Service Y

Service Z

Content from Service X

Content from Service Y

Content from Z

Front-End Tier

Service Tier

Data Tier

Front-End Tier

Service Tier

Data Tier

CachingLight Computation

Orchestration

Front-End Tier

Service Tier

Data Tier

Orchestration

CachingHeavier Computation

Separation of Concerns

Front-End Tier

Service Tier

Data Tier

Orchestration

Data Access(Disk/SSD/RAM)

•Runtime Temperature•Development Temperature

Front-End Tier

Service Tier

Data Tier

Orchestration

Data Access(Disk/SSD/RAM)

Data Source

Service

JDBC ➾ HashMap ➾ JSON

http://gilt.com

Javascript Application

Front End

JSON ➾ HashMap ➾ JSON

Chaos Grows Quickly

Data Source

Service

JDBC ➾ HashMap ➾ JSON

http://gilt.com

Front End

http://gilt.com

Front End

Service

JDBC ➾ HashMap ➾ JSONData Source

Service

http://gilt.com

Front End

Service

http://www.recruitmenttakeout.com/wp-content/uploads/2013/04/brian.png

•Fast

•Service Decomposition

•Implicit core model was good

•Org Scaling

•APIs

•Implicit core model was ... implicit

A data model and APIs for services

(aka, RPC)

http://git-scm.com/images/logos/downloads/Git-Logo-2Color.png

core service

Legacy

core service

Legacy

commons.jar

core clients

core data model

async client framework

Greenfield

core service

Legacy

commons.jar

core clients

core data model

async client framework

Greenfield

•Users•Sales•Products•Skus•Assets•Targeting•Auth

service

client

service

client

•RESTful•Scala clients•All APIs futures-based•Case class schema

service

client

service

Compile/RuntimeDependency

ConsumerServiceClient

Core Core

“Embassy Soil”

Core Core

•Easy functional testing•Service response capture•Test linking•Upgradable•Emergent Regression•Automated upgrades•Compile farmers

Core Core

•Environment ⊕ Config•Live updates•Indirection•Circuit Breaker

Core Core

“All of this is completely wrong.”*

*Not an actual quoteMonday, September 30, 13

1.The network is reliable2.Latency is zero3.Bandwidth is infinite4.The network is secure5.Topology doesn't change6.There is one administrator7.Transport cost is zero8.The network is homogeneous

https://blogs.oracle.com/jag/resource/Fallacies.html

1.The network is reliable

Resolved

2.Latency is zero

Resolved

3.Bandwidth is infinite

Resolved

4.The network is secure

Resolved

5.Topology doesn't change

Resolved

6.There is one administrator

Resolved

7.Transport cost is zero

Resolved

8.The network is homogeneous

Resolved

“Still Wrong.”*

*Possibly an actual quoteMonday, September 30, 13

Convenience Over Correctness

http://steve.vinoski.net/blog/2008/07/01/convenience-over-correctness/Monday, September 30, 13

Database

Database Model

Service Backend

Service Model

Service Frontend

Client/Server Serialization Model

Client Backend

Client Data Model

Client Layer

Public Data Model

Database

Database Model

Service Backend

Service Model

Service Frontend

Client Backend

Client Data Model

Client Layer

Public Data Model

•So many models•Corners are cut•Typesafe helps•Conflation?•Just the data

“Works in Practice for some use cases”

Database

Database Model

Service Backend

Service Model

Service Frontend

Client Backend

Client Data Model

Client Layer

Public Data Model

“Works in Practice for some use cases”“No free silver bullet lunches.”

Database

Database Model

Service Backend

Service Model

Service Frontend

Client Backend

Client Data Model

Client Layer

Public Data Model

However...

•No machine generated stubs•Embassy-Oriented Programming•Lots of indirection•Type-system support for failures

However...

What Actually Sucks about RPC:

What Actually Sucks about RPC:•Remote objects •Failures•Idempotency

Ignoring all that was Too Easy.

Batch Jobs

1.The network is reliable2.Latency is zero3.Bandwidth is infinite4.The network is secure5.Topology doesn't change6.There is one administrator7.Transport cost is zero8.The network is homogeneous

Law of Instrument

Pulling data.

Pushing code.

SOA, Reloaded

SOA, Reloaded•CRUD•Event Streams•Batch Processing•Lambda Architecture•CQRS

Client

Service

Client

Fast Storage

Service

Data Warehouse

Batch Jobs

Clickstream data

http://tech.gilt.com

join us.new york & dublin

"Fast but not Loose" -- Typesafe client/server architecture at Gilt Groupe

Technology

Transcript of "Fast but not Loose" -- Typesafe client/server architecture at Gilt Groupe

Mobile Design at Gilt

Gilt Management Meeting: Practical Considerations for the ... › media › 274434 › ahdb-gilt... · sows • Each 2nd parity sow’s litter is larger than her gilt litter( not

&gilt - legislation.gov.au

Gilt management

Going Reactive in Java with Typesafe Reactive Platform

Akka 2.4 plus new commercial features in Typesafe Reactive Platform

Akka 2.4 plus commercial features in Typesafe Reactive Platform

Real time ecommerce analytics with MongoDB at Gilt Groupe (Michael Bryzek & Michael Nutt)

[Tech Talks] Typesafe Stack Introduction

Introduction to the Typesafe Reactive Platform

2009 Equity Gilt Study

Gilt Groupe - UCLA Econ · Gilt Groupe 4 | P a g e validate the code. Such quality control is an important element leading to performance and customer satisfaction.10 Development

Simplifying development-short - Mirco Dotta (Typesafe)

Indic threads pune12-typesafe stack software development on the jvm

GILT Semester Translation UTA

Typesafe stack - Scala, Akka and Play

Continuous Innovation @ Gilt Groupe

Scaling Scala to the database - Stefan Zeiger (Typesafe)

The Trouble with Types Martin Odersky EPFL and Typesafe.

Typesafe Webinar: Going Reactive with Java 8