Proud to be Polygot!Tugdual “Tug” Grall

@tgrall

Technical Evangelist at .

Wednesday, November 6, 13

{about : “me”}

•Tugdual “Tug” Grall

­ Couchbase

­ Technical Evangelist

­ eXo

­ CTO

­ Oracle

­ Developer/Product Manager

­ Mainly Java/SOA

­ Developer in consulting firms

• Web

­ @tgrall

­ http://blog.grallandco.com

­ tgrall

­ NantesJUG co-founder

­ Pet Project :

­ http://www.resultri.com

­ tug@couchbase.com

­ tugdual@gmail.com

Wednesday, November 6, 13

Modern Applications•Modern Applications Must

­ Manage any type of data

­ Scale

­ Be fault  tolerant

­ Adapt to change

Wednesday, November 6, 13

Standard Architecture•Mono-Technology

­ Same “language/architecture”

­ Single Database : RDBMS

•One size fits all approach

JSP/JSFServices

MessagingJTA

DAO/JPA

Wednesday, November 6, 13

Is it good for me?•Many files

-XML, Sources, ..

• Hibernate/JPA is good for all?

- Impedance mismatch

•Need to scale

- from 0 to 50millions users in 6 weeks (real life)

- can I do that easily with such architecture?

• How to add new features easily?

- code, schema change, ...

- and all this for yesterday!

• Lot of code

Wednesday, November 6, 13

One  size  does  NOT  fit  all!

Wednesday, November 6, 13

Look at our Data....

Wednesday, November 6, 13

0

0.50

1.00

1.50

2.00

2006 2009 2012Source:  IDC  

Trillions  of  G

igabytes  (Ze:

abytes)

Big  Data

Unstructured  and  Semi-­‐Structured  Data

Structured  Data

Text,  Log  Files,  Click  Streams,  Blogs,  

Tweets,  Audio,  Video,  etc.

High Data Variety and Velocity

More  Flexible  Data  Model  Required

Wednesday, November 6, 13

50  Million  Users  in  50  Days

191715131197533/12826242220181614121082/6

Draw  Something  by  OMGPOPDaily  Ac)ve  Users  (millions)

21

2

4

6

8

10

12

14

16

Wednesday, November 6, 13

How do you take this growth?

RDBMS  Scales  UpGet  a  bigger,  more  complex  server

Users

ApplicaXon  Scales  OutJust  add  more  commodity  web  servers

Users

System  CostApplicaMon  Performance  

RelaXonal  Database

Web/App  Server  Tier

System  CostApplicaMon  Performance  

Won’t  scale  beyond  this  point

Wednesday, November 6, 13

NoSQL  Database  Scales  OutCost  and  performance  mirrors  app  Xer

Users

NoSQL  Distributed  Data  Store

Web/App  Server  Tier

ApplicaXon  Scales  OutJust  add  more  commodity  web  servers

Users

System  CostApplicaMon  Performance  

ApplicaMon  Performance  System  Cost

NoSQL Technology Scales Out

Wednesday, November 6, 13

ClouderaHortonworks

Mapr

CouchbaseMongoDB

AnalyXcDatabases

Get  insights  from  data

Real-­‐Xme,  InteracXve  Databases

Fast  access  to  data

NoSQL

OperaMonal  vs.  AnalyMc  Databases

Wednesday, November 6, 13

Polyglot Persistence

Document  &  K/V

•  Products•  User  Profiles•  Game  AcMons•  Sessions•  Shopping  Cart

RDBMS

•  Financial  Data•  ReporMng

Big  Data/Analysis

•  Log  Capture•  RecommendaMons•  AdCampaign

Indexing

•Full  Text  Search•Ad  hoc  queries

Wednesday, November 6, 13

Developer New Skills

• Understand the pros/cons of each solutions

­ Programming API

­ How to access the data? (API, Query Languages)

• Learn how to design data-model

­ De-normalizing and duplicate data is not a problem

• Integrate each solution to your application

Wednesday, November 6, 13

Data Service Layer

Data  Service

•Wrap datastore in Services

- Loosely coupled

Wednesday, November 6, 13

Use Case : Content Driven Site

Wednesday, November 6, 13

Use Case: Mobile Services

PIM  Database

-­‐  Legacy  ApplicaXon-­‐  Product  InformaXon

NoSQL

-­‐  Product  Data-­‐  AddiXonal  Metada

Web/App  Server  Tier Web/App  Server  Tier

Rabbit  MQ

Wednesday, November 6, 13

Now what?

Wednesday, November 6, 13

Why multiple languages?• Sometimes we have no choice

­ HTML/CSS/JavaScript + Server Side

­ Mobile Native Applications

• Java/.Net have not evolved that must these past years

­ Java EE 5.0 is still mainstream (2006)

• Many languages and frameworks

­ Scala, Clojure, Groovy, Dart, Go, Ruby, Python, F#, Erlang

­ Node.js, Rails, Play!, Grails, ...

Wednesday, November 6, 13

Examples• Realtime interaction with WebSockets

- Wait for Java EE 7 to be out?

- Hack your app server and application with Continuation ?

- Use Node.js and Socket.io ?

• Data Collection and Treatment?

- Find a library?

- Create your library?

- Wait for Java 8 and Lambdas?

- Use Scala? ... or other

Wednesday, November 6, 13

Examples

• Simple CRUD Application

­ Use the JavaEE stack with JPA and JSF?

­ Use Spring ?

­ Use tools like Play!, Grails, Rails ?

Wednesday, November 6, 13

How to chose?

•Maturity/Supportability

• Features Set

• Learning Curve

• Productivity

Wednesday, November 6, 13

Polyglot Programming in Action

• Build a distributed database

­ Manage distributed processes

­ Save and retrieve data on disk

­ Cache the data on RAM

­ Build a query engine

Wednesday, November 6, 13

Polyglot Programming in Action• Manage Distributed Process

­ Erlang and OTP (Open Telecom Platform) has been built for that

­ Taking this will bootstrap a project quickly instead of implementing in another language

• Data access and Caching

­ C/C++ is the best/fastest way to interact with I/O

­ Leverage existing Caching layer, for example Memcached written in C

• Query & Indexing

­ Easy to code for developer : Let’s use JavaScript

Wednesday, November 6, 13

ReplicaMon,  Rebalance,    Shard  State  Manager

REST  management  API/Web  UI

8091Admin  Console

Erlang  /O

TP

11210  /  11211Data  access  ports

Object-­‐managedCache

Storage  Engine

8092Query  API

Que

ry  Engine hbp

Data  Manager Cluster  Manager

Cluster  ManagerWriben  in  Erlang

Data  ManagerWriben  in  C/C++

Couchbase Server

Wednesday, November 6, 13

Use Case: Second Screen Gaming

Java/Play  ApplicaXonPython  scrips

AdministraXon

Node.js  +  Socket  IO

Users

Wednesday, November 6, 13

Conclusion• Use the good tool for the good task

­ Do not try to “hack” something when you have a shorter solution

­ Think about your users first... so deliver and get feedback

• Learning is part of our job !

­ And this will help you and your project

•Operations / DevOps

­ Think about it from day one... as a developer

•What about maintenance?

­ Do you think it is easier to maintain complex code?

Wednesday, November 6, 13

Q&A

@tgralltug@couchbase.com

Wednesday, November 6, 13