Proud to be polyglot!

Proud to be Polygot!Tugdual “Tug” Grall

@tgrall

Technical Evangelist at .

Wednesday, November 6, 13

{about : “me”}

•Tugdual “Tug” Grall

Couchbase

Technical Evangelist

eXo

CTO

Oracle

Developer/Product Manager

Mainly Java/SOA

Developer in consulting firms

• Web

@tgrall

http://blog.grallandco.com

tgrall

NantesJUG co-founder

Pet Project :

http://www.resultri.com

[email protected]

[email protected]






mailto:[email protected]




Modern Applications•Modern Applications Must

Manage any type of data

Scale

Be fault tolerant

Adapt to change


Standard Architecture•Mono-Technology

Same “language/architecture”

Single Database : RDBMS

•One size fits all approach

JSP/JSFServices

MessagingJTA

DAO/JPA


Is it good for me?•Many files

-XML, Sources, ..

• Hibernate/JPA is good for all?

- Impedance mismatch

•Need to scale

- from 0 to 50millions users in 6 weeks (real life)

- can I do that easily with such architecture?

• How to add new features easily?

- code, schema change, ...

- and all this for yesterday!

• Lot of code


One size does NOT fit all!


Look at our Data....


0

0.50

1.00

1.50

2.00

2006 2009 2012Source: IDC

Trillions of G

igabytes (Ze:

abytes)

Big Data

Unstructured and Semi-‐Structured Data

Structured Data

Text, Log Files, Click Streams, Blogs,

Tweets, Audio, Video, etc.

High Data Variety and Velocity

More Flexible Data Model Required


50 Million Users in 50 Days

191715131197533/12826242220181614121082/6

Draw Something by OMGPOPDaily Ac)ve Users (millions)

21

2

4

6

8

10

12

14

16


How do you take this growth?

RDBMS Scales UpGet a bigger, more complex server

Users

ApplicaXon Scales OutJust add more commodity web servers

Users

System CostApplicaMon Performance

RelaXonal Database

Web/App Server Tier


Won’t scale beyond this point


NoSQL Database Scales OutCost and performance mirrors app Xer

Users

NoSQL Distributed Data Store

Web/App Server Tier

ApplicaXon Scales OutJust add more commodity web servers

Users


ApplicaMon Performance System Cost

NoSQL Technology Scales Out


ClouderaHortonworks

Mapr

CouchbaseMongoDB

AnalyXcDatabases

Get insights from data

Real-‐Xme, InteracXve Databases

Fast access to data

NoSQL

OperaMonal vs. AnalyMc Databases


Polyglot Persistence

Document & K/V

• Products• User Profiles• Game AcMons• Sessions• Shopping Cart

RDBMS

• Financial Data• ReporMng

Big Data/Analysis

• Log Capture• RecommendaMons• AdCampaign

Indexing

•Full Text Search•Ad hoc queries


Developer New Skills

• Understand the pros/cons of each solutions

Programming API

How to access the data? (API, Query Languages)

• Learn how to design data-model

De-normalizing and duplicate data is not a problem

• Integrate each solution to your application


Data Service Layer

Data Service

•Wrap datastore in Services

- Loosely coupled


Use Case : Content Driven Site


Use Case: Mobile Services

PIM Database

-‐ Legacy ApplicaXon-‐ Product InformaXon

NoSQL

-‐ Product Data-‐ AddiXonal Metada

Web/App Server Tier Web/App Server Tier

Rabbit MQ


Now what?


Why multiple languages?• Sometimes we have no choice

HTML/CSS/JavaScript + Server Side

Mobile Native Applications

• Java/.Net have not evolved that must these past years

Java EE 5.0 is still mainstream (2006)

• Many languages and frameworks

Scala, Clojure, Groovy, Dart, Go, Ruby, Python, F#, Erlang

Node.js, Rails, Play!, Grails, ...


Examples• Realtime interaction with WebSockets

- Wait for Java EE 7 to be out?

- Hack your app server and application with Continuation ?

- Use Node.js and Socket.io ?

• Data Collection and Treatment?

- Find a library?

- Create your library?

- Wait for Java 8 and Lambdas?

- Use Scala? ... or other


Examples

• Simple CRUD Application

Use the JavaEE stack with JPA and JSF?

Use Spring ?

Use tools like Play!, Grails, Rails ?


How to chose?

•Maturity/Supportability

• Features Set

• Learning Curve

• Productivity


Polyglot Programming in Action

• Build a distributed database

Manage distributed processes

Save and retrieve data on disk

Cache the data on RAM

Build a query engine


Polyglot Programming in Action• Manage Distributed Process

Erlang and OTP (Open Telecom Platform) has been built for that

Taking this will bootstrap a project quickly instead of implementing in another language

• Data access and Caching

C/C++ is the best/fastest way to interact with I/O

Leverage existing Caching layer, for example Memcached written in C

• Query & Indexing

Easy to code for developer : Let’s use JavaScript


ReplicaMon, Rebalance, Shard State Manager

REST management API/Web UI

8091Admin Console

Erlang /O

TP

11210 / 11211Data access ports

Object-‐managedCache

Storage Engine

8092Query API

Que

ry Engine hbp

Data Manager Cluster Manager

Cluster ManagerWriben in Erlang

Data ManagerWriben in C/C++

Couchbase Server


Use Case: Second Screen Gaming

Java/Play ApplicaXonPython scrips

AdministraXon

Node.js + Socket IO

Users


Conclusion• Use the good tool for the good task

Do not try to “hack” something when you have a shorter solution

Think about your users first... so deliver and get feedback

• Learning is part of our job !

And this will help you and your project

•Operations / DevOps

Think about it from day one... as a developer

•What about maintenance?

Do you think it is easier to maintain complex code?


Q&A

@[email protected]




Proud to be polyglot!

Technology

Transcript of Proud to be polyglot!