Mission-Critical Cloud/Enterprise Hybrid Deployments...

Mission-CriticalEnterprise/Cloud

Hybrid Applications

Eugene CiuranaOpen-Source Evangelist

SOBA Labs

http://eugeneciurana.com/contact

This presentation is available for download from:http://eugeneciurana.com

LegalAll products and services mentioned

herein as well as their logosare trademarks or registered trademarks

of their respective companies.

Data contained in this presentation servesinformational purposes only.

The case studies in this presentation are based on work performed at public and private companies in the last 18

months. Some names were changed to protect the innocent.

Enjoy the show!

About Eugene...

• 15+ years building mission-critical, high-availability systems

• 14+ years Java work

• Open source evangelist

• Official adoption of open source/Linux at Walmart worldwide

• Trailblazing Hadoop, mongoDB, Mule deployments

• State of the art tech for main line of business roll-outs

• Largest companies in the world• Retail• Finance• Oil• Background: robotics to on-line retail

This presentation is about...

• How to design, implement, and roll-out cloud/enterprise hybrid applications

• Different cloud architectures

• PaaS• SaaS• Private clouds

• Technologies: ESB, clouds, mini-clouds, Java, App Engine, Chef!/Puppet, etc.

• How cloud technologies lower costs

• Understanding the advantages of:

• Cloud deployments• Hybrid deployments distributed in the cloud and enterprise• Hybrid deployments involving Java and non-Java systems

What You’ll Learn

• Identify the applications best suited for cloud deployments

• Identify the advantages of Platform-as-a-Service or Software-as-a-Service resources

• Learn the caveats of cross-platform and cross-language integration

• Learn about high-performance alternatives to XML serialization for data exchange

• Learn how to structure event-based, stateless apps for maximum scalability

What is the Cloud Anyway?

• Ask 10 different people, get 10 different answers

• In general, you may use 4 types of cloud offerings

• Platform as a Service• Software as a Service• Infrastructure as a Service• Private clouds

• Some times you integrate pre-fabricated apps, some times platform, some times both

Cloud Services Features

• Quick deployment of prepackaged components

• Uses commodity, virtualized hardware and network resources

• Amazon Elastic Cloud 2 (EC2) and Simple Storage Service (S3)

• Google App Engine (Python, Java)• Rackspace Cloud Services

• The overall model is “pay as you consume”

• Horizontal scalability is achieved by adding or removing resources as needed

• May host full applications or only services

Cloud Services Features

• Public clouds could replace the data centre

• Basic administration moves to the application owner

• It may move away from the IT team - political fallout

• For the bean counters... it’s an operational expense!

• Tax advantages• Turn on or off as needed• In a tight economy, IT infrastructure ends up under the CFO -

give the guy options

• Assuming sensible SLAs, the ROI is better than for co-located or company-owned data centres

• What do scalability and high-availability mean?

• Scalability: the property of a system to handle bigger amounts of work or to be easily expanded in response to increased demand

• Vertical• Horizontal

Dual Core

Single Processor

16 GB RAM

Virtual Node 0

Virtual Node 2

Virtual Node 1

Dual Core

Dual Processor

32 GB RAM

Virtual Node 0

Virtual Node 2

Virtual Node 1

Scalability and High Availability

Scales up

Virtual Node 3 Node Node Node

Load Balancer

Scales

out

Node Node Node

Load Balancer

Node

Scalability and High Availability

• Availability: how the system provides useful resources over a set period of time

• Uptime != availability• Many factors affect it: network, storage, processor, etc.

Availability Downtime90% 36.5 days99% 4 days

99.9% 9 hours99.99% 53 minutes

99.999% 5.3 minutes99.9999% 32 seconds

•Amazon S3 Outage (summer 2008)•Major companies affected (LeapFrog, eBay, Apple, many others)•Amazon’s SLA? “Sorry - it won’t happen again.”

Can you affordthis?

SLAs and Availability

• Service Level Agreements drive the architectural and technological decisions

• Cloud systems’ scalability characteristics are often pitched

• What is the availability?

• What is the impact on business?• How do you perform disaster recovery?

• What service level are the vendors offering?

• The higher the availability (3-nines, 4-nines, 5-nines), the higher the cost

• Co-lo, data centre deployments still make more sense for 4-nines availability and above

A Hybrid Cloud Architecture

• Many mission-critical systems will live behind the corporate firewall

• The cloud is used for high-load applications and services

• The clouds applications work independently of the data center applications, and vice versa

• Loose coupling over web services• Assume that the data in the cloud is “throwaway” - it’s either

consolidated or sourced in the data centre• The cloud exists mostly for distribution and scalability

A Hybrid Cloud Architecture

services.company.com

Node Node Node Node

Legacy Back-end

Enterprise

Database

Enterprise Service Bus

User

services.company.com

Firewall

Amazon S3http://media.company.com

PNG PNG PNG PNG

PNG PNG PNG PNG

Google App Enginehttp://www.company.com

Node Node Node

Node Node Node

Datastore Datastore

Which Parts Go To The Cloud?

Backbone - message filtering, routing, dispatching, queuing, events

Internet

Load Balancer

Application

Server

Tomcat 6

Services Proxy

Application

Server

Tomcat 6

Load Balancer - Backbone

Mule ESB Mule ESB Mule ESB Mule ESB

Load Balancer - Tailbone

Mule ESB

SOAP, RESTMule ESB

SOAP, REST

Database

Load Balancer - Funnybone

Mule ESB

servlet, MTOMMule ESB

servlet, MTOM

NFS

share

Load Balancer - Message Broker

ActiveMQ ActiveMQ

NFS

share

Which Parts Go To The Cloud?

• Depending on the cloud configuration, load balancing may not be necessary

• Amazon provides Elastic Load Balancers and Elastic IP• Google App Engine provides stateless calls only

• The client itself or Memcache keep state instead• Rackspace provides either “elastic” addresses or explicit load

balancers

• Enterprise Service Bus and other integration may exist in both the cloud and the data centre

• Message routing is OK is if latency is below the SLA requirements

• Strategic partner, application, and services integration may occur 100% on the cloud, with data consolidation behind the firewall

Case Study

• Large consumer and services company

• .Net / Windows servers infrastructure

• Java, ASP.Net, PHP• SQL Server

• Two-tier architecture

• The system “just grew” and became a business

• Server pages, services, media

• 6-month scalability project began in April

• Implementation complete 25.Sep of the same year

Case Study - Objectives

• Stable architecture defined by July

• Low cost

• Build scalability wherever possible

• Optimal data transfer rate for all properties

• Web systems• Media properties

• Meet business requirements and SLAs

• Hard dates set by the business, no input from the technology team

• Milestones: 1.July, 10.Oct

Case Study - Initial Configuration

• System grew “as needed” - no planning

• Combines end-user and internal applications in the same server

• It only scales vertically -- and not very well

• Tightly coupled applications + SQL Server database

• Tightly coupled application servers and services

• Single environment: from development to production without passing Go nor collecting $200

• Disaster recovery?

• 4 hours minimum• From (stale?) backups

Case Study - Phase I Scalability


• Introduces a CDN for asset delivery

• Amazon S3 (optional Cloud Front) for asset delivery• Reduces load on company servers and bandwidth costs

• Introduces database replication for production environments

• Introduces NoSQL storage for write-once, consult-often structured documents

• Establishes a continuous integration environment

• Set tools for hybrid UNIX/Windows environment• UNIX tools take precedence; Cygwin everywhere

• Improved build/release process


Client RelationshipsApp

Dispatcher

CRM

Custom Queuing System

Main App

Search

Queue

StaticFiles(S3)

Reporting

queryreply

Rich Docs(GridFS)

upda

te

NetezzaLucene

ServiceProviders

Various servicesproviders throughoutthe Internet. Someare public, some arepartners

End UsersService Consumers

BrowserRSSOutlookCWSEWS

End Users

Service Consumers

Internal Service

Providers

Heavy web servicesSome XML, some custom

Fire

wal

l

Legend

HTTPSOAPCustom RPCODBC/JDBCDirect/API

Internal End Users


• Fail-over with traditional database replication techniques

Partition 0

Primary Cluster

Node 0 Node 1

DB 0

DB 0b

Partition 1

DB 1

DB 1b

ESB as app services provider

Case Study - Phase II Cloud Deployment


• Web applications move to a uniform technology

• All critical infrastructure moves to Java

• Separation between end-user and services applications

• The database and stored procedures are normalized and optimized

• Applications use common resources via Mule ESB and services

• No more direct calls from apps to databases• Business logic is implemented as stateless POJOs

• Software stack is “best of breed”

• Windows, other servers driven by business requirements• Open source software wherever possible


• Web and other RPC services must coexist

• Different partners use different protocols• Mule transformers take care of all the interfaces so that

development may scale / continue independently of what happens in other layers

• Bandwidth can be expen$ive!

• Data exchange protocols

• Clients: custom, XML, JSON• Images: HTTP, S3• Cloud-to-HQ: JSON, XML/SOAP• HQ data centre: POJOs, JSON, BSON, XML

• Replication strategy: data centre

Phase II Cloud Deployment - Sep 2010

Databases

Search

StaticFiles(S3)

Reporting

ServiceProviders End UsersService

Consumers

BrowserRSSOutlookCWSEWS

Internal Service

Providers

Mule ESB Container: Services, Message Routing, and TransformationsClient

RelationsServices

DispatcherServices

Main App Services

OpenMQOther NewServices

Tomcat App ContainerMain App

(zone instance)Client Relations(Zone Manager)Dispatcher

New Apps

New System

Acquisition(.Net, PHP,

etc.)

cronServices

Rich Documents

(GridFS)

memcached

Local DBs, Other

Resource

Cloud Firewall

Legend: HTTP | Web services (SOAP, REST, JMS, other) | JDBC | Direct/API/Any

EnterpriseServices

Corporate Firewall

End Users

Various servicesproviders throughoutthe Internet. Someare public, some arepartners

Phase II: Hybrid Dev Tools

JavaStatic Typing

VerboseFaster Execution Time

Slower Compile/Build/Test/Deploy CycleSlower Development Time

Harder to read?

PythonDynamic Typing

ConciseSlower Execution Time

Faster Develop/Deploy CycleFaster Development Time

Easier to read?

Excellent class libraryExcellent 3rd party support

Great for writing robust apps

Java and Python Coexist in Tomcat, Mule, and Spring!

Phase II: Hybrid Dev Tools

• Coding time reduced by at least 50%

• Developers are fluent in Java and Python

• Source code reduced by 30% to 75%

• Algorithmic code saw the least reduction• Regular code + API calls average 50%

• Development/testing cycles reduced from 25% to 50%

• Save/test vs. save/build/package/deploy/stop/start

• Can use with or without OSGi

• Language features help to come up with fresh approaches to problem solving

Development Speed

Execution Speed

Java

JavaPython

PythonC

C

Case Study - Phase III Q1 2011

Phase III Data Mining - Q1 2011

DatabasesSearch

Hive

StaticFiles(S3)

ReportingPig

Mule ESB Container: Services, Message Routing, and TransformationsClient

RelationsServices

DispatcherServices

Main AppServices

OpenMQOther NewServices

Tomcat App ContainerMain App

(zone instance)Client Relations(Zone Manager)Dispatcher

New Apps

cronServices

Rich Documents

(GridFS)

memcached

Internal Services

HDFS, GridFS, Data Warehouse Hadoop, DB cluster, computational network

ExternalService or Consumer

Cloud-based MapReduce/NoSQLInfrastructure - expand and contract

capacity as-needed

Hosting Hypervisor (EC2, VMware, Xen)

Linux (CentOS, Ubuntu Server)

Java 6 or later

Unique TCP/IP networking configurationConfig mgmt: Puppet, SOBA, Chef!

Load balancersticky sessions, fail-over,

80:8080

DMZ reverse proxy, Apache gateway :80

memcached

Spring

Tomcat Application Container

Flex HbrntePD4

ML

MuleClient

$FRMWORK

ClientRels Main AppDispatcher $APP

App server configuration web.xml

ulimit configuration



Java 6 or later


Tomcat Application Container


Spring Flex HbrntePD4

ML

MuleClient

$FRMWORK

App server configuration web.xml

3rd PartyServiceSearchReporting



Java 6 or later


memcached

Spring

Mule Services Container

Hbrnte GridFS$FRMWORK

jTDSORCL

v.4

$FRMWORK

Client RelServices

Main AppServices

DispatcherServices

$APPServices

App server configuration mule-config.xml


JMS Container (OpenMQ, ActiveMQ, other)

HTTPTransport

SQLData Srce

RPCTransport

$OTHERTransport

Load balancerstateless round robin,

80:8080

DMZ reverse proxy, Apache gateway :80

Three classes of servers. Services and applications are enabled throughconfiguration changes, not software changes. Every server in a givenclass gets the same exact software.

Each server runs in its own virtual machine. Services can be combined.

Application Server

Internal Features Server

Services Provider

Phase III: System Architecture Q1 2011

Phase III: System Administration

sobaDB192.168.0.42

Other Consumer

192.168.0.42sobaEngine

localhost

UbuntuLandscape

REST SOBA interface - implementation is transparent to caller!http://soba.myserver.com/manage/resource

Firewall Oracle

vm_uuid:b220c8db

Xen Host

SOBA Agent

Xen XML-RPC API

REST SOBA interface

Xen Python

SOBA Python

Amazon EC2

End-user Appami-322ec65b

End-user Appami-322ec65b

EC2 web services API

Phase III: System Administration

Mule-based SOBA Engineabstracts provisioning, configuration, and

monitoring through web services

Java and Python Web Services Interface

CANONICAL LandscapeOther Application

easy integration!

JSONJSON

web

serv

ices

REST

REST

webservices

SOBA Engine

PythonAPI

Native Applicationeasy integration!

Python

dict

amazon EC2 API Xen Server API RackspaceCloud Servers API

SOBA Agent

PythondictEC2

web

serv

ices A

PI

Xen

XML-

RPC

API

REST

JSON

JSON

SOBA Data

mongoDB

EC2 Data

XMLEC2 Query

XML

Config Data

(Puppet?)

Ubuntu Server

Ubuntu Server

Ubuntu Server

REST

DRY Interface

Don't Repeat Yourself!

Provisioning, configurationor monitoring via SOBA is thesame regardless of target:Same API call, same datapayload, same dataformat, etc.

Implementation isabstracted from thecaller!

dict

SOBA Agent

puppetfacter

EnsembleAgent

puppetfacter

SOBA Agent

puppetfacter

Thanks for Coming!

Questions?

Wanna know more about real life cloud, scalable systems?

http://eugeneciurana.com/scalablesystems

Subscribe to the newsletter!

Eugene CiuranaOpen source [email protected]+1 415 387 3800

http://twitter.com/ciurana technology tweets

Mission-Critical Cloud/Enterprise Hybrid Deployments...

Documents

Transcript of Mission-Critical Cloud/Enterprise Hybrid Deployments...