1

Adjusting Carbon Topology to Match High Availability Scenario

Requirements

Afkham Azeez

Director of Architecture

WSO2 Inc

2

About Me

• PMC member Apache Axis, Committer Synapse & Web Services

• Member, Apache Software Foundation• Co-author, Axis2 Web Services• Director of Architecture, WSO2 Inc• Blog: http://blog.afkham.org• Twitter: afkham_azeez

3

Agenda

• A brief look at the WSO2 platform

• Carbon clustering for availability

• Cost of availability & related topologies

4

WSO2 Offerings• WSO2 Carbon

• Full platform of servers for deployment on-premise, in private or public cloud

• Products share a consistent architecture and core platform services (e.g. logging, management, security, identity, caching) through OSGi and the “Carbon Core”

• Includes ESB, AppServer, Data Services, Governance, Identity, Business Process, and more

• WSO2 Stratos• Platform-as-a-Service (PaaS) Foundation

• Supports running servers as elastic, metered, billed, multi-tenant with self-service

• Including all Carbon Servers, PHP, Jetty, and a growing list through a standard Cartridge model

• WSO2 StratosLive• http://stratoslive.wso2.com

• WSO2’s Public PaaS

• An instance of Stratos running in the cloud with all Carbon Servers available

5

Consistent Architecture

• Carbon: A consistent set of class-leading enterprise servers

• The same products run either on-premise or in the cloud, single-tenant or multi-tenant

• Utilize the same Carbon core runtime for a seamless experience

• Stratos: A cloud platform for enterprise, hybrid and public deployment

• Extends the deployment to support full self-service, elastic scaling, metering and billing

• Supports Carbon and native server runtimes

• Including Java and non-Java servers such as Jetty and PHP

• Re-uses the same core Carbon architecture to offer core PaaS services including:

• Identity, Logging, File, Relational Storage, Column Storage, Code Deployment, etc

• Both projects share a common set of OSGi modules and a core runtime architecture

6

WSO2 SOA Platform

7

WSO2 Carbon

8

Availability

The degree to which a system, subsystem, or equipment is in a specified operable and committable state at the start of a mission, when the mission is called for at an unknown, i.e., a random, time.

Simply put, availability is the proportion of time a system is in a functioning condition.

9

Availability

10

Availability

11

High Availability (HA)

A system that is designed for continuous operation in the event of a failure of one or more components. However, the system may display some degradation of service, but will continue to perform correctly.

The proportion of time during which the service is accessible with reasonable response times should be close to 100%.

All single points of failure should be eliminated

12

HA, CO & CA

• Continuous Operation (CO)

• Ability to avoid planned outages.

• hardware and software maintenance carried out while applications remains available users.

• Continuous Availability (CA)

• Combines the characteristics of HA and CO to keep the applications running without any noticeable downtime

• Hot update/ graceful round-robin restart

13

High Availability Techniques

• Redundancy

• Time – retransmit

• Data – e.g. parity bits

• Processing – e.g. redundant nodes

• Diversity

• e.g. Hybrid deployments, do the same thing using different implementations

14

How to decide required availability?

• Average throughput (TPS)

• Max throughput (TPS)

• Monetary value of a transaction

• Average loss & max loss per second of downtime

• Decide on how much to invest on availability based on cost vs. benefit tradeoff

15

Patching Production DeploymentsPatch Distribution Coordinator

3. Push patch 3. Push patch

3. Push patch

3. Push patch

1. Check patch list2.Pull new patch

16

Patching Production DeploymentsPatch Distribution Coordinator

4. Maintenance mode 5. Graceful restart

Round-robin

17

Clustering

• Clustering for scalability

• Clustering for availability

18

Clustering for scalability

18

19

Clustering for availability

19

Group Communication Channel/State replication

20

Carbon Clustering

• Membership types

• Static

• Dynamic

• Hybrid

• Membership modes

• Multicast

• Well-known address

21

Static membership

• Predefined members

• Other (non-predefined) nodes cannot join

M2M1

M3 M4

Static group

N

22

Dynamic membership

• No predefined members

• Nodes can join & leave

M2M1

M3 M4

Dynamic group

NJoin

23

Hybrid membership• Some predefined (well-known) members, and some

dynamic members

• Nodes can join & leave

• Membership revolves around the static members

Hybrid group

N

M2M1

M3 M4

Static members

M6M5

M7

Dynamic members

Join (IP, Port)

24

Multicast based membership management

N

M2

M1

M3

M4

Join (IP, Port)

25

Well-known Address (WKA) based membership management

Hybrid group

NWK2

WK1

WK3 WK4

Static members

M6

M5

M7

Dynamic members

Join (IP, Port)

Notify

26

Multicast vs. WKA

Multicast WKA

All nodes should be in the same subnet Nodes can be in different networks

All nodes should be in the same multicast domain No multicasting requirementMulticasting should not be blocked

No fixed IP addresses or hosts required At least one well-known IP address or host required

Failure of any member does not affect membership discovery

New members can join with some WKA nodes down, but not if all WKA nodes are down

Does not work on IaaSs such as Amazon EC2

IaaS-friendly

Requires keepalived, elastic IPs or some other mechanism for remapping IP addresses of WK members in cases of failure

27

Multicast vs. WKA – how to decide?

• Multicast

• Cluster is going to be setup in a network where multicasting is allowed

• WKA

• Cloud based deployment

• Members are distributed across datacenters & regions

• Multicasting blocked

28

HTTP Session Replication

• catalina-server.xml• <Cluster className="org.wso2.carbon.core.session.CarbonTomcatSimpleTcpCluster"/>

• <Valve className="org.wso2.carbon.tomcat.ext.valves.CarbonTomcatSessionReplicationValve"/>

• web.xml• <distributable/>

29

State Replication

JSR-107/JCacheA standard Java Caching API for use by developers and a standard SPI ("Service Provider

Interface") for use by implementers.

import javax.cache.*

…

CacheManager cacheMgr = Caching.getCacheManager();

Cache<String, Integer> cache =cacheMgr .getCache(cacheName);

cache.put(“key”, sampleValue);

Integer i = cache.get(“key”);

30

State Replication

CarbonContext based API

Cache cache = CarbonContext.getCurrentContext().getCache();

cache.put(“key”, sampleValue);

Integer i = cache.get(“key”);

Axis2 Contexts

Using Axis2 clustering StateManager – axis2.xml

<stateManager class="org.apache.axis2.clustering.state.DefaultStateManager” enable=”true">

31

Elastic Load Balancer 2.0

• New sysadmin-friendly configuration language

• High performance PassThrough transport

• Tenant-aware load balancing

• Ability to dedicate clusters for tenants (private jet mode)

• Improved auto-scaler

• Separate IaaS-aware Cloud controller takes care of spawning new instances on different IaaSs

32

Tenant-aware LB

Private Jet mode

• Analogy• Economy class

• no SLA management, only elasticity

• Business class • elasticity plus SLA guarantees

• Private Jet • Guaranteed isolated VMs or machines for a specific

tenant• Still elastically scaled

34

Private Jet Mode

35

Topologies

• Single node

• Multi-node with LB

• Multi-node with elasticity using ELB

• Management & worker node separated

• Multi-zone or multi-datacenter deployment

• Multi-region

36

Single nodeA

vailab

ilit

y

Cost

LOWEST

HIGHEST

37

Primary-secondary

Primary Secondary

Availab

ilit

y

Cost

LOWEST

HIGHEST

38

Primary-secondary, multiple LB

Primary Secondary

keepalived

Availab

ilit

y

Cost

LOWEST

HIGHEST

39

Active cluster, multiple LB

Active Active

keepalived

Active

Availab

ilit

y

Cost

LOWEST

HIGHEST

40

Management & Worker Node Separation

• Proper separation of concerns - management nodes specialize in management of the setup while worker nodes specialize in serving requests to deployment artifacts

• Only management nodes are authorized to add new artifacts into the system or make configuration changes

• Worker nodes can only deploy artifacts & read configuration

• Lower memory foot in the worker nodes because the management console related OSGi bundles are not loaded

• Improved security - management nodes can be behind the internal firewall & be exposed to clients running within the organization only, while worker nodes can be exposed to external clients.

• Isolation of failures

41

Management & Worker Node SeparationA

vailab

ilit

y

Cost

LOWEST

HIGHEST

42

Regions & Zones

43

Stratos 2.0 Architecture

44

Multi-zone or multi-datacenter Deployment

Region X

Zone 1

Zone 2

Cloud Controller

Availab

ilit

y

Cost

LOWEST

HIGHEST

Multi-region deployment

45

Region X

Zone 1

Zone 2

Region Y

Zone 1

Zone 2

Availab

ilit

y

Cost

LOWEST

HIGHEST

46

Multi-IaaS Deployment

Cloud Controller

Multiple IaaS (hybrid) DeploymentA

vailab

ilit

y

Cost

LOWEST

HIGHEST

47

Private cloud (data center)

Zone 1

Zone 2

Amazon EC2

Zone 1

Zone 2

Rackspace Cloud

Zone 1

Zone 2

48

Cost of Availability

Sin

gle

Nod

e

Prim

ary-

Se

con

dary

, si

ngle

LB

Mul

ti-no

de a

ctiv

e cl

uste

r-

Sin

gle

zon

e

Mul

ti-re

gion

Prim

ary-

Se

con

dary

, w

ith m

ultip

le L

Bs

Mul

ti-zo

ne Mul

ti-Ia

aS

49

HA for the Load Balancer

• Load balancer cluster

• Keepalived

• Elastic IP address

• Round Robin DNS

50

Monitoring Servers

• Monit

• Automatically provide alerts & restart processes when monitored items (e.g. latency) fall below certain thresholds.

• New Relic

• Nagios

51

References

Information on tenant-aware load balancing http://sanjeewamalalgoda.blogspot.com/2012/03/tenant-aware-load-balancer-is-upcoming.html

http://sanjeewamalalgoda.blogspot.com/2012/05/tenant-aware-load-balancer.html

Scaling Stratoshttp://srinathsview.blogspot.com/2012/06/scaling-wso2-stratos.html

http://blog.afkham.org/2011/09/how-to-setup-wso2-elastic-load-balancer.html

http://blog.afkham.org/2011/09/wso2-load-balancer-how-it-works.html

52

Auto-scaler service deployment http://nirmalfdo.blogspot.com/2012/07/autoscaler-service-deployment.html

Auto-scaler service http://nirmalfdo.blogspot.com/2012/07/wso2-autoscaler-service-part-i.html

Automatic failover for WSO2 ELBhttp://gonesimple.org/2012/09/24/automatic-fail-over-for-wso2-elb/

Questions?

53

http://www.flickr.com/photos/oberazzi/