The Next Generation Application Server – How Event Based Processing yields scalability

The Next Generation Application Server – How Event Based Processing yields scalability

Guy KorlandGuy Korland

R&D Team LeaderR&D Team Leader

2

Agenda

• Preview

• The Problem

• The Solution

• Event Containers

• Example

• Benchmarks

• Customer Use Cases

• Summary Q&A

3

About me…

• Core Team Leader – GigaSpaces since 2005

• MSc – Technion (Prof. Roy Friedman)

• PhD candidate – Tel Aviv University (Prof. Nir Shavit)

• Lead Deuce STM – (www.deucestm.org)

Java Software Transactional Memory

http://www.deucestm.org/

4

GigaSpaces XAP – Designed For:

Performance

Scalability

Latency

5

GigaSpaces Evolution

Single space

Load Balance

Partition & Replication

SLA container

Event Container

NG Application

Server

PaaS Cloud

2000 2003 2005 2006 2007 2008 2009

6

Not going to talk about…

• Jini (Java SOA)

• Data Grid implementation.

• Map-Reduce.

• JDBC/JMS/JPA.

• Cloud computing.

• Batch processing.

• Mule ESB.

• WAN vs LAN

• Different languages interoperability.

• TupleSpace model extension.

• JDK improvements (RMI, Reflection, Serialization, Classloading…)

7

Agenda

• Preview

• The Problem

• The Solution


• Example

• Benchmarks


• Summary Q&A

88

Today’s Reality – Tier Based ArchitectureSeparate technology implementation

Bottlenecks in all areas where state is stored, architecture can’t scale linearly!

Separate technology implementation

Separate technology implementation

bo

ttle

nec

ks

bo

ttle

nec

ks

9

AuctionOwner

Traditional Architecture - path to complexity… (marktplaats.nl)

Auction Service

Bid Service

Trade Service

Place bid

Info Service

Timer Service

Auction

Service

Bid

Service

Trade

Service

Info

Service

Timer

Service

B T I

A

B

T

I

T

TT

AA BB TT II

Bidder

Validate

Result

ProcessBid

Bid Accepted

Bid Result

Process Trade

Get Bid Result

10

Traditional Architecture - path to complexity…

10

Business tier

Back-up

Back-up

Redundancy doubles network traffic

Bottlenecks are created

Latency is increased

Separate failover strategy and implementation for each tier

Bidder

AuctionOwner

Auction Service

Bid Service

Trade Service

Info Service

Timer Service

A

B

T

I

T

AA BB TT II

11

BB

Do you see the Problem?

11

Business tierScalability is not linear

Scalability management nightmare

Back-upBack-up

Back-upBack-up

AA BB TT

II

Bidder

AuctionOwner

12

There is a huge gap between peak and average loads

0

100,000,000

200,000,000

300,000,000

400,000,000

500,000,000

600,000,000

700,000,000

800,000,000

900,000,000

1,000,000,000

1,100,000,000

1,200,000,000

1,300,000,000

J-04 M-04 M-04 J-04 S-04 N-04 J-05 M-05 M-05 J-05 S-05 N-05 J-06 M-06 M-06 J-06 S-06 N-06 J-07 M-07 M-07 J-07 S-07

http://www.nyse.com/

13

Bottlenecks, Performance, Scalability and High availability headaches

Bad PublicityRevenue LossCustomer DissatisfactionRegulatory Penalties

15

TBA – Summary

• Historically the following has been done…

– Tune, tune and tune configuration and code

• Once a bottleneck has been resolved, the next one glooms

– Hardware over provision

• To make sure that at peak times the response times were still acceptable

– Hardware upgrades

• To get rid of bottlenecks, whose origin was impossible to track down

– Alternative patterns

• Avoiding 2-phase-commit, using patterns like ‘compensating transactions’

• Using Active/Passive failover, to make the response times faster, risking and

in fact accepting potential data-loss

• Partition the database, but not for size-reasons

16

Agenda

• Preview

• The Problem

• The Solution


• Example

• Benchmarks


• Summary Q&A

17

Event Containers

18

Based on JavaSpaces

C++

19

The Master Worker Pattern

20

GigaSpaces - based on Shared Transactional Memory

Read

Wri

te

• Write – writes a data object

• Read – reads a copy of a data object

• Take – reads a data object and deletes it

• Notify – generates an event on data updates

Not

ify

Write

Take

Write + Read Data Caching

Write + Notify Messaging - Pub/Sub

Write + Take Master Worker

21

Event Containers

22

AB T I

Step 1 – Create a Processing Unit

22

Business tier

Processing Unit

Single model for design, deployment and management

No integration effort

Manage data in memory

Collapse the tiers

Collocate the services

Auction Service

Bid Service

Trade Service

Info Service

Timer Service

A

B

T

I

T

Bidder

AuctionOwner

23

AB T I

Step 2 – Async Persistency

23

Processing Unit

Validate

Process Bid

Process Trade

Process Results

Place Bid

Get Bid Results

Persist for Compliance & Reporting purposes:

- Storing State- Register Orders- etc.

Collocation of data, messaging and services in memory:

Minimum Latency (no network hops)

Maximum Throughput

Auction Service

Bid Service

Trade Service

Info Service

Timer Service

A

B

T

I

T

Bidder

AuctionOwner

24

AB T I

Step 3 – Resiliency

Processing Unit

Single, built-in failover/redundancy investment strategy

Fewer points of failure

Automated SLA driven failover/redundancy mechanism

Continuous High Availability

SLA Driven Container

Backup

AB T I

25

Processing Unit

Step 3 – Resiliency

Automated SLA driven failover/redundancy mechanism

Continuous Availability

Self Healing Capability


Backup

Single, built-in failover/redundancy investment strategy

Fewer integration points mean fewer chances for failure

BackupPrimary

26

Step 4 – Scale

26

Processing Unit

Write Once Scale Anywhere:

Linear scalability

Single monitoring and management engine

Automated, SLA-Driven deployment and management

Scaling policy, System requirements, Space cluster topology

Backup

AB T I

AB T I

Backup

AB T I

AB T I

27

Event Containers

28

Step 5 – Auto Scale Out

29

Processing Unit – Scalability Unit

Single Processing Unit Processing Unit - Scaled

Involves Config Change

No code changes!

30

Processing Unit – High-Availability Unit

Sync Replication

Primary - Processing UnitBusiness logic – Active mode

Backup - Processing UnitBusiness logic – Standby mode

31

Database Integration - Async persistency

Sync Replication

Primary - Processing UnitBusiness logic – Active mode

Backup - Processing UnitBusiness logic – Standby mode

Mirror Process

ORM

Initial Load

Async Replication

Async Replication

32

XAP = Enterprise Grade Middleware

• Scale-out application server

End 2 End scale-out middleware for: Web, Data, Messaging, Business logic

Space Based Architecture – designed for scaling stateful applications In-memory

• Proven performance, Scalability, Low latency, Reliability

• SLA Driven

• Unique database scaling solution that fits cloud environment

In Memory Data Grid

O/R mapping support

• Support major Enterprise languages

Java, .Net, C++

33

Agenda

• Preview

• The Problem

• The Solution


• Example

• Benchmarks


• Summary Q&A

34

Built-in Event Containers

• Polling Container• Notify Container

Processing Unit

PollingEvent Container

ServiceBean

NotifyEvent Container

ServiceBean

Messaging

Take Write Notify

Data

35

Polling Container

• Used for point-to-point messaging• Container polls the Space

for events• Comparable with the

way Ajax works

Processing Unit

PollingEvent Container

ServiceBean

Take Write

36

Notify Container

• Used for publish-subscribe messaging

• Space notifies the container Processing Unit

NotifyEvent Container

ServiceBean

Notify

37

Typical Application

38

Service Grid Summary

Powerful Universal ContainerJava/Net/C++

Distributed

Fault Tolerant

Object based

Transactional

Publish/Subscribe

39

Agenda

• Preview

• The Problem

• The Solution


• Example

• Benchmarks


• Summary Q&A

40

Event Containers

41

@SpaceClass(fifo=true)public class Data { … @SpaceId(autoGenerate = true) public String getId() { return id; } public String setId(String id) {

this.id = id; }

public void setProcessed(boolean processed) { this.processed = processed; } public boolean isProcessed(boolean processed) { return this.processed; }}

The POJO Based Data Domain Model

SpaceClass indicate that this is a SpaceEntry – SpaceClass includes classlevel attributes such as FIFO,Persistent…

SpaceId used to define the key for that entry.

42

@SpaceDataEvent to be called when an event is triggered.

public class DataProcessor{

@SpaceDataEvent public Data processData(Data data){

… data.setProcessed(true);

//updates the space return data;

}}

Data Processor Service Bean

Updates the data in the Space.

43

<bean id="dataProcessor“ class="com.gigaspaces.pu.example1.processor.DataProcessor" />

<os-events:polling-container id="dataProcessorPollingEventContainer" giga-space="gigaSpace">

<os-events:tx-support tx-manager="transactionManager"/>

<os-core:template>

<bean class="org.openspaces.example.data.common.Data">

<property name="processed" value="false"/>

</bean>

</os-core:template>

<os-events:listener>

<os-events:annotation-adapter>

<os-events:delegate ref="dataProcessor"/>

</os-events:annotation-adapter>

</os-events:listener>

</os-events:polling-container>

Wiring Order Processor Service Bean through Spring

The event Template

The event Listener

44

public class DataFeeder {

public void feed(){

Data data = new Data(counter++); data.setProcessed(false);

//feed data gigaSpace.write(data); }}

Data Feeder

Feed Data

45

Remoting – Taking one step forward

Event

46

Remoting – Taking one step forward

Reduce

47

public interface IDataProcessor {

// Process a given Data objectData processData(Data data);

}

Remoting – IDataProcessor Service API

48

@RemotingService public class DataProcessor implements IDataProcessor {

public Data processData(Data data) { …

data.setProcessed(true); return data;

} }

Remoting - DataProcessor Service

49

public class DataFeeder { private IDataProcessor dataProcessor; public void setDataProcessor(…) {

this.dataProcessor = dataProcessor; }

public Data feed(){ Data data = new Data(counter++);

// remoting call return dataProcessor.process(data) }}

Remoting - Order Feeder

50

Summary

51

Agenda

• Preview

• The Problem

• The Solution


• Example

• Benchmarks


• Summary Q&A

52

Scale up Throughput Benchmark – Physical Deployment Topology

Remote (multiple machines , multiple processes)

white box Client

X4450GigaSpaces

4 spaces , one per GSC

X4450GigaSpaces

4 spaces , one per GSC

Switched Ethernet LAN

Embedded (one machine , one process)

X4450 Client

GigaSpaces8 spaces

53

x4450 - stac-sun-1 - Embedded Space - TP vs. Multiple Threads - 8 Partitions

0

200000

400000

600000

800000

1000000

1200000

1400000

1600000

1800000

2000000

1 2 3 4 6 8 10 12 16 20 30

Client Threads

TP

(o

pera

tio

ns/s

ec)

Write TP

Read TP

Take TP

Scale up Throughput Benchmark – Embedded mode

1.8 Million read sec!

1.1 Million write/take

sec!

54

Scale up Throughput Benchmark – Remote mode

x4450 - stac-sun-3 - Remote Space - TP vs. Multiple Threads - 4 partitions

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

100000

1 2 3 4 5 8 12 16 20 24 26 30 34 38 42 46 50 54 58

Client Threads

TP

(o

pera

tio

ns/s

ec) Write TP

Read TP

Take TP 90,00 read sec!

45,00 write/take sec!

55

Agenda

• Preview

• The Problem

• The Solution


• Example

• Benchmarks


• Summary Q&A

56

Event Containers

57

Web Container Grid

58

Web application – Pet Clinic

59

Classic Architecture – Step 1- Request Submission

Web PU

Proxy

Task

Web PU

Proxy

Task

Processing UnitProcessing UnitProcessing UnitProcessing Unit

ServiceBean

ServiceBean

ServiceBean

Replication

ServiceBean

Replication

Primary 1 Backup 1 Primary 2 Backup 2

1. User Click Submits request

TT

Apache Load-Balancer

2. Route Request

Web PU

Proxy

Task

3. invocation 3. invocation

Load

Async Mirroring

Get request and invoke

Service

Data Grid

60

Classic Architecture – Step 2- Retrieve Results

Web PU

Proxy

Task

Web PU

Proxy

Task

Processing UnitProcessing UnitProcessing UnitProcessing Unit

ServiceBean

ServiceBean

ServiceBean

Replication

ServiceBean

Replication

Primary 1 Backup 1 Primary 2 Backup 2

3. User getting Page

TT

Apache Load-Balancer

2. Route Request

Web PU

Proxy

1. Result returned

AggregatedResult

Result

1. Result returned

Result

Reducer

Load

Async Mirroring

Page Generation

Data Grid

61

Web Application Benchmark Results - Capacity

Web Benchmark (pet clinic) - Latency vs. Users

010203040506070

1 2 3 4 5 10 15 20 40 50 60 70 80 90 100

Users

Lat

ency

(ms)

1 Server

2 Servers

3 Servers

62

Web Application Benchmark Results - Capacity

Web Benchmark (pet clinic) - Latency vs. Users

0

500

1000

1500

2000

2500

3000

3500

50 100

150

200

250

500

750

1000

2000

2500

3000

3500

4000

4500

5000

Users

Lat

ency 1 Server

2 Servers

3 Servers

63

Game Server

64

Query Notify

Intercepts updateevents

Game Servers

Table Feeder

Loading Game Tables into the partitioned spaces

Randomly updates the game tables

Publisher (lobby)Game Table Directory

Game Table search

Player search

Partitioned Space

Pub/Sub messaging

Scaling out

Publisher (II)

GameTable

Space Based Architecture – Game Server

65

Publisher Servers

Partitioned SpaceGameTable



Game ServersGigaSpaces Service Grid

Notify / Query

Notify / Query

Uploading 30,000 players for 6000 tables

Randomly updates game tables

Java runtime


Physical backup



Running continues query per user

Space Based Architecture – Game Server

66

Dynamic repartitioning and load sharing I

Partitioned Space

Partitioned Space

Partitioned Space


Indexed Notify / Query template

Notify / Query template

67

Dynamic repartitioning and load sharing II

Partitioned Space

Partitioned Space

Partitioned Space

SLA Driven Container SLA Driven Container

68

Partitioned Space

Backup Space


• 6000 tables

• 30,000 players


Partitioned Space

Backup Space

• 4000 tables

• 20,000 players


Partitioned Space

Backup Space

• 2000 tables

• 10,000 players

Throughput: ~6K/secThroughput: ~12K/secThroughput: ~18K/sec

Scaling

69

Agenda

• Preview

• The Problem

• The Solution


• Example

• Benchmarks


• Summary Q&A

Thank You!

Q&A

Appendix

72

SLA:• Failover policy• Scaling policy• Ststem requirements• Space cluster topology

PU Services beans definition

SLA Driven Deployment

73

Fail-OverFailure

Continuous High Availability

74

VM 1 ,2GGSCGSC

VM 3 , 2GGSCGSC

Dynamic Partitioning = Dynamic Capacity Growth

VM 2 ,2GGSCGSC

Max Capacity=2GMax Capacity=4GMax Capacity=6G

E FPartition 1Partition 1

A BPartition 2Partition 2

C D

Partition 3Partition 3

In some point VM 1 free memory is below 20 % - it about the time to increase the capacity – lets move Partitions 1 to another GSC and

recover the data from the running backup!

Later .. Partition 2 needs to move… After the move ,

data is recovered from the backup

VM 5 , 4GGSCGSCVM 4 ,4GGSCGSCA B


E FPartition 1Partition 1

C D


P - PrimaryP - Primary

B - BackupB - Backup

PP

PP

PP

BB

BB BB

75

Executors

76

Task Executors – Task Execution

Executing a task is done using the execute method

AsyncFuture<Integer>future=gigaSpace.execute(newMyTask(2)

);intresult=future.get();

Processing UnitClient

TaskProxy

Task

Result

1 2

4

3

77

Task Executors – Task Routing

Routing a task can be done in three ways1. Using the task itself2. Passing a POJO to the execute method3. Specifying a routing-parameter in the executemethod

Client

Processing Unit Processing Unit Processing Unit

Task

ClientProxy

Result


Task

Proxy

Task

78

Task Executors – DistributedTask Execution

Executing a distributed task is done using the execute method

AsyncFuture<Integer>future=gigaSpace.execute(newMyDistTask()

);intresult=future.get();

Client


Task Task Task

ClientProxyAggregated

Result


Task Task Task

Proxy

ResultResult

Result

Task

Reducer

79

Task Executors – DistributedTask Routing

Routing a distributed task can be done1. In the same ways as with the plain Task interface2. By broadcasting3. Specifying a number of routing-parameters in the executemethod

Client


Task Task

ClientProxyAggregated

Result


Task Task

Proxy

ResultResult

Task

Reducer

80

Service Executors

81

Service Executors

82

IMDG

Operations

83

IMDG Basic Operations

Application

Space

Take

Application

Space

Read

Application

Space

WriteMultipleApplication

Space

Write

Application

Space

ReadMultiple

Application

Space

TakeMultiple

Application

Space

ExecuteApplication

Space

Notify

84

IMDG Access – Space Operations – Write Operation

write-operation writes a new object to a space

Instantiate an object

Set fields as necessary

Write the object to the space

Auctionauction=newAuction();auction.setType("Bicycle");gigaSpace.write(auction);

Application

Space

Write

85

IMDG Access – Space Operations – Read Operation

read-operation reads an object from a space

A copy of the object is returned

The original copy remains in the space

Build a template/query (more on this later)

Read a matching object from the space

Auctiontemplate=newAuction();AuctionreturnedAuction=gigaSpace.read(template);

Application

Space

Read

86

Object SQL Query Support

Supported Options and QueriesOpeations: =, <>, <,>, >=, <=, [NOT] like, is [NOT] null, IN.

GROUP BY – performs DISTINCT on the POJO properties

Order By (ASC | DESC)

SQLQuery rquery = new SQLQuery(MyPojo.class,"firstName rlike '(a|c).*' or ago > 0 and lastName rlike '(d|k).*'");

Object[] result = space.readMultiple(rquery);

Dynamic Query SupportSQLQuery query = new SQLQuery(MyClass.class,“firstName = ? or lastName = ? and ago>?");

query.setParameters(“david”,”lee”,50);

Supported Options via JDBC APICOUNT, MAX, MIN, SUM, AVG , DISTINCT , Blob and Clob , rownum , sysdate , Table aliases

Join with 2 tables

Non Supported

HAVING, VIEW, TRIGGERS, EXISTS, BETWEEN, NOT, CREATE USER, GRANT, REVOKE, SET PASSWORD, CONNECT USER, ON.

NOT NULL, IDENTITY, UNIQUE, PRIMARY KEY, Foreign Key/REFERENCES, NO ACTION, CASCADE, SET NULL, SET DEFAULT, CHECK.

Union, Minus, Union All.

STDEV, STDEVP, VAR, VARP, FIRST, LAST.

# LEFT , RIGHT [INNER] or [OUTER] JOIN

87

IMDG Access – Space Operations – Take Operation

take-operation takes an object from a space

The matched object is removed from the space

Build a template/query (more on this later)

Take a matching object from the space

Auctiontemplate=newAuction();AuctionremovedAuction=gigaSpace.take(template);

Application

Space

Take

88

IMDG Access – Space Operations – Update Operation

update is equivalent to performing take and write

Executed in a single atomic call

AuctionItemitem=newAuctionItem();item.setType("Bicycle");gigaSpace.write(item);item=gigaSpace.read(item);item.setType("Motorbike");ObjectreturnedObject=space.update(item,null,Lease.Forever,2000L,

UpdateModifiers.UPDATE_OR_WRITE);

Application

Space

Update

89

IMDG Access – Space Operations – Batch API

Apart from the single methods GigaSpaces also provides

batch methods

The methods are:

writeMultiple: writes multiple objects

readMultiple: reads multiple objects

updateMultiple: updates multiple objects

takeMultiple: reads multiple objects and deletes them

Notes:

Performance of the batch operations is generally higher

Requires one call to the space

Can be used with Template matching or SQLQuery

90


• writeMultiple writes the specified objects to the space.

Auction[]auctions=newAuction[]{newAuction(10),newAuction(20)

};

auctions=gigaSpace.writeMultiple(auctions,100);

Application

Space

WriteMultiple

91


• readMultiple reads all the objects matching the specified template from the space.

Auctionauction=newAuction();Auction[]auctions=gigaSpace.readMultiple(auction,100);

Application

Space

ReadMultiple

92


• takeMultiple takes all the objects matching the specified template from the space.

Auctionauction=newAuction();Auction[]auctions=gigaSpace.takeMultiple(auction,100);

Application

Space

TakeMultiple

93


• updateMultiple updates a group of specified objects.

Auction[]auctions=newAuction[]{newAuction(10),newAuction(20)

};

auctions=gigaSpace.updateMultiple(auctions,100);

Application

Space

UpdateMultiple

94

IMDG Summary

Powerful shared memory Service

Distributed

Fault Tolerant

Object based

Single and Batch Operations

Transactional

The Next Generation Application Server – How Event Based Processing yields scalability

Technology

Transcript of The Next Generation Application Server – How Event Based Processing yields scalability