Powering Your Application with XAP

(Using payment processing as an example)

“Don’t call me cache”

Dotan Horovits

Director, Customer Services

Senior solutions architect

About me Dotan Horovits Director, Customer Services Senior system and solutions architect technology evangelist with interest in many things • distributed high-performance systems • big data solutions • cloud technologies • And of course: agile software methodologies • And much more… dotan@gigaspaces.com horovits.wordpress.com @horovits

Short Intro to Caching Evolution

Cache In process caching of Key->Value data

structure

Distribute Cache Partitioned cache

nodes

IMDG Partitioned system

of record

In Memory Application Platform

Collocated IMDG and Processing

Cache

Cache is good for repetitive data reads But it is limited in capacity It also doesn’t handle write-heavy scenarios

Short Intro to Caching Evolution

Cache In process caching of Key->Value data

structure

Distribute Cache Partitioned cache

nodes

IMDG Partitioned system

of record

In Memory Application Platform

Collocated IMDG and Processing

Distribute Cache

Allows you to distribute your cache over numerous machines so you get Increased Capacity But it doesn’t support write heavy scenarios It’s also Limited to query by Id What about the rest of your app? - Business logic & messaging??

Short Intro to Caching Evolution

Cache In process caching of Key->Value data

structure

Distribute Cache Partitioned cache

nodes

IMDG Partitioned system

of record

In Memory Application Platform

Collocated IMDG and Processing

IMDG solves these problems! You get increased capacity IMDG is also a System of Record with:

Query APIs

Optimized data access

Data integrity

It solves your write scalability problem

.

Short Intro to Caching Evolution

Cache In process caching of Key->Value data

structure

Distribute Cache Partitioned cache

nodes

IMDG Partitioned system

of record

In Memory Application Platform

Collocated IMDG and Processing

In Memory Application Platform XAP for end to end scaling

Its an IMDG that hosts your Business logic & has messaging services! It Provides Parallel processing of data You get linear scalability You get high availability

How does XAP work?

Here’s What a Payment Authorization Process Looks Like

Payment Authorization

Request Basic Validation User Profile Check

Merchant Profile Check

Payment Authorization

Approved

Write the Payment Object to XAP

Cash Register

Application Payment

User payment Cluster

@SpaceId()

public Long getId() {

return id;

}

@SpaceRouting

public Long getUserId() {

return userId;

}

}

@SpaceIndex public Long getCardId() {

return cardId;

}

}

The primary key of this object in the grid

The grid will use this attribute to route the object to a particular partition

Payment object

A secondary index for query optimization

Write the Payment Object to XAP

Cash Register

Application

User payment Cluster

Payment

@SpaceId

public Long getId() {

return id;

}

@SpaceRouting

public Long getUserId() {

return userId;

}

@SpaceIndex public Long getCardId() {

return cardId;

}

The primary key of this object in the grid

The grid will use this attribute to route the object to a particular partition

Payment object

A Secondary index for query optimization

Write the Payment Object to XAP

Cash Register

Application

User payment Cluster

Payment

Index

@SpaceId

public Long getId() {

return id;

}

@SpaceRouting

public Long getUserId() {

return userId;

}

@SpaceIndex public Long getCardId() {

return cardId;

}

The primary key of this object in the grid

The grid will use this attribute to route the object to a particular partition

Payment object

A Secondary index for query optimization

Let's Talk About Data Model for a Second

User Id (Routing)| User Name|…

Card Id| Card Data | UserID (Routing)…

Transaction ID | CardId | User ID (Routing)…

1 *

1 *

Data Model

Let's Get Back to the Process

Payment Validation

@EventTemplate

public SQLQuery<Payment> getNewPayment()

{

SQLQuery<Payment> query = new SQLQuery<Payment>(Payment.class," paymentStatus = ? ");

query.setParameter(1, Payment.PaymentStatus.New);

return query;

}

Queue

Payment

Validator

Payment Validation

@SpaceDataEvent

public Payment validatePayment(Payment payment)

{

…

}

private boolean basicPaymentValidation(Payment payment, User user, Card card)

{

… }

Queue

Payment

Class PaymentAuthorization{

Status status;

Boolean userCheck;

Boolean vendorCheck;

}

Validator

Payment Authorization

Basic Validation

if (basicPaymentValidation(payment,user,card)) {

gigaSpace.write(userPaymentMsg);

gigaSpace.write(vendorPaymentMsg);

gigaSpace.write(paymentAuthorization);

payment.setPaymentStatus(Payment.PaymentStatus.Processing);

} else {

payment.setPaymentStatus(Payment.PaymentStatus.SuspectedFraud);

}

Class PaymentAuthorization {

Status status;

Boolean userCheck;

Boolean vendorCheck;

}

User Validation

Vendor Validation

Payment Authorization

Queue

Queue

Queue Validator

User Validation

Class PaymentAuthorization{

Enum Status

Boolean userCheck

Boolean vendorCheck

}

Read Message

Process

Queue

Use

r C

red

it C

ard

P

aym

ent

Querying

@SpaceDataEvent

public void validateUser(UserPaymentMsg event) {

… space.readMultiple(new Card(userId))

space.readMultiple(new Payment(userId))

if (valid()) {

IdQuery<PaymentAutjorzation> idQuery = new

IdQuery<PaymentAuthorization>(PaymentAuthoirzation.class, paymentId);

space.change(idQuery, new ChangeSet().set(“userCheck", true));

} else { ... }

}

User Validation

Read Message

Process

Queue

Use

r C

red

it C

ard

P

aym

ent

Class PaymentAuthorization {

Status status;

Boolean userCheck;

Boolean vendorCheck;

}

Querying

@SpaceDataEvent

public void validateUser(UserPaymentMsg event) {

… space.readMultiple(new Card(userId))

space.readMultiple(new Payment(userId))

if (valid()) {

IdQuery<PaymentAutjorzation> idQuery = new

IdQuery<PaymentAuthorization>(PaymentAuthoirzation.class, paymentId);

space.change(idQuery, new ChangeSet().set(“userCheck", true));

} else { ... }

}

User Validation

Read Message

Process

Queue

Use

r C

red

it C

ard

P

aym

ent

@SpaceDataEvent

public void validateUser(UserPaymentMsg event) {

… space.readMultiple(new Card(userId))

space.readMultiple(new Payment(userId))

if (valid()) {

IdQuery<PaymentAutjorzation> idQuery = new

IdQuery<PaymentAuthorization>(PaymentAuthorization.class, paymentId);

space.change(idQuery, new ChangeSet().set(“userCheck", true));

} else { ... }

}

Class PaymentAuthorization {

Status status;

Boolean userCheck;

Boolean vendorCheck;

}

Querying

Vendor Validation

@SpaceDataEvent

public void validateVendor(VendorPaymentMsg vendorPaymentMsg) {

AsyncFuture<Boolean> future = vendorGigaSpace.execute(

new VendorValidationTask(vendorPaymentMsg),

vendorPaymentMsg.getMerchant());

IdQuery<PaymentAuthorization> idQuery = new

IdQuery<PaymentAuthorization>(PaymentAuthorization.class,

vendorPaymentMsg.getPaymentId());

if (future.get()) gigaSpace.change(idQuery, new ChangeSet().

set("vendorCheck", true));

else gigaSpace.change(idQuery, new ChangeSet().

set("vendorCheck”, false));

}

Read Message

Process

Queue

Task

class PaymentAuthoirzation {

Status status;

Boolean userCheck;

Boolean vendorCheck;

}

Vendor Validation

@SpaceDataEvent

public void processVendorPaymentValidation() {

AsyncFuture<Boolean> result= space.execute(

new DistributedTask(new VendorValidationTask(Payment)))

if (result.get()) {

IdQuery<PaymentAutjorzation> idQuery = new

IdQuery<PaymentAuthorization>(PaymentAuthorization.class,

paymentId);

space.change(idQuery, new ChangeSet().set(“vendorCheck”, true));

} else { ... }

}

Payment Cluster

Task

Vendor Cluster

Class PaymentAuthorization {

Status status;

Boolean userCheck;

Boolean vendorCheck;

}

Vendor Validation

public void validateVendor(VendorPaymentMsg vendorPaymentMsg,

GigaSpace gigaSpace) {

AsyncFuture<Boolean> future = vendorGigaSpace.execute(new

VendorValidationTask(vendorPaymentMsg),

vendorPaymentMsg.getMerchant());

IdQuery<PaymentAuthorization> idQuery = new

IdQuery<PaymentAuthorization>(PaymentAuthorization.class,

vendorPaymentMsg.getPaymentId());

if (future.get()) gigaSpace.change(idQuery, new

ChangeSet().set("vendorCheck", true));

else gigaSpace.change(idQuery, new ChangeSet().set("vendorCheck", false));

}

Payment Cluster Vendor Cluster

Task

Class PaymentAuthorization {

Status status;

Boolean userCheck;

Boolean vendorCheck;

}

Vendor Validation

Payment Cluster Vendor Cluster

Task

public void validateVendor(VendorPaymentMsg vendorPaymentMsg,

GigaSpace gigaSpace) {

AsyncFuture<Boolean> future = vendorGigaSpace.execute(new

VendorValidationTask(vendorPaymentMsg),

vendorPaymentMsg.getMerchant());

IdQuery<PaymentAuthorization> idQuery = new

IdQuery<PaymentAuthorization>(PaymentAuthorization.class,

vendorPaymentMsg.getPaymentId());

if (future.get()) gigaSpace.change(idQuery, new

ChangeSet().set("vendorCheck", true));

else gigaSpace.change(idQuery, new ChangeSet().set("vendorCheck", false));

}

Class PaymentAuthorization {

Status status;

Boolean userCheck;

Boolean vendorCheck;

}

Vendor Validation Check

@Override

public Boolean execute() throws Exception {

return validatePayment(vendorPaymentMsg);

}

Task

Dea

ls

Mer

chan

t

Class PaymentAuthorization {

Status status;

Boolean userCheck;

Boolean vendorCheck;

}

Querying

Vendor Validation Check

Task

Dea

ls

Mer

chan

t

@Override

public Boolean execute() throws Exception {

return validatePayment(vendorPaymentMsg);

}

Class PaymentAuthorization {

Status status;

Boolean userCheck;

Boolean vendorCheck;

}

Querying

Vendor Validation

Payment Cluster Vendor Cluster

Callback Task

public void validateVendor(VendorPaymentMsg vendorPaymentMsg,

GigaSpace gigaSpace) {

AsyncFuture<Boolean> future = vendorGigaSpace.execute(new

VendorValidationTask(vendorPaymentMsg),

vendorPaymentMsg.getMerchant());

IdQuery<PaymentAuthorization> idQuery = new

IdQuery<PaymentAuthorization>(PaymentAuthorization.class,

vendorPaymentMsg.getPaymentId());

if (future.get()) gigaSpace.change(idQuery, new

ChangeSet().set("vendorCheck", true));

else gigaSpace.change(idQuery, new ChangeSet().set("vendorCheck", false));

}

Class PaymentAuthorization {

Status status;

Boolean userCheck;

Boolean vendorCheck;

}

Yey! Payment Has Been Authorized

@EventTemplate

public PaymentAuthorization getNewPayment() {

return new PaymentAuthorization(null, true, true, PaymentAuthorizationStatus.New);

}

Queue

Payment Authorization

Yey! Payment Has Been Authorized

@SpaceDataEvent

public void completePaymentValidation(PaymentAuthorization paymentAuthorization) {

Payment payment = gigaSpace.readById(Payment.class,paymentAuthorization.getPaymentId());

payment.setPaymentStatus(Payment.PaymentStatus.Closed);

gigaSpace.write(payment);

paymentAuthorization.setPaymentAuthorizationStatus(

PaymentAuthorization.PaymentAuthorizationStatus.Done);

}

Queue

Payment Authorization

What’s next?

• Multi-site deployment?

• Persistence to DB for auditing?

• Using Scala? .NET? scripts?

• Schema evolution?

• Running our clusters on the cloud?

• Etc.

Q&A

Source code (Java) publically available on GitHub: https://github.com/Gigaspaces/fraud-detection-example

Thank you!

Dotan Horovits

dotan@gigaspaces.com

horovits.wordpress.com

@horovits