Catch Them in the Act: CEP for Realtime

Ecommerce Influence

Srinath Perera

Director Research WSO2, Apache Member

(@srinath_perera)

srinath@wso2.com

RetailWorld 7B People

- 6.5B have a phone

- 2.5B online

Global E-commerce itself was 1.5

Trillion market (215$/year for

everyone in the world).

Recommendations

One of the main retail

scenarios, can be extended

to targeted marketing and

deals

Search let us make sense of

unstructured Web

In retail, there are too many

choices, Recommendations

help us make a decision.

A Main driver of traffic (e.g.

70% amazon site is devoted

for recommendations, 70%

Netflix sales though

recommendations)

Finding Recommendation

Problem: based on customer activities (purchases, ratings,

views) etc., find out products that are of interest to him

- Done by finding similar items or similar users (common case)

There are often Millions of customers and even millions of

items

Recommendations (Collaborative Filtering)

What if you can freeze time!!

Most solutions are overnight Think how you would buy something!

Research a bit and buy, often overnight is too late.

But not all trends takes time - People change their mind - Trends move fast - React to what customer is doing (do not let

him move away)At CEP speed 400k/sec, if each event

takes a second, it takes 4 days to pass a second in real world!!

Real-time Analytics

Idea is to process data as they are received in streaming fashion

Used when we need - Very fast output - Lots of events (few 100k to millions)- Processing without storing (e.g. too much

data)Two main technologies

- Stream Processing (e.g. Strom, http://storm-project.net/ )

- Complex Event Processing (CEP)http://wso2.com/products/complex-event-processor/

Complex Event Processing

Lambda Architecture with WSO2 Products

WSO2 Big Data Platform

Demo Usecases

Operations Dashboard

Automatic Offers

Recommendations based

on personal trends

Recommendations based

on overall trends

Demo Usecase Architecture

Dashboard

Dashboard to show Realtime Analytics about the site

Example query (e.g. number of purchases or views for each

tag last minute)

from transactionStream[metadata== 'purchase'

or metadata== 'select']#window.time(1 min)

select tag, count(*) as count group by tag

update tagInfo on tagInfo.tag == transactionStream.tag;

Automatic Offers

If user have brought books worth more than 50$ within a

day, given 25% off on the next book

from SalesStream#window.time(1d)

select sum(price) as total having total > 50

insert into ..

Recommendations based on

personal trends

Look at recent history of the user, and find items he has

viewed in last day

Find a user close to his recent items, and recommend

accordingly ( a listener to Recommendations Stream will do

that)

from transactionStream_user#window.time(7d)

select session_id, user_id,

custom:genLSHID(user_id,book_id,metadata) as book_id,

"user" as metadata, "0" as discount

insert into RecommendationsStream;

Recommendations based on

overall trends

Use items sold it last 10 minutes

Find a user close to his recent items, and recommend

accordingly ( a listener to TrendsOutputStream will

do that)

from trendTransactionStream#window.time(10 min)

select session_id, user_id,

custom:genLSHID("trend",book_id,metadata) as book_id,

"trend" as metadata

insert into TrendsOutputStream;

Questions?