Advanced Streaming Analytics - The future of Data Analysis

18
Unleash Your Data Potential

Transcript of Advanced Streaming Analytics - The future of Data Analysis

Unleash Your Data Potential

2

Advanced Streaming AnalyticsWe  analyse  huge  amount  of  data  in  real  3me  so  that  a  firm  can  detect  cri3cal  problems  and  react  to  changing  business  condi3ons  in  seconds  rather  than  days  with  the  current  Data  Analysis  systems

3

Critical Business Activities are not in Real time

Problems

Very Expensive & Complex to implement

Analyse Only Structured Data

Loosing a LOT of Money

Uses Simple Models to analyze

Data Sources Data Warehouse

Data MartAnalysis

ETL

ETL

Tools

Time  is  Money  !!

Analyzing Data can take Days using current systems

4

Help Banking & Telco Save MillionsWhat We Do

Using our architecture, our proprietary and powerful technology and algorithms, We help companies make real time decisions for critical business issues such as fraud detection and Cyber security rather than days it often takes with existing solutions.

Improve  Sales  and  opera7ons

Reduce  Fraud  by  

60%500  Million  Data  proceed  in  1  

second  

Real  7me  Cyber  Security  AFack  

Detec7on

5Architecture

Client  Web  Navigator  /Mobile

Any  type  of  Logs  (CDR,  system  logs,  etc.)

Informa7on  Systems  (Core,  CRM,  etc..-­‐

Real Tim

e Data C

ollection & aggregation

barac

Real  Time  Data  Analysis

Real  Time  Informa7on  &  

Analysis

Dashboards  /  Repor7ng

Integra7on  with  Data  

Visualiza7on

Analy7cal  Data  base  -­‐    Big  Data

Enhance  Models  Daily

Enrich  Data  

6

Description• Every  System  or  machine  has  a  LOG,  we  read  the  logs  directly  in  the  servers  using  our  streamers  and  

we  send  the  important  informa3on  to  a  collector  

• We  extract  valuable  analysis  and  business  informa3on  from  technical  logs  in  this  way  we  don’t  interact  

with  the  Informa3on  system  or  don’t  go  through  all  the  ETL  

• We  transform  logs  into  a  structured  data  sources  to  choose  the  important  informa3on  send  

• We  send  the  data  through  our  streamers  to  a  collector  that  collects  all  the  informa3on  from  million  of  

streamers  

• We  enrich  the  data  from  the  different  informa3on  system  to  get  more  informa3on  

• We  collect  the  data  and  we  apply  advanced  algorithms  to  the  data  to  get  instant  informa3on  :  We  use  

Python  

7Our Streamers

Collector

Real  Time  Analysis  (Python)

Our  Streamers  are  very  light,  are  installed  in  

different  type  of  servers  to  read  the  

informa3on  directly  from  all  type  of  sources,  

parse  it,  encrypt  it  and  send  it  to  a  collector

Storage  of  Analysis

8Technology

• Ultra  lightweight  and  smart  streamers  based  on  Java  language  to  send  selected  and  

encrypted  informa3on  to  a  data  collector  

• We  can  send  up  to  100  Million  events  per  second  from  millions  of  machine  

• Easy  to  scale,  as  you  only  need  to  add  the  streamers  to  the  new  server  

• We  apply  machine  learning  algorithms  before  storing  the  data  to  get  instant  

informa3on  and  analysis  

• We  enhance  our  analysis  models  daily  to  adapt  to  new  behaviors  and  to  improve  the  

models

9Our Strength

CostDrama3cally  decrease  your  storage  cost  by  using  a  Hadoop  and  No-­‐SQL  based  plaSorm  and  have  access  in  real  3me  to  all  your  data

Multiple SourcesAllow  to  query  data  in  real  3me  from  mul3ple  sources  (Hadoop  HDFS,  Cassandra,  SQL,  Oracle,  etc…)  

Easy to implementOur  ready  to  use  plaSorm  is  very  easy  to  implement  and  can  save  70%  of  3me  and  cost  compared  to  open  source  solu3ons

SpeedWe  analyze  data  on  the  fly  by  using  machine  learning  while  data  are  precessed  and  not  a^er  being  stored  which  change  how  the  data  are  analyzed

Streaming DataStream  different  types  of  data  into  the  system  in  real  3me,  analyse  and  then  use  them  to  react  to  your  environment  and  your  users

Machine LearningWe  apply  powerful  machine  learning  algorithms  to  analyse  data  and  we  enhance  our  algorithms  daily

10Changing AnalyticsTradi7onnal Barac

Storage  Cost High Very  Low

Analy7cs Offline Real-­‐3me

Machine  Learning Sample Advanced

Using  Hadoop Yes Yes

Data  Loading  Speed Low   Very  High

Streaming All  File Line  by  line  in  Real  3me

Data  Variety Only  Structured   Non  Structured  &  Structured

Volume Terabyte   Petabyte/Exabyte

Velocity Batch  &  ETL Real-­‐3me

Complex  Query Hours/Days   Minutes

11

Solution (1/2)

12

Benchmark of BaracPerformances

BaracHadoop

Sec

Test  performed  on  6  nodes  on  two  servers  CPU(s):  16,  Thread(s)  per  core:  2,  RAM:  16  GB

Use  Cases

14Big Data in a New Area

Different Use Cases

Fraud DetectionDetect  Fraud  in  

Real3me  in  Telecom,  financial  by  analyzing  

diverse  data

Churn ReductionReduce  churn  by  

integra3ng  mul3ple  datasource  and  real3me  analysis

Network Analysis & Monetization

Analyse  your  Network  in  Real3me  to  detect  any  

problem  and  mone3ze  your  data  to  create  new  revenu

Cyber SecurityDetect  Security  breaches  and  

ahacks  in  real3me

360° ViewReal3me  360°  View  of  your  customer  from  several  

sources  in  real  3me

IoTProcess  and  store  all  the  data  collected  

from  your  IoT  network

CLVCustomer  Value  

Management  with  Personalized  

Marke3ng  &  offers  base  on  diverse  

sources

Predictive MaintenanceUse  data  analysis  to  predict  maintenance  and  detect  failure  in  real3me    before  it  

happens

15

Analyze what the customers watch in real time and Monetize the data Customer View Analytics

Solution

The  telecom  operator  is  using  our  light  streamers  in  his  boxes  to  

send  the  data  about  what  people  are  watching  in  real  time,  enrich  the  data,  anonymize  the  data  and  sell  them  to  media  companies  so  they  can  adapt  their  advertising  in  real  time  

This  is  a  game  changer  in  advertising  worldwide

Challenge•  In  a  very  competitive  environment,  the  telecom  operator  want  to  have  new  sources  of  revenues    

•  Being  a  data  provider  and  the  center  of  data  worldwide  •  Not  share  any  personal  information  about  their  customers

Example

 A  media  company  will  have  the  information  in  real  time  that  

the  people  that  are  watching  now  are  60%  composed  of  women  between  30-­‐40  and  will  then  adapt  the  advertising  to  that  category  to  get  more  conversion  and  interest  

16

Detect early signs of churn in Real timeReal Time Churn

Solution

The  telecom  operator  is  using  our  light  streamers  in  their  internet  

boxes  so  we  can  send  important  information  in  real  time  to  a  collector  and  apply  machine  learning  algorithm  to  detect  any  early  pattern  of  client  to  churn.  

We  enhance  our  models  daily  to  get  better  predictions

Challenge•  In  a  very  competitive  environment,  the  telecom  operator  want  to  detect  early  signs  of  churn  to  take  actions  

•  The  are  able  to  get  the  analysis  but  after  24/72h  (time  for  B.I  to  process)  and  that’s  too  late  for  them

Example

The  Telecom  operator  is  now  able  to  analyse  in  real  time  if  a  customer  closed  his  box  4  times  in  less  than  2  minutes  which  is  an  early  sign  of  client  churn.  The  information  will  get  to  the  

customer  analysis  team  to  take  immediate  action,  correct  the  problem  and  keep  the  customer  happy.

17

Analyse your sales and operations in Real timeReal Time Sales Analysis

Solution

The  hotel  group  is  using  our  streamers  to  read  the  informations  

about  booking  and  operations  directly  from  the  logs  and  send  those  informations  to  a  central  server  for  analysis.  By  using  this  approach,  we  don’t  connect  to  any  information  system  and  we  

read  the  data  directly  from  the  logs  of  the  servers.  We  transform  technical  information  into  a  critical  business  

information  for  the  hotel  central  team

Challenge•  A  Hotel  group  was  growing  by  purchasing  different  hotels  in  Eastern  Europe,  Africa,  Middle  East  and  Asia,  they  have  now  different  Information  systems  

•  Every  region  has  it’s  own  reporting  and  Information  system,  that  was  nearly  impossible  to  get  the  sales  and  operations  of  each  region

Example

They  are  now  able  to  have  real  time  sales  analysis  by  country,  by  region  and  by  zone  to  take  important  sales  and  marketing  decisions.  They  were  also  able  to  detect  a  lot  of  problems  in  

their  regional  B.I  tools

18

[email protected] US

Unleash Your Data Potential