Advanced Streaming Analytics - The future of Data Analysis
-
Upload
barac -
Category
Data & Analytics
-
view
109 -
download
2
Transcript of Advanced Streaming Analytics - The future of Data Analysis
2
Advanced Streaming AnalyticsWe analyse huge amount of data in real 3me so that a firm can detect cri3cal problems and react to changing business condi3ons in seconds rather than days with the current Data Analysis systems
3
Critical Business Activities are not in Real time
Problems
Very Expensive & Complex to implement
Analyse Only Structured Data
Loosing a LOT of Money
Uses Simple Models to analyze
Data Sources Data Warehouse
Data MartAnalysis
ETL
ETL
Tools
Time is Money !!
Analyzing Data can take Days using current systems
4
Help Banking & Telco Save MillionsWhat We Do
Using our architecture, our proprietary and powerful technology and algorithms, We help companies make real time decisions for critical business issues such as fraud detection and Cyber security rather than days it often takes with existing solutions.
Improve Sales and opera7ons
Reduce Fraud by
60%500 Million Data proceed in 1
second
Real 7me Cyber Security AFack
Detec7on
5Architecture
Client Web Navigator /Mobile
Any type of Logs (CDR, system logs, etc.)
Informa7on Systems (Core, CRM, etc..-‐
Real Tim
e Data C
ollection & aggregation
barac
Real Time Data Analysis
Real Time Informa7on &
Analysis
Dashboards / Repor7ng
Integra7on with Data
Visualiza7on
Analy7cal Data base -‐ Big Data
Enhance Models Daily
Enrich Data
6
Description• Every System or machine has a LOG, we read the logs directly in the servers using our streamers and
we send the important informa3on to a collector
• We extract valuable analysis and business informa3on from technical logs in this way we don’t interact
with the Informa3on system or don’t go through all the ETL
• We transform logs into a structured data sources to choose the important informa3on send
• We send the data through our streamers to a collector that collects all the informa3on from million of
streamers
• We enrich the data from the different informa3on system to get more informa3on
• We collect the data and we apply advanced algorithms to the data to get instant informa3on : We use
Python
7Our Streamers
Collector
Real Time Analysis (Python)
Our Streamers are very light, are installed in
different type of servers to read the
informa3on directly from all type of sources,
parse it, encrypt it and send it to a collector
Storage of Analysis
8Technology
• Ultra lightweight and smart streamers based on Java language to send selected and
encrypted informa3on to a data collector
• We can send up to 100 Million events per second from millions of machine
• Easy to scale, as you only need to add the streamers to the new server
• We apply machine learning algorithms before storing the data to get instant
informa3on and analysis
• We enhance our analysis models daily to adapt to new behaviors and to improve the
models
9Our Strength
CostDrama3cally decrease your storage cost by using a Hadoop and No-‐SQL based plaSorm and have access in real 3me to all your data
Multiple SourcesAllow to query data in real 3me from mul3ple sources (Hadoop HDFS, Cassandra, SQL, Oracle, etc…)
Easy to implementOur ready to use plaSorm is very easy to implement and can save 70% of 3me and cost compared to open source solu3ons
SpeedWe analyze data on the fly by using machine learning while data are precessed and not a^er being stored which change how the data are analyzed
Streaming DataStream different types of data into the system in real 3me, analyse and then use them to react to your environment and your users
Machine LearningWe apply powerful machine learning algorithms to analyse data and we enhance our algorithms daily
10Changing AnalyticsTradi7onnal Barac
Storage Cost High Very Low
Analy7cs Offline Real-‐3me
Machine Learning Sample Advanced
Using Hadoop Yes Yes
Data Loading Speed Low Very High
Streaming All File Line by line in Real 3me
Data Variety Only Structured Non Structured & Structured
Volume Terabyte Petabyte/Exabyte
Velocity Batch & ETL Real-‐3me
Complex Query Hours/Days Minutes
12
Benchmark of BaracPerformances
BaracHadoop
Sec
Test performed on 6 nodes on two servers CPU(s): 16, Thread(s) per core: 2, RAM: 16 GB
14Big Data in a New Area
Different Use Cases
Fraud DetectionDetect Fraud in
Real3me in Telecom, financial by analyzing
diverse data
Churn ReductionReduce churn by
integra3ng mul3ple datasource and real3me analysis
Network Analysis & Monetization
Analyse your Network in Real3me to detect any
problem and mone3ze your data to create new revenu
Cyber SecurityDetect Security breaches and
ahacks in real3me
360° ViewReal3me 360° View of your customer from several
sources in real 3me
IoTProcess and store all the data collected
from your IoT network
CLVCustomer Value
Management with Personalized
Marke3ng & offers base on diverse
sources
Predictive MaintenanceUse data analysis to predict maintenance and detect failure in real3me before it
happens
15
Analyze what the customers watch in real time and Monetize the data Customer View Analytics
Solution
The telecom operator is using our light streamers in his boxes to
send the data about what people are watching in real time, enrich the data, anonymize the data and sell them to media companies so they can adapt their advertising in real time
This is a game changer in advertising worldwide
Challenge• In a very competitive environment, the telecom operator want to have new sources of revenues
• Being a data provider and the center of data worldwide • Not share any personal information about their customers
Example
A media company will have the information in real time that
the people that are watching now are 60% composed of women between 30-‐40 and will then adapt the advertising to that category to get more conversion and interest
16
Detect early signs of churn in Real timeReal Time Churn
Solution
The telecom operator is using our light streamers in their internet
boxes so we can send important information in real time to a collector and apply machine learning algorithm to detect any early pattern of client to churn.
We enhance our models daily to get better predictions
Challenge• In a very competitive environment, the telecom operator want to detect early signs of churn to take actions
• The are able to get the analysis but after 24/72h (time for B.I to process) and that’s too late for them
Example
The Telecom operator is now able to analyse in real time if a customer closed his box 4 times in less than 2 minutes which is an early sign of client churn. The information will get to the
customer analysis team to take immediate action, correct the problem and keep the customer happy.
17
Analyse your sales and operations in Real timeReal Time Sales Analysis
Solution
The hotel group is using our streamers to read the informations
about booking and operations directly from the logs and send those informations to a central server for analysis. By using this approach, we don’t connect to any information system and we
read the data directly from the logs of the servers. We transform technical information into a critical business
information for the hotel central team
Challenge• A Hotel group was growing by purchasing different hotels in Eastern Europe, Africa, Middle East and Asia, they have now different Information systems
• Every region has it’s own reporting and Information system, that was nearly impossible to get the sales and operations of each region
Example
They are now able to have real time sales analysis by country, by region and by zone to take important sales and marketing decisions. They were also able to detect a lot of problems in
their regional B.I tools