Embedded Analytics: The Next Mega-Wave of Innovation

[email protected]

Twitter Tag: #briefr

!  Reveal the essential characteristics of enterprise software, good and bad

!  Provide a forum for detailed analysis of today’s innovative technologies

!  Give vendors a chance to explain their product to savvy analysts

!  Allow audience members to pose serious questions... and get answers!


!   June: Intelligence

!   July: Disruption

!   August: Analytics

!   September: Integration

!   October: Database

!   November: Cloud



!   The last ten or so years have seen a massive influx of business intelligence tools: reporting, analytics, data mining, online analytical processing, querying, etc.

!   BI technologies are designed to let organizations take all their capabilities and convert them into knowledge, ultimately getting the the right information to the right people at the right time.

!   Vendors face the challenge of providing organizations with tools robust enough to get at their data and provide the right actionable insight.


John Myers joined Enterprise Management Associates in 2011 as senior analyst of the BI practice area, where he delivers comprehensive coverage of the BI and data warehouse industry. During his career, John spent over ten years working with BI implementations associated with the telecommunications industry. In 2005, John founded the Blue Buffalo Group, a consulting and analysis firm, providing BI expertise to outlets such as BeyeNetwork's Telecom Channel, The Data Warehousing Institute (TDWI) and BillingOSS magazine and go-to-market industry analysis, enabling organizations to penetrate the telecommunications industry vertical.

Analyst: John Myers

! InfoBright’s columnar database is used for applications and data marts that analyze large volumes of machine-generated data.

! InfoBright leverages patented compression techniques and a “knowledge grid” to achieve real-time analytics.

! Infobright offers both an open source and a commercial edition of its software. Both products are designed to handle data volumes up to about 50TB of data.


Susan Davis, Vice President of Marketing at InfoBright, is responsible for the company's marketing strategy and execution. Davis brings more than 25 years of experience in marketing, product management and software development to her role at Infobright. Prior to joining the company, she was vice president of marketing at Egenera and director of product management at Lucent Technologies/Ascend Communications where she was responsible for the release and launch of the telecommunications industry's first commercially available softswitch. She holds a B.S. in economics from Cornell University.


Enabling Real-time Data Analysis

Susan Davis, VP Marketing, Infobright

The Need for Analysis

Ent. Apps market •  Grew to

$115B in 2011

Huge data growth • Machine-generated • Unstructured

SaaS market • 18% growth 2012, projected $22B by 2015

Demand for embedded data analysis

Customers/Users §  Fast access to the data, even

near-real time §  Total flexibility for ad hoc

analysis §  High performance §  Ability to keep longer data

histories §  Less hardware §  No DBA work needed

Technology Provider §  Provide superior analytics for

competitive advantage §  Meet their customers

requirements §  Reduce database costs §  Eliminate need for DBA tuning §  Minimize hardware and

software footprint §  Ease of implementation and

integration with their application

Requirements

Case Study: JDSU

§  Annual revenues exceeded $1.8B in 2011

§  4700 employees are based in over 80 locations worldwide

§  Communications sector offers instruments, systems, software, services, and integrated solutions that help communications service providers, equipment manufacturers, and major communications users maintain their competitive advantage

§  JDSU Service Assurance Solutions §  Ensure high quality of experience (QoE) for wireless voice, data,

messaging, and billing. §  Used by many of the world’s largest network operators

§ New version of Session Trace solution that would: §  Support very fast load speeds to keep up with increasing call

volume and the need for near real-time data access §  Reduce the amount of storage by 5x, while also keeping much

longer data history §  Reduce overall database licensing costs §  Eliminate customers’ “DBA tax,” meaning there should require

zero maintenance or tuning while enabling flexible analysis §  Continue delivering the fast query response needed by

Network Operations Center (NOC) personnel when troubleshooting issues and supporting up to 200 simultaneous users

Telecom Example: JDSU Project Goals

TDR-Store Used by Session Trace Solution

TDR-Store Used by Session Trace Solution

For deployment at Tier 1 network operators, each site

will store between 6 and 45 TB of data, and the total data

volume will range from 700 TB to 1PB of data.

Session Trace Solution

Data Compression & History

•  5X space reduction

•  5X more history online

Getting Data in Quickly

•  Rates of 20,000 TDRs per second (or up to 40,000 database rows per second

•  Appending the new data in less than 10 milliseconds

Reducing Capex & Opex

•  No indexing or tuning required

•  Fewer servers or storage disk required

•  Lower licensing costs than alternatives

Infobright at JDSU

§ Delivers technology solutions that enable and enhance the monetization of internet-distributed video

§ Enables publishers, advertisers, ad networks and media groups to manage, target, display and track advertising in online

Bango: Mobile Payments and Analytics

Bango’s Need Infobright’s Solu6on

A leader in mobile billing and analy/cs services u/lizing a SaaS model

Received a contract with a large media provider §  150 million rows per month §  450GB per month on SQL Server

SQL Server could not support required query performance

Needed a database that could §  scale for much larger data sets §  with fast query response §  with fast implementa/on §  and low maintenance §  in a cost-‐effec/ve solu/on

§  Reduced queries from minutes to seconds

§  Reduced size of one customer’s database from 450 GB to 10 GB for one month of data

Example in Mobile Analytics: Bango

Query SQL Server Infobright 1 Month Report (5MM events) 11 min 10 secs

1 Month Report (15MM events) 43 min 23 secs

Complex Filter (10MM events) 29 min 8 secs

Columnar Database

Designed for fast analy/cs

Deep data compression

Intelligence, not Hardware

Knowledge Grid

Itera/ve Engine

Administra/ve Simplicity

No manual tuning

Minimal ongoing

administra/on

Infobright Analytic Database Technology

Infobright Architecture Overview

Data Packs and Compression

Knowledge Grid Based on MySQL

§  Infobright loader §  High-speed, multi-threaded loader. Load speeds of 80 – 150GB /

hour

§ MySQL loader §  More flexible data formatting options, enhanced error checking. §  Load speed up to about 50GB/hour

§ Distributed Load Processor (DLP) §  Multi-machine data processing engine §  Load speed can exceed 2TB/hour §  Hadoop connector

§ Data Integration tools §  Pentaho, Talend, Informatica, etc

Getting the Data In: Multiple Options

Database server

Distributed Load Processor

Intelligence Not Hardware

• Stores it in the Knowledge Grid (KG) • KG is loaded into memory • Less than 1% of compressed data size

Creates informa/on (metadata) about the

data upon load, automa/cally

• The less data that needs to be accessed, the faster the response

• Sub-‐second responses when answered by the KG

Uses the metadata when processing a query to

eliminate / reduce need to access data

• No need to par//on data, create/maintain indexes, projec/ons or tune for performance

• Ad-‐hoc queries are as fast as sta/c queries, so users have total flexibility

Architecture Benefits

Big Data Analytics: Unique Infobright Features

DomainExpert

•  Web data intelligence

•  Add your domain knowledge

DLP and Hadoop

•  Distributed data processing

•  Simple extract from Hadoop/HDFS

Rough Query

•  Instantaneous drill-down into very large datasets

•  Find the needle in the haystack

Logis6cs, Manufacturing,

Business Intelligence

Online & Mobile Adver6sing/Web Analy6cs

Government U6li6es Research

Financial Services

Telecom & Security

Gaming, Social

Networks

Ø 300 direct and OEM customers across North America, EMEA and Asia Ø 8 of Top 10 Global Telecom Carriers using Infobright via OEM/ISVs

Growing Customer Base across Use Cases and Verticals

Get Started

At infobright.org:

§  Download ICE (Infobright Community Edition)

§  Download an integrated virtual machine from infobright.org

§  Join the forums and learn from the experts!

At Infobright.com

§  Download a free trial of Infobright Enterprise Edition, IEE

§  Download a white paper from the Resource library

§  See the videos at www.youtube.com/infobrightdb

§  Follow us on twitter at twitter.com/infobright

John L Myers Enterprise Management Associates Senior Analyst [email protected]

Pushing Analytics to the “Edge”

© 2012 Enterprise Management Associates, Inc.

Speaker

Slide 29

John Myers joined Enterprise Management Associates in 2011 as senior analyst of the business intelligence (BI) practice area. John has 10+ years of experience working in areas related to business analytics in professional services consulting and product development roles, as well as helping organizations solve their business analytics problems, whether they relate to operational platforms, such as customer care or billing, or applied analytical applications, such as revenue assurance or fraud management.

John L Myers Enterprise Management Associates Senior Analyst

© 2012 Enterprise Management Associates, Inc.

JohnLMyers44

New Definition of Many to Many


There is Big Data and There is LOTS of Data


How to Handle Response Time?


Rather than Center, Push to the “Edge”


Question and Answer

Thank you!

John Myers Senior Analyst [email protected] www.EnterpriseManagement.com JohnLMyer44 twitter JohnLMyers44 Skype



•  What are the types of use cases that InfoBright is getting the most traction from? We have telecom and mobile payment in the case study, but I would be looking for top-5 that may or may not include those two.

•  Are there differences in the geography adoption of InfoBright products? Just wondering about the distribution of particular use cases geographically by region: North America, CALA, EMEA, AsiaPAC.

•  Talk about the attributes of the telecom and mobile payment markets that are “sweet spots” for InfoBright. I would guess it is the “limited” amount of data values (ie., dates, towers, amounts) and the “exploratory” nature (ie.,not set columns of data set).


•  Talk about the choice of MySQL vs. another SQL “interface” for InfoBright. I like the choice, but I would just like to hear the qualitative and quantitative reasons from InfoBright’s perspective.

•  Many people talk about Big-Data requirements (3Vs). What is InfoBright’s specific competitive advantage over other Big Data vendors/players (structured and unstructured)? I am guessing implementation cost, time to implementation and load speed.

•  Why purpose built Columnar over Columnar indexing which has become “popular” from row-based RDBMS vendors?


!   June: Intelligence

!   July: Disruption

!   August: Analytics

!   September: Integration

!   October: Database

!   November: Cloud

Embedded Analytics: The Next Mega-Wave of Innovation

Technology

Transcript of Embedded Analytics: The Next Mega-Wave of Innovation