Embedded Analytics: The Next Mega-Wave of Innovation

Post on 26-May-2015

841 views 0 download

Tags:

description

Could embedded analytics change the way consumers do business? A whole range of Web-based and traditional software providers are now embedding analytical power into their applications such that users can do more complex analysis of their data. The use cases span such industries as eCommerce, telecom, security and other such data-intensive verticals. As a result of this trend, the providers and their customers can gain greater insights about their businesses and thus improve decisions.Check out this episode of The Briefing Room to hear Analyst John Myers of EMA explain how delivering embedded analytics can expand the value of analysis to customers and partners all over the world, while raising the bar for how business is done. Myers will be briefed by Susan Davis of Infobright, who will tout her company’s success in enabling solution providers to deliver real-time analytical capabilities to their customers.

Transcript of Embedded Analytics: The Next Mega-Wave of Innovation

Eric.kavanagh@bloorgroup.com

Twitter Tag: #briefr

!  Reveal the essential characteristics of enterprise software, good and bad

!  Provide a forum for detailed analysis of today’s innovative technologies

!  Give vendors a chance to explain their product to savvy analysts

!  Allow audience members to pose serious questions... and get answers!

Twitter Tag: #briefr

!   June: Intelligence

!   July: Disruption

!   August: Analytics

!   September: Integration

!   October: Database

!   November: Cloud

Twitter Tag: #briefr

Twitter Tag: #briefr

!   The last ten or so years have seen a massive influx of business intelligence tools: reporting, analytics, data mining, online analytical processing, querying, etc.

!   BI technologies are designed to let organizations take all their capabilities and convert them into knowledge, ultimately getting the the right information to the right people at the right time.

!   Vendors face the challenge of providing organizations with tools robust enough to get at their data and provide the right actionable insight.

Twitter Tag: #briefr

John Myers joined Enterprise Management Associates in 2011 as senior analyst of the BI practice area, where he delivers comprehensive coverage of the BI and data warehouse industry. During his career, John spent over ten years working with BI implementations associated with the telecommunications industry. In 2005, John founded the Blue Buffalo Group, a consulting and analysis firm, providing BI expertise to outlets such as BeyeNetwork's Telecom Channel, The Data Warehousing Institute (TDWI) and BillingOSS magazine and go-to-market industry analysis, enabling organizations to penetrate the telecommunications industry vertical.

Analyst: John Myers

! InfoBright’s columnar database is used for applications and data marts that analyze large volumes of machine-generated data.

! InfoBright leverages patented compression techniques and a “knowledge grid” to achieve real-time analytics.

! Infobright offers both an open source and a commercial edition of its software. Both products are designed to handle data volumes up to about 50TB of data.

Twitter Tag: #briefr

Susan Davis, Vice President of Marketing at InfoBright, is responsible for the company's marketing strategy and execution. Davis brings more than 25 years of experience in marketing, product management and software development to her role at Infobright. Prior to joining the company, she was vice president of marketing at Egenera and director of product management at Lucent Technologies/Ascend Communications where she was responsible for the release and launch of the telecommunications industry's first commercially available softswitch. She holds a B.S. in economics from Cornell University.

Twitter Tag: #briefr

Enabling Real-time Data Analysis

Susan Davis, VP Marketing, Infobright

The Need for Analysis

Ent. Apps market •  Grew to

$115B in 2011

Huge data growth • Machine-generated • Unstructured

SaaS market • 18% growth 2012, projected $22B by 2015

Demand for embedded data analysis

Customers/Users §  Fast access to the data, even

near-real time §  Total flexibility for ad hoc

analysis §  High performance §  Ability to keep longer data

histories §  Less hardware §  No DBA work needed

Technology Provider §  Provide superior analytics for

competitive advantage §  Meet their customers

requirements §  Reduce database costs §  Eliminate need for DBA tuning §  Minimize hardware and

software footprint §  Ease of implementation and

integration with their application

Requirements

Case Study: JDSU

§  Annual revenues exceeded $1.8B in 2011

§  4700 employees are based in over 80 locations worldwide

§  Communications sector offers instruments, systems, software, services, and integrated solutions that help communications service providers, equipment manufacturers, and major communications users maintain their competitive advantage

§  JDSU Service Assurance Solutions §  Ensure high quality of experience (QoE) for wireless voice, data,

messaging, and billing. §  Used by many of the world’s largest network operators

§ New version of Session Trace solution that would: §  Support very fast load speeds to keep up with increasing call

volume and the need for near real-time data access §  Reduce the amount of storage by 5x, while also keeping much

longer data history §  Reduce overall database licensing costs §  Eliminate customers’ “DBA tax,” meaning there should require

zero maintenance or tuning while enabling flexible analysis §  Continue delivering the fast query response needed by

Network Operations Center (NOC) personnel when troubleshooting issues and supporting up to 200 simultaneous users

Telecom Example: JDSU Project Goals

TDR-Store Used by Session Trace Solution

TDR-Store Used by Session Trace Solution

For deployment at Tier 1 network operators, each site

will store between 6 and 45 TB of data, and the total data

volume will range from 700 TB to 1PB of data.

Session Trace Solution

Data Compression & History

•  5X space reduction

•  5X more history online

Getting Data in Quickly

•  Rates of 20,000 TDRs per second (or up to 40,000 database rows per second

•  Appending the new data in less than 10 milliseconds

Reducing Capex & Opex

•  No indexing or tuning required

•  Fewer servers or storage disk required

•  Lower licensing costs than alternatives

Infobright at JDSU

§ Delivers technology solutions that enable and enhance the monetization of internet-distributed video

§ Enables publishers, advertisers, ad networks and media groups to manage, target, display and track advertising in online

Bango: Mobile Payments and Analytics

Bango’s  Need   Infobright’s  Solu6on  

A  leader  in  mobile  billing  and  analy/cs  services  u/lizing  a  SaaS  model    

Received  a  contract  with  a  large  media  provider  §  150  million  rows  per  month  §  450GB  per  month  on  SQL  Server  

 

SQL  Server  could  not  support  required  query  performance  

Needed  a  database  that  could  §  scale  for  much  larger  data  sets    §  with  fast  query  response  §  with  fast  implementa/on  §  and  low  maintenance  §  in  a  cost-­‐effec/ve  solu/on  

§  Reduced  queries  from  minutes  to  seconds  

 

§  Reduced  size  of  one  customer’s  database  from  450  GB  to  10  GB  for  one  month  of  data  

Example in Mobile Analytics: Bango

Query   SQL Server   Infobright  1 Month Report (5MM events)   11 min   10 secs  

1 Month Report (15MM events)   43 min   23 secs  

Complex Filter (10MM events)   29 min   8 secs  

Columnar  Database  

Designed  for  fast  analy/cs  

Deep  data  compression  

Intelligence,  not  Hardware  

Knowledge  Grid  

Itera/ve  Engine  

Administra/ve  Simplicity  

No  manual  tuning  

Minimal  ongoing  

administra/on  

Infobright Analytic Database Technology

Infobright Architecture Overview

Data  Packs  and  Compression  

Knowledge  Grid     Based  on  MySQL  

§  Infobright loader §  High-speed, multi-threaded loader. Load speeds of 80 – 150GB /

hour

§ MySQL loader §  More flexible data formatting options, enhanced error checking. §  Load speed up to about 50GB/hour

§ Distributed Load Processor (DLP) §  Multi-machine data processing engine §  Load speed can exceed 2TB/hour §  Hadoop connector

§ Data Integration tools §  Pentaho, Talend, Informatica, etc

Getting the Data In: Multiple Options

Database server

Distributed Load Processor

Intelligence Not Hardware

• Stores  it  in  the  Knowledge  Grid  (KG)  • KG  is  loaded  into  memory  • Less  than  1%  of  compressed  data  size      

Creates  informa/on  (metadata)  about  the  

data  upon  load,  automa/cally  

• The  less  data  that  needs  to  be  accessed,  the  faster  the  response  

• Sub-­‐second  responses  when  answered  by  the  KG  

Uses  the  metadata  when  processing  a  query  to  

eliminate  /  reduce  need  to  access  data  

• No  need  to  par//on  data,  create/maintain  indexes,  projec/ons  or  tune  for  performance  

• Ad-­‐hoc  queries  are  as  fast  as  sta/c  queries,  so  users  have  total  flexibility  

Architecture  Benefits  

Big Data Analytics: Unique Infobright Features

DomainExpert

•  Web data intelligence

•  Add your domain knowledge

DLP and Hadoop

•  Distributed data processing

•  Simple extract from Hadoop/HDFS

Rough Query

•  Instantaneous drill-down into very large datasets

•  Find the needle in the haystack

Logis6cs,  Manufacturing,  

Business  Intelligence    

Online  &  Mobile  Adver6sing/Web  Analy6cs  

Government  U6li6es  Research  

 

Financial  Services  

 

Telecom  &  Security  

 

Gaming,  Social  

Networks  

Ø 300  direct  and  OEM  customers  across  North  America,  EMEA  and  Asia  Ø 8  of  Top  10  Global  Telecom  Carriers  using  Infobright  via  OEM/ISVs  

Growing Customer Base across Use Cases and Verticals

Get Started

At infobright.org:

§  Download ICE (Infobright Community Edition)

§  Download an integrated virtual machine from infobright.org

§  Join the forums and learn from the experts!

At Infobright.com

§  Download a free trial of Infobright Enterprise Edition, IEE

§  Download a white paper from the Resource library

§  See the videos at www.youtube.com/infobrightdb

§  Follow us on twitter at twitter.com/infobright

Twitter Tag: #briefr

John L Myers Enterprise Management Associates Senior Analyst JMyers@EnterpriseManagement.com

Pushing Analytics to the “Edge”

© 2012 Enterprise Management Associates, Inc.

Speaker

Slide 29

John Myers joined Enterprise Management Associates in 2011 as senior analyst of the business intelligence (BI) practice area. John has 10+ years of experience working in areas related to business analytics in professional services consulting and product development roles, as well as helping organizations solve their business analytics problems, whether they relate to operational platforms, such as customer care or billing, or applied analytical applications, such as revenue assurance or fraud management.

John L Myers Enterprise Management Associates Senior Analyst

© 2012 Enterprise Management Associates, Inc.

JohnLMyers44

What is Machine to Machine Big Data

Slide 30 © 2012 Enterprise Management Associates, Inc.

New Definition of Many to Many

Slide 31 © 2012 Enterprise Management Associates, Inc.

There is Big Data and There is LOTS of Data

Slide 32 © 2012 Enterprise Management Associates, Inc.

How to Handle Response Time?

Slide 33 © 2012 Enterprise Management Associates, Inc.

Rather than Center, Push to the “Edge”

Slide 34 © 2012 Enterprise Management Associates, Inc.

Question and Answer

Thank you!

John Myers Senior Analyst JMyers@emausa.com www.EnterpriseManagement.com JohnLMyer44 twitter JohnLMyers44 Skype

Slide 35 © 2012 Enterprise Management Associates, Inc.

Twitter Tag: #briefr

•  What are the types of use cases that InfoBright is getting the most traction from? We have telecom and mobile payment in the case study, but I would be looking for top-5 that may or may not include those two.

•  Are there differences in the geography adoption of InfoBright products? Just wondering about the distribution of particular use cases geographically by region: North America, CALA, EMEA, AsiaPAC.

•  Talk about the attributes of the telecom and mobile payment markets that are “sweet spots” for InfoBright. I would guess it is the “limited” amount of data values (ie., dates, towers, amounts) and the “exploratory” nature (ie.,not set columns of data set).

Twitter Tag: #briefr

•  Talk about the choice of MySQL vs. another SQL “interface” for InfoBright. I like the choice, but I would just like to hear the qualitative and quantitative reasons from InfoBright’s perspective.

•  Many people talk about Big-Data requirements (3Vs).  What is InfoBright’s specific competitive advantage over other Big Data vendors/players (structured and unstructured)? I am guessing implementation cost, time to implementation and load speed.

•  Why purpose built Columnar over Columnar indexing which has become “popular” from row-based RDBMS vendors?

Twitter Tag: #briefr

!   June: Intelligence

!   July: Disruption

!   August: Analytics

!   September: Integration

!   October: Database

!   November: Cloud