Business Analytics: The Big Leap Forwardtimoelliott.com/blog/docs/affecto_keynote.pdfSAP. Oracle....
Transcript of Business Analytics: The Big Leap Forwardtimoelliott.com/blog/docs/affecto_keynote.pdfSAP. Oracle....
-
Business Analytics: The Big Leap Forward
Timo Elliott September 2011
-
2
Top Business Issues 2011
-
3
Business Analytics Provides Great Value
Data is extremely important for competitiveadvantage
Data makes an important contribution to customer relations efforts
Business information has helped manage costs or improve operations
Executives believe companies can benefit greatly from using data, especially information generated within the company
Agree: 69% Agree: 77% Agree: 70%
-
4
Surging Growth in Business Analytics
2009 2010
+3.8%
+13.4%
Gartner: worldwide BI, analytics and performance management software revenue
BI Growth more than tripled between 2009 and 2010!
-
5
Analytics is an Ever-Increasing Share of IT Budget
2009 2010 2011
3.9%
+4.1%
+4.3%
Gartner: worldwide BI, analytics and performance management software revenue
“BI spending has far surpassed IT budget growth overall for several years”
Dan Sommer, Gartner
-
6
Business Analytics Around the World
Business Analytics MarketGrowth 2010
3.0%
3.7%
6.7%
17.8%
18.3%
19.5%
22.9%
Eastern Europe
Japan
Western Europe
North America
Middle East and Africa
Latin America
Asia/Pacific
13.2%
11.6%
Gartner Market Share Analysis: Business Intelligence, Analytics and Performance Management Software, Worldwide, published March 2011
-
77
Market Consolidation Continues
-
8
Business Analytics Market (BI, EPM, Analytic Applications)Share of Market, 2010
Business Analytics Market Shares
SAP
Oracle
SAS Institute
IBM
15.6%
13.2%
11.6%
23%
Gartner Market Share Analysis: Business Intelligence, Analytics and Performance Management Software, Worldwide, published May 2011
The 4 “Megavendors” continued to increase their market share– smaller vendors took from each other
-
9
Business Analytics is Nothing New
The “What” doesn’t fundamentally change — but the “How” does
-
10
Business Analytics Has Struggled to Keep Up
-
11
Reporting
“Typical” Business Intelligence Today
Slow
Painful
Expensive
Operational Data Store
Data Warehouse
Indexes
Aggregates
DataBusiness Applications
Copy
ETLCalculation EngineBusiness Intelligence
Query ResultsQuery
Slow
Painful
Expensive
Operational Data Store
Data Warehouse
Indexes
Aggregates
DataBusiness Applications
Copy
ETL
Calculation EngineBusiness Intelligence
Query ResultsQuery
DataMarts
-
12
What’s the Problem?
Slow Disks & CPUs
I/O Bottleneck
Expensive Memory
Optimized for Transactions
BI is an Afterthought
30 Year-Old Database Design Principles
-
13
A Revolution…
Credit Suisse, “The Need for Speed”
-
14
Today’s Disks Can’t Keep Up With Processing Power
-
15
In-Memory Computing Costs have Plummeted
Oslo Plaza117m
Cost of 1 Mb of memory in 2000: ≈$1
-
16
In-Memory Computing Costs have Plummeted
Cost of 1 Mb of memory today: ≈ ½ cent
My daughter:1m
And shrinking, and shrinking, and shrinking….
Price/performance of in-memory has
DOUBLED in last 9 months
-
17
In-Memory Computing
Operational Data Store
Data Warehouse
Indexes
Aggregates
DataBusiness Applications
Copy
ETL
Calculation EngineBusiness Intelligence
Query ResultsQuery
Up to 1,000x fasterNo optimizations required
DataMarts
-
18
Row vs. Column Databases
My Filing System
My Wife’s Filing System
Row-based Column-based
-
19
Row-Based Data
Wasted space, and a full scan to aggregate any particular field
-
20
Column Data
More efficient data storage, better compression, faster queries
-
21
Data WarehouseData Warehouse
Column Databases
Operational Data Store
Data Warehouse
DataBusiness Applications
Copy
ETL
Calculation EngineBusiness Intelligence
Query ResultsQuery
Up to 1,000x fasterMore data in less space
-
22
Data Warehouse
Massively Parallel Hardware
Operational Data Store
DataBusiness Applications
Copy
ETL
Business IntelligenceQuery Results
Query
Up to 1,000x fasterOptimized for hardware
Calculation Engine
-
23
In-Database Analytics
Forecasting ClusteringAnomalies
Influencers Trends Meaningful or Random?
-
24
In-Database Analytics
-
25
A Database Designed for Business
Volume DriverCyclesDriverForecast DriverForecast AgentsGrowSeasonal ComplexAssortment PlanningCumulateDaysDays OutstandingDiscounted Cash FlowDe-cumulateDelayDelay Debt
Delay StockAnnual DepreciationAnnual DepreciationDiminishing Balance
DepreciationSum of Year DepreciationYear To Date StatisticalYOY/ YOY DifferenceForecast Dual DriverForecast SensitivityFeedFeed OverflowForecastFundsFuture Value
Inflated Cash FlowInternal Rate of ReturnMoving MedianNumber of PeriodsNet Present ValueOutlookPaymentPresent ValueLagLastLeaseLease VariableLinear AverageForecast MixMoving Average/Sum
ProportionRateRepeatSeasonal SimpleSeasonal SimulationStock FlowStock Flow ReverseStock Flow BatchTimeTime SumMax ValueMinimum ValueTransformRounding
Up until now, there’s been a false separation between application logic and database functionality
-
26
Data Warehouse
In-Database Analytics
Operational Data Store
DataBusiness Applications
Copy
ETL
Business IntelligenceQuery Results
Query
Up to 1,000x fasterPush processing down to dedicated hardware, less traffic
Analytic Appliance
Calculation Engine
-
27
Integrating Flows of Data
Incremental loads, replication
-
28
Integrating Flows of Data
-
29
Streaming Data
-
30
Real-Time Data
Operational Data Store
Copy
ETL
Real-time replication — why have a separate operational data store?
DataBusiness Applications
Analytic ApplianceBusiness Intelligence
-
31
Real-Time Analytics on Big Data
-
32
The Basis For Applications of The Future
Copy
Business Applications
Analytic ApplianceBusiness Intelligence
Use a single appliance for both analytics and applications
Data
-
33
Virtuous Circle of Technology
In-Memory
Columnar Databases
Hardware Acceleration
Calculation Engine
Columnar storage increases the amount of data that can be stored in limited memory (compared to disk)
Column databases enable easier parallelization of queries
In-memory processing gives more time for
relatively slow updates to column data
In-memory allows sophisticated calculations
in real-time
Hardware acceleration makes sophisticated
calculations like allocations possible
Each technology works well on its own, but combining them all is the real opportunity — provides all of the upside benefits while mitigating the downsides
-
“By 2012, 70% of Global 1000 organizations will load detailed data into memory as the primary method to optimize BI application performance.”
- Gartner
-
35
6.6
41.9
HANA
Traditional DB
Data Compression with HANA (GB)
3.2
5.1
5.1
1050
1320
2660
Query 1
Query 2
Query 3
Query Run-Time (Seconds)
6.3x Data Compression
369x Average Query Speed-Up
No Schema Changes
Same Data
Same SQL
Immediate Benefits
Large Bank – 1 Month of Customer Information
-
36
-
37
-
38
Extended Architecture
Business ApplicationsAnalytic Appliance
Business Intelligence
Cloud computingUnstructured and personal dataMobile revolutionCollaboration
-
39
Do More, Faster
“The time between ‘event’ and ‘action’ is rapidly closing.”
“In the past, managers could take weeks or days to make important decisions, however to effectively compete globally, some companies are making critical decisions in hours, minutes or even seconds”
Paul Barsch, 2009
“If things seem under control, you’re just not going fast enough.”
-Mario Andretti
-
40
In-Memory Computing is Like Digital Photography
A transformative technology that slowly but surely upturns the whole industry
Faster, Easier, More Convenient
Evolved Faster Than The Alternatives
-
41
It’s All About Flexibility and Evolution
“It's not the strongest that survive, nor the most intelligent, but the one most responsive to change.”
Charles Darwin
-
42
Reality Is, and Always Will be, Messy
Different information sources
Different levels of expertise
Different access devices
Different time horizons
Different levels of analytic need
Differentproject phases
RiskPolitics
But new architectures mean simplification and new opportunities
-
43
What About Big Data / NoSQL / Hadoop?
-
44
What About Big Data / NoSQL / Hadoop?
Complementary technology
Very real value, but immature
Primarily used today for preprocessing unstructured data
Velocity
Volume Variety
New analytic
platforms
HADOOP
-
46
What About Flash Disk / SSDs?
15X
9000X
16X
-
47
“Poor-quality customer data costs U.S. businesses $611 billion a year. Yet nearly half of the companies surveyed admit they have no plans to improve data quality”
The Data Warehousing Institute study
What About Data Qwality?
-
48
Real-Time Data Quality
If everything’s incremental, when do we do data cleansing?
Levels of quality
In-db cleansing
-
49
Applications for Data Stewards
-
50
What About Social Data?
-
51
What About Unstructured Data?
Column stores are good at storing text data.
Can push the text analytics algorithms into the appliance, more flexibility
-
53
Text Data Processing for Unstructured Data
http://experience.sap.com/twitterta/sapsummit.jsp
http://experience.sap.com/twitterta/sapsummit.jsp�
-
54
What About End-User Enablement?
Data Warehouse
ApplicationData DepartmentData
Personal Data
From “self-service BI” to“self-service Data Warehousing”
-
What About Ease of Use?
Top Roadblocks to BI Success
Challenge Rank
Complexity of BI tools and interfaces 1Cost of BI software and per-user licenses 2
Difficulty accessing relevant, timely, or reliable data 3
Insufficient IT staffing or excessive software requirements for IT support 4
Difficulty identifying applications or decisions that can be supported by BI 5
Lack of appropriate BI technical expertise within IT 6
Lack of support from executives or business management 7
Poor planning or management of BI programs 8
Lack of BI technology standards and best practices 9
Lack of training for end users 10
1. Doug Henschen, InformationWeek, “BI Efforts Take Flight”, Oct 13, 2008
-
Donkey Kong
-
Grand Theft Auto
-
58
Progressive Expertise
View Reports Strategic Analysis
-
59
Use the Power to Improve Ease of Use
No longer query –wait – analyze –format …
Iterative feedback loop allows instant feedback and learning
-
60
WYN-WYN-WYNMobile Opportunities
More People, More Often, More Context
-
61
Mobile Isn’t Only About “Mobile”
-
64
10k m
De NHM kijker
Eerste Romeinsenederzetting: “OppidumBatavorum”Jaartal: 12 voor Chr.Afstand: 300 meter
0.3
Augmented Reality
-
65
Filter by: Branch
HighstreetOperations +23%
NE 0.1km
Augmented Corporate Reality
-
66
Augmented Corporate Reality
-
67
Filter by: Maintenance History
Tower Pipe 3Last Maintenance: 2 Weeks
E 0.1km
Photo by Thomas Hawk, Flickr
http://www.flickr.com/photos/thomashawk/�
-
68
Store 23Current sales: $15k
SE 0.1km
Filter by: Store Performance
-
69
“Computers are useless.
- Pablo Picasso
They can only give youanswers.”
-
70
What About Decision Making?
70
We rely onpeople! Source: IDC
-
71
Collaboration Around Data
Supermarine Spitfire
Jay Wright Forrester,Inventor of RAM Memory
-
72
Social Intelligence Needs The New Architectures
Expertise location — Relationship Mining — Social Network Analysis
-
73
Putting Social Into Business Processes
“The big failure of social business is a lack of integration of social tools into the collaborative workflow.”
-
74
Did You Know…
-
75
-
76
The REAL Big Leap Forward
© SAP 2008 / Page 76
Breadth and Sophistication of Possible Analytical Tasks
Perc
enta
ge o
f Use
rs D
oing
or
Thin
king
abo
ut th
ese
task
s
Quantitative Thinking Gap
Huge opportunity to make business people more productive and efficient, increase their satisfaction, save money for the company, and drive more revenue.
-
77
Conclusion
In-memory industry revolutionEvery company in the industry heading the same directionDon’t be the last one shooting on film
Beyond data warehousingBecomes part of the operational systemsPlatform for business applications of the future
Start experimentingThese systems are real, and can provide benefits today
-
Thanks!
Email:[email protected]
BI Blog:timoelliott.com
timoelliott.com/blog/docs/affecto_keynote.pdf
You Should Follow Me on Twitter: @timoelliott
http://timoelliott.com/blog�
Slide Number 1Top Business Issues 2011Business Analytics Provides Great ValueSurging Growth in Business AnalyticsAnalytics is an Ever-Increasing Share of IT BudgetBusiness Analytics Around the WorldMarket Consolidation ContinuesBusiness Analytics Market SharesBusiness Analytics is Nothing NewBusiness Analytics Has Struggled to Keep Up“Typical” Business Intelligence TodayWhat’s the Problem?A Revolution…Today’s Disks Can’t Keep Up With Processing PowerIn-Memory Computing Costs have PlummetedIn-Memory Computing Costs have PlummetedIn-Memory ComputingRow vs. Column DatabasesRow-Based DataColumn DataColumn DatabasesMassively Parallel HardwareIn-Database AnalyticsIn-Database AnalyticsA Database Designed for BusinessIn-Database AnalyticsIntegrating Flows of DataIntegrating Flows of DataStreaming DataReal-Time DataReal-Time Analytics on Big DataThe Basis For Applications of The FutureVirtuous Circle of TechnologySlide Number 34Large Bank – 1 Month of Customer InformationSlide Number 36Slide Number 37Extended ArchitectureDo More, FasterIn-Memory Computing is Like Digital PhotographyIt’s All About Flexibility and EvolutionReality Is, and Always Will be, MessyWhat About Big Data / NoSQL / Hadoop?What About Big Data / NoSQL / Hadoop?What About Flash Disk / SSDs?What About Data Qwality?Real-Time Data QualityApplications for Data StewardsWhat About Social Data?�What About Unstructured Data?Text Data Processing for Unstructured DataWhat About End-User Enablement?What About Ease of Use?Donkey KongGrand Theft AutoProgressive Expertise Use the Power to Improve Ease of UseMobile OpportunitiesMobile Isn’t Only About “Mobile”Slide Number 62Slide Number 63Augmented RealityAugmented Corporate RealityAugmented Corporate RealitySlide Number 67Slide Number 68Slide Number 69What About Decision Making?Collaboration Around DataSocial Intelligence Needs The New ArchitecturesPutting Social Into Business ProcessesDid You Know…Slide Number 75The REAL Big Leap ForwardConclusionSlide Number 78