How to Increase Performance in IBM Cognos

© 2015 IBM Corporation

PDA for CognosCompeting with SQL Server

Sanjeev Datta – Cresco Practice DirectorMark Yingling – IBM Analytics Solution Architect

[email protected]@us.ibm.com

mailto:[email protected]

mailto:[email protected]


Fast on Fast Analytics

The Synergy of IBM Cognos and IBM PureData System for Analytics

3 © 2015 IBM Corporation

IBM Cognos Business Intelligence

Leverage data: Access information in any volume, combination and complexity

Provide insights: Understand your business like never before through self-service analysis at any time on any device

Make confident decisions: Validate your analysis by leveraging predictive information to gain complete visibility into your business

Outperform expectations: Transform your business from a reactive operation to a successful and proactive market leader

A forward-looking view of your business performance through stunning dashboards and reports


Data

Database

Cognos Business Intelligence

RDBMS Adapter

SQL

Data

Database

Data

Data

Growing HistoryMore SourcesRelated DataExtensive TuningInitial Use Case

Limited HistoryTuned Query

SQL SQL

Increased Data Volume

Over time


Multiple Data Sources


RDBMS RDBMS Files Files . . .

External Sources

SQL

• Multiple interfaces• Data movement• Impact on sources • Data consistency• Specialized tuning• Where’s the data?

SQL




RDBMS RDBMS Files Files . . .

• Multiple interfaces• Data movement• Impact on sources • Data consistency• Specialized tuning• Where’s the data?

External Sources

SQL

Data warehouse



• Single database interface• No data movement to Cognos layer for joins, etc.• Standardized administration and tuning• Data quality handled during warehouse load• Reduced impact on source systems• Improved performance for queries and reports


RDBMS

Data warehouse


Requirements Summary• Handle large and growing data volumes

• Integrate into existing environment

• Leverage Cognos and relational database skills

• Provide better and predictable performance

• Minimal to no database and system administration and tuning

• Support for a variety of workloads – queries, reporting, dashboards, analytics

• Simple to get up and running

RDBMS

Data warehouse



Solution

C O G N O S +PureData System for Analytics


Appliance Features Production ready Rack mountable appliance Installed in a standard, customer provided rack Entire integrated appliance tested and packaged at the factory Full function Netezza Platform Software (NPS) with IBM Netezza Analytics Self Encrypting Drives; Up to 16TB1 of user data

Ease of Use Same ease of use and features as larger appliances

- Load and go with no tuning or administration Installation by IBM or an IBM Partner certified to install the N3001-001

Availability & Support Highly available, Full redundancy

− All redundant hardware, 4 disk spares, hot swap power supply Remote access for support; Call Home enabled

1 Assuming 4X compression

PureData System for Analytics N3001-001Bringing speed and simplicity to midsize organizations for big outcomes


IBM Netezza Analytics

Bring the analytics to the data not the data to the analytics

Included

Features

Built-in, in-database analytic functions- Data mining, prediction, transformations,

statistics, geospatial, data preparation Full integration with tools for BI &

visualization- IBM Cognos, Microstrategy, Business

Objects, SAS, MS Excel, SSRS, Kognitio, Qlikview

Full integration with tools for model building & scoring- IBM SPSS, SAS, Open Source R, Fuzzy

Logix Full integration for custom analytics

- Open Source R, Java, C, C++, Python, LUA

Data Preparation

Predictive Analytics

Geospatial Analytics

Advanced Statistics


Big Data and Business Intelligence Ready

Real-time AnalyticsInfoSphere Streams Developer Edition 2 users, non-production licenses

Business Intelligence Cognos software, 5 Analytics User licenses, plus 1 Analytics Administrator license

Hadoop Data ServicesIBM BigInsights v4 for Apache Hadoop® to manage ~100 TB of Hadoop data

Included with the PureData System for Analytics N3001

Data Integration & TransformationInfoSphere DataStage 280 PVUs, 2 concurrent Designer Client licenses and InfoSphere Data Click

Data Warehouse Appliance

Up to 16TB capacity for your Data Warehouse / Data Mart

IBM Fluid Query

Supporting Hadoop Solutions and Streaming Analytics

Open Source “R”

Netezza Analytics


Use cases

Features

Business IntelligenceThe power of IBM Cognos with PureData for Analytics

Leading Business Intelligence- Interactive analysis- Compelling visualizations - web, mobile or email- Enterprise scalability

Optimized for PureData for Analytics- Offers high performing OLAP over relational

experience- Cognos Dynamic Query Mode extends benefits of

PureData by adding in-memory & caching on top of already fast appliance performance

- Exploits Netezza analytic in-database functions

Rapid deployment of answers to key business questions

Included with PureData for Analytics:IBM Cognos Business Intelligence 10.2.1

5 Analytics User licenses, 1 Analytics Administrator license1

Included

Reporting, analysis, scorecards, dashboards Data visualization Mobile business intelligence … and many others

1PureData System for Analytics N3001 must be the data source for Cognos.


When is PureData System for Analytics a Good Fit

Data Volume– At least 0.5 TB of data

Performance– The existing data warehouse/mart solution is not performing– Lots of aggregate tables are required to make the DW perform

• Increases tuning effort and reduces flexibility Maintenance & Customer skill set

– Many resources are required to maintain the data warehouse (>1-2 DBA’s)– The skill set required to tune the existing system is high– The DBA team is slow to react to new business requirements and resulting

query patterns


Dynamic Query Mode is optimized for PDA

Offers a high-performing OLAP Over Relational experience via hybrid

SQL/MDX techniques

Avoids redundant queries through security-aware metadata, data, and

query plan cache management

Provides built-in query visualization tool

Leverages 64-bit architecture

Uses JDBC connection to Netezza

Advanced sorting behavior that aligns DMR queries with other OLAP data

sources


Executing a Dimensionally Modeled Relational (DMR) report with Dynamic Query Mode

Dimensional report results in MDX query against execution engine If the dimension and measure data is in cache, query is computed directly without

accessing database If the data is not in the cache the necessary data is gathered with a relational

SQL query

Using Cognos with PureData for Analytics


High performance analytics over growing data volumes

Aggregate awarenessAggregate acceleration

Optimize in-memory caching with in-database processing

Dynamic Cubes


• Security is applied on top of the caches, so all users benefit

BI query service

Database

Warehouse

Aggregates

Netezza Data Warehouse

Result Set Cache

Expression Cache

Member Cache

Query Data Cache

Aggregate Cache

Over 80% of queries are < 3 seconds

Over half of queries are sub-second

Dynamic Cubes find the shortest path to the answer


TPC-DS 10 TB warehouse performance with Dynamic Cubes

28.8 billion row fact table 65 million members in largest

dimension (Customer)

Subsequent open

First open


Example: Kerberos authentication is a key new feature in Netezza 7.2. The Cognos and Netezza engineering teams collaborated to ensure that Netezza 7.2 and Cognos BI 10.2.2 provided a seamless Kerberos single-sign on experience before either 7.2 or 10.2.2 were released.

The Cognos and Netezza engineering teams can easily collaborate to resolve an IBM technical service request.– There is significantly more barriers to technical support when multiple

vendors are involved.

Collaboration between the IBM Labs

Cognos and Netezza engineers work together to ensure customer success


Integrating Netezza Analytics into Cognos

Netezza Analytic Functions are available as Stored Procedures or UDFs

Create Mining results in Netezza tables and access them during report generation

Read-Only analytic Stored Procedures and UDX can be executed directly from Cognos reports


Benefits of a SPSS Modeler and PureData System for Analytics

Visual, Easy to Use Interface – Faster time to solution and understanding– Expand to Line of Business users

Scalable and Optimized for PureData System for Analytics– Limited/no data movement – analysis executed within the DB

(SQL Pushback, UDFs, In-database Mining, In-database Scoring)– No programming - SQL is automatically generated– Analytics run10x-100x faster

Analytics Flexibility and Deployment– Executed on a purpose built appliance (powered by Netezza)– SPSS Algorithms and Netezza Analytics available– Works with SPSS greater SPSS portfolio

Fortune 100 telco company using SPSS Modeler and Netezza– Scoring 100M customers, 1 model + 10 predictors < 4 seconds!– Scoring 100M customers, 20 models + 20 predictors < 10 seconds!

Performance and Ease of Use


Qualcomm responds to business needs more quickly with PureData System for Analytics

Time reduced to daysfrom months which was spent on development

600 times faster query performance

Solution components • IBM PureData System for Analytics (powered by

Netezza technology)• IBM Cognos® Business Intelligence

“By having an optimized and integrated system, we now can leverage all our data to look for new opportunities and focus our attention where we will see the most return.”

- Kim Konotchick, Senior IT Manager, Qualcomm

Time reduced to daysto market on new solutions


FleetRisk Advisors help trucking operators prevent more accidents with stronger and faster risk prediction models

20% reductionin the incidence of minor accidents

80% reductionin serious accidents amount trucking company customers

30% increasein driver retention rates, with commensurate decreases in recruiting and training costs

Solution components • IBM® PureData™ System for Analytics (powered by Netezza® technology)• IBM SPSS® Collaboration and Deployment Services• IBM SPSS Modeler• IBM SPSS Modeler Desktop• IBM SPSS Modeler Server

“Our new solution has enabled us to push the boundaries of predictive risk analysis, which has translated into real value for our trucking operator customers that rely on it.”

—Patrick Ritto, chief technology officer

25 © 2015 IBM Corporation25

Cognos and Netezza – a blazing combination

5 reasons to use Cognos BI with Netezza

1. Interactive analysis – engaging self-service interfaces2. Enterprise scalability – supports thousands of users3. Compelling visualizations – on the web, mobile, or emailed4. Optimized queries – intelligently balances local and remote data

processing5. No wait time – instantaneous responses when in-memory cache

is leveraged

C O G N O S + Blazing Results=

Netezza Solutions

100+ Joint Customers



PureData System for Analytics N3001-001 vs

Microsoft SQL Server


https://www.rocksolidsql.com/News/News.aspx?NewsCategoryKey=9785e502-23c2-4ea4-baf6-1b01927d14d1

Aging Install Base 80%+ 2008 or Older


SQL Server PDA Mini (N3001-001)

…and hope it runs! …turn the key and Go!!


Four Things to Know about SQL Server 2008 R2

1. OLTP optimized solution • Microsoft SQL Server 2008 R2 is an OLTP optimized databases. Its optimizer is not designed

or built to handle the complex queries inherent in analytic workloads.

2. No HA Built In• The SQL Server SMP based Fast Track Solution is a single server only. There are no HA

capabilities. Can you trust your mission critical warehouse to a system with no HA capability?

3. Scalability is Limited• Both the single server and the software limit the scalability of the solution. You cannot simply

add more resources and grow the system and expect more performance out of the system due to the inherent limitations of an SMP architecture for data warehousing.

4. Inefficient data compression• SQL Server uses some old, inefficient algorithms that rely on the data in the table being

always in sorted order to work at all. If the data is random, just like it is generated in an OLTP system, or batched into a DWH, then SQL Server will get very little, if any, compression.


Mini Appliance test results

IBM PureData System for Analytics

Mini Appliance (N3001-001)MS SQL Server

3seconds

1Avnet beta test performed using customer workload on PureData System for Analytics N3001-001 compared to MS SQL Server 2008

384seconds

What could you do if your queries were 127x faster?

vs.

Avnet beta test using customer workload


PDA Mini Appliance test results

1GrassRoots beta test performed using customer workload on PureData System for Analytics N3001-001 compared to MS SQL Server 2008

What could you do if your queries were 44x faster?

IBM PureData System for Analytics

Mini Appliance (N3001-001)MS SQL Server

10seconds

444seconds vs.

'We were blown away by the performance, we loaded 600 million records and the Netezza Mini Mako appliance performed 45x faster than our MSSQL 2008 instance. What was more impressive was how quick the Netezza Mini was when used with Tableau. Even with 600 million records, we

were able to use Tableau in an almost interactive fashion. No more waiting for minutes for the data to be retrieved and visualized"

Grass Roots beta test using customer workload


Lets Look At How PureData System for Analytics stacks up to

The Latest Windows Columnar Competitive Database



Test Scenarios

Workload tested– Two sets of queries

• Sales report style queries (80%)• Data Scientist style queries (20%)

Two modes of execution tested– Serial Execution test (single user test)

• Used to isolate single query performance• Single connection iterates all queries in the workload

– Heavy Mixed Throughput test (30 user test)• Time to complete a set number of reports for each user


The Systems Tested (Initially)

Current Columnar WindowsCompetitive database

PureData System for Analytics N3001-001

Fully HA appliance2x x3650M4 (20 cores) 128GB RAM24x 600GB HDD

System (hardware & software) Cost $170,000Maintenance for 3 years $51,000Total Cost over 3 years $221,000

Single server, no HA 20 cores Intel Ivy Bridge EX 512 GB RAM 8 x 900GB HDD

Server Cost $40,500Software Cost $137,500Maintenance for 3 years $111,000Total Cost over 3 years $289,000

30% more expensive, and much slower

for systems compared

PureData Analytics N3001-001 vs. 2014 Columnar Database Competitor (Intermediate and Complex Analytics workload) Based on IBM internal tests comparing PureData System For Analytics with a comparable competitor configuration (version available as of 03/15/2015) executing a materially identical analytics workload in a controlled laboratory environment. Test measured 30, 20, and 10 concurrent user report throughput to execute identical SQL query workloads on the same data. Competitor configuration includes competitor recommended software options and features. IBM configuration: PureData System for Analytics N3001-001. Results may not be typical and will vary based on actual workload, configuration, applications, queries and other variables in a production environment. Users of this document should verify the applicable data for their specific environment. Contact IBM and see what we can do for you. Pricing is based on publically available information and published list prices as of 3/15/2015. . Total cost over 3 years takes into account cost of hardware, software, and maintenance.


Testing

PureData System for Analytics N3001-001

Single server, no HA 20 cores Intel Ivy Bridge EX 512 GB RAM 8 x 900GB HDD

Initially, one of the Data Scientist style queries never finished on the columnar database, but completed in seconds on PureData System for

Analytics





Hardware and Software Tuning vs. No TuningPureData System for Analytics

N3001-001


System (hardware & software) Cost $170,000Maintenance for 3 years $51,000Total Cost over 3 years $221,000

Single server, no HA 20 cores Intel Ivy Bridge EX 512 GB RAM 8 x 900GB HDD 1.2TB High IOPS FlashServer Cost $58,000Software Cost $137,500Maintenance for 3 years $111,000Total Cost over 3 years $306,500

After adding flash storage – PLUS a week of expert tuning, the team was able to get the

“problem” query to run

No indexes, no aggregates, no tuning at all…

Faster with no tuning.




No Tuning for IBMPureData System for Analytics

N3001-001

Single server, no HA 20 cores Intel Ivy Bridge EX 512 GB RAM 8 x 900GB HDD 1.2TB High IOPS Flash


1TB 30 User Concurrent Execution Workload

159 qphqueries per Hour

181 qphqueries per Hour

14% Slower, 39% More expensivePLUS more tuning required

for systems compared



Over a week of expert tuning PLUS adding Flash on Competitor


PDA Mini Beats The Windows Competitive Database

Pure Data System to Analytics is Simply Faster – for systems compared– Faster to get up and running– Faster and easier to get blazing performance– Faster performing

And More Importantly, Pure Data System to Analytics– Costs Less– Does not force sacrifices in performance

Not just faster and easier than the older, traditional Row Store version -- but also the new Columnar version as well


PDA The Smart Choice Over SQL Server

Proven architecture for complex analytics on large data volumes

True appliance simplicity, quick time to value

In-database analytics functions to keep processing within the database

Simple: No indexes, No Tuning!

Includes entitlements for• Business Intelligence – Cognos

• Data Integration and Transformation – InfoSphere DataStage

• Hadoop Data Services – BigInsights

• Real Time Analysis – InfoSphere Streams …


Q&A


Tactical Institute Utilizes IBM Watson Analytics to Gain CrimeAnalysis – June 1 at 11 AM CST

Complimentary Workshop: PureData Analytics – June 22 in Dallas, TX

Register for these events & more: crescointl.com/events

Or RSVP email [email protected]

Upcoming events

Sanjeev Datta


Thanks for joining!Cresco brings your data and business together via Analytics Expertise in software management, technical and management consulting, training and support.

Contact Us >> Chat with us on www.crescointl.com> Call 844.6.CRESCO> Email [email protected]

Sanjeev Datta

How to Increase Performance in IBM Cognos

Data & Analytics

Transcript of How to Increase Performance in IBM Cognos