High Performance BI

8/10/2019 High Performance BI

1/320


2/320

20002011 MicroStrategy, Incorporated. All rights reserved.

This Course (course and course materials) and any Software are provided as is and without express or limited

warranty of any kind by either MicroStrategy, Inc. or anyone who has been involved in the creation, production, ordistribution of the Course or Software, including, but not limited to, the implied warranties of merchantability andfitness for a particular purpose. The entire risk as to the quality and performance of the Course and Software is with

you. Should the Course or Software prove defective, you (and not MicroStrategy, Inc. or anyone else who has been

involved with the creation, production, or distribution of the Course or Software) assume the entire cost of allnecessary servicing, repair, or correction.

In no event will MicroStrategy, Inc. or any other person involved with the creation, production, or distribution of theCourse or Software be liable to you on account of any claim for damage, including any lost profits, lost savings, or otherspecial, incidental, consequential, or exemplary damages, including but not limited to any damages assessed against orpaid by you to any third party, arising from the use, inability to use, quality, or performance of such Course andSoftware, even if MicroStrategy, Inc. or any such other person or entity has been advised of the possibility of suchdamages, or for the claim by any other party. In addition, MicroStrategy, Inc. or any other person involved in thecreation, production, or distribution of the Course and Software shall not be liable for any claim by you or any other

party for damages arising from the use, inability to use, quality, or performance of such Course and Software, basedupon principles of contract warranty, negligence, strict liability for the negligence of indemnity or contribution, thefailure of any remedy to achieve its essential purpose, or otherwise.

The information contained in this Course and the Software are copyrighted and all rights are reserved byMicroStrategy, Inc. MicroStrategy, Inc. reserves the right to make periodic modifications to the Course or the Softwarewithout obligation to notify any person or entity of such revision. Copying, duplicating, selling, or otherwisedistributing any part of the Course or Software without prior written consent of an authorized representative ofMicroStrategy, Inc. are prohibited. U.S. Government Restricted Rights. It is acknowledged that the Course andSoftware were developed at private expense, that no part is public domain, and that the Course and Software areCommercial Computer Software provided with RESTRICTED RIGHTS under Federal Acquisition Regulations andagency supplements to them. Use, duplication, or disclosure by the U.S. Government is subject to restrictions as setforth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFAR 252.227-7013et. seq. or subparagraphs (c)(1) and (2) of the Commercial Computer SoftwareRestricted Rights at FAR 52.227-19, asapplicable. Contractor is MicroStrategy, Inc., 1850 Towers Crescent Plaza, Tysons Corner, Virginia 22182. Rights are

reserved under copyright laws of the United States with respect to unpublished portions of the Software.

Copyright Information

All Contents Copyright 2011 MicroStrategy Incorporated. All Rights Reserved.

Trademark Information

MicroStrategy, MicroStrategy 6, MicroStrategy 7, MicroStrategy 7i, MicroStrategy 7i Evaluation Edition,MicroStrategy 7i Olap Services, MicroStrategy 8, MicroStrategy 9, MicroStrategy Distribution Services, MicroStrategyMultiSource Option, MicroStrategy Command Manager, MicroStrategy Enterprise Manager, MicroStrategy Object

Manager, MicroStrategy Reporting Suite, MicroStrategy Power User, MicroStrategy Analyst, MicroStrategy Consumer,MicroStrategy Email Delivery, MicroStrategy BI Author, MicroStrategy BI Modeler, MicroStrategy Evaluation Edition,MicroStrategy Administrator, MicroStrategy Agent, MicroStrategy Architect, MicroStrategy BI Developer Kit,

MicroStrategy Broadcast Server, MicroStrategy Broadcaster, MicroStrategy Broadcaster Server, MicroStrategyBusiness Intelligence Platform, MicroStrategy Consulting, MicroStrategy CRM Applications, MicroStrategy Customer

Analyzer, MicroStrategy Desktop, MicroStrategy Desktop Analyst, MicroStrategy Desktop Designer, MicroStrategyeCRM 7, MicroStrategy Education, MicroStrategy eTrainer, MicroStrategy Executive, MicroStrategy Infocenter,MicroStrategy Intelligence Server, MicroStrategy Intelligence Server Universal Edition, MicroStrategy MDX Adapter,MicroStrategy Narrowcast Server, MicroStrategy Objects, MicroStrategy OLAP Provider, MicroStrategy SDK,MicroStrategy Support, MicroStrategy Telecaster, MicroStrategy Transactor, MicroStrategy Web, MicroStrategy WebBusiness Analyzer, MicroStrategy World, Application Development and Sophisticated Analysis, Best In BusinessIntelligence, Centralized Application Management, Information Like Water, Intelligence Through Every Phone,Intelligence To Every Decision Maker, Intelligent E-Business, Personalized Intelligence Portal, Query Tone, Rapid


3/320

Application Development, MicroStrategy Intelligent Cubes, The Foundation For Intelligent E-Business, The IntegratedBusiness Intelligence Platform Built For The Enterprise, The Platform For Intelligent E-Business, The ScalableBusiness Intelligence Platform Built For The Internet, Industrial-Strength Business Intelligence, Office Intelligence,

MicroStrategy Office, MicroStrategy Report Services, MicroStrategy Web MMT, MicroStrategy Web Services, PixelPerfect, Pixel-Perfect, MicroStrategy Mobile, MicroStrategy Integrity Manager and MicroStrategy Data MiningServices are all registered trademarks or trademarks of MicroStrategy Incorporated.

All other company and product names may be trademarks of the respective companies with which they are associated.Specifications subject to change without notice. MicroStrategy is not responsible for errors or omissions.MicroStrategy makes no warranties or commitments concerning the availability of future products or versions thatmay be planned or under development.

Patent Information

This product is patented. One or more of the following patents may apply to the product sold herein: U.S. Patent Nos.6,154,766, 6,173,310, 6,260,050, 6,263,051, 6,269,393, 6,279,033, 6,567,796, 6,587,547, 6,606,596, 6,658,093,6,658,432, 6,662,195, 6,671,715, 6,691,100, 6,694,316, 6,697,808, 6,704,723, 6,741,980, 6,765,997, 6,768,788,

6,772,137, 6,788,768, 6,798,867, 6,801,910, 6,820,073, 6,829,334, 6,836,537, 6,850,603, 6,859,798, 6,873,693,

6,885,734, 6,940,953, 6,964,012, 6,977,992, 6,996,568, 6,996,569, 7,003,512, 7,010,518, 7,016,480, 7,020,251,7,039,165, 7,082,422, 7,113,993, 7,127,403, 7,174,349, 7,181,417, 7,194,457, 7,197,461, 7,228,303, 7,260,577, 7,266,181,7,272,212, 7,302,639, 7,324,942, 7,330,847, 7,340,040, 7,356,758, 7,356,840, 7,415,438, 7,428,302, 7,430,562,7,440,898, 7,486,780, 7,509,671, 7,516,181, 7,559,048, 7,574,376, 7,617,201, 7,725,811, 7,801,967, 7,836,178, 7,861,161,7,861,253, 7,881,443, 7,925,616, 7,945,584, 7,970,782 and 8,005,870. Other patent applications are pending..

How to Contact Us

MicroStrategy Education Services1850 Towers Crescent Plaza

Tysons Corner, VA 22182Phone: 877.232.7168

Fax: 703.848.8602E-mail: [email protected]

http://www.microstrategy.com/education

MicroStrategy Incorporated1850 Towers Crescent Plaza

Tysons Corner, VA 22182Phone: 703.848.8600

Fax: 703.848.8610E-mail: [email protected]

http://www.microstrategy.com


4/320


5/320

2011 MicroStrategy, Inc. 5

TABLEOFCONTENTS

Preface Course Description.................................................................... 11

Who Should Take this Course............................................... 13Course Prerequisites ............................................................. 13

Follow-Up Courses ................................................................ 13

Course Objectives ................................................................. 14

About the Course Materials ......................................................... 15

Content Descriptions ............................................................. 15

Learning Objectives............................................................... 15

Lessons ................................................................................. 15

Opportunities for Practice ...................................................... 16

Typographical Standards....................................................... 16

Other MicroStrategy Courses ...................................................... 19

Core Courses......................................................................... 19

1. Introduction to HighPerformance

Lesson Description ................................................................... 21

Lesson Objectives ................................................................. 22

The High Performance Initiative .................................................. 23

Steps Taken in the High Performance Direction.................... 24

Best Practices Lead to Optimal Performance........................ 26

MicroStrategy High Performance Highlights................................ 27

Components of Performance....................................................... 29

Course Structure.................................................................... 30

Lesson Summary......................................................................... 33


6/320


7/320


8/320

Table of Contents Deploying MicroStrategy High Performance BI

8 2011 MicroStrategy, Inc.

Web Server Configuration ......................................................... 143

JVM Settings........................................................................ 143

MicroStrategy Web Pool Sizes ............................................ 144

Using a Separate Web Server for Static Content ................ 146

Logging and Statistics.......................................................... 147

JavaScript ............................................................................ 147Lesson Summary....................................................................... 148

5. Data Presentation Lesson Description ................................................................. 151

Lesson Objectives ............................................................... 152

High Performance Reports ........................................................ 153

Report Execution Flow......................................................... 153

Report Configuration Techniques to Optimize Performance154

High Performance Dashboards ................................................. 159

The Dashboard Execution Flow........................................... 159Dataset Techniques to Optimize Performance.......................... 162

Dashboard Data Preparation Steps..................................... 162

Reducing the Number of Datasets in a Dashboard ............. 163

Reducing the Amount of Data in a Dataset.......................... 167

Using Intelligent Cubes........................................................ 169

Design Techniques to Optimize Performance ........................... 172

General Performance Topics............................................... 172

DHTML Performance Topics ............................................... 184

Flash Performance Topics................................................... 187

Optimizing Performance for Mobile ........................................... 194

Execution Workflow for Mobile Devices............................... 194

Improving Execution Time for a Mobile Document or Dashboard195

General Design Best Practices for Documents and Dashboards200

MicroStrategy Mobile 9.2.1 Performance Optimizations...... 201

Lesson Summary....................................................................... 203

6. Data WarehouseAccess

Lesson Description ................................................................. 207

Lesson Objectives ............................................................... 208

Introduction to Data Warehouse Access ................................... 209

Database Query Performance ............................................. 209

SQL Generation Algorithm................................................... 210

Data Architecture Optimizations ................................................ 213

Report and Schema Design Optimizations................................ 219


9/320


10/320

Table of Contents Deploying MicroStrategy High Performance BI

10 2011 MicroStrategy, Inc.


11/320


PREFACE

Course Description

After companies reach their business goals, the next area to focus on isincreasing productivity. MicroStrategy software is designed and implemented

with performance in mind and delivers top performance to scale.MicroStrategy out-of-the-box software is optimized for a typical use casereflecting the most frequently requested hardware and functionality. Sometuning may be required to achieve optimal performance.

This course includes the best practices of implementing MicroStrategy BI andprovides the information necessary to build your applications withperformance in mind, tuning MicroStrategy for optimal performance.

In this course, you will first be introduced to the MicroStrategy HighPerformance initiative, the main factors behind it, and the most important

outcomes of this initiative to date. Then, you will learn about the componentsof performance and about recommendations on how to deploy a HighPerformance MicroStrategy platform by using features and settings in the fivemain areas of a BI system: In-memory caching and cubes, data transfer, systemconfiguration, reports, dashboards, mobile applications and data sourceaccess.


12/320


13/320

Deploying MicroStrategy High Performance BI Preface

2011 MicroStrategy, Inc. Who Should Take this Course 13

Who Should Take this Course

This course is designed for:

Administrators

Project Managers

Developers

Architects

Course Prerequisites

Before starting this course, you should know all topics covered in the followingcourses:

MicroStrategy Administration: Configuration and Security

MicroStrategy Administration: Application Management

MicroStrategy Report Services: Dynamic Dashboards

Follow-Up CoursesAfter taking this course, you might consider taking the following course:

MicroStrategy Mobile for Apple iPad and iPhone


14/320

Preface Deploying MicroStrategy High Performance BI

14 Course Objectives 2011 MicroStrategy, Inc.

Course Objectives

After completing this course, you will be able to:

Describe the MicroStrategy High Performance initiative. Understand themain impact to your environment provided by the MicroStrategy initiative.Learn the high-level topics covered by the chapters of this course. (Page 22)

Describe different levels of caching in a MicroStrategy environment.Understand best practices for leveraging caching for high performance.(Page 36)

Understand the different instances of data transfer and their impact on theBI system performance. Describe key network concepts and networkperformance recommendations. Apply best practices techniques when

working with Distribution Services. (Page 92)

List the components of performance, understand the main performancerecommendations for server specification, system configuration, and theWeb environment. (Page 114)

Describe report and dashboard execution flow. Understand therecommendations for designing high performance reports and dashboards.(Page 152)

Apply the learned skills to optimize report queries to reduce the database

execution time. (Page 208)

Understand performance testing, so you can choose the right type of tests forspecific performance requirements. Design, implement, execute, andanalyze performance tests in a correct way. (Page 254)


15/320


2011 MicroStrategy, Inc. About the Course Materials 15

About the Course Materials

This course is organized in to lessons and reference appendices. Each lessonfocuses on major concepts and skills that help you to better understand

MicroStrategy products and use them to implement MicroStrategy projects.The appendices provide you with supplemental information to enhance yourknowledge of MicroStrategy products.

Content Descriptions

Each major section of this course begins with a Description heading. TheDescription introduces you to the content contained in that section.

Learning Objectives

Learning objectives enable you to focus on the key knowledge and skills youshould obtain by successfully completing this course. Objectives are providedfor you at the following three levels:

CourseYou will achieve these overall objectives by successfullycompleting all the lessons in this course. The Course Objectives heading inthis Preface contains the list of course objectives.

LessonYou will achieve these main objectives by successfully completingall the topics in the lesson. You can find the primary lesson objectivesdirectly under the Lesson Objectives heading at the beginning of eachlesson.

Main TopicYou will achieve this secondary objective by successfullycompleting the main topic. The topic objective is stated at the beginning ofthe topic text. You can find a list of all the topic objectives in each lessonunder the Lesson Objectives heading at the beginning of each lesson.

Lessons

Each lesson sequentially presents concepts and guides you with step-by-stepprocedures. Illustrations, screen examples, bulleted text, notes, and definitiontables help you to achieve the learning objectives.


16/320


16 About the Course Materials 2011 MicroStrategy, Inc.

Opportunities for Practice

A Workshop is a reinforcement and assessment activity that follows two ormore lessons. Because a Workshop covers content and applied skills presentedin several lessons, it is a separate section on the level of a lesson.

The following sections within lessons provide you with opportunities toreinforce important concepts, practice new product and project skills, andmonitor your own progress in achieving the lesson and course objectives:

Review

Case Study

Business Scenario

Exercises

Typographical Standards

The following sections explain the font style changes, icons, and different typesof notes that you see in this course.

Actions

References to screen elements and keys that are the focus of actions are in boldArial font style. The following example shows this style:

Click Select Warehouse.

Code

References to code, formulas, or calculations within paragraphs are formatted

in regular Courier.New font style. The following example shows this style:

Sum(sales)/number of months


17/320


2011 MicroStrategy, Inc. About the Course Materials 17

Data Entry

References to literal data you must type in an exercise or procedure are in boldArial typeface. References to data you type in that could vary from user to useror system to system is in bold italic Arial font style. The following example

shows this style:

Type copy c:\filename d:\foldername\filename.

Keyboard Keys

References to a keyboard key or shortcut keys are in uppercase letters in boldArial font style. The following example shows this style:

Press CTRL+B.

New Terms

New terms to note are in regular italic font style. These terms are defined whenthey are first encountered in the course material. The following example showsthis style:

The aggregation levelis the level of calculation for the metric.

Notes and Warnings

A note icon indicates helpful information.

A warning icon calls your attention to very important information thatyou should read before continuing the course.

Heading Icons

The following heading icons are used to indicate specific practice and reviewsections:

Precedes a Review section


18/320


18 About the Course Materials 2011 MicroStrategy, Inc.

Precedes a Case Study

Precedes a Business Scenario

Precedes Exercises


19/320


2011 MicroStrategy, Inc. Other MicroStrategy Courses 19

Other MicroStrategy Courses

Core Courses Implementing MicroStrategy: Development and Deployment

MicroStrategy Architect: Project Design Essentials

MicroStrategy Desktop: Advanced Reporting

MicroStrategy Desktop: Reporting Essentials

MicroStrategy Mobile for Apple iPad and iPhone

MicroStrategy Report Services: Document Essentials

MicroStrategy Report Services: Dynamic Dashboards

MicroStrategy Web for Professionals

MicroStrategy Web for Reporters and Analysts

All courses are subject to change. Please visit the MicroStrategy website for the latest educationofferings.


20/320


20 Other MicroStrategy Courses 2011 MicroStrategy, Inc.


21/320


1

INTRODUCTIONTOHIGH

PERFORMANCE

Lesson Description

This lesson provides an introduction to the MicroStrategys High Performanceinitiative. In this lesson, you will learn about the factors that drove

MicroStrategy to pursue the High Performance initiative and about thebenefits that the results of this initiative can bring to your environment. Inaddition, you will learn about the MicroStrategy components of performanceand how this course is structured.


22/320

Introduction to High Performance Deploying MicroStrategy High Performance BI1

22 Lesson Objectives 2011 MicroStrategy, Inc.

Lesson Objectives

After completing this lesson, you will be able to:

Describe the MicroStrategy High Performance initiative. Understand the mainimpact to your environment provided by the MicroStrategy initiative. Learnthe high-level topics covered by the chapters of this course.

After completing the topics in this lesson, you will be able to:

Understand the factors behind MicroStrategys push for developing theHigh Performance initiative. (Page 23)

List the most important achievements of the MicroStrategy HighPerformance initiative. (Page 27)

Understand how this course is structured based on the MicroStrategycomponents of performance. (Page 29)


23/320

Deploying MicroStrategy High Performance BI Introduction to High Performance 1

2011 MicroStrategy, Inc. The High Performance Initiative 23

The High Performance Initiative

After completing this topic, you will be able to:

Understand the factors behind MicroStrategys push for developing the HighPerformance initiative.

Performance is a combination of speed and scale. Speed is important because itaffects the user experience. Speed is typically measured as user wait time orresponse time. Scale is important because it determines how many people cansubmit requests or reports to a business intelligence (BI) system, how manyreports can be supplied by the system, and how much data can be accessed.Most BI technologies can do one or the other, but not both effectively.MicroStrategy has always been focused on high performance, with thecombination of high speed at high scale.

Recent third-party reports have documented increased dissatisfaction with theBI system performance1. While MicroStrategy is recognized as the leader inhigh performance, the company continues to re-evaluate its performance toensure customers are achieving the highest performance possible with theMicroStrategy platform.

MicroStrategy Positioning on Performance Matters


24/320


24 The High Performance Initiative 2011 MicroStrategy, Inc.

Starting in 2007, MicroStrategy surveyed customer applications. It quicklybecame clear that there is a continuing trend and inherent demand towardsimproved BI performance. This insight led to the creation of a dedicated highperformance initiative. Based on the results of this survey, the goals of thisinitiative were to:

Deliver up to 10x faster BI applicationsTodays BI applications mustefficiently access terabytes of data. Since most competing BI tools do notinclude performance acceleration engines, average BI application queryresponse times often range from 10 seconds to one minute or more.MicroStrategys high performance initiative will set a new performancestandard, aiming to deliver up to 10x faster query response time at any datascale.

Provide faster than 3-second response time for most predictablequeries and analysesMicroStrategy research has found that mostbusiness queries are predictable. Business people often run similar reports

on a daily, weekly, or monthly basis to understand operationalperformance. Using its memory technology to cache computations andplace the results into server memory, MicroStrategy can dramaticallyaccelerate repetitive operational reports as well as most subsequentanalyses.

Provide faster than 5-second response time for the majority of ad hocqueriesBy optimizing and accelerating all aspects of its BI platform, fromSQL generation to SQL execution to data rendering, MicroStrategy seeks toenable 50% of all ad hoc queries to return in less than 5 seconds.

Steps Taken in the High Performance Direction

MicroStrategys high performance initiative includes the formation of its HighPerformance and Scalability Lab, the creation of a dedicated PerformanceEngineering team, and specific R&D efforts solely focused on providingMicroStrategy customers with the highest levels of performance for BIapplications of all sizes.


25/320


2011 MicroStrategy, Inc. The High Performance Initiative 25

MicroStrategy High Performance and Scalability Lab

MicroStrategy built a state-of-the-art research laboratory equipped with thelatest database hardware, software, and performance testing tools.MicroStrategys multi-million dollar research laboratory has more than 100

servers running tests 24x7 on all supported platforms, including Linux,Microsoft Win64, Oracle Solaris, IBM AIX, and HP-UX.MicroStrategys standard benchmark test lines have two IntelNehalem-based Intelligence Servers with eight cores and 144 GB RAM eachfor Intelligence Server, four Web servers with eight cores each, eight cores forthe MicroStrategy metadata, and eight cores to simulate user load. The entiretest station contains 112 cores in total.

In addition, MicroStrategy currently has at least two enterprise-class databaseservers to support its testing efforts, including a four-node Oracle RACcluster with 96 cores in total, 256 GB RAM on each node, and 18 TB of usable

space; as well as a Teradata 13 5555H server, with eight cores, 48 GB RAM, and18 TB of usable space.

MicroStrategy Performance Engineering Team

MicroStrategy has a dedicated team of performance engineers who workclosely with selected customers to understand and document the currentperformance development of BI system configurations. This team runshundreds of performance tests each week to identify and eliminate system

bottlenecks and to build accurate system performance profiles. In addition, theteam is authoring technical notes, articles, and best practice documents to helpcustomers maximize performance of their BI applications.

Professional Services

MicroStrategy consulting services can help customers build high performanceBI applications as well as audit and fine-tune existing applications through:

Capacity PlanningMicroStrategys capacity planning-related services

involve a holistic approach to matching existing infrastructure to currentand growing BI requirements, including a quantitative assessment ofcurrent system capacity and potential bottlenecks.

High Performance TuningMicroStrategys high performance-relatedservices involve a methodical approach to tuning MicroStrategy softwarefor maximum performance, including all components of the BI ecosystemsuch as relational database management systems.


26/320


26 The High Performance Initiative 2011 MicroStrategy, Inc.

Best Practices Lead to Optimal Performance

For most business intelligence customers, their initial requirement is to satisfytheir functional requirements and solve the problem at hand. MicroStrategysBI platform is recognized as one of the most complete and technically

advanced in the industry.

The second requirement is stellar performance. After companies reach theirbusiness goals, they want to reach them faster and do more in the sameamount of time for increased productivity. MicroStrategys software isdesigned and implemented with performance in mind and delivers topperformance at scale. MicroStrategy customers implement a wide range of BIfunctionality in their applicationsfrom the mainstream to the specialized,from the conservative to the innovative use of BI. It is important to recognizethat depending on the application, different optimizations are needed foroptimal performance.

MicroStrategy software often allows multiple ways to implement a certainreporting requirement providing our customers with the utmost flexibility.Designing BI applications with performance in mind improves applicationperformance. MicroStrategy out-of-the-box software is optimized for a typicaluse case reflecting the most frequently requested hardware and functionality.Some additional tuning may be required to achieve optimal performance.


27/320


2011 MicroStrategy, Inc. MicroStrategy High Performance Highlights 27

MicroStrategy High Performance Highlights


List the most important achievements of the MicroStrategy High Performanceinitiative.

By optimizing and accelerating all aspects of its BI platform, from SQLgeneration to data rendering, MicroStrategy seeks to enable 50% of all ad hocqueries to return in less than 5 seconds. Specific areas of R&D include:

Faster Database QueriesMicroStrategys ROLAP technology leveragesdatabase engines for complex calculations and data joins, employing thelatest techniques to reduce processing time and optimize overall query

performance, including: database-specific optimizations, workloadbalancing across multiple databases using improved aggregate awareness,and reduction on the number of SQL passes for sophisticated analyses toimprove database query time by 75%.

Larger Data CachesMicroStrategy is further enhancing its data loadingalgorithms. The latest enhancements in MicroStrategy show performanceimprovements of over 30% in cube data load time.

Dynamic Sourcing OptimizationsThese optimizations improve systemperformance by increasing the number of reports that can be directed to

Intelligent Cubes and result in dynamic sourcing of reports from IntelligentCubes to be up to 80% faster.

Faster Metadata ResponsesOptimizations have been made to enablefaster project start-up and administration. Projects are loaded in parallelrather than serially, reducing the Intelligence Server startup time.Additional enhancements that have resulted in faster metadata operationsare as follows:

Configuration Wizard has been enhanced to support the process ofupgrading MicroStrategy projects and environments to the most recentversion of MicroStrategy.

Architect operations, such as dragging tables from the warehouse intothe Graphical Architect interface and schema manipulations, are up to20% faster.

Object migration processes have been optimized to support migration ofobjects across unrelated projects.


28/320


28 MicroStrategy High Performance Highlights 2011 MicroStrategy, Inc.

A new algorithm optimizes the processing of security filters.

Project upgrade process is significantly faster.

Reduced threshold XML size results in less data to transfer.

New index optimizations on metadata tables reduce the metadata accesstime.

Health Center Integration with High Performance InitiativeHealthCenter proactively collects customer data from customer systems. Based onthis data, Health Center identifies performance bottlenecks andrecommends steps to resolve those bottlenecks. This integration brings youcloser to MicroStrategy Technical Support, while providing you a betterunderstanding of your environment.

For more information on Health Center, refer to the MicroStrategyAdministration: Configuration and Security course. Faster Web InteractivityThe objective is to continue to provide the

richest user interactivity with rapid performance on Web browsers andmobile devices by minimizing data transfers, streamlining processing,optimizing page construction, rendering pages faster, and loadingon-demand Javascript.

Faster DashboardsDashboardApps can contain an entire days worth ofinformation within a single dashboard for even the most novice BI users.Further enhancements include on-demand data transfer technology,enhanced data compression algorithms, and optimized rendering

algorithms.

Providing fast speed and greater capacity to customers deployments have beenand continue to be at the core of MicroStrategy technology.


29/320


30/320


30 Components of Performance 2011 MicroStrategy, Inc.

Course Structure

The remainder of this course is divided into the following chapters:

Caching and Intelligent Cubes

Data transfer

System architecture and configuration

Data presentation

Data warehouse access

Performance testing methodology

Workshop

Caching and Intelligent Cubes

MicroStrategys memory technology is engineered to meet the increaseddemand for higher BI performance, which is driven by the rapid expansion ofboth data volumes and the number of BI users in organizations acrossindustries. MicroStrategy accelerates performance by pre-calculatingcomputations and placing the results into its memory acceleration engine todramatically improve real-time query performance. In this chapter, you willlearn about the main performance recommendations when using caches and

Intelligent Cubes.

Data Transfer

Data transfers over one or more networks are a very important component of aBI implementation. A slow or poorly tuned network performance in any ofthose transfers will translate into poor performance from a report ordashboard execution perspective. In this chapter, you will learn about the mainrecommendations to improve network performance and distribution services

executions.


31/320


2011 MicroStrategy, Inc. Components of Performance 31

System Architecture and Configuration

Successful BI applications accelerate user adoption and enhance productivity,resulting in demand for more users, data, and reports. MicroStrategy providesthe ability to adapt quickly to constant changes and evolve along with business

requirements. MicroStrategy Intelligence Server has been proven in real-worldscenarios to deliver the highest performance at scale with the fewest serversand minimum IT overhead. This lesson introduces you to important conceptsregarding tuning your MicroStrategy platform, including recommendations forthe Intelligence Server, Web server, client machines, and configuringclustering.

Data Presentation

Dashboards provide graphical, executive views into KPIs, enabling quickbusiness insights. MicroStrategy enables higher performing dashboards,averaging 30-45% faster execution and interactivity. Using new compressionmethods, MicroStrategy dashboards have a smaller footprint than everbeforeup to 55% smallerresulting in faster delivery using less networkbandwidth. Dashboards deliver ever more analysis and data for end-users. Inthis chapter, you will learn about the report and dashboard execution flows,and about the key recommendations for data set and design optimization forpresenting data in Web and mobile devices. These series of design techniqueswill help you develop reports and dashboards, while keeping performance inmind.

Data Warehouse Access

High performance BI starts with optimizing SQL queries to retrieve resultsfrom the database as quickly as possible. BI performance is dependent largelyon the time that the queries take to execute in the database. An averagereporting request usually takes 40 seconds to complete, out of which 34seconds, or 85% of the query time, is spent executing in the database.Therefore, it is critical to optimize report queries to reduce database executiontime. In this chapter, you will learn about the main recommendations toretrieve source data for reports and documents in an efficient manner.


32/320


32 Components of Performance 2011 MicroStrategy, Inc.

Performance Testing Methodology

Questions about the performance of the system need to be answered withvarious performance testing. Unlike the straightforward feature testing thatverifies the functional correctness of the product, performance testing needs

special design and is more complex to execute and analyze. The goal of thislesson is to provide you with a good understanding of performance testing, soyou can choose the right type of tests for specific performance requirements.

Workshop

The three-hour workshop will enable you to experience the performance gainsof redesigning a MicroStrategy dashboard, by following the methodologypresented in this course.

This course manual includes the best practices of implementing MicroStrategyBI and provides the information necessary to build your applications withperformance in mind and to tune MicroStrategy for optimal performance.


33/320


2011 MicroStrategy, Inc. Lesson Summary 33

Lesson Summary

In this lesson, you learned:

MicroStrategy continues to remain focused on delivering highperformance, with the combination of high speed at high scale.

Starting in 2007, MicroStrategy surveyed customers and found out therewas an inherent demand towards improved BI performance. This insightled to the creation of a dedicated High Performance initiative.

The goals of the High Performance initiative are to deliver up to 10x fasterBI applications, provide faster than three-second response time for mostpredictable queries and analyses, and provide faster than five-secondresponse time for the majority of ad hoc queries.

MicroStrategys High Performance initiative includes the formation of itsHigh Performance and Scalability Lab, the creation of a dedicatedPerformance Engineering team, and specific R&D efforts solely focused onproviding MicroStrategy customers with the highest levels of performancefor BI applications of all sizes.


34/320


34 Lesson Summary 2011 MicroStrategy, Inc.


35/320


2

CACHINGANDINTELLIGENT

CUBES

Lesson Description

In this lesson, you will learn about the concept of computational distance andthe advantages of the 64-bit applications. Next, you will learn how to reduce

the computational distance and increase performance by applying best practicetechniques to the different levels of caching in a MicroStrategyenvironmentreport, document, element, and object caching.

Additionally, you will learn about Intelligent Cubes, understand the process ofpublishing cubes in memory, and identify the usage scenarios that representbest candidates for taking advantage of Intelligent Cubes. You will aso learnabout the memory required for Intelligent Cubes in Intelligence Server, and thetuning techniques that must be applied when designing cubes and reports.Finally, you will be presented with the performance considerations whendesigning reports that utilize Intelligent Cubes.


36/320

Caching and Intelligent Cubes Deploying MicroStrategy High Performance BI2

36 Lesson Objectives 2011 MicroStrategy, Inc.

Lesson Objectives

After completing this lesson, you will be able to:

Describe different levels of caching in a MicroStrategy environment.Understand best practices for leveraging caching for high performance.

After completing the topics in this lesson, you will be able to:

Understand the concept of computational distance and its impact on thesystem response time. (Page 38)

Understand the importance of 64-bit systems in reducing the computationaldistance. (Page 40)

Define caching, list and define cache types, and set cache properties. (Page41)

Understand the best practices to leverage report caching. (Page 42)

Understand the best practices to leverage document caching. (Page 50)

Understand object caching and the best practices to leverage object caching.(Page 53)

Understand element caching and the best practices to leverage elementcaching. (Page 54)

Understand the rules of thumb to size caches in your system. (Page 57)

Understand the Intelligent Cubes functionality and how it complements theMicroStrategy caching strategy. (Page 59)

Understand the process of publishing an Intelligent Cube. (Page 61)

Identify the usage scenarios that represent best candidates to takeadvantage of Intelligent Cubes. (Page 68)

Understand the memory requirements for Intelligent Cubes and the tuningtechniques that must be applied when designing cubes and reports. (Page73)


37/320

Deploying MicroStrategy High Performance BI Caching and Intelligent Cubes 2

2011 MicroStrategy, Inc. Lesson Objectives 37

Refresh cube data using the different options available in the incrementalrefresh feature. (Page 78)

Understand the performance considerations when designing reports toutilize Intelligent Cubes. (Page 85)


38/320


38 Computational Distance 2011 MicroStrategy, Inc.

Computational Distance


Understand the concept of computational distance and its impact on thesystem response time.

Any BI system consists of a series of processes and tools that take raw data atthe very bottomat the transactional level in a databaseand use varyingtechnologies to transform that data to the finished answer that the user needs.At every step along the way, some kind of processing is done on the followingcomponentsthe database, network, BI application, or the browser.

The concept of computational distance refers to the length in terms ofsystems, transformations, and other processes that the data must undergofrom its lowest level, all the way to being rendered on a browser as shown inthe image below:

Computational Distance

The longer the computational distance is for a given report, the longer it willtake to execute and render. The preceding image shows a hypothetical exampleof a report that runs in 40 seconds. Each processing step on that report, such asaggregation, formatting, and rendering, adds to the report's computationaldistance, increasing the report overall execution time.


39/320


2011 MicroStrategy, Inc. Computational Distance 39

Computational distance offers a useful framework from a performanceoptimization perspective because it tells us that to improve the performance ofa report or dashboard, you must focus on reducing its overall computationaldistance. The following image shows different techniques such as caching,cubes, and aggregation that can be used to optimize performance for the 40second hypothetical report:

Reducing the Computational Distance of a Report

This lesson focuses on the following two key computational distance reductiontechniques offered in the MicroStrategy platformcaching and IntelligentCubes.


40/320


40 The Importance of 64-Bit Systems 2011 MicroStrategy, Inc.

The Importance of 64-Bit Systems


Understand the importance of 64-bit systems in reducing the computationaldistance.

The server memory (RAM) is a critical resource for effective performanceoptimization strategies involving caching and Intelligent Cubes. A system withlarge memory resources enables architects to design more caches andIntelligent Cubes that can source an increasingly larger set of reports anddashboards. In turn, as more reports and dashboards are sourced by eithercaches or Intelligent Cubes, the average user wait time is reduced. Thefollowing image illustrates the increase in the % of reports sourced fromin-memory BI. Average response time drops with an increase in the amount ofserver RAM:

Average Wait Time Versus In Memory BI Usage

By dramatically increasing the amount of addressable RAM in a server, the64-bit technology is the ultimate high performance enabler. MicroStrategyproject implementations will have significantly more opportunities for

performance optimization using both caching and Intelligent Cubes when theyare deployed using 64-bit server technologies.


41/320


2011 MicroStrategy, Inc. Introduction to Caching 41

Introduction to Caching


Define caching, list and define cache types, and set cache properties.

Cachingis the retention of a recently used object in memory for the purpose ofimproving query response time for future requests. Caching enables users toretrieve results from in-memory stored files rather than executing queriesagainst a database. The MicroStrategy architecture offers several caching levelswithin the different functional layers. This section covers the key cachinglevelsreport, document, element, and object caching from the perspective ofperformance optimization.

The following image displays the different levels of caching in theMicroStrategy platform:

Cache Levels in the MicroStrategy Environment


42/320


42 Report Caches 2011 MicroStrategy, Inc.

Report Caches


Understand the best practices to leverage report caching.

Report Caching Overview

MicroStrategy report caches store all report results in Intelligence Server forfast retrieval. Because they reduce computational distance to a bare minimum,reports that can execute against caches will have better response times than

reports that execute against cubes, aggregate tables, or fact tables on the datawarehouse. Reports that are very frequently run, and have fixed andpredictable characteristics are perfect candidates for using MicroStrategy'scaching technology. User experience can be further enhanced by schedulingthe reports to run during batch window times. This way, by pre-caching reportsin batch, even the first report execution wait time can be minimized.

Report Cache Flow

The following image illustrates how report caching works:How Report Caching Works?


43/320


2011 MicroStrategy, Inc. Report Caches 43

The basic workflow is as follows:

1 A user runs a report for the first time.

2 The report runs against the data warehouse database.

3 Intelligence Server caches the result set and also returns them to the user.

4 A second user subsequently runs the same report.

5 Intelligence Server searches its report caches.

6 Intelligence Server uses the report cache matching algorithm to determineif it can reuse an existing cache.

7 If the report cache matching algorithm shows that the cache is not valid,

Intelligence Server queries the data warehouse. However, if the cache isdetermined to be valid, Intelligence Server retrieves the result set from thecache.

The greater the number of long-running reports in the project, the greater theimpact is on the performance. However, with report caching, you can reducethe number of report requests that have to be processed against the datawarehouse and thereby improve processing time. The following imageillustrates how you can achieve better performance through report caching:

Report Caching Example


44/320



In this example, 20 users need to view the same data from the same reporteach day. This report ranks product sales to display the best-selling andworst-selling items, and it runs against a data warehouse table that containstwo million rows. Given the qualifications it is performing and the volume ofdata in the table, this report takes 10 minutes to run. If each user queries thedata warehouse for the result set, collectively, their requests consume over twohours of processing time. However, if you configure caching for this report,only the first user who runs the report queries the data warehouse. The other19 users can access the cache created by the first user. The processing time isreduced to 10 minutes plus the time for cache retrieval, which in most casestakes only a few seconds.

Being able to cache report results ensures that redundant queries are not sentto the data warehouse and raises the level of performance as it reducesprocessing time.

Report Caching Best Practices

In an ideal scenario, all reports that a user in a system could possibly needwithin any given day should be pre-cached. This type of implementation wouldpractically guarantee very fast report response times for all users in the system.However, more practically, there are three key factors that make pre-cachingevery single report an impossibility:

the degree of personalization and security for the reports

the amount of RAM available on the Intelligence Server

the available batch window time

In general, MicroStrategy has found that because of the above three factors, ina typical BI implementation about 10% of reports will benefit from caching.The rest of this subsection explains in detail some important considerationsand the best practices to leverage report caching for performance optimization.

Enabling Caching for Frequently Used Reports Only

From a performance perspective, given that not every report can be cached, itis important to prioritize and choose which reports are good candidates to becached. The frequency of use, especially for shared reports is a goodprioritization criteria. Report cache creation and usage consume serverresources such as RAM and disk space. As a result, MicroStrategy recommendsusing true cache reuse as a criteria for choosing which reports to cache.


45/320



Typical MicroStrategy implementations involve shared reports that aremanaged through a development, test, and production cycle; and personalreports that are created by users directly in the production environment. Giventhe typical configuration described above, the following report cachingrecommendations should be considered:

1 Caching all reports at the project level is only practical for small projectsinvolving limited personal object creation.

2 Not all shared reports and documents are good caching candidates if thediversity of content to be viewed by users is high. For example, a promptedreport that is heavily personalized by each user will not be a good candidatefor caching, but a prompted report in which 90% of job executions use thesame prompt answers could be a candidate for caching.

3 Caching should be disabled at the report object template level for report

wizard, report builder, and blank report templates. These templates areused to create and save personal reports.

4 The Maximum number of caches setting in the Project Configuration Editorenables administrators to control cache creation per project. It isrecommended to use this setting to limit the number of caches allowed.

You can use MicroStrategy Enterprise Manager to analyze system usage dataand identify the reports that fall within the frequently used shared reportscategory and will therefore be good caching candidates.

Disabling Caches for Highly Prompted Reports

Caching for prompted reports is tied to specific prompt answers. This meansthat for a prompted report cache to be reused, the exact same prompt answeras the one with which the cache was created must be provided by the user. Anyother answer will cause Intelligence Server to reject the cache and issue a SQLstatement against the data warehouse. For a highly prompted report to beeffectively cached, every possible combination of prompt answers would needto be pre-cached, making this a highly impractical, if not impossible,proposition given RAM and batch window limitations.


46/320


47/320



4 Click OKto save your settings.

XML Caching

You can view the sizes of both the binary and XML caches for a givenreport in the Cache Monitor.

Allocating Enough Memory for Report Caches

Report caches are first created in memory within Intelligence Server. They arelater backed up to disk and reloaded into memory when accessed.

If Intelligence Server memory is insufficient to hold all new requested reportcaches, the least recently used caches are unloaded automatically frommemory to make space for the new caches.


48/320



The process of loading a cache from disk can add significant delays to the userwait time. Large cache sizes take longer time to load and may displace othercaches that will need to be moved back from memory to disk to make space.This process degrades performance both for the user who requested the cachedreport and for users who would have been able to hit the now displaced caches.

Cache Load/Unload From Disk

Based on the above considerations, from a performance perspective it isimportant to run MicroStrategy in a 64-bit environment whenever possiblegiven the significantly larger amounts of addressable memory it offers. If this isnot possible, you must at least ensure that all available Intelligence Servermemory is used optimally.

Memory capacity planning for report caches can be extremely challenging, but

at a high level, a good estimate of the amount of memory needed to effectivelycache a given set of reports can be obtained as follows:

1 Determine which reports should be cached on a given project. For example,you can use Enterprise Manager to determine the most frequently usedshared reports as discussed earlier.

2 Generate caches for those reports and record cache sizes (both binary andXML) using Cache Monitor. Note that in the case of prompted reports, youwill need to generate caches for each prompt answer you wish to cache.

3 If you plan to generate some user-specific report caches, you must add thisvariable into your total cache memory utilization estimates.

After you have estimated the total amount of RAM that will be used for reportcaching you can define it in Intelligence Server.


49/320



To define the memory available for report caching:

1 In Desktop, launch the Project Configuration Editor.

2 In the Project Configuration Editor, in the Categories list, expand Caching,expandResult Caches, and select Storage.

3 Under Datasets, in the Maximum RAM usage (MBytes) text box, type theamount of Intelligence Server RAM you want to allocate for report cachestorage.

The maximum value you can define is 64 GB.4 Click OKto save your settings

Maximum RAM Usage Setting

Practicing Cache Maintenance

Implementing the right cache strategy is very important for a successful BIdeployment. The list below contains recommendations for cache maintenance:

Deleting Obsolete CachesEstablish a cache deletion/regenerationstrategy to ensure users can access the most recent data available. UseSchedule Administrative Tasks in Desktop to schedule cache deletion.

Allocating Enough Disk Space for CachesEnsure enough disk space isallocated to store all cache files. Otherwise, new caches will not be created.In addition, in a clustered environment, ensure that all nodes have thecorrect read and write permission to the location where caches reside.


50/320


50 Document Caching 2011 MicroStrategy, Inc.

Document Caching


Understand the best practices to leverage document caching.

Document Caching Overview

Report Services documents can be cached to improve system performance andmaintain a low memory footprint of the Intelligence Server. Document cachesare created at run time or on a schedule and behave in a manner similar to

report caches. At run time, document caches are created only in MicroStrategyWeb. When you execute a prompted document in MicroStrategy Web, an XMLdocument cache is generated. When you reprompt the document, a new XMLcache is generated with the data reflecting new prompt answers.

To enable document caching, you must first enable report caching. For aspecific document cache to be generated, all underlying dataset caches mustexist or be generated at the document run time. If caching for one of thedatasets is disabled or fails for any reason, the document cache for thatdocument will notbe generated.

Unlike report caches, which are created only as binary files, document cachescan exist in the following formats: PDF, Excel, XML/Flash, and HTML. Thefollowing image shows the document caching options in the ProjectConfiguration Editor:

Document Caching - Creation Subcategory


51/320


2011 MicroStrategy, Inc. Document Caching 51

For Dashboards, the creation of new caches is basically based on the differentviews you choose on the interfaceEditable, Interactive, Flash, and so forth.This logic is followed for any type of documentssingle-layout andmulti-layout, and single panel and multi-panel. Having these differences doesnot change the cache creation behavior.

In multi-layout dashboards, a cache for Flash view is created for eachlayout. However, switching between layouts in Flash view will notincrease the hit count of the caches as it would if you do the same thingin Editable or Interactive view. This is due to an optimization for Flashcontent, where the Flash content is kept in the Intelligence Servermemory for faster processing and rendering.

In addition to the document cache storage location, you can specify themaximum RAM usage and the maximum number of caches, as shown in thefollowing image:

Document Caching - Storage Subcategory


52/320


52 Document Caching 2011 MicroStrategy, Inc.

Document Caching Best Practices

In terms of performance, the most important considerations for documentcaching are listed below:

Unless there is a low number of relatively simple documents in a project, itis not recommended to enable project-level document caching.

Allocate sufficient memory for document caches to avoid file swapping,which negatively affects performance. To determine how much memorymight be needed, you must consider the cache format that will be created:

XML caches consume more memory than report binary caches.Depending on the operating system, the memory consumption can varyfrom two to four times the size of the file. For example, in anIntelligence Server running on Oracle Solaris, a 10 KB file cache will use40 KB of RAM.

Binary, PDF, and Excel caches usually use the same amount of memoryas their file sizes

Avoid generating document caches on a per-user basis.

Schedule the documents to run during the batch window. By pre-cachingthese documents, the wait time for the first user accessing these documentsis completely eliminated.

Avoid enabling the Automatic option to send documents to History List.When you send the document to a History List, a cached document instance

is stored directly in the user's History List. This cache will not be reusedduring regular execution of the document. Therefore, if you send thedocuments automatically to History List, the execution of these documentswill not create caches that can be reused across users.


53/320


2011 MicroStrategy, Inc. Object Caches 53

Object Caches


Understand object caching and the best practices to leverage object caching.

Object Caching Overview

Anobject cache is a recently used object definition stored in the memory ofDesktop and Intelligence Server. You can create object caches for bothapplication and schema objects. The example below illustrates how object

cache works:

1 A user opens the Report Editor.

2 The collection of objects displayed in the Report Editor make up thereports definition. If no object cache for the report exists in the Desktop orthe Intelligence Server memory, the object request is sent to the metadata.

3 The report object definition is retrieved from the metadata and displayed tothe user in the Report Editor.

4 An object cache is created in memory of the Intelligence Server and theDesktop machines. If the same user requests the same object, the objectcache on the Desktop machine satisfies that request. If a different userrequests the same object, the object cache in Intelligence Server satisfiesthat request.

Object Caching Best Practices

Allocate enough memory in Intelligence Server for the storage of object caches.Tuning this setting to allocate memory greater than the 100 MB default isrecommended when dealing with complex schemas with a large number ofobjects.


54/320


54 Element Caches 2011 MicroStrategy, Inc.

Element Caches


Understand element caching and the best practices to leverage elementcaching.

Element Caching Overview

An element cache is a recently used attribute element list stored in memory forfaster retrieval. When it hits a cache, a request for attribute elements runs

against Intelligence Server memory instead of issuing a SQL statement againstthe database, thereby reducing response times significantly.

Between 25% and 30% of all queries issued against the database in a typical BIimplementation are element queries as the following figure shows:

Element Queries as a % of All Database Queries

This means that performance gains on element browsing will have a significantimpact on overall system performance.


55/320


2011 MicroStrategy, Inc. Element Caches 55

Element caches tend to be especially useful when users execute reports ordocuments that contain attribute element list prompts. The speed at whichusers see the options for elements in lists and thus, the overall usability of thereport or dashboard is greatly enhanced by caching the elements present in theprompt.

For more details on the element browsing query flow, refer to theMicroStrategy Administration: Configuration and Security course.

Element Caching Best Practices

Using Intelligent Cubes to Eliminate Element Database

Queries

The execution of a typical prompted report sourced directly against the datawarehouse generates two types of querieselement queries to populate theprompt and report queries to calculate the report results. On the other hand, areport sourced from an Intelligent Cube retrieves both the prompt elementsand the report dataset directly from the cube, alleviating the load on the datawarehouse and improving overall system performance.

Setting the Same Element Display Size for MicroStrategy Web

and MicroStrategy Desktop

The amount of memory that element caches use on Intelligence Server (set atthe project level) and on the MicroStrategy Desktop machine can be controlled.To avoid the risk of memory swapping and performance degradation, it isimportant to optimize the use of the memory assigned to element caches. Ifdifferent element display sizes are set on Intelligence Server and MicroStrategyDesktop, the element caches cannot be shared between those two components,and the risk of memory swapping increases.


56/320


56 Element Caches 2011 MicroStrategy, Inc.

Instead, defining identical element display sizes on both Intelligence Serverand MicroStrategy Desktop allows element cache sharing and optimizes theuse of a limited resource, as illustrated in the following image:

Inefficient and Efficient Use of Element Caching

Assigning Enough Memory for Element Caching

You should allocate sufficient memory in the server for the storage of elementcaches. Default values tend to be low. The table below shows the recommendedsettings for element caching. You should consider allocating even morememory for cases when some attributes contain large amounts of elements.

Element Caching Recommended Settings

Default Setting Recommended Setting

Web Desktop Web Desktop

Element Display Size 30 1000 50 50

Element Cache Size 120 4000 200 200

Avg. Amount of Data

(512 bytes/element)

60 KB 2,000 KB 100 KB


57/320


2011 MicroStrategy, Inc. Cache Sizing Recommendations 57

Cache Sizing Recommendations


Understand the rules of thumb to size caches in your system.

MicroStrategy recommends the following cache sizing to achieve highperformance on a MicroStrategy deployment:

Report CachesFor each application, counting 10 reports per user at anaverage report size of 50 KB, the total memory for result caches per user perproject should be about 500 KB.

Document CachesDocument cache would require about 100 MB for 100

documents cached in a project with an average document size of 1 MB.

Element CachesAn average element cache memory requirement isabout 10 MB, for a display size of 50 elements. For 100 attributes, as seen ina typical implementation, it is 512 bytes per element.

Object CachesObject cache calculation is based on the metadata size.Object caches should range from 20-50% of the metadata size. A metadatasize of 1 GB should have an object cache of 500 MB, which corresponds to50% of the metadata size.

Intelligent CubesCubes would take about 25 GB for an average of 50

cubes per project at 500 MB per cube. Intelligent Cubes have querycharacteristics like databases, but because the cubes reside in the mainmemory, they have the performance characteristics of caches. Thus, theyare not a part of the caching layer but are extremely important to providesuperior performance.


58/320


58 Cache Sizing Recommendations 2011 MicroStrategy, Inc.

The following image summarizes the recommended cache sizes:

Recommended Cache Sizes

Use the following Caching Size Worksheet to estimate initial cache settingsbased on your BI Implementation size:

Caching Size Worksheet


59/320


60/320


61/320


62/320


62 The Intelligent Cube Publication Process 2011 MicroStrategy, Inc.

Within this process and from a performance perspective, cube designers mustconsider two critical factorsoverall cube publication times (horizontal axis onthe graph) and peak memory usage (vertical axis on the graph). The followingsub-sections cover the trade-offs and the impact that different settings andapproaches have on these two critical factors.

Peak Memory Usage

As the previous cube publication process image shows, because of the dataprocessing that the Analytical Engine must perform to create the necessarydata structures to support the cube, the construction of a cube requires morememory than what is ultimately used by a fully published cube.

This is an important factor to consider when calculating how much server

memory might be needed to create Intelligent Cubes to support a set of reportsand dashboards. Exceeding the available memory in a server during cubepublication does not cause the process to fail, but it does cause the process tobecome extremely slow. The reason for this is that the operating system usesmemory swapping to disk which severely impacts cube publication times.

Based on internal benchmarks, as shown in the following image, the peakmemory used by Intelligence Server is expected to be two to five times the cubesize while publishing a cube:

Peak Memory to Cube Size Ratio

The variations in this ratio are due to the types of attributes, number of distinctattribute elements, and the normalization techniques used to generate thecube.


63/320


2011 MicroStrategy, Inc. The Intelligent Cube Publication Process 63

The study represented by the above image was run in MicroStrategyversion 9.0.1.

Intelligent Cube Data Normalization TechniquesThe process of publishing cubes, which involves querying lookup tables,dimensional tables, and fact tables from databases to perform necessary joinsand data aggregation, produces an intermediate table similar in structure tothe following table:

This intermediate table includes all the data for a final cube, and it is referredto as the Base Cube Table. The size of the Base Cube Table is usually much

larger than the source table in the data warehouse, because of the additionalinformation generated by data joins and metric aggregations. This tableincludes a significant amount of redundant information as the attribute formsare repeated multiple times.

MicroStrategy uses an optimization process referred to as normalization tooptimize the size of the Base Cube Table. The final output of the normalizationprocess is a set of tables which includes one lookup table for each attribute andone result table.

Intermediate Table

A1_ID A1_DES A1_... A2_ID A2_DESC ... Metric A MetricB

001 A1111... A1111 101 A2222...

002 A1111... A1111 102 A2222...

003 A1111... ... 103 A2222...

004 A1111... ... 104 A2222...

005 ... ... 405 A2222...


64/320



The Normalized Cube Tables are sent to the Analytical Engine, which convertsthe data into the final Intelligent Cube structure. The final Intelligent Cubestructure looks similar to the normalized structure, although the underlyingdata type of each table cell is different from the Base Cube Table or theNormalized Cube Tables, as shown in the following table:

Normalized Cube Tables

Data normalization can be performed at various steps of the cube publicationprocess and can be controlled through the VLDB settings. The list belowdescribes the four VLDB options for Intelligent Cube normalization:

Do not normalize Intelligent Cube dataThe data is not normalizedwhen it is retrieved from the database. Normalization occurs inside theAnalytical Engine, only after all resulting Intelligent Cube data has beenretrieved from the database. This type of normalization is not

recommended.

Normalize Intelligent Cube data in Intelligence ServerThe data isnormalized by the database class component of Intelligence Server whilefetching data from the database. This type of normalization hasdemonstrated superior performance in regards to memory consumption inthe following situations:

When the peak memory usage for cube publication is expected to be low

When publishing cubes with a large number of attributes

When publishing cubes with attributes that have string IDs

This is the default setting.

A1_ID A2_ID MetricA MetricB ...

A1Index001 A1Index002




... ...


65/320



Normalize Intelligent Cube data in the databaseThe data is normalizedon the database side. The resulting cube data is stored in a temporary table.Then, data is normalized using multiple SELECT statements. Thisnormalization may cause memory consumption to be higher at publicationif Attribute strings or VARCHAR data types are used. This option isrecommended when publishing cubes with attributes that have multipleforms.

For data warehouse databases which are not optimized for large datainserts (such as Netezza) publishing a cube with SQL normalization maytake longer.

Normalize Intelligent Cube data in database using relationshiptablesThe data is normalized on the database side. This approach canprovide improved performance in scenarios where cube data includes alarge ratio of repeating data, dimensions include a large number ofattributes, and attribute lookup tables are much smaller than fact tables.

Direct loading of dimensional data and filtered fact dataThere is noneed to normalize the data. It generates optimized SQL that retrieves datadirectly from a data warehouse and transforms the data as SQL passes inthe following manner:

Attribute passesOne pass for each attribute

Dimension passesOne pass for each dimension

Metric passesOne pass for all metrics in the Intelligent Cube

While this option may provide faster cube publication as compared toIntelligence Server Normalization (the default setting), peak memoryconsumption for cube publication will be higher.

To define the normalization method to be used during cube publication:

1 In MicroStrategy Desktop, edit an Intelligent Cube.

2 On the Data menu, select VLDB Properties.

3 In the VLDB Properties window, expand Query Optimizations, and selectData population for Intelligent Cubes.

4 Clear the Use default inherited value - [Default Settings]check box toselect any of the non-default options.


66/320



5 Click the normalization method you want the cube to use duringpublication.

6 Click Save and Close to save your settings.

VLDB Property for Cube Normalization Methods

Because every environment is unique, there are key factors to consider whendetermining which normalization will be more appropriate for a givenimplementation. The list below emphasizes these factors:

Database and MicroStrategy HardwareThe normalization process isconducted on either a database hardware or a MicroStrategy hardware,depending on the normalization method used. Therefore, the hardwarepower ratio between the database server and Intelligence Server can affectthe outcome of the comparison.

Network SpeedDatabase normalization retrieves only the normalizeddata. Intelligence Server normalization retrieves the entire Base CubeTable, which is usually larger than the normalized data. On a slowernetwork, it is expected that database normalization performs better.

Database Insert PerformanceDatabase normalization requires insertoperations, which can affect cube normalization performance on databasesthat have a slow insert performance.

Outer JoinsDatabase normalization may require outer joins to beconducted on the database side, which may require a large amount oftemporary table space. This behavior creates a scalability limitationbecause the size of the outer join can be unpredictable.


67/320



Intermediate Table Type VLDB Property

The Intermediate Table Type VLDB property enables you to choose betweencreating derived tables or true temporary tables when intermediate tables arecreated during cube publication. True temporary tables implement each pass

of a multi-pass SQL in separate tables. They represent a feasible option in mostcases and are the default setting. Derived tables, instead of issuing multipletables for multi pass SQL, generate a single, large SQL pass. Because there areno CREATE and DROP TABLE statements, the queries may run faster.However, because multiple cross joins result in temporary space issues,derived tables may not be a feasible option to use in all cases.

To define the intermediate table type:

1 In Desktop, edit an Intelligent Cube.

2 On the Data menu, select VLDB Properties.

3 In the VLDB Properties window, expand Tables, and select IntermediateTable Type.

4 Clear the Use default inherited value - [DBMS level] check box and selectany of the non-default options.

5 Click Save and Closeto save your settings.

The following image shows the Intermediate Table Type VLDB property:

Intermediate Table Type VLDB property


68/320


68 When to Use Intelligent Cubes? 2011 MicroStrategy, Inc.

When to Use Intelligent Cubes?


Identify the usage scenarios that represent best candidates to take advantage ofIntelligent Cubes.

As with report caching, not all reports and dashboards in a project will be goodcandidates for Intelligent Cubes. The decision of when to create IntelligentCubes to optimize dashboard and report execution performance requires someanalysis. The next sections in this document go through a simple frameworkthat will help project designers with this decision.

Before designing cubes, you should look at the system usage to analyze thedistribution of reports by frequency of usage and by average user wait time.This analysis generally provides a good indication of which reports will benefitfrom cube creation. The following graph illustrates the distribution of reportsin a system:

Distribution of Reports in a Project

On the lower part of the graph, represented by good performance, are thereports that already execute within a satisfactory response time. These reportsare not good candidates for Intelligent Cubes because it is unlikely that theirperformance can be significantly improved.


69/320


2011 MicroStrategy, Inc. When to Use Intelligent Cubes? 69

Similarly, reports that are only infrequently used (located at the far left of thegraph) are not good candidates for Intelligent Cubes, because it is unnecessaryto spend memory resources creating cubes to support reports that will not beused frequently.

The far right of the graph displays the very frequently executed reports, whichare not good candidates for Intelligent Cubes either. For those reports, youshould use caching whenever possible, as it generally offers betterperformance.

Lastly, represented in dark grey in the graph are a group of frequently runreports with increasing levels of user wait time. These reports are the goodcandidates for Intelligent Cubes. These reports can fall into three differentcategories that can be addressed by individual design strategies to buildIntelligent Cubes:

Highly used reports

Multiple overlapping reports

Ad-hoc analysis reports

Using Cubes for Highly Used Prompted Reports

Highly used prompted reports with high average wait times are greatcandidates for Intelligent Cube optimization. In a sense, those are the most

expensive reports in a BI application and tend to show a high level of variabilitythat is introduced predominantly by the use of prompts and security filters.Caching will not be effective for this class of reports because of the abovecombination of prompts and security filters. As a result, most reports in thiscategory will submit many highly similar database queries, each incurring theaverage wait time, causing significant database workload.


70/320


70 When to Use Intelligent Cubes? 2011 MicroStrategy, Inc.

For this category of reports, the recommendation is to create one IntelligentCube specifically designed to support each highly used prompted report. Thefollowing image shows the use of cubes for frequently used prompted reports:

Using Cubes for Highly Used Prompted Reports

This is done by creating one cube containing data for all prompt answers and

security filter combinations for any of these reports. As a result, the databaseonly incurs a one-time database query. After that, all report requests areanswered by the Intelligent Cube with a greatly improved overall responsetime. This strategy should be effective when applied to the top reports byhighest total wait time.

Using Cubes for Highly Used Overlapping Reports

This set of reports is usually characterized by the fact that most reports arereally variations of an original report design. This is a common scenariowhereby users create personalized versions of IT supplied reports.


71/320


2011 MicroStrategy, Inc. When to Use Intelligent Cubes? 71

In one case of a financial customer, over 1300 user-created reports could betraced to a small originating core set of only 12 reports. Because it tends to useup valuable memory with almost identical data structures, caching is veryineffective for this list of independent reports. Without caching, these reportscause a heavy workload on the database, which spends cycles executing manyhighly similar queries each with the average wait time.

For this category of reports, the recommendation is to create an I

High Performance BI

Documents

Transcript of High Performance BI