#Kscope To Slice or Not to Slice? The Next Advancement in ASO Cubes Angela Wilcox BI Architect...
-
Upload
horatio-stafford -
Category
Documents
-
view
215 -
download
0
description
Transcript of #Kscope To Slice or Not to Slice? The Next Advancement in ASO Cubes Angela Wilcox BI Architect...
#Kscope
To Slice or Not to Slice?
The Next Advancement in ASO Cubes
Angela WilcoxBI ArchitectMedAssets
#Kscope
Developing Essbase Applications
● Like the best, most advanced Essbase conference there ever could be
● Advanced content● Good practices● Written by some of the most well
known Essbase developers● Source code at
www.developingessbasebook.com● You should buy it
#Kscope
ASO Cube Slices
Only available in ASO cubes
Allows true incremental loads Allows true regional clears Real time and near-real time now possible
Greatest thing since sliced bread! Really!
#Kscope
Steps to Using Slices
STEP 1: Load data into the cube and aggregate using your normal processes
STEP 2: Define your new load and define it to load to a slice● Instead of loading in the main cube it will be situated
next to it● Aggregate views will be created automatically
#Kscope
Steps to Using Slices Cont…
STEP 3: Multiple slices can be added in the same set of loads
STEP 4: Eventually performance degradation will be noted and can be statistically proven in the statistics
STEP 5: Merge the slices together● Merge into a single slice or● Merge the slices into the larger historical slice
#Kscope
Steps to Using Slices, Cont.
STEP 6: Aggregations must be dropped when the merge occurs – so a re-aggregation of data will need to happen when the merge is completed.
The specifics of which options to deploy for an installation will be unique and need to be considered.
Look at a real-life case study from JCP
#Kscope
Use Case
Discuss and examine production code for each of the following tasks:● Creating the Main Database● A Week in the Life● Merging Slices● Updating Dimensions● Aggregating the Cube● Creating a Slice● Clearing Slices
#Kscope
Requirements
The requirements for this implementation had some unique characteristics that no previous implementation had:● Essbase = database of record● Data from twenty-five disparate sources● The staging tables would hold one week only and
then be cleared● Two years of history plus the one year building
#Kscope
Requirements, Cont
Granular Dimension● Six levels● Million plus members
Large weekly load volumes● Fifty million records to start● Three hundred million records when fully deployed
No downtime during Monday and Thursday updates
#Kscope
Requirements, Cont
Dimensional updates ● Weekends only● Cube allowed offline
Essbase Classic Add-in Templates● Template Only● No Ad-hoc● SLA less than 30 seconds
#Kscope
View Before Processing Starts
Statistics regarding the slices and incremental data are provided in EAS. ● Open EAS.● Drill down to the database level of the cube.● Right-click on the database and select Edit > Properties.● Select the Statistics Tab.
#Kscope
A Week in the Life
Cube 1 Saturday am
alter database ${APP_NAME}.${DB_NAME} clear aggregates ; export database ${APP_NAME}.${DB_NAME} level0 data to
data_file '${APP_NAME}.export.txt' ;
Dimensions of Cube1 Total Stored
Metrics 171 169
Dimension1 215 215
Dimension2 6 6
Dimension3 25 11
Dimension4 228 228
Dimension5 1159 1148
Dimension 6 1039504 843779
#Kscope
A Week - Merging Slices
alter database ${APP_NAME}.${DB_NAME} merge all data;
We did not use – but available:● alter database ${APP_NAME}.${DB_NAME} merge
all data remove_zero_values;● alter database ${APP_NAME}.${DB_NAME} merge
incremental data;
#Kscope
A Week Sunday - Updating Dimensions
import database ${APP_NAME}.${DB_NAME} dimensions connect as ${USER}identified by ‘${PWD}’ using server rules_file '${LD_RULE1}', connect as ${USER}identified by ‘${PWD}’ using server rules_file '${LD_RULE2}’on error append to '${ESS_ERROUT}' ;
#Kscope
A Week Sunday – Aggregating Data
execute aggregate build on database ${APP_NAME}.${DB_NAME} using view_file ‘${AGG_VIEW}’;
Views vs. Default Aggregation● Behavior Known?● Size of Cube?● SLA met with default?
#Kscope
A Week Sunday – Create Slices
alter database ${APP_NAME}.${DB_NAME} initialize load_buffer with buffer_id 1 resource_usage 1.0 property ignore_missing_values;import database ${APP_NAME}.${DB_NAME} data connect as ${USER} identified by '${PWD}' using server rules_file ${LD_RULE} to load_buffer with buffer_id 1 on error write to '${ESS_ERROR_DIR}${JOB_FILE_PREFIX}.${APP_NAME}.${LD_RULE}.err' ;import database ${APP_NAME}.${DB_NAME} data from load_buffer with buffer_id 1 create slice;
#Kscope
Slice Behavior
Behavior not always as expected
Load #Cell Count Result
1 75,000 1 Slice with 75,000 cells
2 400,0001 Slice with 475,00 cells (merged because slice one had less than 100,000 cells)
3 1,000,0002 Slices; 1 with 1,000,000 cells and 1 with 475,000 cells (nothing was merged because load #3 is greater than two times larger than the existing slice)
4 300,0002 Slices; 1 with 1,000,000 cells and 1 with 775,000 cells (load 4 is merged with loads 1 and 2 because that slice is less than 2 times the size of the new slice)
#Kscope
Weekend Setup Recap BIG ASO Saturday:
● Clear Aggregates● Backup Using Export● Drop Aggregates● Merge Slices
Sunday● Build Dimensions● Re-Aggregate● Load Data
SMALL ASO Saturday:
● Clear Aggregates● Backup Using Export● Drop Aggregates● Merge Slices● Re-Aggregate
Sunday● Build Dimensions
● Load Data
#Kscope
When Things Go Wrong - Clearing Slices import database ${APP_NAME}.${DB_NAME}
data from load_buffer with buffer_id 1 override all data;
import database ${APP_NAME}.${DB_NAME} data from load_buffer with buffer_id 1 override incremental data;
import database ${APP_NAME}.${DB_NAME} data from load_buffer with buffer_id 1 subtract values create slice;
#Kscope
Clearing Slices, Cont
alter database ${APP_NAME}.${DB_NAME} clear data in region ‘{2011W48}’ physical;
alter database ${APP_NAME}.${DB_NAME} clear data in region ‘{2011W48}’; ● Logical Clear - Creates a slice● Zeros
A number of things will automatically happen if there are multiple slices in the cube when this command is issued:● Merge the slices● Clear the specified region● Re-materialize all aggregate views
#Kscope
Questions?
Most of the Information presented here is available in the “Developing Essbase Applications” – see my chapter!