OBIEE UDML XML

Basel · Baden · Bern · Lausanne · Zürich · Düsseldorf · Frankfurt/M. · Freiburg i. Br. · Hamburg · München · Stuttgart · Wien

Scripting OBIEE – Is UDML and XML all you need ?

Andreas Nobbmann Consultant Business Intelligence [email protected]

Brighton, May 14th, 2009

© 2009 Scripting OBIEE – Is UDML and XML all you need ? 2

Why scripting OBIEE ?


Agenda

Data is always part of the game.

What is UDML ?

How to use UDML ?

Good examples for UDML usage

XML

Good examples for XML usage

The last word


Agenda


What is UDML ?

How to use UDML ?


XML


The last word


What is UDML ?

Commonly known as:

Scripting and definition language to describe databases or perform certain actions against databases

E.g. DDL creation etc.

In OBIEE:

„Undocumented“ feature in BI Server repository to manage mouseless administrative scripting

Oracle recommends for example to merge Oracle BI repositories by using UDML

UDML = Universal Database Markup Language


What to use it for

Connection pools

Users / groups, LDAP server definitions, Security settings, privileges

Initialization blocks, variables

physical, aliases, logical tables, views...

Primary / secondary keys, Foreign key, complex joins

Dimensions / hierarchies

External name settings (DISPLAY NAME CN_.. / CD_..)

.......

In general for scripting or describing everything stored in your Oracle BI repository file


Agenda


What is UDML ?

How to use UDML ?


XML


The last word


How to use UDML ? – Ad Hoc

Simply do copy (CTRL+C) on the object you want to script

Paste (CTRL+V) it to your favoured editor, edit script and copy back to repository


How to use UDML ? – Ad Hoc

Attention: Copy is context sensitive, means

place your cursor on the father of the object you want to insert the UDML

If new Business Model (BM)

do not click on any object and paste into the BM window

If new table source

Go to BM and paste it into

And so on

Remark: users and groups cannot be copied this way, they can only be merged into repositories


How to use UDML ? – UDML generation

Use nQUDMLGen.exe generates UDML from your repository

- N does not generate upgrade IDs (needed for versioning)

- Q without security objects

- S only security objects (overwrites Q parameter)

-N use if you want to create a repository from the scratch, OBIA creates new object ID‘s

c:\OracleBI\server\Bin\nQUDMLgen.exe –U Administrator –P Administrator –R c:\OracleBI\server\repository\paint.rpd –O paint_udml


How to use UDML ? – merging repositories

Use nQUDMLExec.exe merges UDML into repository and optionally backs it up

- I input script name

- B optional base repository name

- O output repository name

-8 is for UTF-8

c:\OracleBI\server\Bin\nQUDMLexec.exe –U Administrator –P Administrator –I c:\OracleBI\server\repository\ChangeRepo_2_prodDB_UDML.txt -B c:\OracleBI\server\repository\sales.rpd –O c:\OracleBI\server\repository\sales_new.rpd

Remark: always make a backup before working with nQUDMLExec.


How to use UDML ? – update via data source

Use nQUDMLcli.exe adding UDML to OBIEE repository via Data Source

-D Data Source name as in odbcad32.exe (ODBC Data Source Administrator)

-w optional passphrase for SSL connections

-P if you leave empty it uses the profile of the client you are connecting from

c:\OracleBI\server\Bin\nQUDMLcli.exe –U Administrator –P Administrator -D AnalyticsWeb –I c:\OracleBI\server\repository\ChangeRepo_2_prodDB_UDML.txt

Remark: sometimes this tool is not running because of lacking privileges. Copy it with all DLLs named NQSUDMLxx.dll into another directory and restart.


Agenda


What is UDML ?

How to use UDML


XML


The last word

© 2009

„Good“ examples for UDML usage

Parsing your repository

Development street

Automating import of user groups

Update row counts

Add icons to your repository

To be continued .....


Good UDML examples – Parsing your repository

Download UDML parser jar-ball from http://code.google.com/p/udmlparser/.

Copy it to a directory you want

Use nQUDMLgen.exe with options –N –Q -8 for the repository

Fire java –jar udmlparser0_4.jar –udml=YourUDMLfile.txt – rpdxml=YourUDML_XML.xml against YourUDMLFile.txt.

You get a well formed XML, which is of course editable and transformable via Style Sheets etc.

Thanks to Daniel Galassi for the parser


UDML examples – Development street

Assume we have three environments, where DWHDEV, DWHTST and DWHPRD, are the data sources names defined within TNSNAMES.ora

Export the UDML from your DEV repository with nQUDMLgen.exe

c:\OracleBI\server\Bin\nQUDMLgen.exe –U Administrator –P Administrator –R c:\OracleBI\server\repository\sales.rpd –O sales_dev_udml

search for the connection pool (DECLARE CONNECTION POOL)

Copy the whole block of the connection pool into two text files


UDML examples – Development street – cont.

In the first file replace the text within the {} with DWHTST

In the second file replace it with DWHPRD

Adapt your user settings as required

Save them as for example DBConnectInfos_UDML_DWHTST and DBConnectInfos_UDML_DWHPRD

Generate the repository sales_tst.rpd by using nQUDMLExec

c:\OracleBI\server\Bin\nQUDMLexec.exe –U Administrator –P Administrator –I c:\OracleBI\server\repository\DBConnectInfos_UDML_ DWHTST -B c:\OracleBI\server\repository\sales.rpd –O c:\OracleBI\server\repository\sales_tst.rpd

Thanks to Venkat, John and Mark for their blogs around that !


UDML examples – Development street – cont.

Another way is to copy this into the files


UDML examples – Automating user group import

Get your user groups from your table with a simple SQL

Select ‘DECLARE SECURITY ROLE “’ ¦¦ usergroup ¦¦ ‘”AS “’ ¦¦ usergroup ¦¦ ‘UPGRADE ID 78077 description{Data Security Group ’ ¦¦ usergroup ¦¦ ‘} PRIVILEGES (READ);’ from DWH_APP.usergroups

Save the result of your SQL in a text file and update your repository with nQUDMLexec

c:\OracleBI\server\Bin\nQUDMLexec.exe –U Administrator –P Administrator –I c:\OracleBI\server\repository\UserGroups_UDML -B c:\OracleBI\server\repository\sales.rpd –O c:\OracleBI\server\repository\sales_UserGroups.rpd

Thanks to Venkat for his blog around that !

© 2009 Scripting OBIEE – Is UDML and XML all you need ?

UDML examples – update row counts

2 general steps to do so normally

Right click on table and do update row count

Or use tools / update all row counts

WAITING

When using hints in Oracle or an event table for Cache you can get the updated row count from there and copy the results into a UDML file

DECLARE TABLE "U".."SYSTEM"."S_ETL_DAY" AS "S_ETL_DAY" UPGRADE ID 80890 NO INTERSECTION ROW COUNT 13394.000000 LAST UPDATED '2009-04-29 18:22:43‘ PRIVILEGES ( READ);


UDML examples – making the repository intuitive

Assign the dollar sign to all fact tables

DECLARE TABLE "Paint"..."Fact" AS "Fact" UPGRADE ID 554 ICON INDEX 102 HAVING

Assign the stop sign to the database connections when you work on them for a longer period of time

DECLARE DATABASE "Forecast" AS "Forecast" UPGRADE ID 162 ICON INDEX 92 TYPE XMLDirect

Other examples:

Question mark (Index 90), globe (Index 81), work cone (index 81) etc.


Agenda


What is UDML ?

How to use UDML


XML


The last word


XML

Requests

Prompts, filters

Dashboards, dashboard pages

User objects, group objects

Privileges

Descriptions etc.

Is used for everything defined in the presentation catalog

23


XML

3 ways to insert XML into catalog

Advanced tab in request

Paste XML and

Click Set XML

In catalog manager

Edit and paste

Or directly in the file system under

<OracleBIData>\web\catalog\<YourCatalog>\path_to_your_report\YourReport

Open with editor

Paste XML and save

24


Agenda


What is UDML ?

How to use UDML


XML


The last word

© 2009


Automatically create pivot-table label heading style

Format all columns with Percent in name

Format all columns depending on their data type

Editing XML directly in your editor

To be continued .....

26 Scripting OBIEE – Is UDML and XML all you need ?


XML examples – create pivot-table

27


XML examples – create pivot-table

Here the XML source:

28


XML examples – create pivot-table – column formulas

Now open a good XML editor (for example XMLSpy) and create a XSLT stylesheet

Add namespaces, start at / elements, then match all saw:report elements and copy them to target

With this part of the stylesheet you copy all the column formulas.

<saw:column formula=""D2 Market"."M01 Market"" columnID="c0"/> .....

@* = all attributes Copy all saw:report elements

29


XML examples – create pivot-table – parameters

The next part is related to parameters needed for formatting purpose

You see the parameters

aggRule aggregation rule within Pivot

hAlign Allignment

fontStyle Font Style

3 = # of views in compoundView

parameters

30


XML examples – create pivot-table – compoundView

Next is the standard compoundView

Here also the pivotTableView1 is added

31


XML examples – create pivot-table – makePivot

next part is creating the Pivot table in XML dynamically

Parameters are used with { }

Parameter {aggRule}

32


XML examples – format all Percent columns

Let‘s take a look on the pattern functionalities of XML

We define the rule for the percent pattern on template saw:column

<xsl:if test="contains(@formula, 'Percentage')"> <xsl:attribute name="percent" select="'true'"/> </xsl:if>

Pretty easy, isn‘t it ?

pattern

33


XML examples – format on data type

Let‘s derive the data type from the UDML-XML we parsed from the original sales sample

Goal: define all columns with data type double like 1.000.000,00

34



Get presentation column depending on your input column on Xpath level PresentationColumnMappingID

Get the logical column from your UDML data by searching the presentation column on Xpath level LogicalColumnCalculation

35


XML examples – format on data type – cont.

Get the physical column by searching for the logical column on Xpath level PhysicalColumnID

and derive the data type PhysicalColumnDatatype from physical column

36



The test result looks like that

of course it can be further extended by formatting per data type

37


XML examples – editing directly in your editor

If you have a good editor (like UltraEdit etc.) installed on your PC then it probably has the column mode,

you can put your cursor to a column and edit multiple columns at the same time

38


Agenda


What is UDML ?

How to use UDML


XML


The last word


The last word

UDML and XML are really very useful for scripting

Good news: everything can be automated by scripts

Cache Seeding, Backup, Synchronizing styles & skins with OC4J, users & user groups, repository, presentation catalog ...

UDML&XML provide a good approach for creating repositories and reports automatically from the metadata

I am working on it

Good news: new tool Content Accelerator Framework (CAF) published within 10.1.3.4.1:

Venkat blogged already about it

http://oraclebizint.wordpress.com/2009/05/05/oracle-bi-ee-101341-content-accelerator-framework-caf-synchronizer-a-reviewpreview-xudml/

My resumee:

Scripting with OBIEE is fun !

Basel · Baden · Bern · Lausanne · Zürich · Düsseldorf · Frankfurt/M. · Freiburg i. Br. · Hamburg · München · Stuttgart · Wien

Thank you very much!

? www.trivadis.com


If you have any questions or want the scripts

[email protected]

Trivadis AG Basel Consultant Business Intelligence

+41-79 264 8807

Please feel free to drop me a note or give me a call


Appendix


Error Messages UDML parser


UDML Parser error messages

UDML Parser Messages This category contains message numbers 28xxx.

28001 Near Line @1%d, Near (@2%s): String must be in a single line. Cause. The named string must be on a single line. Response. Remove all the new lines within this string.

28002 Near Line @1%d, Near (@2%s): End of file has been reached. Cause. The singly or doubly quoted string does not have a matching quote. Response. Add single or double quote at the end.

28003 Near Line @1%d, near (@2%s): Syntax error @3%s. Cause. A keyword was misspelled or an unexpected identifier was encountered. Response. Verify the syntax and correct any errors before retrying the action.

28004 Near Line @1%d, (@2%s): Duplicate definition. Cause. The same object was defined more than once. Response. Change the name of one of the objects.


UDML Parser error messages – cont.

28005 Near Line @1%d, (@2%s): Multiple matches found in the current repository. Cause. More than one match was found in the repository. The repository may be corrupted. Response. Repair the repository. Restore it from a back-up, or use the Check Consistency command in the Administration Tool to identify compilation errors. Correct any problems before retrying your actions.

28006 Near Line @1%d, (@2%s): Not defined. Cause. The named symbol is not defined either in the script or the repository. Response. Define the named symbol in the script or create an object in the repository. If necessary, contact Siebel Systems technical support for assistance.

28007 Near Line @1%d, (@2%s): This data type is not supported. Cause. The named data type is not supported. Response. Correct the data type and retry the action. If necessary, contact Siebel Systems technical support for assistance.

28008 Near Line @1%d, (@2%s): This database type is not supported. Cause. The named database type is not supported. Response. Correct the database type and retry the action. If necessary, contact Siebel Systems technical support for assistance.



28009 Near Line @1%d, (@2%s): This connection type is not supported. Cause. The named connection type is not supported. Response. Correct the connect type and retry the action. If necessary, contact Siebel Systems technical support for assistance.

28010 Near Line @1%d, (@2%s): It cannot be a parent or container of itself. Cause. A circular definition was found. Response. Correct the appropriate definitions to break the circle. If necessary, contact Siebel Systems technical support for assistance.

28012 Near Line @1%d, (@2%s): This role is not defined. Cause. The required role is missing. Each relationship needs to have two roles. This is an internal error. Response. Contact Siebel Systems technical support for assistance.

28013 Near Line @1%d, (@2%s): This object privilege cannot be used for none set. Cause. This object privilege is wrong. This is an internal error. Response. Contact Siebel Systems technical support for assistance.

28014 Near Line @1%d, (@2%s): This object privilege cannot be used for read set. Cause. This object privilege is wrong. This is an internal error. Response. Contact Siebel Systems technical support for assistance.



28015 Near Line @1%d, (@2%s): This object privilege cannot be used for read_write set. Cause. This object privilege is wrong. This is an internal error. Response. Contact Siebel Systems technical support for assistance.

28016 Near Line @1%d, (@2%s): This object privilege cannot be used for erase set. Cause. This object privilege is wrong. Response. Contact Siebel Systems technical support for assistance.

28017 Near Line @1%d, (@2%s): This object privilege cannot be used for grant set. Cause. This object privilege is wrong. Response. Contact Siebel Systems technical support for assistance.

28018 Near Line @1%d, (@2%s): This object privilege cannot be used for all set. Cause. This object privilege is wrong. Response. Contact Siebel Systems technical support for assistance.

28019 Near Line @1%d: In the metadata expression {@2%s ......}, the following error occurred: @3%s@4%s. Cause. The named expression has one or more errors. This can be caused by spelling errors or objects referenced in the expression that are missing. Response. Ignore and then reconstruct the problem expressions.



28020 Near Line @1%d, (@2%s): is already in the repository and it's ID is different from the designated ID. Cause. An object of the same name exists in the repository but with a different ID. The repository may be corrupted. Response. Repair the repository. Restore it from a back-up, or use the Check Consistency command in the Administration Tool to identify compilation errors. Correct any problems before retrying your actions.

28021 Near Line @1%d, (@2%s): is an invalid feature. Cause. The system could not find the named feature. Response. Verify the spelling of the named feature.

28022 Near Line @1%d, (@2%s): is an invalid value for the feature (@3%s) Cause. The specified value is invalid for the named Boolean feature. Response. Valid values include words like YES, NO, ON or OFF. Make the appropriate changes before retrying the action.

28023 Near Line @1%d, (@2%s): is an invalid value repository property name. Cause. The property name is wrong. Response. Use 'CustomPresentationLayer'.

28024 Near Line @1%d, (@2%s): This alias appears more than once. Cause. The alias appeared twice in the repository, either as the name or alias of an object of the same type. Response. Find the duplicates in the repository using the Query tool and make the appropriate changes.



28025 Near Line @1%d, (@2%s): This alias is the same as an object in the script. Cause. The alias has the same name as another object of the same type in the script. Response. Correct the script so the alias and the object use different names.

28026 Near Line @1%d, (@2%s): This alias is the same as an object in the repository. Cause. The alias has the same name as another object of the same type in the repository. Response. Correct the repository so the alias and the object use different names.

28027 Near Line @1%d: The Administrator security role does not inherit from any other roles. Cause. The Administrators role inherited from another role. Response. The Administrator security role can only be a root level role.

28028 Near Line @1%d: (@2%s): Invalid DateTime format. Cause. The format of the Datetime string is invalid. Response. Correct the format of the Datetime string and retry the action.

28029 Invalid level relationship to @1%s found. Please recreate the level(s) relationships. Cause. The level with the required ID does not exist in the repository. The repository may be corrupted. Response. Repair the repository. Restore it from a back-up, or use the Check Consistency command in the Administration Tool to identify compilation errors. Correct any problems before retrying your actions.



28030 Error: The object @1%s(@2%s) does not exist. If it is on-line mode, please check in changes from the File menu. Cause. The object with the required ID does not exist in the repository. The object may not be checked in. Response. If the repository is in on-line mode, use the Check-in function from the File menu to make the object available.

28031 Error: Cannot open the file: @1%s. Cause. Cannot open the named UDML script file. The file may not exist, or may be in use. Response. Verify that the file exists and is not in use by another process. If the problem persists, contact Siebel Systems technical support for assistance.

Source: http://download.oracle.com/docs/cd/E12100_01/books/AnyMsg/AnyMsgMessages10.html (Siebel 2004)


How to use UDML ? – equalize repositories

Use equalizerpds.exe equalizes 2 repositories into a new one

-X treats name-wise similar facts as one (e.g facts Factxxxx as Fact)

-Y equals the string sets

-O does not update directly, creates a new repository

c:\OracleBI\server\Bin\equalizerpds.exe –A Administrator –B Administrator -C c:\OracleBI\server\repository\Sales_Version1.rpd -D Administrator –E Administrator -F c:\OracleBI\server\repository\Sales_Version2.rpd –X -Y

OBIEE UDML XML

Documents

Transcript of OBIEE UDML XML