Working with Sessions

By PenchalaRaju.Yanamala

This chapter includes the following topics:

Working with Sessions OverviewCreating a Session TaskEditing a SessionUnderstanding Buffer MemoryConfiguring Automatic Memory SettingsConfiguring Performance DetailsUsing Pre- and Post-Session SQL CommandsUsing Pre- and Post-Session Shell CommandsValidating a SessionStopping and Aborting a SessionWorking with Session ParametersMapping Parameters and Variables in SessionsAssigning Parameter and Variable Values in a SessionHandling High Precision Data

Working with Sessions Overview

A session is a set of instructions that tells the Integration Service how and when to move data from sources to targets. A session is a type of task, similar to other tasks available in the Workflow Manager. In the Workflow Manager, you configure a session by creating a Session task. To run a session, you must first create a workflow to contain the Session task.

When you create a Session task, enter general information such as the session name, session schedule, and the Integration Service to run the session. You can select options to run pre-session shell commands, send On-Success or On-Failure email, and use FTP to transfer source and target files.

Configure the session to override parameters established in the mapping, such as source and target location, source and target type, error tracing levels, and transformation attributes. You can also configure the session to collect performance details for the session and store them in the PowerCenter repository. You might view performance details for a session to tune the session.

You can run as many sessions in a workflow as you need. You can run the Session tasks sequentially or concurrently, depending on the requirement.

The Integration Service creates several files and in-memory caches depending on the transformations and options used in the session.

Creating a Session Task

You create a Session task for each mapping you want the Integration Service to run. The Integration Service uses the instructions configured in the session to move data from sources to targets.

You can create a reusable Session task in the Task Developer. You can also create non-reusable Session tasks in the Workflow Designer as you develop the

workflow. After you create the session, you can edit the session properties at any time.

Note: Before you create a Session task, you must configure the Workflow Manager to communicate with databases and the Integration Service. You must assign appropriate permissions for any database, FTP, or external loader connections you configure.

Related Topics: Working with Connection Objects

Steps to Create a Session Task

Create the Session task in the Task Developer or the Workflow Designer. Session tasks created in the Task Developer are reusable.

To create a Session task:

1. In the Workflow Designer, click Tasks > Create. 2. Select Session Task for the task type.3. Enter a name for the Session task.4. Click Create. 5. Select the mapping you want to use in the Session task and click OK.6. Click Done.Related Topics: Reusable Workflow Tasks

Editing a Session

After you create a session, you can edit it. For example, you might need to adjust the buffer and cache sizes, modify the update strategy, or clear a variable value saved in the repository.

Double-click the Session task to open the session properties. The session has the following tabs, and each of those tabs has multiple settings:

General tab. Enter session name, mapping name, and description for the Session task, assign resources, and configure additional task options.Properties tab. Enter session log information, test load settings, and performance configuration.Config Object tab. Enter advanced settings, log options, and error handling configuration.Mapping tab. Enter source and target information, override transformation properties, and configure the session for partitioning.Components tab. Configure pre- or post-session shell commands and emails.Metadata Extension tab. Configure metadata extension options.

You can edit session properties at any time. The repository updates the session properties immediately.

If the session is running when you edit the session, the repository updates the session when the session completes. If the mapping changes, the Workflow Manager might issue a warning that the session is invalid. The Workflow Manager then lets you continue editing the session properties. After you edit the

session properties, the Integration Service validates the session and reschedules the session.

Related Topics: Validating a SessionSession Properties Reference

Applying Attributes to All Instances

When you edit the session properties, you can apply source, target, and transformation settings to all instances of the same type in the session. You can also apply settings to all partitions in a pipeline. You can apply reader or writer settings, connection settings, and properties settings.

For example, you might need to change a relational connection from a test to a production database for all the target instances in a session. You can change the connection value for one target in a session and apply the connection to the other relational target objects.

Table 7-1. Apply All Options

Setting Option DescriptionReader Writer

Apply Type to All Instances

Applies a reader or writer type to all instances of the same object type in the session. For example, you can apply a relational reader type to all the other readers in the session.

Reader Writer

Apply Type to All Partitions

Applies a reader or writer type to all the partitions in a pipeline. For example, if you have four partitions, you can change the writer type in one partition for a target instance. Use this option to apply the change to the other three partitions.

ConnectionsApply Connection Type

Applies the same type of connection to all instances. Connection types are relational, FTP, queue, application, or external loader.

ConnectionsApply Connection Value

Apply a connection value to all instances or partitions. The connection value defines a specific connection that you can view in the connection browser. You can apply a connection value that is valid for the existing connection type.

ConnectionsApply Connection Attributes

Apply only the connection attribute values to all instances or partitions. Each type of connection has different attributes. You can apply connection attributes separately from connection values.

ConnectionsApply Connection Data

Apply the connection value and its connection attributes to all the other instances that have the same connection type. This option combines the connection option and the connection attribute option.

ConnectionsApply All Connection Information

Applies the connection value and its attributes to all the other instances even if they do not have the same connection type. This option is similar to Apply Connection Data, but it lets you change the connection type.

Properties Apply Attribute to all Instances

Applies an attribute value to all instances of the same object type in the session. For example, if you have a relational target you can choose to truncate a table before you load data. You can apply the attribute value to all the relational targets in the session.

Properties Apply Attribute to all Partitions

Applies an attribute value to all partitions in a pipeline. For example, you can change the name of the reject file name in one partition for a target instance, then apply the file name change to the other partitions.

Applying Connection Settings

When you apply connection settings you can apply the connection type, connection value, and connection attributes. You can only apply a connection value that is valid for a connection type unless you choose the Apply All Connection Information option. For example, if a target instance uses an FTP connection, you can only choose an FTP connection value to apply to it. The Apply All Connection Information option lets you apply a new connection type, connection value, and connection attributes.

Applying Attributes to Partitions or Instances

When you apply attributes to all instances or partitions in a session, you must open the session and edit one of the session objects. You apply attributes or properties to other instances by choosing an attribute in that object and selecting to apply its value to the other instances or partitions.

To apply attributes to all instances or partitions:

1. Open a session in the workspace.2. Click the Mappings tab.

3.

Choose a source, target, or transformation instance from the Navigator. Settings for properties, connections, and readers or writers might display, depending on the object you choose.

Understanding Buffer Memory

When you run a session, the Integration Service process starts the Data Transformation Manager (DTM). The DTM allocates buffer memory to the session at run time based on the DTM Buffer Size setting in the session properties.

The DTM divides the memory into buffer blocks as configured in the Default Buffer Block Size setting in the session properties. The reader, transformation, and writer threads use buffer blocks to move data from sources to targets. The buffer block size should be larger than the precision for the largest row of data in a source or target.

The Integration Service allocates at least two buffer blocks for each source and target partition. Use the following calculation to determine buffer block requirements:

[(total number of sources + total number of targets)* 2] = (session buffer blocks)

For example, a session that contains a single partition using a mapping that contains 50 sources and 50 targets requires a minimum of 200 buffer blocks.

[(50 + 50)* 2] = 200

You configure buffer memory settings by adjusting the following session properties:

DTM Buffer Size. The DTM buffer size specifies the amount of buffer memory the Integration Service uses when the DTM processes a session. Configure the DTM buffer size on the Properties tab in the session properties.Default Buffer Block Size. The buffer block size specifies the amount of buffer

memory used to move a block of data from the source to the target. Configure the buffer block size on the Config Object tab in the session properties.

The Integration Service specifies a minimum memory allocation for the buffer memory and buffer blocks. By default, the Integration Service allocates 12,000,000 bytes of memory to the buffer memory and 64,000 bytes per block.

If the DTM cannot allocate the configured amount of buffer memory for the session, the session cannot initialize. Usually, you do not need more than 1 GB for the buffer memory.

You can configure a numeric value for the buffer size, or you can configure the session to determine the buffer memory size at run time.

Configuring Automatic Memory Settings

You can configure the Integration Service to determine buffer memory size and session cache size at run time. When you run a session, the Integration Service allocates buffer memory to the session to move the data from the source to the first transformation and from the last transformation to the target. It also creates session caches in memory. Session caches include index and data caches for the Aggregator, Rank, Joiner, and Lookup transformations, as well as Sorter and XML target caches. The values stored in the data and index caches depend on the requirements of the transformation. For example, the Aggregator index cache stores group values as configured in the group by ports, and the data cache stores calculations based on the group by ports. When the Integration Service processes a Sorter transformation or writes data to an XML target, it also creates a cache.

You configure buffer memory and cache memory settings in the transformation and session properties. When you configure buffer memory and cache memory settings, consider the overall memory usage for best performance.

You enable automatic memory settings by configuring a value for the Maximum Memory Allowed for Auto Memory Attributes and the Maximum Percentage of Total Memory Allowed for Auto Memory Attributes. If you set Maximum Memory Allowed for Auto Memory Attributes or the Maximum Percentage of Total Memory Allowed for Auto Memory Attributes to zero, the Integration Service uses default values for memory attributes that you set to auto.

Related Topics: Session Caches

Configuring Buffer Memory

The Integration Service can determine the memory requirements for the following buffer memory:

DTM Buffer SizeDefault Buffer Block Size

You can configure DTM buffer size and the default buffer block size in the session properties. If you specify a numeric value that is less than 12MB for the DTM buffer size, the Integration Service updates the DTM buffer size to 12MB.

When the session requires more memory than the value you configure for the DTM buffer size, session performance decreases and the session can fail.

If the session is configured to retry on deadlock and the value for the DTM buffer size is less than what the session requires, the Integration Service writes the following message in the session log:

WRT_8193 Deadlock retry will not be used. The free buffer pool must be at least [number of bytes] bytes. The current size of the free buffer pool is [number of bytes] bytes.

To configure automatic memory settings for the DTM buffer size:

1. Open the session, and click the Config Object tab. 2. Enter a value for the Default Buffer Block Size.You can specify auto or a numeric value. If you enter 2000, the Integration Service interprets the number as 2,000 bytes. Append KB, MB, or GB to the value to specify other units. For example, specify 512MB.3. Click the Properties tab.4. Enter a value for the DTM buffer size.You can specify auto or a numeric value. If you enter 2000, the Integration Service interprets the number as 2,000 bytes. Append KB, MB, or GB to the value to specify other units. For example, specify 512MB.

Note: If you specify auto for the DTM buffer size or the default Buffer Block Size, enable automatic memory settings by configuring a non-zero value for the Maximum Memory Allowed for Auto Memory Attributes and the Maximum Percentage of Total Memory Allowed for Auto Memory Attributes. If you do not enable automatic memory settings after you specify auto for the DTM buffer size or the default Buffer Block Size, the Integration Service uses default values.

Configuring Session Cache Memory

The Integration Service can determine memory requirements for the following session caches:

Lookup transformation index and data cachesAggregator transformation index and data cachesRank transformation index and data cachesJoiner transformation index and data cachesSorter transformation cacheXML target cache

You can configure auto for the index and data cache size in the transformation properties or on the mappings tab of the session properties.

Configuring Maximum Memory Limits

When you configure automatic memory settings for session caches, configure the maximum memory limits. Configuring memory limits allows you to ensure that you reserve a designated amount or percentage of memory for other processes. You can configure the memory limit as a numeric value and as a percent of total memory. Because available memory varies, the Integration Service bases the percentage value on the total memory on the Integration Service process machine.

For example, configure automatic caching for three Lookup transformations in a session. Then, configure a maximum memory limit of 500 MB for the session. When you run the session, the Integration Service divides the 500 MB of allocated memory among the index and data caches for the Lookup transformations. The maximum memory limit for the session does not apply to transformations that you did not configure for automatic cashing.

When you configure a maximum memory value, the Integration Service divides memory among transformation caches based on the transformation type.

When you configure a maximum memory, you must specify the value as both a numeric value and a percentage. When you configure a numeric value and a percent, the Integration Service compares the values and determines which value is lower. The Integration Service uses the lesser of these values as the maximum memory limit.

When you configure automatic memory settings, the Integration Service specifies a minimum memory allocation for the index and data caches. By default, the Integration Service allocates 1megabyte to the index cache and 2 megabytes to the data cache for each transformation instance. If you configure a maximum memory limit that is less than the minimum value for an index or data cache, the Integration Service overrides the value based on the transformation metadata.

When you run a session on a grid and you configure Maximum Memory Allowed For Auto Memory Attributes, the Integration Service divides the allocated memory among all the nodes in the grid. When you configure Maximum Percentage of Total Memory Allowed For Auto Memory Attributes, the Integration Service allocates the specified percentage of memory on each node in the grid.

Configuring Automatic Memory Settings for Session Caches

To use automatic memory settings for session caches, configure the caches for auto and configure the maximum memory size.

To configure automatic memory settings for session caches:

1. Open the transformation in the Transformation Developer or the Mappings tab of the session properties.

2. In the transformation properties, select or enter auto for the following cache size settings:

Index and data cacheSorter cacheXML cache

3. Open the session in the Task Developer or Workflow Designer, and click the Config Object tab.

4. Enter a value for the Maximum Memory Allowed for Auto Memory Attributes.If you enter 2000, the Integration Service interprets the number as 2,000 bytes. Append KB, MB, or GB to the value to specify other units. For example, specify 512MB. This value specifies the maximum amount of memory to use for session caches. If you set the value to zero, the Integration Service uses default values for memory attributes that you set to auto.

5. Enter a value for the Maximum Percentage of Total Memory Allowed for Auto Memory Attributes.

This value specifies the maximum percentage of total memory the session caches may use. If the value is set to zero, the Integration Service uses default values for memory attributes that you set to auto.

Configuring Performance Details

You can configure a session to collect performance details and store them in the PowerCenter repository. Collect performance data for a session to view performance details while the session runs. Write performance data for a session in the PowerCenter repository to store and view performance details for previous session runs.

If you want to write performance data to the repository you must perform the following tasks:

Configure the session to collect performance data.Configure the session to write performance data to repository.Configure Integration Service to persist run-time statistics to the repository at the verbose level.

The Workflow Monitor displays performance details for each session that is configured to collect or write performance details.

To configure performance details:

1. In the Workflow Manager, open the session properties and select the Properties tab.

2. Select Collect performance data to view performance details while the session runs.

3. Select Write Performance Data to Repository to store and view performance details for previous session runs.

You must also configure the Integration Service to store the run-time information at the verbose level.4. Click OK.