Fast Write for Teradata

CUSTOMER

SAP BusinessObjects Predictive Analytics 3.12017-10-26

Fast Write for TeradataAutomated Analytics Configuration Guide

Content

1 What's New in Fast Write for Teradata. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 About this document. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 Purpose. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

4 Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74.1 Main components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .74.2 Technology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

5 Performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

6 Perimeter of Usage and Limitations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

7 Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127.1 Installing Fastload. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

To install fastload on Windows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12To install fastload on UNIX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13

7.2 Checking the Installation of Fastload. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137.3 Teradata Patches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157.4 Automated Analytics Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Checking the Activation of Fast Write?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

8 Fast Write Usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178.1 Logs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .178.2 Tips and Tricks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

9 Advanced Setup and Tuning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

10 Main Options for Integration Tuning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2110.1 Log and Temporary Objects Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2110.2 Logon Mechanism. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2110.3 Changing Paths & Protocols. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22

11 Main Options for Performance Tuning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

12 Main options for Debug and Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

13 Annex A - Default fastload Script. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2513.1 Built-in Template. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2513.2 Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2 C U S T O M E RFast Write for Teradata

Content

14 Annex B - List of Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Fast Write for TeradataContent C U S T O M E R 3

1 What's New in Fast Write for Teradata

Links to information about the new features and documentation changes for Fast Write for Teradata Guide.

SAP Predictive Analytics 30

What's New Link to More Information

Suppression of any references to Solaris,HPUX & Windows 32 bits as supported platforms.

Performance [page 10]

Perimeter of Usage and Limitations [page 11]

Setup [page 12]

Teradata Patches [page 15]

SAP Predictive Analytics 2.3

What's New Link to More Information

The list of supported platforms and compatibility with Teradata has been updated.

Perimeter of Usage and Limitations [page 11]


What's New in Fast Write for Teradata

2 About this document

This document is addressed to SAP internals and partners who want to understand the Automated Analytics feature Fast Write for Teradata.

In this document, fastload references the Teradata import tool and Fast Write references the Automated Analytics feature built on top of fastload.

Complete documentation can be found on the SAP Help Portal at http://help.sap.com/pa

Fast Write for TeradataAbout this document C U S T O M E R 5

http://help.sap.com/pa

3 Purpose

The application is historically not designed to write a large amount of data in a DBMS.

Writing large amount of data in DBMS typically concerns scoring processes and the In-database feature of the application manages this process entirely in DBMS, thus avoiding slow response.

However, with the integration of new algorithms and the increase of complexity and size of the models, there are now some contexts where the In-database Apply feature cannot be used and the application must write directly in the DBMS, slowing down the score process. Social is a typical example of algorithms suffering from this issue.

The feature Fast Write for Teradata addresses this issue for Teradata and is available since version 6.1 of the application.


Purpose

4 Architecture

4.1 Main components

ODBC

The fastload technology is a fast import technology. It does not provide any way to request meta data information about tables. This is why the Fast Write mode still needs an ODBC connection.

Fast Write

The Fast Write module manages metadata information about target table, prepares a special fast communication channel, generates a fastload script instrumented to make fastload fetch data from a special communication channel and feed this channel as fast as possible. This module also monitors the fastload task in order to provide the user with some feedback.

Fast Write for TeradataArchitecture C U S T O M E R 7

Fastload

This is the best-in-class Teradata’s import technology. It executes the fastload script generated by the application and is able to send data to Teradata using a proprietary protocol.

This tool has a plugin facility (INMOD setup) allowing data to be fetched from an external library.

KxFastLoadBridge

This is a plugin for fastload following Teradata’s INMOD protocol. This plugin connects back to Automated Analytics and fetches data from a special communication channel.

Fastload Script

Except for the INMOD plugin usage, the fastload script generated by the application uses the standard and basic fastload features.

Fast Write Data Stream

The fast communication channel used by Automated Analytics to send data to KxFastLoadBridge is a named pipe.

4.2 Technology

The Fast Write feature uses the best Teradata import tool: fastload. The fastload technology allows the import of huge datasets in a Teradata database with a specialized and efficient proprietary protocol.

Advantages

Among several advantages, fastload is:

● A very common and natural tool in Teradata’s ecosystem. Its installation is straightforward and often, already done.

● A cornerstone tool for Teradata. Therefore, it always benefits from best-in-class Teradata technology (parallel import for example) and is always up-to-date regarding latest Teradata DBMS features.


Architecture

Drawbacks

For the following reasons, the detection of the availability of fastload and the activation of Fast Write are not automatic

● fastload is a Teradata proprietary tool complex to manage and to integrate. Initializing a fastload task is quite heavy with an important bootstrap time and it may be counterproductive to use it for small datasets.

● Due to specificities of the fastload protocol, a bad tuning may use too many resources in Teradata DBMS and therefore slow other activities.

● fastload tasks are typically a sparse resource in a Teradata DBMS. Number of concurrent and available fastload tasks may be small (default is 15).

For more information, contact SAP Support.

Fast Write for TeradataArchitecture C U S T O M E R 9

5 Performance

Compared to ODBC, the default tuning of Fast Write allows you to enhance performances by a coefficient of 100 minimum. A fastload task comprise of two steps: the acquisition phase and the application phase.

Phase I: Acquisition Phase

During the acquisition phase, rows are sent as fast as possible to the DBMS and are stored in a temporary spool space. This phase is heavily dependent on the actual network bandwidth usage and the capacity of the client to send data as fast as possible.

Phase II: Application Phase

During the application phase, the data stored in the temporary space is dispatched into Teradata nodes. This phase does not use any network resource and is entirely done inside Teradata’s DBMS.

Test platforms

In a day-to-day basis, the application uses small test platforms for Teradata. These platforms have hardware limitations and for these platforms, phase II takes some time. Using a true Teradata hardware, this phase II is almost free.


Performance

6 Perimeter of Usage and Limitations

Supported Platforms

Client OS Teradata Client Teradata DBMS

Windows 64-bit TTUF 13.1 Teradata 13.1

Linux 64-bit TTUF 13.1 Teradata 13.1








Caution1. For TTUF 14.10, patch for fastload 14.10.00.03 is mandatory

Compatibility with TD 14.0 & TD 14.1 & TD 15.0 & TD 15.10

Fast Write is automatically compatible with the enhancements on huge resultsets provided in TD 14.0, on extended object names provided in TD 14.1, and with the enhancements provided in TD 15.0 and TD 15.10.

Limitations

The limitations of Fast Write are the limitations of fastload:

● FastWrite is activated only on non-existing or empty tables.● Blobs cannot be used.

Fast Write for TeradataPerimeter of Usage and Limitations C U S T O M E R 11

7 Setup

ODBC

The fastload tool does not provide any metadata information. Therefore, the ODBC technology is used to discover metadata information about data (list and type of fields) and setup fastload tasks.

This is why the ODBC driver must be properly setup even if Fast Write is activated.

NoteODBC installation and setup are described in the support document Connecting to your Database Management System.

32/64-bit fastload Packages

Fastload is a 32-bit technology. Even on a 64-bit OS and with Automated Analytics in 64-bit, fastload and all its dependencies are 32-bit Teradata packages.

Globally, to be able to use the Fast Write feature, the following Teradata packages need to be installed:

● fastload 32-bit● fastload dependencies● fdicu 32-bit● teragss 32-bit● cliv2 32-bit● DataConnector (piom) 32-bit● tdodbc 64-bit● tdodbc dependencies● tdicu 64-bit● teragss 64-bit

7.1 Installing Fastload

7.1.1 To install fastload on Windows

The fastload packages can be found on the Teradata CD labeled:

Teradata Tools and Utilities Load/unload for Windows


Setup

Release 13.10.01 Volume: 01/03

1. Insert the CD and select Custom Setup.2. When prompted to select packages, select fastload.

All dependencies are automatically selected.3. Confirm the summary dialog.

The installation is then straightforward.4. Once fastload has been set up, check that the installation has been correctly executed (see the topic To

Check the Installation of Fastload).

7.1.2 To install fastload on UNIXThe fastload packages can be found on the Teradata CD labeled:

Teradata Tools and Utilities Load/Unload for HP-UX & Linux or Load/Unload for AIX & Solaris Release 13.10.01 Volume: 02/03 or 03/03

1. Insert the CD that contains the Teradata Client Software to install. When inserted, the CD will allow selecting the Teradata packages to install.

2. Select fastload, which is the only top-level package needed for the application. The dependent packages tdicu, TeraGSS, DataConnector and cliv2 will automatically be selected.

3. Once fastload has been set up, check that the installation has been correctly executed.

7.2 Checking the Installation of FastloadIt is very important that fastload is properly set up for the Fast Write feature to work. If the following procedure fails, stop the Fast Write setup and contact support.

1. Open a command window (on Windows) or an interactive shell (on UNIX).2. At prompt, type fastload. If fastload is properly set up, you should see:

Fast Write for TeradataSetup C U S T O M E R 13

3. Type .logon <Teradata host name>/<user>,<password>

Note<Teradata host name> follows Teradata naming policy or can be an IP address as shown in the example above.

4. Check if the logon is successful (FDL4808 LOGON successful).

5. Type .logoff; to disconnect

6. Check that the whole fastload task has terminated properly: the message Highest return code encountered = ‘0’ should be displayed.


Setup

CautionAny error in this procedure is fatal. The fastload tool must be properly setup before Fast Write can be activated.

7.3 Teradata Patches

It is always a good idea to update your fastload installation with up-to-date patches. The process to download and apply Teradata patches depends on each Teradata site and must be done by an authorized user.

Here are the patched versions of Teradata packages the application is currently using to validate Fast Write:

OS Architecture Package Version

Linux 32 tdicu 13.10.00.02

Linux 32 TeraGSS 13.10.04.01

Linux 32 cliv2 13.10.00.11-1

Linux 32 DataConnector 13.10.00.09-1

Linux 32 fastload 13.10.00.12-1

Windows 32 tdicu 13.10.00.02

Windows 32 TeraGSS 13.10.05.1

Windows 32 cliv2 13.10.00.08

Windows 32 piom 13.10.00.09

Windows 32 fastload 13.10.00.13

7.4 Automated Analytics Setup

Fast Write must be manually activated for each ODBC connection. This is done by editing the configuration file Fast Write.cfg and adding the following line for each ODBC connection that should use Fast Write:ODBCStoreSQLMapper.<ODBC DSN>.SupportBulkImport=true.

ExampleIf the connection ODBC to accelerate is BigData, the line to add is: ODBCStoreSQLMapper.BigData.SupportBulkImport=true

NoteInstead of <ODBC Name>, the wildcard * can be used and means ‘all ODBC connections’.

Fast Write for TeradataSetup C U S T O M E R 15

7.4.1 Checking the Activation of Fast Write?

A simple way is to use the application to make a data transfer from sample files to Teradata.

1. Launch SAP BusinessObjects Predictive Analytics.

2. Select Toolkit> Perform a Data Transfer .3. Choose the sample file Census01.csv4. Transfer it using a Teradata ODBC connection with Fast Write enabled. The transfer should be very fast

(~10s).5. Click the detailed log button to see the whole logs for Fast Write. Actual values of throughput and times

depend on the current configuration.


Setup

8 Fast Write Usage

Standard Usage

After the initial setup, the Fast Write feature is designed to be automatically used in a transparent way. Specifically, there is no special action associated to Fast Write in the graphical interface.

8.1 Logs

Fast Write uses a complex architecture. Each component of this architecture provides logs in order to ease support and tuning.

Globally, the application drives these components to name all these logs using the same default policy. Each log is named KxFastLoad_<Unique_Session_Number>_<topic>.

Fastload Temporary Log Tables

For each fastload task, the fastload tool creates two log tables named:

● KxFastLoad_<Unique_Session_Number>_Log1● KxFastLoad_<Unique_Session_Number>_Log2

These tables are populated only in case of a data issue (record rejected due to constraint violation, bad data format, …) and actual content can be analyzed only by a DBA. The database used to store these tables is the same as the target table.

These tables are automatically deleted if no issue is detected.

Temporary Objects and Logs

For each Fast Write session, three files are created:

● KxFastLoad_<Unique_Session_Number>_script.fld: the fastload script generated by the application

● KxFastLoad_<Unique_Session_Number>_log: the log file generated by fastload● KxFastLoad_<Unique_Session_Number>_err: the log file generated by KxFastLoadBridge

These files are created in the standard temp directory and are automatically deleted if no issue is detected.

Several configuration options allow tuning the name of these files, their folder,… and are described in the Annex B - List of Options.

Fast Write for TeradataFast Write Usage C U S T O M E R 17

Logs & Error Management

In case of any error or warning (logon issue, network or DBMS issue, bad format, ...) reported by fastload, the application does not delete any log and displays a message asking to collect these files and send them to the SAP support.

Example

8.2 Tips and Tricks

Tips and Tricks

The fastload technology uses a special import feature of the Teradata DBMS; while running it sets up the target table in a specific state (as well as both log tables). While in this state, a table cannot be used for the standard SELECT,INSERT,UPDATE or DELETE information. Any attempt at doing that will generate the following diagnostic: Operation not allowed: <table name>l is being Loaded.

After an error in a fastload task, the target and log tables are kept in this special ‘loading state’ and therefore cannot be used. Only way to go further is then to delete the target and log tables.

Cleaning Process After an Error:

1. Check if a fastload process is still running (rare condition). If yes, this process must be killed before going further.

2. Drop the target table.3. Drop the two log tables KxFastLoad_<unique_number>_Log1 and KxFastLoad_<unique_number>_Log2.

Always Provide Logon Information

With the Teradata ODBC driver, it is common to store the logon information (user name and password) in the ODBC setup dialog (Windows) or the odbc.ini file (UNIX). When used, this allows you to type the logon credentials and quickly connecting to Teradata.


Fast Write Usage

This facility is not managed by the fastload tool (ODBC setup and fastload setup are separated); when Fast Write is used, the logon credentials MUST be provided.

Otherwise, on Windows, an error message is displayed.

NoteIf providing the credentials is an issue, some tuning options allow you to control this check and to define a default user and a default password.

And on UNIX, an error is triggered.

Automated Analytics Metadata/Small Tables and Fast Write Performances

The Fast Write feature is really fast but needs some bootstrap time and important resources. As a result, using it for small tables is counterproductive.

Some Automated Analytics metadata tables are always small and are automatically not eligible for the Fast Write mode; however the application does not evaluate the size of regular data to see if it is worth using Fast Write .

Models also need a special attention when saved in metadata: the first save will be done automatically with the Fast Write mode; but saving the model a second time in the same table will not use it and will be slow.

NoteTo always benefit from the Fast Write mode, always save your models in new tables.

Fast Write for TeradataFast Write Usage C U S T O M E R 19

9 Advanced Setup and Tuning

The Fast Write feature has many options allowing you to tune various aspects of this module.

Default suitable values are built-in in the application but for specific purposes or contexts, each option can be overridden.

An option has a name and a value.

To Override an Option Value:

1. Open the configuration file Fast Write.cfg.2. Add a line:

Fast Write.<ODBC Connection Name>.<Option>=<Value>For example:Fast Write.BigData.DropTempFiles=false

NoteThe wildcard * can be used instead of <ODBC Connection Name> meaning ‘all connections’.


Advanced Setup and Tuning

10 Main Options for Integration Tuning

10.1 Log and Temporary Objects Management

DropLogTablesBefore

The default behavior is to drop log tables before any action. Change the value to No if this cleaning must not be done.

LogSchemaName

The log tables are stored in the same schema as the target table. Change the value of this option to force a specific schema where all log tables will be stored.

TempPrefix

KxFastLoad_<Unique_Number>_is the default prefix added before every temporary object name. Change the value of this option to force a user defined prefix.

You should use this option only for debug purposes; it will force a single name for all fastload tasks and forbid to have several fastload tasks at the same time.

10.2 Logon Mechanism

Host

The Teradata host to contact is automatically inferred from the current ODBC connection. If the host is not properly guessed, use this option to force the correct host name.

UserName, Password

The user/password sent to fastload are the same as the ones used by the current ODBC connection. You can change these values to force a specific user name/password.

LogMech, LogData, Logon

Fast Write uses the default fastload mechanism for logon. Use these options to force a specific logon mechanism and its parameters (Active Directory,..). Please consult the fastload documentation for proper usage of these parameters.

CheckEmptyUserName

Fast Write checks that the user name is not empty. Set this option to deactivate this test.

DefaultUserName

This option allows you to define a default user name. This default user name will be used by Fast Write when no user name is provided. Note that this option is used only if the option CheckEmptyUserName has been deactivated.

Fast Write for TeradataMain Options for Integration Tuning C U S T O M E R 21

DefaultPassword

This option allows you to define a default password. This password will be used by Fast Write when no password is provided.

10.3 Changing Paths & Protocols

FastLoadPath

The standard path for the fastload tool is "" on Windows and /usr/bin on UNIX. Change this option if fastload has not been set up in the default folders.

FastLoadName

Fastload is the name of the tool launched by Fast Write. Change this option if a non-standard name is used.

KxFastLoadBridgeFullPath

Change this option if your installation is not standard and if the location of KxFastLoadBridge cannot be automatically found.

EXITProtocol

Fast Write automatically manages extended fastload protocol coming with TD 14.0 & TD 14.1.If for any reason, this automatic choice is not properly done, this option allows you to force a given protocol. Possible values are indicated in the following table:

EXIT Compatible with all versions of Teradata

EXIT64 Available only in TD 14.1. 64-bits events are managed.

EXITEON Available only in TD 14.1. EON events are managed. Note that EXITEON implies EXIT64.


Main Options for Integration Tuning

11 Main Options for Performance Tuning

TimeBetweenPerformancesInfos

Automated Analytics displays some performance information (Nb rows inserted, Current Nb Rows /s) every ten seconds. This option allows you to change this delay.

BlockSize

Rows are encoded in a binary format suitable for fastload and sent by blocks to fastload. The default size of a block is 1048576 (1MB). Change this option to tune the size of the blocks.

MaxQueueSize

Binary blocks are asynchronously sent to fastload, meaning that some blocks may be queued waiting until fastload can treat them. The maximum size of this queue is 20. Basic statistics about usage of this queue are available in the application log:

● maximum actual size,● time spent in full queue state (fastload overloaded),● time spent with no block available for fastload (starvation).

Both options MaxQueueSize and BlockSize allow you to tune the block queue and maximizing throughput of rows.

SessionMin and SessionMax

These parameters handle the same name fastload parameters.

The concept of low level fastload sessions is key to maximizing the performances. Changing these options must be validated by an SAP representative or a Teradata administrator.

The default value for SessionMin is 1, SessionMax is 4. This setup always allows you to initiate a Fast Write session:

● with an immediate performance improvement, even for ‘small’ datasets,● without consuming too many Teradata’s resources.

You can change these parameters with the help of a Teradata DBA or your application support if the number of low level fastload sessions is an identified issue or if full maximization of performances is required, especially for huge data sets.

Fast Write for TeradataMain Options for Performance Tuning C U S T O M E R 23

12 Main options for Debug and Support

TemplateScriptFullPath

The fastload script used is generated from a built-in template script.

It is possible to force the usage of a specific template script by using this option.

DropTempFiles

All temporary files are automatically deleted when no issue is detected. It can be useful to force this option to No in order to check the logs for a successful session.

ReportFastloadWarning

Fastload is a complex tool reporting various conditions depending on actual production setup. The default behavior is to report any warning emitted by fastload. It may be useful to deactivate this reporting if the warning condition has no practical impact. This deactivation must be done with the counsel of SAP.


Main options for Debug and Support

13 Annex A - Default fastload Script

Usage

The fastload script actually used by is generated from a built-in template.

This template has many %OptionName% patterns. Each pattern is replaced by its actual value at generation time. The actual value is either generated by the application or found in the Fast Write.cfg configuration file.

Each %<OptionName>% pattern is a possible <OptionName> option.

13.1 Built-in Template

** Generated by %KXEN_VERSION% ** %DATE%SET SESSION CHARSET “%SESSIONCHARSET%”; SHOW VERSION; SESSIONS %SESSIONMAX% %SESSIONMIN%; ** ** Connection step**** These 3 lines will be used if the options LogMech,LogData,Logon are set *** LOGMECH %LOGMECH%; *** LOGDATA %LOGDATA%; *** LOGON %LOGON%; LOGON %HOST%/%USERNAME%,%PASSWORD%; ** The initial cleaning lines are not added if the DropLogTablesBefore option is unset** Cleaning logs** DROP TABLE %FULLLOGTABLENAME1%; DROP TABLE %FULLLOGTABLENAME2%; ** Define all fields to work with** DEFINE %FIELDSDEFINITIONS%** Usage of INMOD feature INMOD=%KXFASTLOADBRIDGEFULLPATH%; ** NOTIFY allows monitoring session steps ** %EXIT% specifies an external handler to publish notifications** TEXT allows giving parameters to establish communications with KXEN NOTIFY HIGH %EXIT% % KXFASTLOADBRIDGEFULLPATH % TEXT “PP=%PRIVATEPREFIX% TL=%TRACELEVEL% TF=%TRACEFILE% TE=%TRACESTDERR%";** The actual start of the fastload session ** BEGIN LOADING %FULLTABLENAME% ERRORFILES % FULLLOGTABLENAME1% % FULLLOGTABLENAME2% INDICATORS; ** The INSERT statement using fastload fields defined by 'DEFINE' statement** INSERT INTO %FULLTABLENAME% (%FIELDNAMES%) VALUES(%FIELDSDEFINES%);** End of job ** END LOADING;

Fast Write for TeradataAnnex A - Default fastload Script C U S T O M E R 25

13.2 Example

** Generated by KXEN InfiniteInsight - KXEN Internal for development purpose only - Valid to 2013-01-15 6.1.0-b1 ** 2012-07-18 10:03:32**SET SESSION CHARSET "ASCII";SHOW VERSION; ** Connection step**LOGON 10.1.1.218/kxenodbc,kxenodbc;** Cleaning logs** DROP TABLE KxFastLoad_1_7308_Log1; DROP TABLE KxFastLoad_1_7308_Log2;** Define all fields to work with**** With usage of INMOD feature**DEFINEf0 (BIGINT), f1 (BIGINT),f2 (BIGINT), f3 (BIGINT), f4 (BIGINT), f5 (BIGINT), f6 (BIGINT),f7 (BIGINT), f8 (BIGINT), f9 (BIGINT) INMOD=KxFastLoadBridge.dll; ** NOTIFY allows to monitor session steps** EXIT specifies an external handler to publish notifications ** TEXT allows to give parameters to establish communications with KXEN NOTIFY HIGH EXIT KxFastLoadBridge.dll TEXT "PP=KxFastLoad_1_7308 TL=Detailed TF=Pub TE=On"; ** The actual start of fastload session**BEGIN LOADING acgafwdo ERRORFILES KxFastLoad_1_7308_Log1, KxFastLoad_1_7308_Log2 INDICATORS; ** The INSERT statement using fastload fields defined by 'DEFINE' statement **INSERT INTO acgafwdo (id, i1, i2, i3, i4, i5, i6, i7, i8, i9) VALUES (:f0, :f1, :f2, :f3, :f4, :f5, :f6, :f7, :f8, :f9); ** End of job **END LOADING; QUIT


Annex A - Default fastload Script

14 Annex B - List of Options

Option Category Default value Comment

fastloadName Integration fastload Name of fastload executable

fastloadPath Integration "" or /usr/bin Path of fastload executable

EXITProtocol Integration EXIT or EXITEON

FullKxInfosScriptPath

Integration <temp dir>/%TempPrefix%_trc.txt

Trace from KxFastLoadBridge

TemplateScriptFullPath

Integration Allows you to use a specific template

TempPrefix Integration KxFastLoad_<Unique number>

Added to every temp or log object

Date Script Current date

DropLogTablesBefore Script Yes Drop log tables before any action

EXITProtocol Script EXITEON for TTUF 14.10. EXIT otherwise

Extended Object Names (EON) are new feature of Teradata 14.1

FieldNames Script generated from target table definition

Field names in fastload syntax

FieldsDefinition Script generated from target table definition

Field definitions in fastload format

FieldValues Script generated from target table definition

Field values in fastload syntax

FullLogTableName1 Script %LogSchemaName%.%TempPrefix%_log1

Log table generated by fastload

FullLogTableName1 Script %LogSchemaName%.%TempPrefix%_log2

Log table generated by fastload

FullResultScriptPath

Script <temp dir>/%TempPrefix%_log.txt

fastload execution log

FullScriptPath Script <temp dir>/%TempPrefix%_script.fld

Full path of generated script

FullTableName Script %SchemaName%.%TableName%

Target table specification

Host Script Teradata host used by ODBC connection

Allows to force a Teradata gateway

KxFastLoadBridgeFullPath

Script generated from Automated Analytics installation

Full path of KxFastLoadBridge

Fast Write for TeradataAnnex B - List of Options C U S T O M E R 27

LogData Script Use this option and the 2 following options to setup an advanced authentication scheme (cf fastload documentation)

LogMech Script

Logon Script

CheckEmptyUserName Integration true

DefaultUserName Integration empty

DefaultPassword Integration empty

LogSchemaName Script generated from Target table Allows you to force a specific schema for log tables

LogTableName1 Script %TempPrefix%_<Unique number>_log1

LogTableName2 Script %TempPrefix%_<Unique number>_log2

Password Script password from ODBC connection

SchemaName Script generated from Target table Schema part of target table spec

SessionCharset Script ASCII or UTF16 Can be also UTF8

TableName Script The target table Table part of target table spec

UserName Script user name from ODBC connection

DropTempFiles Support Yes Temp files are kept only in case of error

DumpOptionsBeforefastload

Support No Dump all options in SQL log

Mode Support Advanced ‘Synchronous’ allows you to use a slower but simpler fastload dialog

TraceFile Support Pub Managed by Automated Analytics only

TraceLevel Support Detailed Managed by Automated Analytics only

BlockSize Tuning 1048576 (1MB) Size of a fastload binary block

MaxQueueSize Tuning 20 Max # of blocks waiting to be sent

SessionMax Tuning Use these parameters with caution. Cf fastload documentation


Annex B - List of Options

SessionMin Tuning Use these parameters with caution. Cf fastload documentation

TimeBetweenPerformancesInfos

Tuning 10s

Fast Write for TeradataAnnex B - List of Options C U S T O M E R 29

Important Disclaimers and Legal Information

Coding SamplesAny software coding and/or code lines / strings ("Code") included in this documentation are only examples and are not intended to be used in a productive system environment. The Code is only intended to better explain and visualize the syntax and phrasing rules of certain coding. SAP does not warrant the correctness and completeness of the Code given herein, and SAP shall not be liable for errors or damages caused by the usage of the Code, unless damages were caused by SAP intentionally or by SAP's gross negligence.

AccessibilityThe information contained in the SAP documentation represents SAP's current view of accessibility criteria as of the date of publication; it is in no way intended to be a binding guideline on how to ensure accessibility of software products. SAP in particular disclaims any liability in relation to this document. This disclaimer, however, does not apply in cases of willful misconduct or gross negligence of SAP. Furthermore, this document does not result in any direct or indirect contractual obligations of SAP.

Gender-Neutral LanguageAs far as possible, SAP documentation is gender neutral. Depending on the context, the reader is addressed directly with "you", or a gender-neutral noun (such as "sales person" or "working days") is used. If when referring to members of both sexes, however, the third-person singular cannot be avoided or a gender-neutral noun does not exist, SAP reserves the right to use the masculine form of the noun and pronoun. This is to ensure that the documentation remains comprehensible.

Internet HyperlinksThe SAP documentation may contain hyperlinks to the Internet. These hyperlinks are intended to serve as a hint about where to find related information. SAP does not warrant the availability and correctness of this related information or the ability of this information to serve a particular purpose. SAP shall not be liable for any damages caused by the use of related information unless damages have been caused by SAP's gross negligence or willful misconduct. All links are categorized for transparency (see: http://help.sap.com/disclaimer).


Important Disclaimers and Legal Information

http://help.sap.com/disclaimer/

Fast Write for TeradataImportant Disclaimers and Legal Information C U S T O M E R 31

go.sap.com/registration/contact.html

© 2017 SAP SE or an SAP affiliate company. All rights reserved.No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company. The information contained herein may be changed without prior notice.Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors. National product specifications may vary.These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. All other product and service names mentioned are the trademarks of their respective companies.Please see http://www.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.

https://go.sap.com/registration/contact.html

https://go.sap.com/registration/contact.html

http://www.sap.com/corporate-en/legal/copyright/index.epx

http://www.sap.com/corporate-en/legal/copyright/index.epx

Fast Write for Teradata

Documents

Transcript of Fast Write for Teradata