Using SAS Software to Retrieve - Stevens Institute of ... · PDF fileAll FTP requests that are...

Using SAS® Software to Retrieve

Thomson Reuters Tick-History Data TS-810

i

Using SAS® Software to Retrieve Thomson Reuters Tick-History Data

Table of Contents

Introduction .................................................................................................................... 1

Important Considerations ............................................................................................. 1

Using Macros to Submit and Retrieve TRTH Requests ............................................. 2

Initial Setup ................................................................................................................ 2

Submission Request and Retrieval ........................................................................... 7

Example ................................................................................................................... 11

Appendix: Macro Definitions ..................................................................................... 12

%ASSIGNMENTS ................................................................................................... 12

Description ........................................................................................................ 12

Syntax ............................................................................................................... 12

Parameters ........................................................................................................ 12

%EXEC .................................................................................................................... 13

Description ........................................................................................................ 13

Syntax ............................................................................................................... 13

Parameters ........................................................................................................ 13

%EXECFTP ............................................................................................................. 14

Description ........................................................................................................ 14

Syntax ............................................................................................................... 14

Parameters ........................................................................................................ 14

%GET ...................................................................................................................... 15

Description ........................................................................................................ 15

Syntax ............................................................................................................... 15

Parameters ........................................................................................................ 15

%GETRESULTS ...................................................................................................... 16

Description ........................................................................................................ 16

Syntax ............................................................................................................... 16

Parameters ........................................................................................................ 16

1


Introduction

“Thomson Reuters Tick History provides millisecond-time-stamped tick data going back over eleven years, covering 45

million OTC and exchange-traded instruments worldwide.”1

The current process for importing Thomson Reuters Tick History (TRTH) data into SAS® requires that you place a request

using the TRTH Web interface. Then you wait for the query to finish before downloading the data onto a local drive. The

data are then imported into a SAS interface by using a point-and-click wizard or by submitting SAS code. A SAS data set

is created, which is then ready for analysis within the SAS system.

A program is available that uses the TRTH application programming interface to request and fetch tick-history data from

within SAS. Several new dictionary macro definitions (%ASSIGNMENTS, %EXEC, %EXECFTP, %GET,

%GETRESULTS) are also available, which provide supporting functionality to perform this task. This program removes a

substantial amount of processing time that is required to request and fetch data using the TRTH Web interface. It does

this by enabling you to request and fetch the data using SAS macro functions, which store the data locally in the form of a

SAS data set. Furthermore, the code enables you to submit and monitor multiple FTP requests simultaneously for queries

that fetch large amounts of data. This enables you to submit and track requests without having to wait for the query to

finish processing on the server. User credentials are saved within the SAS session and are automatically resubmitted to

avoid the issue of credentials timing out.

This paper shows you how to run the code and macros that enable you to request and fetch tick-history data from within

SAS. This document is intended for programmers who understand the TRTH data source and the interface it provides.

Information about each macro is provided in the Appendix of this paper. You can download the code and macros2 from the

SASTRTHAPI_v1.2.zip file.

Before you get started, read the important considerations that are explained in the next section. Then, follow the step-by-

step instructions. Sample code is provided in the final section of this paper.

Important Considerations

In order to run the code and macros that enable you to request and fetch tick-history data from within SAS, the following

software requirements apply:

Operating Systems: Windows 32-bit and 64-bit platforms

SAS System Software: SAS® 9.3

SAS Software: SAS Display Manager and SAS® Enterprise Guide® (No Profile Selected)

Third-Party Software: 7-Zip open-source file-compression software, which you can download from the 7Zip Web site

Additional Requirements: A valid TRTH user account

1 Thomson Reuters, “Thomson Reuters Tick History.” 2012. Available at thomsonreuters.com/products_services/financial/financial_products/a-z/tick_history/. 2 The sample files and code examples are provided by SAS Institute Inc. "as is" without warranty of any kind, either express or implied, including but not limited

to the implied warranties of merchantability and fitness for a particular purpose. Recipients acknowledge and agree that SAS Institute shall not be liable for any

damages whatsoever arising out of their use of this material. In addition, SAS Institute will provide limited support for the materials contained herein.

http://support.sas.com/resources/papers/SASTRTHAPI_v1.2.zip

http://www.7-zip.org/

http://thomsonreuters.com/products_services/financial/financial_products/a-z/tick_history/

2


Note the following:

The password that is associated with the TRTH account should not include special characters, such as single

quotation marks ( ‘ ), double quotation marks ( “ ), ampersands ( & ), or percent signs ( % ).

Most fields are case sensitive. Therefore, ensure that the exact values of the parameters are passed to the

%GET, %EXEC, and %EXECFTP macros.

Ensure that there are no spaces in the macro variable TRTHdir and that the variable ends with a backslash (\).

Otherwise, the macros will not run correctly.

All FTP requests that are made via the %EXECFTP macro are recorded in the data set TRTH.FTPRequests.

Once recorded in the data set, the requests can be tracked and then accessed via the %GETRESULTS macro.

The requests made via %EXEC are not recorded because they are purged as soon as %GETRESULTS is

submitted for the relevant request ID.

Supported values for the REQTYPE, MSGTYPE, and FIELDLIST parameters in %EXEC and %EXECFTP are

provided in the data sets TRTH.gettypes and TRTH.reqtypes. These data sets are created when you run the

macro %ASSIGNMENTS.

Using Macros to Submit and Retrieve TRTH Requests

The following steps show you how to use the macros that you downloaded from the SASTRTHAPI_v1.2.zip file to submit

and retrieve TRTH requests. The first section guides you through the initial setup. This setup only needs to be completed

at the beginning of each SAS session. The next section guides you through submitting and retrieving requests. The last

section provides an example.

Initial Setup

1. Extract the files from the SASTRTHAPI_v1.2.zip file to a directory of your choice. Ensure that there are no spaces in

any of the folder names.



3


2. Start a SAS session in either SAS Enterprise Guide or SAS Display Manager.

3. Assign a global macro variable TRTHdir to the directory that you specified in Step 1. Make sure that there are no

spaces in any of the folder names and that the path ends in a backslash (\). The directory path must also be

recognized by the server from which the code is executed as a fully qualified path. An example is shown in the

following displays.

For example, submit the following: %let TRTHdir=C:\TRTH\ThisFolderNameHasNoSpacesInIt\;

4


4. Submit the code: %include "&TRTHdir.TRTHFiles\TRTHMacros.sas";

5


5. Navigate to the directory and executable 7z.exe. Note that the .exe extension might not appear depending on your

folder settings. In this case, select the file called 7z for 7-Zip.

6


6. Submit the macro %ASSIGNMENTS with the appropriate parameters.

7


Two tables are created: TRTH.gettypes and TRTH.reqtypes

Submission Request and Retrieval

1. To send a request for a single security for a single day, submit the macro %EXEC with the appropriate parameters.

To send a request for a single security for more than one day, submit the macro %EXECFTP with the appropriate

parameters. The parameters REQTYPE (request type), MSGTYPE (message type), and FIELDLIST (list of fields to

display), which are used in both macros, are discussed in Step 2.

8


Example for Requesting Tick-History Data for a Single Security for a Single Day The following code requests “Price” and “Volume” data for “Trades” of type “Time and Sales” for security BHP.AX on the date 2011-12-15 between the hours of 00:00:00 and 23:59:59.999. %exec(BHP.AX,2011-12-15,00:00:00,23:59:59.999,TimeAndSales,Trade,%bquote(Price,Volume))

If the security exists, a data set TRTH.VerifyRIC is included in the output that shows details of the security.

2. If you are unsure of the combination of supported values that are available for the REQTYPE, MSGTYPE, and

FIELDLIST parameters, submit the macro %GET. Give the first parameter the value MessageTypes and give the

second parameter a value of RequestType from the list of supported request types. This list was created in Step 6 in

the “Initial Setup” section and stored in the data set TRTH.reqtypes.

9


For example, you want to request “TimeAndSales” data but do not know which values are supported for the

parameters MSGTYPE and FIELDLIST. Submitting the following code provides the list of supported values for

REQTYPE “TimeAndSales”:

%get(MessageTypes,TimeAndSales)

The TRTH.MessageTypes data set is created, which includes the list of supported values.

10


3. Fetch the results from the preceding request by submitting the macro %GETRESULTS. For example, submit the following: %getresults(&reqID)

The results are stored in the TRTH library with a data set name of the security and submission date.

11


Example

This example summarizes the preceding step-by-step instructions and provides some additional examples.

The following code provides the required initial setup:

%let TRTHdir=C:\TRTH\ThisFolderNameHasNoSpacesInIt\; %include "&TRTHdir.TRTHFiles\TRTHMacros.sas"; %assignments(TRTHUserName,TRTHPassword,C:\Program Files\7-Zip\7z.exe)

You might be interested in data of type “Time and Sales” but might not be aware of the parameter values that are

supported for message type and field list returned, which are the fifth and sixth parameters in the macro %EXEC. The

following code retrieves a list of supported values for these fields.

%get(MessageTypes,TimeAndSales)

The following code requests “Price” and “Volume” data for “Trades” of type “Time and Sales” for security BHP.AX on the

date 2011-12-15 between the hours of 00:00:00 and 23:59:59.999.

%exec(BHP.AX,2011-12-15,00:00:00,23:59:59.999,TimeAndSales,Trade,%bquote(Price,Volume)) %getresults(&reqID)

The additional %EXECFTP macro requests “Price” data for “Trades” of type “Time and Sales” for security BHP.AX for the

period 2011-12-15 00:00:00 to 2011-12-31 23:59:59.999.

%execftp(BHP.AX,2011-12-15,00:00:00,2011-12-31,23:59:59.999,TimeAndSales,Trade,Price)

Because the amount of data requested is much larger than the %EXEC request, additional parameters can be specified to

indicate how many attempts should be made to request the result from the server and how long to wait between each

attempt. In the following example, 10 attempts will be made, with a wait time of 60 seconds between each attempt.

%getresults(&reqID,10,6)

12


Appendix: Macro Definitions

%ASSIGNMENTS

Description

The %ASSIGNMENTS macro creates most of the global macro variables that are used by the other macro definitions.

It also sets some global options and creates up to three SAS data sets—TRTH.Gettypes, TRTH.Reqtypes, and

TRTH.FTPRequests. The data set TRTH.Gettypes provides a list of valid values that can be passed to the parameter

GETTYPE in the macro %GET. The data set TRTH.Reqtypes provides a list of valid values that can be passed to the

parameter REQTYPE in the macros %GET, %EXEC, and %EXECFTP. The data set TRTH.FTPRequests is also

created if it does not already exist. This data set records all of the FTP requests that are submitted.

Syntax

%assignments(_uid,_pw,_compexe)

Parameters

_uid The TRTH user name.

_pw The TRTH password.

_compexe The complete file path including the executable to 7z.exe.

13


%EXEC

Description

The %EXEC macro definition submits a tick-history request to the Thomson Reuters server using the credentials that

are supplied in %ASSIGNMENT for a single security for a single day and time range. A successful submission

assigns a request ID into the macro variable reqID that is associated with this request. This request ID can then be used to fetch the underlying data by submitting %getresults(&reqID).

Syntax

%exec(sec,date,starttime,endtime,reqtype,msgtype,%bquote(fieldlist))

Parameters

SEC The name of the security.

DATE The date of inquiry in the format 'yyyy-mm-dd'.

STARTTIME The start time for the above date in the format 'hh:mm:ss.sss'.

ENDTIME The end time for the above date in the format 'hh:mm:ss.sss'.

REQTYPE The request type. See TRTH.Reqtypes for a list of acceptable values.

MSGTYPE Message type—For a list of acceptable values, submit %get(MessageTypes,reqtype), where reqtype is the request type

specified above.

FIELDLIST The list of fields to be returned—Ensure that the fields are separated by commas and contained within %bquote(). The valid values can be ascertained by

submitting %get(MessageTypes,reqtype), where reqtype is the request

type specified above.

14


%EXECFTP

Description

The %EXECFTP macro definition submits a tick history request to the Thomson Reuters server for a single security

for a user-specified date-time period. A successful submission assigns a request ID into the macro variable reqID,

which is associated to this request. This request ID can then be used to fetch the underlying data by calling the macro %getresults(&reqID). Because FTP requests can take time to process on the Thomson Reuters server, all

FTP requests, including the request ID are recorded in TRTH.FTPRequests. This enables the user to refer back to a particular request in order to retrieve the results by submitting %getresults(&reqID).

Syntax

%execftp(sec,startdate,starttime,enddate,endtime,reqtype,msgtype,fieldlist)

Parameters

SEC The name of the security.

STARTDATE The start date of inquiry in the format 'yyyy-mm-dd'.

STARTTIME The start time for the start date in the format 'hh:mm:ss.sss'.

ENDDATE The end date of inquiry in the format 'yyyy-mm-dd'.

ENDTIME The end time for the end date in the format 'hh:mm:ss.sss'.

REQTYPE The request type. See TRTH.Reqtypes for a list of acceptable values.

MSGTYPE Message type—For a list of acceptable values, submit %get(MessageTypes,reqtype), where reqtype is the request type

specified above.

FIELDLIST The list of fields to be returned—Ensure that the fields are separated by commas and contained within %bquote(). The valid values can be ascertained by

submitting %get(MessageTypes,reqtype), where reqtype is the request

type specified above.

15


%GET

Description

The %GET macro fetches the list of possible values for the value of GETTYPE. The parameter REQTYPE is required

only when GETTYPE equals “MessageTypes.” It can be left as blank or not included for all other cases, for example, %get(Currencies).

This macro is useful in finding the supported values of REQTYPE, GETTYPE, and FIELDLIST parameters used in

%EXEC and %EXECFTP.

Syntax

%get(gettype,reqtype)

Parameters

GETTYPE The supported values for GETTYPE are provided in the data set TRTH.Gettypes

which is created when the macro %ASSIGNMENTS is submitted.

REQTYPE The supported values for REQTYPE are provided in the data set TRTH.Reqtypes,

which is created when the macro %ASSIGNMENTS is submitted and needs to be

provided only when GETTYPE is equal to “MessageTypes”.

16


%GETRESULTS

Description

The %GETRESULTS macro fetches the results from the server that is associated with REQID. These results are

saved locally as a file with the extension csv.gz and stored in the file location specified by TRTHdir in the directory TRTHOut. If the compressed file is successfully extracted, then the delimited file is also stored in the same location.

A successful import stores a data set in the library TRTH that contains the name of the security (with special

characters stripped out) for the specified date-time period.

Syntax

%getresults(reqID,attempts,wait)

Parameters

REQID The request ID generated from %EXEC or %EXECFTP.

ATTEMPTS The number of times to resubmit the retrieval of results.

WAIT The amount of time, in seconds, to wait between resubmissions.

SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are trademarks of their respective companies.

Copyright © 2012 SAS Institute Inc., Cary, NC USA. All rights reserved.

Using SAS Software to Retrieve - Stevens Institute of ... · PDF fileAll FTP requests that are...

Documents

Transcript of Using SAS Software to Retrieve - Stevens Institute of ... · PDF fileAll FTP requests that are...