SCREEN SCRAPING USING “IBM PERSONAL COMMUNICATION” scraping IBM 5250.pdf · SCREEN SCRAPING...

www.tobuku.com

Version 1.0 - 1 -

SCREEN SCRAPING USING “IBM PERSONAL

COMMUNICATION”

Feb 2011 Level: √

By : Feri Djuandi Beginner Intermediate Expert

Platform : Visual C# 2010

INTRODUCTION

This article explains the programming technique used with IBM Personal Communications to

provide a way for users and programmers to access the iSeries host screen with a set of

functions that can be called from a custom application running on a workstation.

The Personal Communications is a terminal emulation program as shown in the following

example screen shots.

www.tobuku.com

Version 1.0 - 2 -

In a normal way, users are using this emulation program to access the IBM iSeries screens to

interact with the host. However it is possible to create a custom application with a nice

Windows graphical look rather than the original textual green screen. This leads us to a

programming technique so called screen scraping.

NOTE:

In relation to this context, IBM documentation uses the term of

“presentation space” rather than “terminal emulator”. In this article we

will use both terms interchangeably which have the same meanings.

Screen scraping is programming that translates between legacy application programs (written to

communicate with now generally obsolete input/output devices and user interfaces) and new

user interfaces so that the logic and data associated with the legacy programs can continue to

be used. Screen scraping is sometimes called advanced terminal emulation. A program that does

screen scraping must take the data coming from the legacy program that is formatted for the

screen of an older type of terminal such as an IBM 3270 or IBM 5250 and reformat it for a

Windows user or someone using a Web browser. The program must also reformat user input

from the newer user interfaces (such as a Windows graphical user interface or a Web browser)

so that the request can be handled by the legacy application as if it came from the user of the

older device and user interface.1

Data scraping is most often done to either (1) interface to a legacy system which has no other

mechanism which is compatible with current hardware, or (2) interface to a third-party system

which does not provide a more convenient Application Programming Interface (API). In the

second case, the operator of the third-party system may even see screen scraping as unwanted,

due to reasons such as increased system load, the loss of advertisement revenue, or the loss of

control of the information content.

Data scraping is generally considered an ad hoc, inelegant technique, often used only as a "last

resort" when no other mechanism is available.2

In spite of the pros and cons, the screen scraping is no doubt provides a simple and quick way to

bridge the modern Windows applications and the legacy system. The easiness becomes the

main reason why this technique is still used now. The screen scraping technique promotes the

reusability. Meaning the use of screen scrapper program does not require any modification or

enhancement of the original programs or screens in the host, so we can use the existing ones.

The development is only on the custom applications, while everything remains the same on the

host side. It significantly reduces the development cost because normally the development

efforts of COBOL or RPG programs are very expensive, while developing custom applications

(e.g. C++, VB) are typically less costly.

In the market today, there are choices for the programmers to select screen scraper software

with various features and platform supported. IBM Personal Communications is one of them.

With no intention promoting it as the best among the others, I see some reasons why we may

want to use this technique:

1 A definition from Whatis.com

2 Comments found in Wikipedia

www.tobuku.com

Version 1.0 - 3 -

� Free license. The IBM Personal Communications is a companion client application to the

IBM iSeries. It is a freeware program that does not obligate the user to pay for using it

(that’s what I read from resources in the Internet). This is a significant benefit. There are

some screen scraper programs in the market sold with different levels of price. Multiply

by the number of users in the company, and then you will end up with a surprising huge

amount. If we can have a free software, why should buy?

� Compatibility. This client program is developed by IBM to access its host machine. We

can be almost 100% sure of its compatibility. Surprisingly the IBM Personal

Communications works fine with a wide range of Windows programming language like

C++, VB6 up to .NET. This program was written many years ago which I believe having no

certain view of the technology development today, but I am amazed this program can

collaborate with Ms Visual C# 2010 nicely!

� Fast and stable. The IBM emulation program has been being used by many users for

quite long time until today through various version releases. It is proven stable and fast.

� Easy to develop. The API is quite easy to be used with the custom application. The

technical documentation and sample programs are available in the Internet in an easy to

understand fashion. Many chat rooms in the Web discussing the topic with assorted

technical cases.

EMULATOR API

There are numbers of API come with the IBM Personal Communications product where each

interface has a specific set of functions and may be used for different purposes. A programmer

only needs to choose an interface that best matches the functional requirements of the

application, although some applications may use more than one interface to achieve the desired

results. The programming interfaces are:

� Emulator High Level Language API (EHLLAPI)

This API provides functions to access the emulator such as characters on the host

screen. It also provides functions for sending keystrokes to the host, intercepting user-

entered keystrokes, querying the status of the host session, uploading and downloading

files, and other functions. This interface is often used for automated custom

applications which read host screens and enter keystrokes without direct user

intervention.

� Dynamic Data Exchange (DDE)

This interface is similar to the EHLLAPI interface in that it provides a programmable

means to read the host screen, send keystrokes, and perform related functions. It has

some additional functions for access to the emulator clipping rectangle, intercepting

mouse events, and adding/removing commands on the emulator menu bar.

� Personal Communications Session API (PCSAPI)

This interface is used to start, stop, and control emulator sessions and settings.

� Server-Requestor Programming Interface (SRPI)

www.tobuku.com

Version 1.0 - 4 -

This interface is used in cooperation with an IBM Enhanced Connectivity Facility (ECF)

application running on a host system. This API provides functions for writing

synchronous call-return interfaces to remote server programs.

� IBM Personal Communications Host Access Class Library (ECL)

It is a set of objects that allow application programmers and scripting language writers

to access host applications easily and quickly. Personal Communications supports three

different ECL layers (C++ objects, ActiveAutomation (OLE), and LotusScript Extension

(LSX)).

With no intention to disregard the other APIs, I choose EHLLAPI in this article for discussion

because I find it the most convenient to use with the custom applications than the others.

EHLLAPI PROGRAMMING

The model of the screen scrapper programming is shown in the following picture.

As seen in the picture, the custom application interacts with the host through the API callings.

The API will then communicates with the presentasion space to send the commands or

characters and receives back the response from the iSeries. The API does not directly access the

iSeries, instead still uses the presentation space in the middle. In this structure the custom

application may simulate the key strokes by sending the characters to the API functions as if the

user entering the keys through the terminal emulator. The host will not be aware whether the

interactions are done through the custom application or the emulator.

Requirement in the EHLLAPI programming:

� The IBM Personal Communication used in this article is version 5.7. I have no experience

in using the software other than the specified version, but I suppose there won’t be

many differences in the API architecture so you may follow the The API is installed

together with the emulation program so basically there is no additional installation or

plug in required for this purpose. Simply install Personal Communication, and that’s it. It

must be done either on the programmer’s computer (for development use) and on the

users’ computers (when the custom application is released for production).

Presentation

space

Custom

Applications API

Host

iSeries system

IBM PERSONAL

COMMUNICATION

www.tobuku.com

Version 1.0 - 5 -

� Any development tool can be used so long it supports the API DLL calls. The code

examples presented in this article are using Ms Visual C# 2010 Express. The reason why

I use this programming language is only for the convenience sake. You may use other

language if you personally are more familiar with it.

The EHLLAPI is implemented in a form of a DLL file. The file named PCSHLL32.DLL is located in

the ..\Program Files\IBM\Client Access\Emulator directory. Ensure this directory is referenced

in the PATH system variable so that the DLL file is accessible from any application working

directory. To verify whether the path is specified in the system variable, simply open the

command prompt and type the PATH command.

In the custom application, this API file must be referenced by declaring a command line

something like this:

...

[DllImport("PCSHLL32.dll")]

public static extern UInt32 hllapi(out UInt32 Func, StringBuilder Data,

out UInt32 Length, out UInt32 RetC);

...

Notice that inside the PCSHLL32.DLL file there is a function called hllapi. This is the one and the

only function that we use to perform the API call. All interactions to the iSeries system are done

through this function. What controls or differentiates the actions is the parameters sent to this

function. The calling of the hllapi function follows the syntax:

hllapi(out <function_number>, <data>, out <length>, out <ps_position> )

The hllapi function receives various combinations of parameter values that distinguish the

action to be performed. The hllapi function takes four parameter as follow:

1. Function number (integer). This number determines the action that the API will do for

the caller. For instance 1 to connect the presentation space, 2 to disconnect from the

presentation space, 3 to send key, etc. There is a command table in the Appendix that

lists the parameters for specified commands.

2. Data (string). The character or text that wishes to be sent to the presentation space or

read from.

3. Length (integer). The length of the corresponding character or text.

4. Position (integer). The position of the cursor in the presentation space.

NOTE:

The presentation space position is calculated using the formula:

Position = ( (row – 1) * screen_width) + col

row = vertical position of the cursor from the top edge, starts from 1 to

the maximum height of the screen.

col = horizontal position of the cursor from the left edge, starts from 1

to the maximum width of the screen.

Screen_width depends on the size chosen for the presentation space, but

typically the width is 80 characters.

www.tobuku.com

Version 1.0 - 6 -

The EHLLAPI allows a lot number of commands to be invoked by the caller. However among the

available commands there are some tasks most commonly used in the custom application:

� Connecting to the presentation space

� Disconnecting from the presentation space

� Reading text from the presentation space

� Sending keys

� Waiting the response from the host

� Getting the current cursor position

� Moving the cursor position

The next sections in this article will explain the commands in more detail.

www.tobuku.com

Version 1.0 - 7 -

Connecting to the Presentation Space Establishing the connection from a custom application to the host presentation space is the first

step that must be carried out before any other command can be run.

EHLLAPI function parameters:

Parameters Values

Function number 1

Data string The letter of the presentation space ID (PSID), i.e. A, B, C …

Length 1 for standard interface; 4 for enhanced interface

PS Position NA

Return value:

Return Description

0 The Connect Presentation Space function was successful; the host presentation space

is unlocked and ready for input.

1 An incorrect host presentation space ID was specified. The specified session either

does not exist or is a logical printer session. This return code could also mean that the

API Setting for DDE/EHLLAPI is not set on.

4 Successful connection was achieved, but the host presentation space is busy.

5 Successful connection was achieved, but the host presentation space is locked (input

inhibited).

9 A system error was encountered.

11 This resource is unavailable. The host presentation space is already being used by

another system function.

Example:

...

StringBuilder Data = new StringBuilder(4);

Data.Append(psid.ToString()); //the presentation space ID

UInt32 rc = 0; //NA

UInt32 f = 1;

UInt32 l = 4;

ret = hllapi(out f, Data, out l, out rc);

...

NOTE:

Before the custom application connects to the presentation space, be sure

that the IBM Personal Communications program is already running and

connected to the host.

Read the Appendix for the explanation of command methods to start the

emulation program

www.tobuku.com

Version 1.0 - 8 -

Disconnecting from the Presentation Space Terminating the connection from the host presentation space after the custom application

finishes.


Parameters Values

Function number 2

Data string NA

Length NA

PS Position NA

Return value:

Return Description

0 The Disconnect Presentation Space function was successful.

1 The program was not currently connected to the host presentation space.


Example:

...


Data.Append(sessId); //NA

UInt32 rc = 0; //NA

UInt32 f = 2;

UInt32 l = 4; //NA


...

www.tobuku.com

Version 1.0 - 9 -

Reading Text Reading the characters from a static text or field content at a certain position in the

presentation space.


Parameters Values

Function number 8

Data string After the function called, this parameter contains the string read

from the presentation space.

Length Length of the target data string.

PS Position Position within the host presentation space of the first character.

Return value:

Return Description

0 The host presentation space contents were copied to the application program. The

target presentation space was active, and the keyboard was unlocked.

1 The program is not connected to a host session.

2 An error was made in specifying string length, or the sum of (Length - 1) + PS position is

greater than the size of the connected host presentation space.

4 The host presentation space contents were copied. The host presentation space was

waiting for host response.

5 The host presentation space was copied. The keyboard was locked.

7 The host presentation space position is not valid.


Example:

...


UInt32 rc = (UInt32)((rowPos - 1) * SCREEN_WIDTH + colPos);

UInt32 f=8;

UInt32 l = (UInt32)textLen;


txt = Data.ToString().Substring(0, (int)textLen);

...

www.tobuku.com

Version 1.0 - 10 -

Sending Keys Sending keystrokes to the current cursor position. This task can be two meanings: (1) sending a

command key (e.g. Enter, Tab, F3), or (2) sending text (e.g. entering a field)


Parameters Values

Function number 3

Data string A string of keystrokes, maximum 255. Uppercase and lowercase

ASCII characters are represented literally. Function keys and shifted

function keys are represented by mnemonics. See Keyboard

Mnemonics in the Appendix.

Length Length of the source data string.

PS Position NA

Return value:

Return Description

0 The keystrokes were sent; status is normal.


2 An incorrect parameter was passed to EHLLAPI.

4 The host session was busy; all of the keystrokes could not be sent.

5 Input to the target session was inhibited or rejected; all of the keystrokes could not be

sent.


Example:

...

StringBuilder Data = new StringBuilder(str.Length);

Data.Append(str);

UInt32 rc=0; //NA

UInt32 f=3;

UInt32 l=(UInt32) str.Length;


...

www.tobuku.com

Version 1.0 - 11 -

Waiting the Response

After a command is sent to the host, there may be a certain time for the host to process the

request until it can respond again for the subsequent command. During the period, the custom

application must wait for the host until it completes the work. The custom application should

not send another command or text to the host because they are potentially lost or abandoned

by the host.


Parameters Values

Function number 4

Data string NA

Length NA

PS Position NA

Return value:

Return Description

0 The keyboard is unlocked and ready for input.

1 The application program is not connected to a valid session.

4 Timeout while still in XCLOCK (X []) or XSYSTEM.

5 The keyboard is locked.


Example:

...

StringBuilder Data = new StringBuilder(0); //NA

UInt32 rc = 0; //NA

UInt32 f = 4;

UInt32 l = 0; //NA

ret = hllapi(out f, Data, out l, out rc); ...

www.tobuku.com

Version 1.0 - 12 -

Getting the Current Cursor Position To get the current position of the cursor in the presentation space.


Parameters Values

Function number 7

Data string NA

Length NA

After the function called, it holds the host presentation space

position of the cursor

PS Position NA

Return value:

Return Description

0 The Query Cursor Location function was successful.

1 The program is not currently connected to a host session.


Example:

...


UInt32 rc = 0; //NA

UInt32 f = 7;

l = 0; //return position


...

www.tobuku.com

Version 1.0 - 13 -

Moving the Cursor Position To move the cursor to a new location in the presentation space.


Parameters Values

Function number 40

Data string NA

Length NA

PS Position New cursor position in the presentation space

Return value:

Return Description

0 Cursor was successfully located at the specified position.


4 The session is busy.

7 A cursor location less than 1 or greater than the size of the connected host

presentation space was specified.


Example:

...


UInt32 rc = (UInt32)p; //the new position

UInt32 f = 40;

UInt32 l = 0; //NA


...

www.tobuku.com

Version 1.0 - 14 -

APPENDIX

EHLLAPI Functions Number

Action Function Number

Connect Presentation Space 1

Disconnect Presentation Space 2

Send Key 3

Wait 4

Copy Presentation Space 5

Search Presentation Space 6

Query Cursor Location 7

Copy Presentation Space to String 8

Set Session Parameters 9

Query Sessions 10

Reserve 11

Release 12

Copy OIA 13

Query Field Attribute 14

Copy String to Presentation Space 15

Pause 18

Query System 20

Reset System 21

Query Session Status 22

Start Host Notification 23

Query Host Update 24

Stop Host Notification 25

Search Field 30

Find Field Position 31

Find Field Length 32

Copy String to Field 33

Copy Field to String 34

Set Cursor 40

Start Close Intercept 41

www.tobuku.com

Version 1.0 - 15 -

Query Close Intercept 42

Stop Close Intercept 43

Start Keystroke Intercept 50

Get Key 51

Post Intercept Status 52

Stop Keystroke Intercept 53

Query Communication Event 81

Stop Communication Notification 82

Send File 90

Receive File 91

Cancel File Transfer 92

Convert Position or Convert RowCol 99

Connect Window Services 101

Disconnect Window Service 102

Query Window Coordinates 103

Window Status 104

Change Switch List LT Name 105

Change PS Window Name 106

Start Playing Macro 110

For the more detail specification of this function, please refer to the official document in the

Reference.

www.tobuku.com

Version 1.0 - 16 -

Keyboard Mnemonic Use these mnemonic codes to send the special key strokes to the presentation space.

Meaning Mnemonic Code

@ @@

Alt @A

Alt Cursor @$

AEhllapiFuncntion @A@Q

Backspace @<

Backtab (Left Tab) @B

Clear @C

Cmd (function) Key @A@Y

Cursor Down @V

Cursor Left @L

Cursor Right @Z

Cursor Select @A@J

Cursor Up @U

Delete @D

Dup @S@x

End @q

Enter @E

Erase EOF @F

Erase Input @A@F

Field Exit @A@E

Field Mark @S@y

Field - @A@-

Field + @A@+

Help @H

Hexadecimal @A@X

Home @0 ('@' followed by a zero)

Insert @I

Insert Toggle @A@I

Local Print @P

www.tobuku.com

Version 1.0 - 17 -

New Line @N

Page Up @u

Page Down @v

Print (PC) @A@t

Print Screen @A@T

Record Backspace @A@<

Reset @R

Shift @S

Sys Request @A@H

Tab (Right Tab) @T

Test @A@C

PA1 @x

PA2 @y

PA3 @z

PA4 @+

PA5 @%

PA6 @&

PA7 @' ('@' followed by a single quote)

PA8 @(

PA9 @)

PA10 @*

PF1/F1 @1

PF2/F2 @2

PF3/F3 @3

PF4/F4 @4

PF5/F5 @5

PF6/F6 @6

PF7/F7 @7

PF8/F8 @8

PF9/F9 @9

PF10/F10 @a

PF11/F11 @b

www.tobuku.com

Version 1.0 - 18 -

PF12/F12 @c

PF13/F13 @d

PF14/F14 @e

PF15/F15 @f

PF16/F16 @g

PF17/F17 @h

PF18/F18 @i

PF19/F19 @j

PF20/F20 @k

PF21/F21 @l

PF22/F22 @m

PF23/F23 @n

PF24/F24 @o

www.tobuku.com

Version 1.0 - 19 -

Starting the Personal Communications Program The IBM Personal Communications is the standard 5250 terminal emulator used in this article.

Before the custom application can perform the EHLLAPI function calls, the terminal emulation

program should be opened first because all the EHLLAPI function calls are done against the

program.

In a default installation, the Personal Communications program is located under the ..\Program

Files\IBM\Client Access\Emulator directory. The emulation program itself is called PCSWS.EXE.

If the terminal emulator is not started, then few lines of code need to be added within the

custom application to open the PCSWS program. Normally it can be done by a simple shell

command.

The Personal Communications program receives some parameters when it is launched. The

basic syntax to execute this program is:

PCSWS.EXE <WS_file>

The WS file is a workstation profile file that is created when the Personal Communications is run

in the first time. This workstation profile file contains the connection information like the system

name, workstation ID, etc as shown in the dialog window below. The screen should be familiar

to anyone who works with the iSeries quite often.

So when the PCSWS program is execute, it requires the WS file to connect to the host already

specified before. Besides the WS file name, there are other parameters that can be added to the

command when calling the PCSWS program.

� To suppress the IBM logo, add the parameter /Q to the command

Example:

www.tobuku.com

Version 1.0 - 20 -

..\PCSWS.EXE C:\AppData\TCPIP1.WS /Q

� To start the PCSWS program as an icon, not as a window, add the parameter /I to the

command.

Example:

..\PCSWS.EXE C:\AppData\TWINAX.WS /I

� To start a hidden PCSWS program, not as an icon or a window, add the parameter /H to

the command.

Example:

..\PCSWS.EXE C:\AppData\TWINAX.WS /H

www.tobuku.com

Version 1.0 - 21 -

REFERENCE

� Emulator Programming, Personal Communications for Windows, Version 5.9 (Document

Number SC31-8478-09). Tenth Edition, July 2006

http://publib.boulder.ibm.com/infocenter/pcomhelp/v5r9/topic/com.ibm.pcomm.doc/

books/html/emulator_programming.htm

� IBM i Access for Windows, sample programs are available to help develop IBM i Access

for Windows applications using Visual Basic

http://www-03.ibm.com/systems/i/software/access/windows/toolkit/vb.html

� Using EHLLAPI in C#, sample program by Luca Bertoncini

http://www.codeproject.com/KB/cs/all_ehllapi.aspx

SCREEN SCRAPING USING “IBM PERSONAL COMMUNICATION” scraping IBM 5250.pdf · SCREEN SCRAPING...

Documents

Transcript of SCREEN SCRAPING USING “IBM PERSONAL COMMUNICATION” scraping IBM 5250.pdf · SCREEN SCRAPING...