Introduction to R in IBM SPSS Modeler:

20
Introduction to R in IBM SPSS Modeler: A guide for SPSS Users Wannes Rosius/Belgium/IBM Goal of this guide Although there are several very good articles and blogs related to IBM SPSS Modeler, in my role as technical professional for IBM Analytical solutions, we still see lots of people struggling with both R and the integration between IBM SPSS Modeler and R. The idea of this document is certainly not to replace these very useful links listed below, but to enhance these in a way that people knowing IBM SPSS Modeler, with only a very limited knowledge of R, can use this integration. Going through sections 2, 3 and 4, the reader should be able to understand at a high level the R integration within SPSS and to (re)create some very basic R models within SPSS, even if you have only a basic knowledge of R. In section 5 you will learn more detailed tips, tricks and other things. This part is for the experienced user and can be interpreted as a list of loose things which might help you get up to speed with some more detailed functionalities of the integration, and understand some pitfalls. At every point in the document, we try to include R examples to the reader that could be easily copied into the appropriate R node in IBM SPSS Modeler. Unless speci ed otherwise, these code snippets are always based on the telco.sav dataset which can be found in the demo folder of your SPSS Modeler installation. After the source node, attach a type node, and thereafter the appro- priate R node. However, sometimes there are just abstracts of code to show you the idea. It will be clearly mentioned when the code is incomplete. You will nd these codes back into several code frames throughout this document. Furthermore, all the SPSS streams and assets are embedded in the pdf symbolized by . You can access them by right clicking within this pdf document. Some useful links Essentials for R - Installation Instructions User Guide: IBM SPSS Modeler 18 R Nodes Modeler essentials for R Downloads SPSS Modeler and R integration - Getting started

Transcript of Introduction to R in IBM SPSS Modeler:

Page 1: Introduction to R in IBM SPSS Modeler:

Introduction to R in IBM SPSS Modeler

A guide for SPSS Users

Wannes RosiusBelgiumIBM

Goal of this guide

Although there are several very good articles and blogs related to IBM SPSS Modeler in my roleas technical professional for IBM Analytical solutions we still see lots of people struggling withboth R and the integration between IBM SPSS Modeler and R

The idea of this document is certainly not to replace these very useful links listed below butto enhance these in a way that people knowing IBM SPSS Modeler with only a very limitedknowledge of R can use this integration

Going through sections 2 3 and 4 the reader should be able to understand at a high level theR integration within SPSS and to (re)create some very basic R models within SPSS even if youhave only a basic knowledge of R

In section 5 you will learn more detailed tips tricks and other things This part is for theexperienced user and can be interpreted as a list of loose things which might help you get up tospeed with some more detailed functionalities of the integration and understand some pitfalls

At every point in the document we try to include R examples to the reader that could be easilycopied into the appropriate R node in IBM SPSS Modeler Unless specied otherwise these codesnippets are always based on the telcosav dataset which can be found in the demo folder of yourSPSS Modeler installation After the source node attach a type node and thereafter the appro-priate R node However sometimes there are just abstracts of code to show you the idea It willbe clearly mentioned when the code is incomplete You will nd these codes back into several codeframes throughout this document Furthermore all the SPSS streams and assets are embedded

in the pdf symbolized by You can access them by right clicking within this pdf document

Some useful links

bull Essentials for R - Installation Instructions

bull User Guide IBM SPSS Modeler 18 R Nodes

bull Modeler essentials for R Downloads

bull SPSS Modeler and R integration - Getting started

IBM SPSS Modeler and R

Contents

1 System Setup 311 Installing R 312 Enabling the R nodes 3

2 R basics 3

3 The basics of R nodes in IBM SPSS Modeler 531 The R nodes 532 Simple R code example 5

321 modelerData 6322 modelerDataModel 7323 modelerModel 8

33 Some general remarks 1034 Read data options 11

4 Custom Dialog builder 1141 Tools 1242 Custom dialog 1243 Simple example 12

5 Tips amp tricks Some more detailed 1451 R code 14

511 ibmspsscf70 library 14512 Some useful parts of R code 15

52 Custom Dialog builder 17521 How to save and share a custom dialog 17522 Link to dialog and script 17

53 What about SQL Pushback Hadoop pushback 1854 What about real-time scoring and Solution Publisher 1955 Something more about the metadata in modeler and the consequences on R integration 19

Page 2 of 20

IBM SPSS Modeler and R

1 System Setup

Let us start with the setup of your system For now we assume that you have a valid installationof IBM SPSS Modeler on your machine For more installation topics we refer to the InstallationInstructions

11 Installing R

Depending on the version of your IBM SPSS Modeler you will now have to install dierent versionsof R

SPSS Version R version R download link

160 2152 httpscranr-projectorgbinwindowsbaseold2152170 31 httpscranr-projectorgbinwindowsbaseold310171 31 httpscranr-projectorgbinwindowsbaseold310180 32 httpscranr-projectorgbinwindowsbaseold320

Once you downloaded and installed you will have a working R instance on your machineserverLike SPSS Modeler you can have several versions of R installed on your machine without anyproblem

12 Enabling the R nodes

You will need to install the IBM SPSS Modeler essentials for R You can nd these here on theSPSS Community Downloads page Click 2 Get Essentials for SPSS and then click the buttonGet R Essentials for SPSS Modeler This will take you to github and you will be able to selectand download the Modeler 18 Essentials for R for a variety of platforms If you require Essentialsfor R for earlier Modeler versions there is also a link to legacy versions

Run this execution le The installation will ask you the path of your R installation and thepath to the bin les of your SPSS Modeler installation (Note that in the prelled path it is thedefault path to a ModelerServer and you will need to change this if you want to congure yourclient) This installation will place the R nodes in your SPSS Modeler node palette and it willalso include necessary R libraries in your R installation folder

2 R basics

There is already an over13ow of R courses (publicly) available through several channels so wewould certainly not want to replace these In it also not very important that you are an R expertto follow this document However there are still some basics of R code and R terminology usersneed to understand in order to exploit the integration of R and IBM SPSS Modeler For thissection let us open R in its original GUI Therefore go to the R installation folder and openbinx64RGUIexe A window will be opened looking like this

Page 3 of 20

IBM SPSS Modeler and R

This is the R console ready for commands to run You might often hear the term RStudio whichis nothing more than a development environment on top of this R gui Installation of RStudio isnot required for this introduction but might be handy for further useWe will start the R introduction by stating R is a powerful programming language and environmentfor statistical computing and graphics An important part within that last phrase is that R is aprogramming language unlike IBM SPSS Modeler That means it is built on objects that aredened by the user As an example assume the following R code (feel free to type it within theR console to see the R outputs)

1 x lt- 1+1

2 y lt- 2x

3 xyVector lt- c(xy)

4 z lt- mean(xyVector)

5 print(z)

Here x is an object This statement will ll the object x with the value of the evaluated formula1 + 1 being 2 So whenever the program refers to x it will be interpreted as 2 In the secondline we will dene y as twice the value of x In the third line we create a vector containing thecontent of x and y to calculate the mean of these 2 objects and place it in an object z

The operator lt- could also be replaced by = but for various reasons lots of R users pre-fer this way of writing (actually it is not exactly the same but that could be ignored for thepurpose of this document) If you feel more comfortable in using = please do so

Like we lled x y and z with some numbers any R object can be lled with a variety of typesHere is a list of the most important for our purposes

Vector is a sequence of data elements of the same type (eg numeric or character) This includesvectors of length 1 which can be interpreted as just being numbers You can create a vectorwith the R function c() So in the example code above all the values of x y and z arevectors of length one xyVector is a vector of length 2 containing the values of (the vector)x followed by (the vector) y Trying to link it back to SPSS you can interpret a vector asthe values of a single data column

Data frame is a list of vectors of equal length If you look at a vector as the values of a variablea data frame could be interpreted as a 2-dimensional dataset with columns (the number ofvectors) and lines (the size of each vector)

1 n lt- c(2 3 5 3 9) A first vector of 5 numeric values

2 n2 lt- c(1 3 2 5 4) A second vector of 5 numeric values

3 s lt- c(aa bb cc aa zz) A third vector of 5 string values

4 b lt- c(TRUE FALSE TRUE TRUE TRUE) A fourth vector of 5 flag values

5 Data lt- dataframe(n s b New = n+n2) A data frame containing 4 vectors

Page 4 of 20

IBM SPSS Modeler and R

6 Note n+n2 will be a new vector called New with the sum of the n + n2 c(3 6 7 8 13)

7

8 dim(Data) Will show you it is a 5x4 dataset

9 Data[24] Will give back the value on the 2nd line the 3rd column

10 colnames(Data) Will give the column names as a vector (nsb New)

11 Data$n[1] Will give back the first value of the vector n within the data frame

12

13 iris predefined data frame

There are also several pre-dened data frames installed within R One of them is called irisSometimes this document will refer back to iris

Model class which is actually a specic list containing predened objects dening a statisticalmodel For example a linear model class will be a list containing among others the coecientsof the regression model

List is an ordered collection of objects As an example you can have a list where the rst elementis a vector the second is a data frame and the third is a model Note that a data frame isa special type of a list where all the elements are vectors of equal sizes

3 The basics of R nodes in IBM SPSS Modeler

31 The R nodes

Once the installation for the R essentials are done you will see 3 new nodes in your node palettesThere is also a 4th R node which is the R nugget The dierence between and understanding ofthese 4 objects are essential

Output with this node data will be sent to R but it will never go back to SPSS (as it is aterminal node) The only thing that can go back to SPSS is the outputs generated by Rthatwill be presented within an SPSS output window

Transform data will go from SPSS to R but will also go back to SPSS after which the SPSSprocess can be continued

Model like the output node this is a terminal node so data will not go back to SPSS Howeverthere will be a reusable R object created within a nugget

Nugget similar to Transform node with the dierence that there is a reusable R object that canbe used in the R code

Node

Name R output node R transform node R model node R syntax nodePalette tab Output Record Ops Model NAData back to SPSS No Yes No YesReusable R object No No Create Use

32 Simple R code example

Let us start with saying that all the examples in this section are intentionally kept very simple soas to explain the interaction in a functional and structured way and be simple enough for non Rprogrammers We are certainly aware that most of the R code snippets we show in this chapter

Page 5 of 20

IBM SPSS Modeler and R

could also easily be implemented using standard SPSS Modeler functionality

There are 3 very important and reserved R objects that you should keep in mind when youuse the SPSS Modeler R integration Here is a brief description of these 3 after which we will gointo more detail for each of them

modelerData This is an R data frame that will be lled by the data entering in this R node Thisdata frame can be used and changed within your R code Eventually it will also be the dataframe that will be sent back to SPSS Modeler as a dataset Note that it will only containthe content of the data not (necessarily) the data column names and other metadata items

modelerDataModel This is also an R data frame containing the metadata of the data that is sentto R and back to SPSS Modeler It contains most of the information that you may expectwithin an SPSS Modeler type node This will be the object that will be most strange forexperienced R users

modelerModel this is an R object that can be lled by the user by any type of object you wantIt does not need to have a certain structure It will be calculated in the R model node afterwhich it will be saved within the R nugget where it can be used in the R-syntax

Note that R code is case-sensitive and therefore so are these object names In the following sectionswe will explain the usage of these objects

321 modelerData

modelerData is the R data frame that will be lled by the dataset entering the SPSS Modeler nodeit comes from So you can use this data frame to perform the desired calculations transformationsand outputs in R

Place the following code in an R output node

1 Print the first 6 lines of the data

2 head(modelerData)

3

4 Give a summary of the data

5 summary(modelerData)

6

7 create a histogram of the variable tenure

8 hist(modelerData$tenure xlab = years main = Tenure histogram)

9

10 change the tenure unit from months to years

11 modelerData$tenure lt- modelerData$tenure1212

13 recreate the histogram now in months

14 hist(modelerData$tenure xlab = months main = Tenure histogram)

Execution of this node will result in an SPSS Modeler output window in which all the R outputswill be assembled These will always be divided in 2 tabs Text output and Graph output

Page 6 of 20

IBM SPSS Modeler and R

In this case the text output is linked to the code on line 2 and 5 rst it prints the rst 6 lines(head) of the data next it will give summary statistics for each column

The graph output are two histograms One for the tenure in months the other for the samecolumn but after redening it by dividing the original value by twelve to give the tenure in years(note the X-axis scale)

As shown in the example stream Explain modelerDatastr you can also copy exactly this samecode into a transform node and attach a table node to it After running this table node you willnot see any R output (as none is expected) That means that even though the output code hasrun no outputs will be given However the data frame of modelerData will be send back to SPSSModeler In this case you will see the value of tenure being divided by 12

322 modelerDataModel

Metadata is very important in SPSS Modeler Let us for simplicity say that within modeler meta-data is represented by the type node With metadata we mean the type of each of the variablesin the dataset (numeric 13ag String storage ) At all times modeler will know exactly all themetadata at every step in the stream

R does not handle the metadata in a similar way as SPSS Modeler We already explainedmodelerDataModel taking over the role of the type node This is done by a data frame(=dataset) of the following structure

X1 X2 Xn

fieldName region tenure agefieldLabel Geographic indicator Months with service Age in years

fieldStorage real real realfieldMeasure nominal continuous continuousfieldFormat standard standard standardfieldRole input input input

So this means that this dataset will always have 6 lines with xed names (yes in R also thelines have names) The thing with this dataset is that it is completely the responsibility of theuser to align this metadata with the appropriate data So that means if we would like to add avariable with R the user must also manually add a column in modelerDataModel to make suremodelerData correctly goes back to SPSS Modeler In the earlier example above we did not makeany changes to the modelerDataModel and it was also not needed as the metadata did not change(dividing a number by 12 will not change the metadata) Now let us continue on the previousexample But now rather than changing the value of tenure in the same data variable we willcreate another one As a result we would have to update the metadata

1 Create the vector of tenure in years

2 Rcolumn lt- modelerData$tenure123

4 Paste this vector to the right of the dataset

5 modelerData lt- cbind(modelerDataRcolumn)

6

7 create the metadata for the column to add

8 newVar lt- c(fieldName=tenureYears fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

9

10 paste the new column metadata to the existing metadata

11 modelerDataModel lt- cbind(modelerDataModelnewVar)

Running a table node downstream of this transform node will show you the new variable withthe name tenureYears There are some important things to realize in this

Page 7 of 20

IBM SPSS Modeler and R

bull fieldName and fieldStorage are the only 2 required rows that needs to be lled in for anynew column In the code we left all the other lines empty meaning they will be lled in bythe stream default For a list of available values we refer to the user guide

bull As modelerDataModel is only useful when you go back to SPSS Modeler you will generallyonly usechange this object in non-terminal R-nodes It might still be handy to use it interminal nodes if the value of the modelerDataModel is important for your output (egrun a histogram of all continuous variables)

bull When data will go back to SPSS modeler it will be the content of the data frame excludingthe column- and row names That means that even though the column in the modelerData

will be called Rcolumn the name of the column in SPSS will only be dened by the metadatawithin the row fieldName In this case it is called tenureYears

bull The only link between modelerData and modelerDataModel is the order of the columns Itwill not look by name The rst column in the data will be given the metadata of the rstcolumn of modelerDataModel In case the metadata (modelerDataModel ) does not matchthe modelerData an error is thrown The table below shows schematic how this works

modelerData modelerDataModel

R

RName1 RNamenx11 x1n

x21 x2n

xm11 xm1n

xm1 xmn

X1 Xn

fieldName Name1 NamenfieldLabel

fieldStorage xxx xxxfieldMeasure fieldFormat

fieldRole

SPSS

Name1 Namenx11 x1n

x21 x2n

xm1 xmn

Note that only the names of the modelerDataModel are used

Since this concept is very strange to standard R users We found this part the most dicult toexplain To people who know SPSS you can summarize it as modelerDataModel taking over therole of the type node

You can nd all streams and R scripts explaining modelerDataModel here

323 modelerModel

modelerModel is the R object that is stored within the R nugget This object will be populatedwithin the R model node after which you could use modelerModel within the R nugget for scoringThis very much works the same way as IBM Modeler works You ask a model node to calculatea formula after which that formula will be stored within the nugget together with the way itshould be used to calculate a scoring

You will only use this object within the R model node and nugget Note that within the Rmodel node there are 2 syntax window

Page 8 of 20

Explain modelerDataModelxml

data0001dat

data0002dat

IBM SPSS Modeler and R

R model building is to calculate whatever you want to store within modelerModel that couldbe used within your nugget calculations to score your data This can be any object withinR As any SPSS Model builder node this will be a terminal code meaning that from thiscode no data will go back to SPSS modeler unless some outputs and things that are storedwithin the modelerModel object

R model scoring is the syntax to dene how you will use the object modelerModel containingthe content you stored to it in the R model building syntax to derive the new data Apartfrom the use of modelerModel this is very similar to the R transform node

Let us start with a simple example where we would like to create a basic linear model for thevariable tenure The formula of this model should be saved in the modelerModel after which itcan be used in the scoring

1 Create the model and save it in modelerModel

2 modelerModel lt- lm(tenure ~ age + region + ed + income data= modelerData)

3

4 Add some summary of the model in the nugget

5 summary(modelerModel)

6

7 together with a histogram

8 hist(modelerModel$residuals main = residual histogram)

9

10 and the residual vs actuals scatterplot

11 plot(modelerData$tenure modelerModel$fittedvalues xlab = actual ylab = predicted )

12

13 All of these output will be stored in the modeler nugget tabs

1 Use the model to make a prediction and add it to the existing data

2 pred lt- predict(modelerModel modelerData)

3 modelerData lt- cbind(modelerDatapred)

4

5 Take care of the metadata

6 newVar lt-c(fieldName=$L-tenure fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

It is important to note that modelerModel can be lled with any type of object but will veryoften be of a model class but does not have to be In the previous example the object stored toit was clearly a (statistical) model In the next example we will just save 2 numbers within themodelerModel object Imagine we want to calculate the z-values of a certain variable In orderto create the z-values we need the mean and the standard deviation of the column We will storeboth of these within modelerModel after which we will use them in the scoring syntax1 Thisexample shows you do not need to store a statistical model within your modelerModel objectbut it really can be any R object

1 calculate mean and standard deviation

2 M lt- mean(modelerData$tenure)3 SD lt- sd(modelerData$tenure)4

5 and save it in a list called modelerModel

6 modelerModel lt- list(avg = M sDev = SD)

1 calculate z scores using the elements in modelerModel

2 zTenure lt- (modelerData$tenure - modelerModel$avg)modelerModel$sDev3 modelerData lt- cbind(modelerDatazTenure)

4

5 define new metadata column and add it

6 newVar lt- c(fieldName=zTenure fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

1Note that there is a very good reason this is not combined into an R Transform node explained in 34

Page 9 of 20

IBM SPSS Modeler and R

You can nd all streams and R scripts explaining modelerModel here

33 Some general remarks

bull Although it might seems this way you are not required to build modelerData from theexisting data within that frame modelerData will be lled with the dataset you have inModeler however nothing stops you to throw that data away in R and dene some new datacoming from another data source in R As an example imagine this link from the WeatherCompany website This will give the weather history in Brussels Belgium in the monthNovember 2015 Now we can use R code as a source node by just overwriting modelerData

and redening modelerDataModel

1 Define the link

2 linkPath lt- httpwwwwundergroundcomhistoryairportEBBR20151101

MonthlyHistoryhtmlreq_city=Brusselsampformat=1

3 Read the data as csv

4 modelerData lt- readcsv(linkPath)

5 modelerData[1] lt- asDate(modelerData[1])

6

7 Redefining modelerDataModel all are real numbers except the first column is the date

8 modelerDataModel lt- asdataframe(t(dataframe(fieldName = colnames(modelerData)

fieldLabel = fieldStorage = c(daterep(realncol(modelerData)-1))

fieldMeasure = fieldFormat = fieldRole = )))

As you can see this code does not use the old denition of the dened R objects butcompletely redenes them Placing this in a R transform node will give back this newdataset to modeler So in this way you can use this approach to create an R input node

You can nd an example here

bull Within an R model node there is place for 2 scripts The building script will be the scriptthat will be populated within the R nugget It will not be run when you run the model nodeAs a result of this these 2 scripts are independent The only thing they share is the valueof the object modelerModel which is saved within the nugget when running the buildingsyntax and picked up within the R scoring syntaxThis also means that eventual R-libraries that are required should be loaded in both scriptsTake for example a model for a random forest

1 Load the library

2 library(randomForest)

3 Create the model and save it in modelerModel

4 modelerModel lt- randomForest(tenure ~ age + region + ed + income data= modelerDatantree

=50)

1 Load the library

2 library(randomForest)

3

4 Use the model to make a prediction and add it to the existing data

5 predlt- predict(modelerModel modelerData)

6 modelerData lt- cbind(modelerDatapred)

7

8 Take care of the metadata

9 newVar lt- c(fieldName=$RF-tenurefieldLabel=fieldStorage=realfieldMeasure=fieldFormat=fieldRole=)

10 modelerDataModel lt- cbind(modelerDataModelnewVar)

bull Talking about libraries and package A package is a collection of R objects dened for acertain purpose These often are specic statistical functionalities like randomForest in theexample above A basic R installation comes with the standard packages however there aremany more packages available made available by the R community on CRAN

Page 10 of 20

Explain modelerModelxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

R as sourcexml

data0001dat

data0002dat

IBM SPSS Modeler and R

Packages needs to be installed and made locally available in libraries Once the package isinstalled on the system as a library you can load this library in any R session by the codelibrary(ltnamegt)To install a package you have several options The easiest is probably to write a codelike installpackages(randomForest) within R You will have to select a CRAN mirrorwhere this library will be downloaded from and the download will go automatically Nor-mally you will only have to do this onceAlthough possible it is not recommended to run this package installation command fromwithin SPSS The reason is that these libraries will than be saved in a temporary folder andafterwards be deleted If you still want to this through SPSS you will have to hard codethe installation path

34 Read data options

Something we ignored until now are the settings within the node under Read data options Thebasics of the R integration with Modeler can be done without the knowledge of this as it requiressome more advanced R knowledge The user guide still has a good explanation about these items

However there is one more thing that might be important For modeler version 17 and lowerthe R integration of non-terminal nodes (ie transform and nuggets) will by default be done inbatches of 1000 The reason for this was to allow these R nodes to work on hadoop and otherclustered environments As a result of this it is very important to realize that any R code thatwould span over multiple lines of data would lead to false results For a workaround for this werefer to 511

Take as an example the z scores above If we would calculate the mean and the standard de-viation of the variable in a non-terminal node it would start with running this code for the rst1000 lines of data So that leads to a specic mean deviation and corresponding z-scores How-ever the next 1000 lines a new mean and deviation would be calculated and the z scores will bebased on these

As a solution the means and standard deviations are calculated in the R model (ie a termi-nal node) over all the data and used in the R nugget to calculate the z scores

Note that this approach may lead to a very slow integration between SPSS and R in the caseof streaming R nodes in a local non-clustered environment However as from IBM SPSS Modelerversion 171 there is a default option not to use this approach of batch processing or to increasethe batch size For the lower versions there is a workaround possible if you still want to increasethis batch size or turn it o (see later)

4 Custom Dialog builder

The Custom Dialog Builder allow you to create and manage R nodes with prefilled R code to use insideIBM SPSS Modeler streams In this way users can create their own nodes You can start the CustomDialog builder in the Tools menu under rdquoCustom Dialog Builderrdquo

When opening a custom dialog builder you will see a 2 windows One of them is the custom dialogitself the other is the toolset to populate the dialog

Page 11 of 20

IBM SPSS Modeler and R

41 Tools

The tools window is a list of items you can place within your dialog This include among others the fieldchooser Check and combo boxes Text and number controls and tabs You can select any and drag themonto the dialog itselfOnce you have any item in the dialog you can select it and you will see the item properties These are theproperties of this specific item and will change dependent on the type of item it is The most importantare the identifier (the way it will be referenced within the script) and the Title (the one that will be visualin the dialog)

42 Custom dialog

The big gray window is the dialog itself For the moment it is empty as it should be populated with itemsfrom the Tools listClicking on this gray dialog will show you the dialog properties below As main items this includes thename and title of the dialog the script itself and the type and position of the created nodeWith regards to the script to be written The global rule is that you reference to the items within thedialog using their identifier between double percentages (rdquoltidentifiergtrdquoOnce you finished creating the custom node you can install it by pressing the green arrow in the toolbarYou can also save intermediate versions to the disk

43 Simple example

Let us create a custom dialog for the randomForest model created earlier in section 33 Below you willfind a step by step approachThe most important thing we have to wonder is what within this code we want flexible for the user Inthe case of this model there might be 3 things that we want flexibel the input variables the target andthe number of trees in our forest

First fill in the Custom dialog properties as indicated

Page 12 of 20

IBM SPSS Modeler and R

In our fixed example the target is tenure but a user might chooseany other field As a result we will place a field chooser on the dialogChange the properties like shown The variable filter properties allowsyou to select only categorical variables

In our fixed example the inputs are age income but a user mightchoose any other field As a result we will place a field chooser on thedialog The biggest difference is that now we can select several variablesas there might be different inputs To make it easier we will separatethese values by a + Therefore change the properties like shown

As a third custom choice we would like to add the number of trees inour forest In the original script is was 50 so we will choose this as thedefault However users may choose any integer value between 1 and1000 Add a number control on the dialog and change the properties

So now the dialog is ready and we need to add the script to it Go tothe Edit options and choose rdquoScript Templaterdquo This will bring you toan empty window for the script In this case (as we selected we wanted2 scripts) there is a tab for the building code and one for the scoringscript If the coring script is greyed out you did set the dialog propertyrdquoScore from the Modelrdquo to True

Page 13 of 20

IBM SPSS Modeler and R

Let us start with the scoring script as this is easier The only thingwhich will need to be adapted for custom input is the variable namethat will be send back to SPSS So copy the scoring code and changethe name tenure to TARGET (this is the name of the identifier of thetarget)

Fill in the code for the building script and change in a similar wayas above the values for target and intput variables together with thenumber of trees Afterwards press OK to close the script window

Being back at your Custom Dialog builder save the dialog in any appro-priate location Also deploy the dialog by clicking on the green deployarrow in the toolbar Close the Dialog builder

Back to the stream you will now see the new node in the model paletteYou can use this node within your stream

You can find the resulted cfe file here (place this in the correct location see 521) and a stream

where it is deployed

5 Tips amp tricks Some more detailed

51 R code

511 ibmspsscf70 library

Let us now have a more detailled view about what actually happens with the code First of all it is worthto check what happens when you do the R installation correctly This will install the by IBM delivered Rpackage ibmspsscf70 in the library folder of your R installation This library contains several functionsto handle the data traffic between SPSS and R

Running any R node in SPSS will not only run the code you write but it will also run some extracode behind the scenes You can see this code in the rdquoConsole outputrdquo window of the R node Lookingfor example at this tab for an R nugget you will see that your code will be something like

1 modelerModel lt- ibmspsscfoutputGetModel()

2 while(ibmspsscfdataHasMoreData())

3 modelerDataModel lt- ibmspsscfdatamodelGetDataModel()

4 modelerData lt- ibmspsscfdataGetData(rowCount=1000 missing=NA rDate=None

logicalFields=FALSE)

5

Page 14 of 20

app-extensionxml

cdb_peerjar

comspsssharedcustom_guiui_builderpeersRStatsApplierPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsApplierPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsApplierPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                    
                

comspsssharedcustom_guiui_builderpeersCDExtensionPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class CDExtensionPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static String                         RBUILDER                            private                         static String                         SCORE_SYNTAX                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            public void CDExtensionPeer()                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private String                         replaceFieldSeparator(String String)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$1                         implements javaawteventActionListener     void ActionHandler$1(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$2class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$2                         implements javaawteventActionListener     void ActionHandler$2(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsUIPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsUIPeer                         implements compaswframeworkuiswingspiExtensionObjectSwingUIPeer compaswframeworkcommonextensionspiInteractorListener compaswcorecontainerContainerSetListener                             private ActionHandler                         actionHandler                            public                         static                         final String                         UPDATE_CONTAINER = updateContainer                            public void RStatsUIPeer()                            public void                         initSwingUI(compaswframeworkuiswingExtensionObjectSwingUI compaswframeworkcommoncoreProperties compaswframeworkuiswingExtensionObjectSwingUIContext)                            public void                         disposeSwingUI()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            public void                         handleAction(String String compaswframeworkcommoncoreProperties)                            public void                         containerChanged(compaswcorecontainerContainerSetEvent)                            public void                         updateContainer(compaswcorecontainerContainerSetEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsProcessPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsProcessPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsProcessPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersCopyrightclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         abstract                         interface Copyright                         extends annotationAnnotation                             public                         abstract String                         value()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxControl                         extends javaxswingJTextArea                         implements compaswframeworkuiswingspiPropertyControl                             private                         static                         final String                         editScoreSyntax = editScoreSyntax                            private                         static                         final String                         editScoreSyntaxTitle = editScoreSyntaxTitle                            private                         static                         final String                         editScoreSyntaxWarning = editScoreSyntaxWarning                            private                         static                         final String                         editScoreSyntaxContinueBtn = editScoreSyntax_Continue                            private                         static                         final String                         editScoreSyntaxCancelBtn = editScoreSyntax_Cancel                            private javaxswingJPanel                         containerPanel                            private javaxswingJButton                         editScoreSyntaxBtn                            private compaswframeworkuiswingPropertyControlContext                         propertyControlContext                            public void ManagedScoreSyntaxControl()                            public void                         disposeControl()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public javaxswingJComponent                         getUIComponent()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            public boolean                         isControlEnabled()                            public void                         setControlEnabled(boolean)                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxPanel                         implements compaswframeworkuiswingspiManagedPanel                             public void ManagedScoreSyntaxPanel()                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ROutputFileChooserControl                         implements compaswframeworkuiswingspiPropertyControl compaswuicommoncontrolControlListener                             public                         static                         final String                         OUTPUT_TYPE = output_type                            public                         static                         final String                         OUTPUT_TYPE_GRAPH = Graph                            public                         static                         final String                         GRAPH_OUTPUT_FILE_TYPE = graph_output_file_type                            public                         static                         final String                         TEXT_OUTPUT_FILE_TYPE = text_output_file_type                            public                         static                         final String                         OUTPUT_FILE_TYPE_HTML = HTML                            public                         static                         final String                         OUTPUT_FILE_TYPE_TEXT = Text                            public                         static                         final String                         OUTPUT_FILE_TYPE_COU = Cou                            public                         static                         final String                         FILENAME = OutputFilename                            public                         static                         final String                         OUTPUT_FILE_CHOOSER_BUTTON = OutputFileChooserButton                            public                         static                         final char                         BACK_SLASH_CHAR = 92                            public                         static                         final char                         FORWARD_SLASH_CHAR = 47                            private ActionHandler                         actionHandler                            private javaxswingJLabel                         fileNameLabel                            private javaxswingtextJTextComponent                         textComponent                            private javaxswingAbstractButton                         browseButton                            private compaswuiswingcontrolDefaultControlPanel                         mainPanel                            private String                         propertyName                            private compaswframeworkuiswingPropertyControlContext                         context                            public void ROutputFileChooserControl()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            private void                         initComponent()                            private compaswframeworkuiswingSwingResourceProvider                         getSwingResourceProvider()                            private String                         getLabelFromSwingResourceProvider(String)                            private Object                         getSwingUiProperty(String)                            protected String                         getTextComponentValue()                            public javaxswingJComponent                         getUIComponent()                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public void                         setControlEnabled(boolean)                            public boolean                         isControlEnabled()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public void                         disposeControl()                            public void                         controlValueChanged(compaswuicommoncontrolControlEvent)                            private String                         removeExtension(String)                            public void                         controlSelectionChanged(compaswuicommoncontrolControlEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandlerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ActionHandler                             private compaswframeworkuiswingExtensionObjectSwingUI                         extensionObjectSwingUI                            private boolean                         bPrintAll                            private int                         printScope    String                         actionErrorMessage                            public void ActionHandler(compaswframeworkuiswingExtensionObjectSwingUI)                            public void                         handleAction(String String)                            private void                         handleExportHTML(String)                            private void                         handleExportText(String)                            private void                         handlePrint(String String)                            private void                         handlePrintView()                            private void                         handlePageSetup(String)                            private void                         handleSaveNode(String)                            private void                         handlePreviewTable(String)                            private void                         handleSaveToProject(String)                            private void                         handleHelpButton()                            private void                         exportHTMLFile(String)                            private void                         saveTextFile(String String)                            private javaxswingJPanel                         createPrintViewPanel(javautilList)                            public String                         launchSaveFileChooser(String String[])                            private void                         configureSaveFileChooser(javaxswingJFileChooser)                            private void                         configureSaveChooserCallback(compaswcoreapplicationcallbackFileSystemChooserRequest)                            private String                         getSaveDialogTitle()                            private String                         getApproveButtonText()                            private boolean                         useApplicationFileChoosers()                            private compaswframeworkuicommonUISession                         getUISession()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            private String                         getTextContainerData(String)                            private boolean                         isExportText(String)                            private boolean                         isExportHTML()                            private boolean                         isSelectNode()                            private compaswuiswingcontrolModelViewerPanel                         getModelViewerPanel()                            private String                         getDefaultFileName()                            private void                         handleUpdateContainer(String)                    
                

comspsssharedcustom_guiui_builderpeersManagedBasePanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedBasePanel                         extends javaxswingJPanel                         implements javaawtprintPrintable     void ManagedBasePanel(javaawtLayoutManager)                            public int                         print(javaawtGraphics javaawtprintPageFormat int)                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$1                     
                

comspsssharedcustom_guiui_builderpeersUIUtilitiesclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class UIUtilities                             public void UIUtilities()                            public                         static javaxswingJButton                         createButton(compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComboBox                         createComBox(compaswframeworkuiswingSwingResourceProvider String int javaxswingDefaultComboBoxModel)                            public                         static javaxswingDefaultComboBoxModel                         createEnumComboBoxModel(compaswframeworkcommoncoreEnumerationValueDescriptor)                            public                         static javaxswingAbstractButton                         addAccessKey(javaxswingAbstractButton compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingtextJTextComponent                         createTextFiled(compaswframeworkuiswingSwingResourceProvider String int)                            public                         static javaxswingJComponent                         addTooltip(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComponent                         addDescription(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static                         transient void                         fitAllComponents(javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxPreferSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setPreferSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMaxSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMaxSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMinSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMinSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                    
                

comspsssharedcustom_guiui_builderpeersManagedHTMLOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedHTMLOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedHTMLOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedModelOutputPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedModelOutputPanel                         implements compaswframeworkuiswingspiManagedPanel compaswframeworkcommonextensionspiInteractorListener                             public                         static                         final String                         BUILD_TEXT_OUTPUT = buildTextOutputDisplay                            public                         static                         final String                         BUILD_CONSOLE_OUTPUT = buildConsoleOutputDisplay                            public                         static                         final String                         BUILD_HTML_OUTPUT = buildHTMLOutputDisplay                            public                         static                         final String                         BUILD_MV_OUTPUT = buildModelViewerDisplay                            public                         static                         final String                         APPLIER_CONSOLE_OUTPUT = applierConsoleOutputDisplay                            public                         static                         final String                         BUILD_TEXT_CONTAINER_NAME = TextOutput                            public                         static                         final String                         BUILD_CONSOLE_CONTAINER_NAME = ConsoleOutput                            public                         static                         final String                         BUILD_HTML_CONTAINER_NAME = HTMLOutput                            public                         static                         final String                         BUILD_PMML_CONTAINER_NAME = PMML                            public                         static                         final String                         BUILD_STAT_XML_CONTAINER_NAME = StatXML                            public                         static                         final String                         APPLIER_CONSOLE_CONTAINER_NAME = ApplierConsoleOutput                            private ManagedTextOutputArea                         textOutput                            private ManagedTextOutputArea                         consoleOutput                            private ManagedHTMLOutputArea                         htmlOutput                            private ManagedEmptyOutputArea                         emptyOutput                            private javautilHashMap                         componentMap                            private compaswframeworkuiswingManagedPanelContext                         managedPanelContext                            public void ManagedModelOutputPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private compaswframeworkuiswingspiManagedUIElement                         setupTextOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupConsoleOutputControl(String)                            public void                         updateConsoleOutputControl(String)                            private compaswframeworkuiswingspiManagedUIElement                         setupHTMLOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupModelViewerControl()                            public String                         getTextContainerData(String)                            public                         static boolean                         exportToHTML(compaswframeworkcommoncontainerContainer String)                            public                         static javaioFile                         getHTMLFileTest(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractHTMLCompoundContainer(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractZip(String String boolean)                         throws javaioIOException                            public                         static javaioFile                         getHTMLFile(String String boolean)                         throws javaioIOException                    
                

comspsssharedcustom_guiui_builderpeersManagedTextOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedTextOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedTextOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$ButtonsActionclass

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$ButtonsAction                         implements javaawteventActionListener                             private void ManagedScoreSyntaxControl$ButtonsAction(ManagedScoreSyntaxControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersManagedEmptyOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedEmptyOutputArea                         extends javaxswingJPanel                         implements compaswframeworkuiswingspiManagedUIElement                             public void ManagedEmptyOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public javaxswingJComponent                         getComponent()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ROutputFileChooserControl$1                         implements javaawteventActionListener     void ROutputFileChooserControl$1(ROutputFileChooserControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

extensionxml

imagescopygif

imagesdeletegif

imagesfileMenugif

imagesgenerateMenugif

imageshelp_buttongif

imageslg_gm_rpng

imageslg_r_cdb_gmgif

imageslg_r_cdb_modgif

imageslg_r_cdb_outputgif

imageslg_r_cdb_processgif

imagespreview_tablegif

randomForestxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

IBM SPSS Modeler and R

6 Your R code7

8 ibmspsscfdatamodelSetDataModel(modelerDataModel)

9 ibmspsscfdataSetData(modelerData)

10

All the functions starting with ibmspsscf are functions within this library It is this part of the codethat is responsible for the transfer of the data and the metadata to R Also you will see a while loopindicating that the data will go to R in batches of 1000 The other options are the values of the data readoptions within the nodeThe last lines of the code prepares the data to be send back to SPSS Also note the closing brace whichis the end of the while loop

Now since this library will be loaded with every interaction between SPSS and R you are free to use thesefunctions within your code as well So if you would like to avoid these runs in batches of 1K and you donot have version 171 available you can start your R code with another loop to first continue filling themodelerData and only than start your actual code In practice that means that in your R node you startwith

1 while(ibmspsscfdataHasMoreData())

2

3 modelerData lt-rbind(modelerDataibmspsscfdataGetData(rowCount=100000missing=NArDate=None

logicalFields=FALSE))

4

5

6 Some more code using modelerData which is now the complete dataset

512 Some useful parts of R code

Make sure a package is installed Whenever you have created any R node using packages you have tomake sure that anyone using this node has this package on his machine installed if needed withouttoo much user interference Therefore you can use this part of code

1 packages lt- function(x)

2 x lt- ascharacter(matchcall()[[2]])

3 if (require(xcharacteronly=TRUE))

4 installpackages(pkgs=xrepos=httpcranr-projectorg)

5 require(xcharacteronly=TRUE)

6

7

8

9 packages(rpart)

It will firstly verify if the library is installed on the system if not it will silently install it fromthe given CRAN mirror (you can change this to another mirror or a local repository if needed)Installation will only happen the first time the node will be used

Create the metadata corresponding to the R data frame Sometimes your data is very much trans-formed compared to the original data that it is difficult to build your metadata starting from theoriginal one So you might want to change the metadata to link to the data in R independent ofthe original This is in particularly useful if you want to use R as a sort node Below is a functionthat asks for a data frame and creates modelerData and modelerDataModel accordingly

1 sendToModeler lt- function (dataFrame)

2 if(isnull(dim(dataFrame)))

3 stop(Invalid data received not a dataframe)

4 if (dim(dataFrame)[1]lt=0)

5 print(Warning modelerData has no line all fieldStorage fields set to strings)

6 getStorage lt- function(x)return(string)

7 else

8 getStorage lt- function(x)

9 x lt- unlist(x)

10 res lt- NULL

11 if x is a factor typeof will return rsquointegerrsquo so we handle this case first

Page 15 of 20

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 2: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

Contents

1 System Setup 311 Installing R 312 Enabling the R nodes 3

2 R basics 3

3 The basics of R nodes in IBM SPSS Modeler 531 The R nodes 532 Simple R code example 5

321 modelerData 6322 modelerDataModel 7323 modelerModel 8

33 Some general remarks 1034 Read data options 11

4 Custom Dialog builder 1141 Tools 1242 Custom dialog 1243 Simple example 12

5 Tips amp tricks Some more detailed 1451 R code 14

511 ibmspsscf70 library 14512 Some useful parts of R code 15

52 Custom Dialog builder 17521 How to save and share a custom dialog 17522 Link to dialog and script 17

53 What about SQL Pushback Hadoop pushback 1854 What about real-time scoring and Solution Publisher 1955 Something more about the metadata in modeler and the consequences on R integration 19

Page 2 of 20

IBM SPSS Modeler and R

1 System Setup

Let us start with the setup of your system For now we assume that you have a valid installationof IBM SPSS Modeler on your machine For more installation topics we refer to the InstallationInstructions

11 Installing R

Depending on the version of your IBM SPSS Modeler you will now have to install dierent versionsof R

SPSS Version R version R download link

160 2152 httpscranr-projectorgbinwindowsbaseold2152170 31 httpscranr-projectorgbinwindowsbaseold310171 31 httpscranr-projectorgbinwindowsbaseold310180 32 httpscranr-projectorgbinwindowsbaseold320

Once you downloaded and installed you will have a working R instance on your machineserverLike SPSS Modeler you can have several versions of R installed on your machine without anyproblem

12 Enabling the R nodes

You will need to install the IBM SPSS Modeler essentials for R You can nd these here on theSPSS Community Downloads page Click 2 Get Essentials for SPSS and then click the buttonGet R Essentials for SPSS Modeler This will take you to github and you will be able to selectand download the Modeler 18 Essentials for R for a variety of platforms If you require Essentialsfor R for earlier Modeler versions there is also a link to legacy versions

Run this execution le The installation will ask you the path of your R installation and thepath to the bin les of your SPSS Modeler installation (Note that in the prelled path it is thedefault path to a ModelerServer and you will need to change this if you want to congure yourclient) This installation will place the R nodes in your SPSS Modeler node palette and it willalso include necessary R libraries in your R installation folder

2 R basics

There is already an over13ow of R courses (publicly) available through several channels so wewould certainly not want to replace these In it also not very important that you are an R expertto follow this document However there are still some basics of R code and R terminology usersneed to understand in order to exploit the integration of R and IBM SPSS Modeler For thissection let us open R in its original GUI Therefore go to the R installation folder and openbinx64RGUIexe A window will be opened looking like this

Page 3 of 20

IBM SPSS Modeler and R

This is the R console ready for commands to run You might often hear the term RStudio whichis nothing more than a development environment on top of this R gui Installation of RStudio isnot required for this introduction but might be handy for further useWe will start the R introduction by stating R is a powerful programming language and environmentfor statistical computing and graphics An important part within that last phrase is that R is aprogramming language unlike IBM SPSS Modeler That means it is built on objects that aredened by the user As an example assume the following R code (feel free to type it within theR console to see the R outputs)

1 x lt- 1+1

2 y lt- 2x

3 xyVector lt- c(xy)

4 z lt- mean(xyVector)

5 print(z)

Here x is an object This statement will ll the object x with the value of the evaluated formula1 + 1 being 2 So whenever the program refers to x it will be interpreted as 2 In the secondline we will dene y as twice the value of x In the third line we create a vector containing thecontent of x and y to calculate the mean of these 2 objects and place it in an object z

The operator lt- could also be replaced by = but for various reasons lots of R users pre-fer this way of writing (actually it is not exactly the same but that could be ignored for thepurpose of this document) If you feel more comfortable in using = please do so

Like we lled x y and z with some numbers any R object can be lled with a variety of typesHere is a list of the most important for our purposes

Vector is a sequence of data elements of the same type (eg numeric or character) This includesvectors of length 1 which can be interpreted as just being numbers You can create a vectorwith the R function c() So in the example code above all the values of x y and z arevectors of length one xyVector is a vector of length 2 containing the values of (the vector)x followed by (the vector) y Trying to link it back to SPSS you can interpret a vector asthe values of a single data column

Data frame is a list of vectors of equal length If you look at a vector as the values of a variablea data frame could be interpreted as a 2-dimensional dataset with columns (the number ofvectors) and lines (the size of each vector)

1 n lt- c(2 3 5 3 9) A first vector of 5 numeric values

2 n2 lt- c(1 3 2 5 4) A second vector of 5 numeric values

3 s lt- c(aa bb cc aa zz) A third vector of 5 string values

4 b lt- c(TRUE FALSE TRUE TRUE TRUE) A fourth vector of 5 flag values

5 Data lt- dataframe(n s b New = n+n2) A data frame containing 4 vectors

Page 4 of 20

IBM SPSS Modeler and R

6 Note n+n2 will be a new vector called New with the sum of the n + n2 c(3 6 7 8 13)

7

8 dim(Data) Will show you it is a 5x4 dataset

9 Data[24] Will give back the value on the 2nd line the 3rd column

10 colnames(Data) Will give the column names as a vector (nsb New)

11 Data$n[1] Will give back the first value of the vector n within the data frame

12

13 iris predefined data frame

There are also several pre-dened data frames installed within R One of them is called irisSometimes this document will refer back to iris

Model class which is actually a specic list containing predened objects dening a statisticalmodel For example a linear model class will be a list containing among others the coecientsof the regression model

List is an ordered collection of objects As an example you can have a list where the rst elementis a vector the second is a data frame and the third is a model Note that a data frame isa special type of a list where all the elements are vectors of equal sizes

3 The basics of R nodes in IBM SPSS Modeler

31 The R nodes

Once the installation for the R essentials are done you will see 3 new nodes in your node palettesThere is also a 4th R node which is the R nugget The dierence between and understanding ofthese 4 objects are essential

Output with this node data will be sent to R but it will never go back to SPSS (as it is aterminal node) The only thing that can go back to SPSS is the outputs generated by Rthatwill be presented within an SPSS output window

Transform data will go from SPSS to R but will also go back to SPSS after which the SPSSprocess can be continued

Model like the output node this is a terminal node so data will not go back to SPSS Howeverthere will be a reusable R object created within a nugget

Nugget similar to Transform node with the dierence that there is a reusable R object that canbe used in the R code

Node

Name R output node R transform node R model node R syntax nodePalette tab Output Record Ops Model NAData back to SPSS No Yes No YesReusable R object No No Create Use

32 Simple R code example

Let us start with saying that all the examples in this section are intentionally kept very simple soas to explain the interaction in a functional and structured way and be simple enough for non Rprogrammers We are certainly aware that most of the R code snippets we show in this chapter

Page 5 of 20

IBM SPSS Modeler and R

could also easily be implemented using standard SPSS Modeler functionality

There are 3 very important and reserved R objects that you should keep in mind when youuse the SPSS Modeler R integration Here is a brief description of these 3 after which we will gointo more detail for each of them

modelerData This is an R data frame that will be lled by the data entering in this R node Thisdata frame can be used and changed within your R code Eventually it will also be the dataframe that will be sent back to SPSS Modeler as a dataset Note that it will only containthe content of the data not (necessarily) the data column names and other metadata items

modelerDataModel This is also an R data frame containing the metadata of the data that is sentto R and back to SPSS Modeler It contains most of the information that you may expectwithin an SPSS Modeler type node This will be the object that will be most strange forexperienced R users

modelerModel this is an R object that can be lled by the user by any type of object you wantIt does not need to have a certain structure It will be calculated in the R model node afterwhich it will be saved within the R nugget where it can be used in the R-syntax

Note that R code is case-sensitive and therefore so are these object names In the following sectionswe will explain the usage of these objects

321 modelerData

modelerData is the R data frame that will be lled by the dataset entering the SPSS Modeler nodeit comes from So you can use this data frame to perform the desired calculations transformationsand outputs in R

Place the following code in an R output node

1 Print the first 6 lines of the data

2 head(modelerData)

3

4 Give a summary of the data

5 summary(modelerData)

6

7 create a histogram of the variable tenure

8 hist(modelerData$tenure xlab = years main = Tenure histogram)

9

10 change the tenure unit from months to years

11 modelerData$tenure lt- modelerData$tenure1212

13 recreate the histogram now in months

14 hist(modelerData$tenure xlab = months main = Tenure histogram)

Execution of this node will result in an SPSS Modeler output window in which all the R outputswill be assembled These will always be divided in 2 tabs Text output and Graph output

Page 6 of 20

IBM SPSS Modeler and R

In this case the text output is linked to the code on line 2 and 5 rst it prints the rst 6 lines(head) of the data next it will give summary statistics for each column

The graph output are two histograms One for the tenure in months the other for the samecolumn but after redening it by dividing the original value by twelve to give the tenure in years(note the X-axis scale)

As shown in the example stream Explain modelerDatastr you can also copy exactly this samecode into a transform node and attach a table node to it After running this table node you willnot see any R output (as none is expected) That means that even though the output code hasrun no outputs will be given However the data frame of modelerData will be send back to SPSSModeler In this case you will see the value of tenure being divided by 12

322 modelerDataModel

Metadata is very important in SPSS Modeler Let us for simplicity say that within modeler meta-data is represented by the type node With metadata we mean the type of each of the variablesin the dataset (numeric 13ag String storage ) At all times modeler will know exactly all themetadata at every step in the stream

R does not handle the metadata in a similar way as SPSS Modeler We already explainedmodelerDataModel taking over the role of the type node This is done by a data frame(=dataset) of the following structure

X1 X2 Xn

fieldName region tenure agefieldLabel Geographic indicator Months with service Age in years

fieldStorage real real realfieldMeasure nominal continuous continuousfieldFormat standard standard standardfieldRole input input input

So this means that this dataset will always have 6 lines with xed names (yes in R also thelines have names) The thing with this dataset is that it is completely the responsibility of theuser to align this metadata with the appropriate data So that means if we would like to add avariable with R the user must also manually add a column in modelerDataModel to make suremodelerData correctly goes back to SPSS Modeler In the earlier example above we did not makeany changes to the modelerDataModel and it was also not needed as the metadata did not change(dividing a number by 12 will not change the metadata) Now let us continue on the previousexample But now rather than changing the value of tenure in the same data variable we willcreate another one As a result we would have to update the metadata

1 Create the vector of tenure in years

2 Rcolumn lt- modelerData$tenure123

4 Paste this vector to the right of the dataset

5 modelerData lt- cbind(modelerDataRcolumn)

6

7 create the metadata for the column to add

8 newVar lt- c(fieldName=tenureYears fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

9

10 paste the new column metadata to the existing metadata

11 modelerDataModel lt- cbind(modelerDataModelnewVar)

Running a table node downstream of this transform node will show you the new variable withthe name tenureYears There are some important things to realize in this

Page 7 of 20

IBM SPSS Modeler and R

bull fieldName and fieldStorage are the only 2 required rows that needs to be lled in for anynew column In the code we left all the other lines empty meaning they will be lled in bythe stream default For a list of available values we refer to the user guide

bull As modelerDataModel is only useful when you go back to SPSS Modeler you will generallyonly usechange this object in non-terminal R-nodes It might still be handy to use it interminal nodes if the value of the modelerDataModel is important for your output (egrun a histogram of all continuous variables)

bull When data will go back to SPSS modeler it will be the content of the data frame excludingthe column- and row names That means that even though the column in the modelerData

will be called Rcolumn the name of the column in SPSS will only be dened by the metadatawithin the row fieldName In this case it is called tenureYears

bull The only link between modelerData and modelerDataModel is the order of the columns Itwill not look by name The rst column in the data will be given the metadata of the rstcolumn of modelerDataModel In case the metadata (modelerDataModel ) does not matchthe modelerData an error is thrown The table below shows schematic how this works

modelerData modelerDataModel

R

RName1 RNamenx11 x1n

x21 x2n

xm11 xm1n

xm1 xmn

X1 Xn

fieldName Name1 NamenfieldLabel

fieldStorage xxx xxxfieldMeasure fieldFormat

fieldRole

SPSS

Name1 Namenx11 x1n

x21 x2n

xm1 xmn

Note that only the names of the modelerDataModel are used

Since this concept is very strange to standard R users We found this part the most dicult toexplain To people who know SPSS you can summarize it as modelerDataModel taking over therole of the type node

You can nd all streams and R scripts explaining modelerDataModel here

323 modelerModel

modelerModel is the R object that is stored within the R nugget This object will be populatedwithin the R model node after which you could use modelerModel within the R nugget for scoringThis very much works the same way as IBM Modeler works You ask a model node to calculatea formula after which that formula will be stored within the nugget together with the way itshould be used to calculate a scoring

You will only use this object within the R model node and nugget Note that within the Rmodel node there are 2 syntax window

Page 8 of 20

Explain modelerDataModelxml

data0001dat

data0002dat

IBM SPSS Modeler and R

R model building is to calculate whatever you want to store within modelerModel that couldbe used within your nugget calculations to score your data This can be any object withinR As any SPSS Model builder node this will be a terminal code meaning that from thiscode no data will go back to SPSS modeler unless some outputs and things that are storedwithin the modelerModel object

R model scoring is the syntax to dene how you will use the object modelerModel containingthe content you stored to it in the R model building syntax to derive the new data Apartfrom the use of modelerModel this is very similar to the R transform node

Let us start with a simple example where we would like to create a basic linear model for thevariable tenure The formula of this model should be saved in the modelerModel after which itcan be used in the scoring

1 Create the model and save it in modelerModel

2 modelerModel lt- lm(tenure ~ age + region + ed + income data= modelerData)

3

4 Add some summary of the model in the nugget

5 summary(modelerModel)

6

7 together with a histogram

8 hist(modelerModel$residuals main = residual histogram)

9

10 and the residual vs actuals scatterplot

11 plot(modelerData$tenure modelerModel$fittedvalues xlab = actual ylab = predicted )

12

13 All of these output will be stored in the modeler nugget tabs

1 Use the model to make a prediction and add it to the existing data

2 pred lt- predict(modelerModel modelerData)

3 modelerData lt- cbind(modelerDatapred)

4

5 Take care of the metadata

6 newVar lt-c(fieldName=$L-tenure fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

It is important to note that modelerModel can be lled with any type of object but will veryoften be of a model class but does not have to be In the previous example the object stored toit was clearly a (statistical) model In the next example we will just save 2 numbers within themodelerModel object Imagine we want to calculate the z-values of a certain variable In orderto create the z-values we need the mean and the standard deviation of the column We will storeboth of these within modelerModel after which we will use them in the scoring syntax1 Thisexample shows you do not need to store a statistical model within your modelerModel objectbut it really can be any R object

1 calculate mean and standard deviation

2 M lt- mean(modelerData$tenure)3 SD lt- sd(modelerData$tenure)4

5 and save it in a list called modelerModel

6 modelerModel lt- list(avg = M sDev = SD)

1 calculate z scores using the elements in modelerModel

2 zTenure lt- (modelerData$tenure - modelerModel$avg)modelerModel$sDev3 modelerData lt- cbind(modelerDatazTenure)

4

5 define new metadata column and add it

6 newVar lt- c(fieldName=zTenure fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

1Note that there is a very good reason this is not combined into an R Transform node explained in 34

Page 9 of 20

IBM SPSS Modeler and R

You can nd all streams and R scripts explaining modelerModel here

33 Some general remarks

bull Although it might seems this way you are not required to build modelerData from theexisting data within that frame modelerData will be lled with the dataset you have inModeler however nothing stops you to throw that data away in R and dene some new datacoming from another data source in R As an example imagine this link from the WeatherCompany website This will give the weather history in Brussels Belgium in the monthNovember 2015 Now we can use R code as a source node by just overwriting modelerData

and redening modelerDataModel

1 Define the link

2 linkPath lt- httpwwwwundergroundcomhistoryairportEBBR20151101

MonthlyHistoryhtmlreq_city=Brusselsampformat=1

3 Read the data as csv

4 modelerData lt- readcsv(linkPath)

5 modelerData[1] lt- asDate(modelerData[1])

6

7 Redefining modelerDataModel all are real numbers except the first column is the date

8 modelerDataModel lt- asdataframe(t(dataframe(fieldName = colnames(modelerData)

fieldLabel = fieldStorage = c(daterep(realncol(modelerData)-1))

fieldMeasure = fieldFormat = fieldRole = )))

As you can see this code does not use the old denition of the dened R objects butcompletely redenes them Placing this in a R transform node will give back this newdataset to modeler So in this way you can use this approach to create an R input node

You can nd an example here

bull Within an R model node there is place for 2 scripts The building script will be the scriptthat will be populated within the R nugget It will not be run when you run the model nodeAs a result of this these 2 scripts are independent The only thing they share is the valueof the object modelerModel which is saved within the nugget when running the buildingsyntax and picked up within the R scoring syntaxThis also means that eventual R-libraries that are required should be loaded in both scriptsTake for example a model for a random forest

1 Load the library

2 library(randomForest)

3 Create the model and save it in modelerModel

4 modelerModel lt- randomForest(tenure ~ age + region + ed + income data= modelerDatantree

=50)

1 Load the library

2 library(randomForest)

3

4 Use the model to make a prediction and add it to the existing data

5 predlt- predict(modelerModel modelerData)

6 modelerData lt- cbind(modelerDatapred)

7

8 Take care of the metadata

9 newVar lt- c(fieldName=$RF-tenurefieldLabel=fieldStorage=realfieldMeasure=fieldFormat=fieldRole=)

10 modelerDataModel lt- cbind(modelerDataModelnewVar)

bull Talking about libraries and package A package is a collection of R objects dened for acertain purpose These often are specic statistical functionalities like randomForest in theexample above A basic R installation comes with the standard packages however there aremany more packages available made available by the R community on CRAN

Page 10 of 20

Explain modelerModelxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

R as sourcexml

data0001dat

data0002dat

IBM SPSS Modeler and R

Packages needs to be installed and made locally available in libraries Once the package isinstalled on the system as a library you can load this library in any R session by the codelibrary(ltnamegt)To install a package you have several options The easiest is probably to write a codelike installpackages(randomForest) within R You will have to select a CRAN mirrorwhere this library will be downloaded from and the download will go automatically Nor-mally you will only have to do this onceAlthough possible it is not recommended to run this package installation command fromwithin SPSS The reason is that these libraries will than be saved in a temporary folder andafterwards be deleted If you still want to this through SPSS you will have to hard codethe installation path

34 Read data options

Something we ignored until now are the settings within the node under Read data options Thebasics of the R integration with Modeler can be done without the knowledge of this as it requiressome more advanced R knowledge The user guide still has a good explanation about these items

However there is one more thing that might be important For modeler version 17 and lowerthe R integration of non-terminal nodes (ie transform and nuggets) will by default be done inbatches of 1000 The reason for this was to allow these R nodes to work on hadoop and otherclustered environments As a result of this it is very important to realize that any R code thatwould span over multiple lines of data would lead to false results For a workaround for this werefer to 511

Take as an example the z scores above If we would calculate the mean and the standard de-viation of the variable in a non-terminal node it would start with running this code for the rst1000 lines of data So that leads to a specic mean deviation and corresponding z-scores How-ever the next 1000 lines a new mean and deviation would be calculated and the z scores will bebased on these

As a solution the means and standard deviations are calculated in the R model (ie a termi-nal node) over all the data and used in the R nugget to calculate the z scores

Note that this approach may lead to a very slow integration between SPSS and R in the caseof streaming R nodes in a local non-clustered environment However as from IBM SPSS Modelerversion 171 there is a default option not to use this approach of batch processing or to increasethe batch size For the lower versions there is a workaround possible if you still want to increasethis batch size or turn it o (see later)

4 Custom Dialog builder

The Custom Dialog Builder allow you to create and manage R nodes with prefilled R code to use insideIBM SPSS Modeler streams In this way users can create their own nodes You can start the CustomDialog builder in the Tools menu under rdquoCustom Dialog Builderrdquo

When opening a custom dialog builder you will see a 2 windows One of them is the custom dialogitself the other is the toolset to populate the dialog

Page 11 of 20

IBM SPSS Modeler and R

41 Tools

The tools window is a list of items you can place within your dialog This include among others the fieldchooser Check and combo boxes Text and number controls and tabs You can select any and drag themonto the dialog itselfOnce you have any item in the dialog you can select it and you will see the item properties These are theproperties of this specific item and will change dependent on the type of item it is The most importantare the identifier (the way it will be referenced within the script) and the Title (the one that will be visualin the dialog)

42 Custom dialog

The big gray window is the dialog itself For the moment it is empty as it should be populated with itemsfrom the Tools listClicking on this gray dialog will show you the dialog properties below As main items this includes thename and title of the dialog the script itself and the type and position of the created nodeWith regards to the script to be written The global rule is that you reference to the items within thedialog using their identifier between double percentages (rdquoltidentifiergtrdquoOnce you finished creating the custom node you can install it by pressing the green arrow in the toolbarYou can also save intermediate versions to the disk

43 Simple example

Let us create a custom dialog for the randomForest model created earlier in section 33 Below you willfind a step by step approachThe most important thing we have to wonder is what within this code we want flexible for the user Inthe case of this model there might be 3 things that we want flexibel the input variables the target andthe number of trees in our forest

First fill in the Custom dialog properties as indicated

Page 12 of 20

IBM SPSS Modeler and R

In our fixed example the target is tenure but a user might chooseany other field As a result we will place a field chooser on the dialogChange the properties like shown The variable filter properties allowsyou to select only categorical variables

In our fixed example the inputs are age income but a user mightchoose any other field As a result we will place a field chooser on thedialog The biggest difference is that now we can select several variablesas there might be different inputs To make it easier we will separatethese values by a + Therefore change the properties like shown

As a third custom choice we would like to add the number of trees inour forest In the original script is was 50 so we will choose this as thedefault However users may choose any integer value between 1 and1000 Add a number control on the dialog and change the properties

So now the dialog is ready and we need to add the script to it Go tothe Edit options and choose rdquoScript Templaterdquo This will bring you toan empty window for the script In this case (as we selected we wanted2 scripts) there is a tab for the building code and one for the scoringscript If the coring script is greyed out you did set the dialog propertyrdquoScore from the Modelrdquo to True

Page 13 of 20

IBM SPSS Modeler and R

Let us start with the scoring script as this is easier The only thingwhich will need to be adapted for custom input is the variable namethat will be send back to SPSS So copy the scoring code and changethe name tenure to TARGET (this is the name of the identifier of thetarget)

Fill in the code for the building script and change in a similar wayas above the values for target and intput variables together with thenumber of trees Afterwards press OK to close the script window

Being back at your Custom Dialog builder save the dialog in any appro-priate location Also deploy the dialog by clicking on the green deployarrow in the toolbar Close the Dialog builder

Back to the stream you will now see the new node in the model paletteYou can use this node within your stream

You can find the resulted cfe file here (place this in the correct location see 521) and a stream

where it is deployed

5 Tips amp tricks Some more detailed

51 R code

511 ibmspsscf70 library

Let us now have a more detailled view about what actually happens with the code First of all it is worthto check what happens when you do the R installation correctly This will install the by IBM delivered Rpackage ibmspsscf70 in the library folder of your R installation This library contains several functionsto handle the data traffic between SPSS and R

Running any R node in SPSS will not only run the code you write but it will also run some extracode behind the scenes You can see this code in the rdquoConsole outputrdquo window of the R node Lookingfor example at this tab for an R nugget you will see that your code will be something like

1 modelerModel lt- ibmspsscfoutputGetModel()

2 while(ibmspsscfdataHasMoreData())

3 modelerDataModel lt- ibmspsscfdatamodelGetDataModel()

4 modelerData lt- ibmspsscfdataGetData(rowCount=1000 missing=NA rDate=None

logicalFields=FALSE)

5

Page 14 of 20

app-extensionxml

cdb_peerjar

comspsssharedcustom_guiui_builderpeersRStatsApplierPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsApplierPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsApplierPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                    
                

comspsssharedcustom_guiui_builderpeersCDExtensionPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class CDExtensionPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static String                         RBUILDER                            private                         static String                         SCORE_SYNTAX                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            public void CDExtensionPeer()                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private String                         replaceFieldSeparator(String String)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$1                         implements javaawteventActionListener     void ActionHandler$1(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$2class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$2                         implements javaawteventActionListener     void ActionHandler$2(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsUIPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsUIPeer                         implements compaswframeworkuiswingspiExtensionObjectSwingUIPeer compaswframeworkcommonextensionspiInteractorListener compaswcorecontainerContainerSetListener                             private ActionHandler                         actionHandler                            public                         static                         final String                         UPDATE_CONTAINER = updateContainer                            public void RStatsUIPeer()                            public void                         initSwingUI(compaswframeworkuiswingExtensionObjectSwingUI compaswframeworkcommoncoreProperties compaswframeworkuiswingExtensionObjectSwingUIContext)                            public void                         disposeSwingUI()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            public void                         handleAction(String String compaswframeworkcommoncoreProperties)                            public void                         containerChanged(compaswcorecontainerContainerSetEvent)                            public void                         updateContainer(compaswcorecontainerContainerSetEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsProcessPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsProcessPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsProcessPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersCopyrightclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         abstract                         interface Copyright                         extends annotationAnnotation                             public                         abstract String                         value()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxControl                         extends javaxswingJTextArea                         implements compaswframeworkuiswingspiPropertyControl                             private                         static                         final String                         editScoreSyntax = editScoreSyntax                            private                         static                         final String                         editScoreSyntaxTitle = editScoreSyntaxTitle                            private                         static                         final String                         editScoreSyntaxWarning = editScoreSyntaxWarning                            private                         static                         final String                         editScoreSyntaxContinueBtn = editScoreSyntax_Continue                            private                         static                         final String                         editScoreSyntaxCancelBtn = editScoreSyntax_Cancel                            private javaxswingJPanel                         containerPanel                            private javaxswingJButton                         editScoreSyntaxBtn                            private compaswframeworkuiswingPropertyControlContext                         propertyControlContext                            public void ManagedScoreSyntaxControl()                            public void                         disposeControl()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public javaxswingJComponent                         getUIComponent()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            public boolean                         isControlEnabled()                            public void                         setControlEnabled(boolean)                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxPanel                         implements compaswframeworkuiswingspiManagedPanel                             public void ManagedScoreSyntaxPanel()                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ROutputFileChooserControl                         implements compaswframeworkuiswingspiPropertyControl compaswuicommoncontrolControlListener                             public                         static                         final String                         OUTPUT_TYPE = output_type                            public                         static                         final String                         OUTPUT_TYPE_GRAPH = Graph                            public                         static                         final String                         GRAPH_OUTPUT_FILE_TYPE = graph_output_file_type                            public                         static                         final String                         TEXT_OUTPUT_FILE_TYPE = text_output_file_type                            public                         static                         final String                         OUTPUT_FILE_TYPE_HTML = HTML                            public                         static                         final String                         OUTPUT_FILE_TYPE_TEXT = Text                            public                         static                         final String                         OUTPUT_FILE_TYPE_COU = Cou                            public                         static                         final String                         FILENAME = OutputFilename                            public                         static                         final String                         OUTPUT_FILE_CHOOSER_BUTTON = OutputFileChooserButton                            public                         static                         final char                         BACK_SLASH_CHAR = 92                            public                         static                         final char                         FORWARD_SLASH_CHAR = 47                            private ActionHandler                         actionHandler                            private javaxswingJLabel                         fileNameLabel                            private javaxswingtextJTextComponent                         textComponent                            private javaxswingAbstractButton                         browseButton                            private compaswuiswingcontrolDefaultControlPanel                         mainPanel                            private String                         propertyName                            private compaswframeworkuiswingPropertyControlContext                         context                            public void ROutputFileChooserControl()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            private void                         initComponent()                            private compaswframeworkuiswingSwingResourceProvider                         getSwingResourceProvider()                            private String                         getLabelFromSwingResourceProvider(String)                            private Object                         getSwingUiProperty(String)                            protected String                         getTextComponentValue()                            public javaxswingJComponent                         getUIComponent()                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public void                         setControlEnabled(boolean)                            public boolean                         isControlEnabled()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public void                         disposeControl()                            public void                         controlValueChanged(compaswuicommoncontrolControlEvent)                            private String                         removeExtension(String)                            public void                         controlSelectionChanged(compaswuicommoncontrolControlEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandlerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ActionHandler                             private compaswframeworkuiswingExtensionObjectSwingUI                         extensionObjectSwingUI                            private boolean                         bPrintAll                            private int                         printScope    String                         actionErrorMessage                            public void ActionHandler(compaswframeworkuiswingExtensionObjectSwingUI)                            public void                         handleAction(String String)                            private void                         handleExportHTML(String)                            private void                         handleExportText(String)                            private void                         handlePrint(String String)                            private void                         handlePrintView()                            private void                         handlePageSetup(String)                            private void                         handleSaveNode(String)                            private void                         handlePreviewTable(String)                            private void                         handleSaveToProject(String)                            private void                         handleHelpButton()                            private void                         exportHTMLFile(String)                            private void                         saveTextFile(String String)                            private javaxswingJPanel                         createPrintViewPanel(javautilList)                            public String                         launchSaveFileChooser(String String[])                            private void                         configureSaveFileChooser(javaxswingJFileChooser)                            private void                         configureSaveChooserCallback(compaswcoreapplicationcallbackFileSystemChooserRequest)                            private String                         getSaveDialogTitle()                            private String                         getApproveButtonText()                            private boolean                         useApplicationFileChoosers()                            private compaswframeworkuicommonUISession                         getUISession()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            private String                         getTextContainerData(String)                            private boolean                         isExportText(String)                            private boolean                         isExportHTML()                            private boolean                         isSelectNode()                            private compaswuiswingcontrolModelViewerPanel                         getModelViewerPanel()                            private String                         getDefaultFileName()                            private void                         handleUpdateContainer(String)                    
                

comspsssharedcustom_guiui_builderpeersManagedBasePanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedBasePanel                         extends javaxswingJPanel                         implements javaawtprintPrintable     void ManagedBasePanel(javaawtLayoutManager)                            public int                         print(javaawtGraphics javaawtprintPageFormat int)                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$1                     
                

comspsssharedcustom_guiui_builderpeersUIUtilitiesclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class UIUtilities                             public void UIUtilities()                            public                         static javaxswingJButton                         createButton(compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComboBox                         createComBox(compaswframeworkuiswingSwingResourceProvider String int javaxswingDefaultComboBoxModel)                            public                         static javaxswingDefaultComboBoxModel                         createEnumComboBoxModel(compaswframeworkcommoncoreEnumerationValueDescriptor)                            public                         static javaxswingAbstractButton                         addAccessKey(javaxswingAbstractButton compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingtextJTextComponent                         createTextFiled(compaswframeworkuiswingSwingResourceProvider String int)                            public                         static javaxswingJComponent                         addTooltip(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComponent                         addDescription(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static                         transient void                         fitAllComponents(javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxPreferSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setPreferSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMaxSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMaxSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMinSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMinSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                    
                

comspsssharedcustom_guiui_builderpeersManagedHTMLOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedHTMLOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedHTMLOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedModelOutputPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedModelOutputPanel                         implements compaswframeworkuiswingspiManagedPanel compaswframeworkcommonextensionspiInteractorListener                             public                         static                         final String                         BUILD_TEXT_OUTPUT = buildTextOutputDisplay                            public                         static                         final String                         BUILD_CONSOLE_OUTPUT = buildConsoleOutputDisplay                            public                         static                         final String                         BUILD_HTML_OUTPUT = buildHTMLOutputDisplay                            public                         static                         final String                         BUILD_MV_OUTPUT = buildModelViewerDisplay                            public                         static                         final String                         APPLIER_CONSOLE_OUTPUT = applierConsoleOutputDisplay                            public                         static                         final String                         BUILD_TEXT_CONTAINER_NAME = TextOutput                            public                         static                         final String                         BUILD_CONSOLE_CONTAINER_NAME = ConsoleOutput                            public                         static                         final String                         BUILD_HTML_CONTAINER_NAME = HTMLOutput                            public                         static                         final String                         BUILD_PMML_CONTAINER_NAME = PMML                            public                         static                         final String                         BUILD_STAT_XML_CONTAINER_NAME = StatXML                            public                         static                         final String                         APPLIER_CONSOLE_CONTAINER_NAME = ApplierConsoleOutput                            private ManagedTextOutputArea                         textOutput                            private ManagedTextOutputArea                         consoleOutput                            private ManagedHTMLOutputArea                         htmlOutput                            private ManagedEmptyOutputArea                         emptyOutput                            private javautilHashMap                         componentMap                            private compaswframeworkuiswingManagedPanelContext                         managedPanelContext                            public void ManagedModelOutputPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private compaswframeworkuiswingspiManagedUIElement                         setupTextOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupConsoleOutputControl(String)                            public void                         updateConsoleOutputControl(String)                            private compaswframeworkuiswingspiManagedUIElement                         setupHTMLOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupModelViewerControl()                            public String                         getTextContainerData(String)                            public                         static boolean                         exportToHTML(compaswframeworkcommoncontainerContainer String)                            public                         static javaioFile                         getHTMLFileTest(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractHTMLCompoundContainer(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractZip(String String boolean)                         throws javaioIOException                            public                         static javaioFile                         getHTMLFile(String String boolean)                         throws javaioIOException                    
                

comspsssharedcustom_guiui_builderpeersManagedTextOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedTextOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedTextOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$ButtonsActionclass

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$ButtonsAction                         implements javaawteventActionListener                             private void ManagedScoreSyntaxControl$ButtonsAction(ManagedScoreSyntaxControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersManagedEmptyOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedEmptyOutputArea                         extends javaxswingJPanel                         implements compaswframeworkuiswingspiManagedUIElement                             public void ManagedEmptyOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public javaxswingJComponent                         getComponent()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ROutputFileChooserControl$1                         implements javaawteventActionListener     void ROutputFileChooserControl$1(ROutputFileChooserControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

extensionxml

imagescopygif

imagesdeletegif

imagesfileMenugif

imagesgenerateMenugif

imageshelp_buttongif

imageslg_gm_rpng

imageslg_r_cdb_gmgif

imageslg_r_cdb_modgif

imageslg_r_cdb_outputgif

imageslg_r_cdb_processgif

imagespreview_tablegif

randomForestxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

IBM SPSS Modeler and R

6 Your R code7

8 ibmspsscfdatamodelSetDataModel(modelerDataModel)

9 ibmspsscfdataSetData(modelerData)

10

All the functions starting with ibmspsscf are functions within this library It is this part of the codethat is responsible for the transfer of the data and the metadata to R Also you will see a while loopindicating that the data will go to R in batches of 1000 The other options are the values of the data readoptions within the nodeThe last lines of the code prepares the data to be send back to SPSS Also note the closing brace whichis the end of the while loop

Now since this library will be loaded with every interaction between SPSS and R you are free to use thesefunctions within your code as well So if you would like to avoid these runs in batches of 1K and you donot have version 171 available you can start your R code with another loop to first continue filling themodelerData and only than start your actual code In practice that means that in your R node you startwith

1 while(ibmspsscfdataHasMoreData())

2

3 modelerData lt-rbind(modelerDataibmspsscfdataGetData(rowCount=100000missing=NArDate=None

logicalFields=FALSE))

4

5

6 Some more code using modelerData which is now the complete dataset

512 Some useful parts of R code

Make sure a package is installed Whenever you have created any R node using packages you have tomake sure that anyone using this node has this package on his machine installed if needed withouttoo much user interference Therefore you can use this part of code

1 packages lt- function(x)

2 x lt- ascharacter(matchcall()[[2]])

3 if (require(xcharacteronly=TRUE))

4 installpackages(pkgs=xrepos=httpcranr-projectorg)

5 require(xcharacteronly=TRUE)

6

7

8

9 packages(rpart)

It will firstly verify if the library is installed on the system if not it will silently install it fromthe given CRAN mirror (you can change this to another mirror or a local repository if needed)Installation will only happen the first time the node will be used

Create the metadata corresponding to the R data frame Sometimes your data is very much trans-formed compared to the original data that it is difficult to build your metadata starting from theoriginal one So you might want to change the metadata to link to the data in R independent ofthe original This is in particularly useful if you want to use R as a sort node Below is a functionthat asks for a data frame and creates modelerData and modelerDataModel accordingly

1 sendToModeler lt- function (dataFrame)

2 if(isnull(dim(dataFrame)))

3 stop(Invalid data received not a dataframe)

4 if (dim(dataFrame)[1]lt=0)

5 print(Warning modelerData has no line all fieldStorage fields set to strings)

6 getStorage lt- function(x)return(string)

7 else

8 getStorage lt- function(x)

9 x lt- unlist(x)

10 res lt- NULL

11 if x is a factor typeof will return rsquointegerrsquo so we handle this case first

Page 15 of 20

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 3: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

1 System Setup

Let us start with the setup of your system For now we assume that you have a valid installationof IBM SPSS Modeler on your machine For more installation topics we refer to the InstallationInstructions

11 Installing R

Depending on the version of your IBM SPSS Modeler you will now have to install dierent versionsof R

SPSS Version R version R download link

160 2152 httpscranr-projectorgbinwindowsbaseold2152170 31 httpscranr-projectorgbinwindowsbaseold310171 31 httpscranr-projectorgbinwindowsbaseold310180 32 httpscranr-projectorgbinwindowsbaseold320

Once you downloaded and installed you will have a working R instance on your machineserverLike SPSS Modeler you can have several versions of R installed on your machine without anyproblem

12 Enabling the R nodes

You will need to install the IBM SPSS Modeler essentials for R You can nd these here on theSPSS Community Downloads page Click 2 Get Essentials for SPSS and then click the buttonGet R Essentials for SPSS Modeler This will take you to github and you will be able to selectand download the Modeler 18 Essentials for R for a variety of platforms If you require Essentialsfor R for earlier Modeler versions there is also a link to legacy versions

Run this execution le The installation will ask you the path of your R installation and thepath to the bin les of your SPSS Modeler installation (Note that in the prelled path it is thedefault path to a ModelerServer and you will need to change this if you want to congure yourclient) This installation will place the R nodes in your SPSS Modeler node palette and it willalso include necessary R libraries in your R installation folder

2 R basics

There is already an over13ow of R courses (publicly) available through several channels so wewould certainly not want to replace these In it also not very important that you are an R expertto follow this document However there are still some basics of R code and R terminology usersneed to understand in order to exploit the integration of R and IBM SPSS Modeler For thissection let us open R in its original GUI Therefore go to the R installation folder and openbinx64RGUIexe A window will be opened looking like this

Page 3 of 20

IBM SPSS Modeler and R

This is the R console ready for commands to run You might often hear the term RStudio whichis nothing more than a development environment on top of this R gui Installation of RStudio isnot required for this introduction but might be handy for further useWe will start the R introduction by stating R is a powerful programming language and environmentfor statistical computing and graphics An important part within that last phrase is that R is aprogramming language unlike IBM SPSS Modeler That means it is built on objects that aredened by the user As an example assume the following R code (feel free to type it within theR console to see the R outputs)

1 x lt- 1+1

2 y lt- 2x

3 xyVector lt- c(xy)

4 z lt- mean(xyVector)

5 print(z)

Here x is an object This statement will ll the object x with the value of the evaluated formula1 + 1 being 2 So whenever the program refers to x it will be interpreted as 2 In the secondline we will dene y as twice the value of x In the third line we create a vector containing thecontent of x and y to calculate the mean of these 2 objects and place it in an object z

The operator lt- could also be replaced by = but for various reasons lots of R users pre-fer this way of writing (actually it is not exactly the same but that could be ignored for thepurpose of this document) If you feel more comfortable in using = please do so

Like we lled x y and z with some numbers any R object can be lled with a variety of typesHere is a list of the most important for our purposes

Vector is a sequence of data elements of the same type (eg numeric or character) This includesvectors of length 1 which can be interpreted as just being numbers You can create a vectorwith the R function c() So in the example code above all the values of x y and z arevectors of length one xyVector is a vector of length 2 containing the values of (the vector)x followed by (the vector) y Trying to link it back to SPSS you can interpret a vector asthe values of a single data column

Data frame is a list of vectors of equal length If you look at a vector as the values of a variablea data frame could be interpreted as a 2-dimensional dataset with columns (the number ofvectors) and lines (the size of each vector)

1 n lt- c(2 3 5 3 9) A first vector of 5 numeric values

2 n2 lt- c(1 3 2 5 4) A second vector of 5 numeric values

3 s lt- c(aa bb cc aa zz) A third vector of 5 string values

4 b lt- c(TRUE FALSE TRUE TRUE TRUE) A fourth vector of 5 flag values

5 Data lt- dataframe(n s b New = n+n2) A data frame containing 4 vectors

Page 4 of 20

IBM SPSS Modeler and R

6 Note n+n2 will be a new vector called New with the sum of the n + n2 c(3 6 7 8 13)

7

8 dim(Data) Will show you it is a 5x4 dataset

9 Data[24] Will give back the value on the 2nd line the 3rd column

10 colnames(Data) Will give the column names as a vector (nsb New)

11 Data$n[1] Will give back the first value of the vector n within the data frame

12

13 iris predefined data frame

There are also several pre-dened data frames installed within R One of them is called irisSometimes this document will refer back to iris

Model class which is actually a specic list containing predened objects dening a statisticalmodel For example a linear model class will be a list containing among others the coecientsof the regression model

List is an ordered collection of objects As an example you can have a list where the rst elementis a vector the second is a data frame and the third is a model Note that a data frame isa special type of a list where all the elements are vectors of equal sizes

3 The basics of R nodes in IBM SPSS Modeler

31 The R nodes

Once the installation for the R essentials are done you will see 3 new nodes in your node palettesThere is also a 4th R node which is the R nugget The dierence between and understanding ofthese 4 objects are essential

Output with this node data will be sent to R but it will never go back to SPSS (as it is aterminal node) The only thing that can go back to SPSS is the outputs generated by Rthatwill be presented within an SPSS output window

Transform data will go from SPSS to R but will also go back to SPSS after which the SPSSprocess can be continued

Model like the output node this is a terminal node so data will not go back to SPSS Howeverthere will be a reusable R object created within a nugget

Nugget similar to Transform node with the dierence that there is a reusable R object that canbe used in the R code

Node

Name R output node R transform node R model node R syntax nodePalette tab Output Record Ops Model NAData back to SPSS No Yes No YesReusable R object No No Create Use

32 Simple R code example

Let us start with saying that all the examples in this section are intentionally kept very simple soas to explain the interaction in a functional and structured way and be simple enough for non Rprogrammers We are certainly aware that most of the R code snippets we show in this chapter

Page 5 of 20

IBM SPSS Modeler and R

could also easily be implemented using standard SPSS Modeler functionality

There are 3 very important and reserved R objects that you should keep in mind when youuse the SPSS Modeler R integration Here is a brief description of these 3 after which we will gointo more detail for each of them

modelerData This is an R data frame that will be lled by the data entering in this R node Thisdata frame can be used and changed within your R code Eventually it will also be the dataframe that will be sent back to SPSS Modeler as a dataset Note that it will only containthe content of the data not (necessarily) the data column names and other metadata items

modelerDataModel This is also an R data frame containing the metadata of the data that is sentto R and back to SPSS Modeler It contains most of the information that you may expectwithin an SPSS Modeler type node This will be the object that will be most strange forexperienced R users

modelerModel this is an R object that can be lled by the user by any type of object you wantIt does not need to have a certain structure It will be calculated in the R model node afterwhich it will be saved within the R nugget where it can be used in the R-syntax

Note that R code is case-sensitive and therefore so are these object names In the following sectionswe will explain the usage of these objects

321 modelerData

modelerData is the R data frame that will be lled by the dataset entering the SPSS Modeler nodeit comes from So you can use this data frame to perform the desired calculations transformationsand outputs in R

Place the following code in an R output node

1 Print the first 6 lines of the data

2 head(modelerData)

3

4 Give a summary of the data

5 summary(modelerData)

6

7 create a histogram of the variable tenure

8 hist(modelerData$tenure xlab = years main = Tenure histogram)

9

10 change the tenure unit from months to years

11 modelerData$tenure lt- modelerData$tenure1212

13 recreate the histogram now in months

14 hist(modelerData$tenure xlab = months main = Tenure histogram)

Execution of this node will result in an SPSS Modeler output window in which all the R outputswill be assembled These will always be divided in 2 tabs Text output and Graph output

Page 6 of 20

IBM SPSS Modeler and R

In this case the text output is linked to the code on line 2 and 5 rst it prints the rst 6 lines(head) of the data next it will give summary statistics for each column

The graph output are two histograms One for the tenure in months the other for the samecolumn but after redening it by dividing the original value by twelve to give the tenure in years(note the X-axis scale)

As shown in the example stream Explain modelerDatastr you can also copy exactly this samecode into a transform node and attach a table node to it After running this table node you willnot see any R output (as none is expected) That means that even though the output code hasrun no outputs will be given However the data frame of modelerData will be send back to SPSSModeler In this case you will see the value of tenure being divided by 12

322 modelerDataModel

Metadata is very important in SPSS Modeler Let us for simplicity say that within modeler meta-data is represented by the type node With metadata we mean the type of each of the variablesin the dataset (numeric 13ag String storage ) At all times modeler will know exactly all themetadata at every step in the stream

R does not handle the metadata in a similar way as SPSS Modeler We already explainedmodelerDataModel taking over the role of the type node This is done by a data frame(=dataset) of the following structure

X1 X2 Xn

fieldName region tenure agefieldLabel Geographic indicator Months with service Age in years

fieldStorage real real realfieldMeasure nominal continuous continuousfieldFormat standard standard standardfieldRole input input input

So this means that this dataset will always have 6 lines with xed names (yes in R also thelines have names) The thing with this dataset is that it is completely the responsibility of theuser to align this metadata with the appropriate data So that means if we would like to add avariable with R the user must also manually add a column in modelerDataModel to make suremodelerData correctly goes back to SPSS Modeler In the earlier example above we did not makeany changes to the modelerDataModel and it was also not needed as the metadata did not change(dividing a number by 12 will not change the metadata) Now let us continue on the previousexample But now rather than changing the value of tenure in the same data variable we willcreate another one As a result we would have to update the metadata

1 Create the vector of tenure in years

2 Rcolumn lt- modelerData$tenure123

4 Paste this vector to the right of the dataset

5 modelerData lt- cbind(modelerDataRcolumn)

6

7 create the metadata for the column to add

8 newVar lt- c(fieldName=tenureYears fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

9

10 paste the new column metadata to the existing metadata

11 modelerDataModel lt- cbind(modelerDataModelnewVar)

Running a table node downstream of this transform node will show you the new variable withthe name tenureYears There are some important things to realize in this

Page 7 of 20

IBM SPSS Modeler and R

bull fieldName and fieldStorage are the only 2 required rows that needs to be lled in for anynew column In the code we left all the other lines empty meaning they will be lled in bythe stream default For a list of available values we refer to the user guide

bull As modelerDataModel is only useful when you go back to SPSS Modeler you will generallyonly usechange this object in non-terminal R-nodes It might still be handy to use it interminal nodes if the value of the modelerDataModel is important for your output (egrun a histogram of all continuous variables)

bull When data will go back to SPSS modeler it will be the content of the data frame excludingthe column- and row names That means that even though the column in the modelerData

will be called Rcolumn the name of the column in SPSS will only be dened by the metadatawithin the row fieldName In this case it is called tenureYears

bull The only link between modelerData and modelerDataModel is the order of the columns Itwill not look by name The rst column in the data will be given the metadata of the rstcolumn of modelerDataModel In case the metadata (modelerDataModel ) does not matchthe modelerData an error is thrown The table below shows schematic how this works

modelerData modelerDataModel

R

RName1 RNamenx11 x1n

x21 x2n

xm11 xm1n

xm1 xmn

X1 Xn

fieldName Name1 NamenfieldLabel

fieldStorage xxx xxxfieldMeasure fieldFormat

fieldRole

SPSS

Name1 Namenx11 x1n

x21 x2n

xm1 xmn

Note that only the names of the modelerDataModel are used

Since this concept is very strange to standard R users We found this part the most dicult toexplain To people who know SPSS you can summarize it as modelerDataModel taking over therole of the type node

You can nd all streams and R scripts explaining modelerDataModel here

323 modelerModel

modelerModel is the R object that is stored within the R nugget This object will be populatedwithin the R model node after which you could use modelerModel within the R nugget for scoringThis very much works the same way as IBM Modeler works You ask a model node to calculatea formula after which that formula will be stored within the nugget together with the way itshould be used to calculate a scoring

You will only use this object within the R model node and nugget Note that within the Rmodel node there are 2 syntax window

Page 8 of 20

Explain modelerDataModelxml

data0001dat

data0002dat

IBM SPSS Modeler and R

R model building is to calculate whatever you want to store within modelerModel that couldbe used within your nugget calculations to score your data This can be any object withinR As any SPSS Model builder node this will be a terminal code meaning that from thiscode no data will go back to SPSS modeler unless some outputs and things that are storedwithin the modelerModel object

R model scoring is the syntax to dene how you will use the object modelerModel containingthe content you stored to it in the R model building syntax to derive the new data Apartfrom the use of modelerModel this is very similar to the R transform node

Let us start with a simple example where we would like to create a basic linear model for thevariable tenure The formula of this model should be saved in the modelerModel after which itcan be used in the scoring

1 Create the model and save it in modelerModel

2 modelerModel lt- lm(tenure ~ age + region + ed + income data= modelerData)

3

4 Add some summary of the model in the nugget

5 summary(modelerModel)

6

7 together with a histogram

8 hist(modelerModel$residuals main = residual histogram)

9

10 and the residual vs actuals scatterplot

11 plot(modelerData$tenure modelerModel$fittedvalues xlab = actual ylab = predicted )

12

13 All of these output will be stored in the modeler nugget tabs

1 Use the model to make a prediction and add it to the existing data

2 pred lt- predict(modelerModel modelerData)

3 modelerData lt- cbind(modelerDatapred)

4

5 Take care of the metadata

6 newVar lt-c(fieldName=$L-tenure fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

It is important to note that modelerModel can be lled with any type of object but will veryoften be of a model class but does not have to be In the previous example the object stored toit was clearly a (statistical) model In the next example we will just save 2 numbers within themodelerModel object Imagine we want to calculate the z-values of a certain variable In orderto create the z-values we need the mean and the standard deviation of the column We will storeboth of these within modelerModel after which we will use them in the scoring syntax1 Thisexample shows you do not need to store a statistical model within your modelerModel objectbut it really can be any R object

1 calculate mean and standard deviation

2 M lt- mean(modelerData$tenure)3 SD lt- sd(modelerData$tenure)4

5 and save it in a list called modelerModel

6 modelerModel lt- list(avg = M sDev = SD)

1 calculate z scores using the elements in modelerModel

2 zTenure lt- (modelerData$tenure - modelerModel$avg)modelerModel$sDev3 modelerData lt- cbind(modelerDatazTenure)

4

5 define new metadata column and add it

6 newVar lt- c(fieldName=zTenure fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

1Note that there is a very good reason this is not combined into an R Transform node explained in 34

Page 9 of 20

IBM SPSS Modeler and R

You can nd all streams and R scripts explaining modelerModel here

33 Some general remarks

bull Although it might seems this way you are not required to build modelerData from theexisting data within that frame modelerData will be lled with the dataset you have inModeler however nothing stops you to throw that data away in R and dene some new datacoming from another data source in R As an example imagine this link from the WeatherCompany website This will give the weather history in Brussels Belgium in the monthNovember 2015 Now we can use R code as a source node by just overwriting modelerData

and redening modelerDataModel

1 Define the link

2 linkPath lt- httpwwwwundergroundcomhistoryairportEBBR20151101

MonthlyHistoryhtmlreq_city=Brusselsampformat=1

3 Read the data as csv

4 modelerData lt- readcsv(linkPath)

5 modelerData[1] lt- asDate(modelerData[1])

6

7 Redefining modelerDataModel all are real numbers except the first column is the date

8 modelerDataModel lt- asdataframe(t(dataframe(fieldName = colnames(modelerData)

fieldLabel = fieldStorage = c(daterep(realncol(modelerData)-1))

fieldMeasure = fieldFormat = fieldRole = )))

As you can see this code does not use the old denition of the dened R objects butcompletely redenes them Placing this in a R transform node will give back this newdataset to modeler So in this way you can use this approach to create an R input node

You can nd an example here

bull Within an R model node there is place for 2 scripts The building script will be the scriptthat will be populated within the R nugget It will not be run when you run the model nodeAs a result of this these 2 scripts are independent The only thing they share is the valueof the object modelerModel which is saved within the nugget when running the buildingsyntax and picked up within the R scoring syntaxThis also means that eventual R-libraries that are required should be loaded in both scriptsTake for example a model for a random forest

1 Load the library

2 library(randomForest)

3 Create the model and save it in modelerModel

4 modelerModel lt- randomForest(tenure ~ age + region + ed + income data= modelerDatantree

=50)

1 Load the library

2 library(randomForest)

3

4 Use the model to make a prediction and add it to the existing data

5 predlt- predict(modelerModel modelerData)

6 modelerData lt- cbind(modelerDatapred)

7

8 Take care of the metadata

9 newVar lt- c(fieldName=$RF-tenurefieldLabel=fieldStorage=realfieldMeasure=fieldFormat=fieldRole=)

10 modelerDataModel lt- cbind(modelerDataModelnewVar)

bull Talking about libraries and package A package is a collection of R objects dened for acertain purpose These often are specic statistical functionalities like randomForest in theexample above A basic R installation comes with the standard packages however there aremany more packages available made available by the R community on CRAN

Page 10 of 20

Explain modelerModelxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

R as sourcexml

data0001dat

data0002dat

IBM SPSS Modeler and R

Packages needs to be installed and made locally available in libraries Once the package isinstalled on the system as a library you can load this library in any R session by the codelibrary(ltnamegt)To install a package you have several options The easiest is probably to write a codelike installpackages(randomForest) within R You will have to select a CRAN mirrorwhere this library will be downloaded from and the download will go automatically Nor-mally you will only have to do this onceAlthough possible it is not recommended to run this package installation command fromwithin SPSS The reason is that these libraries will than be saved in a temporary folder andafterwards be deleted If you still want to this through SPSS you will have to hard codethe installation path

34 Read data options

Something we ignored until now are the settings within the node under Read data options Thebasics of the R integration with Modeler can be done without the knowledge of this as it requiressome more advanced R knowledge The user guide still has a good explanation about these items

However there is one more thing that might be important For modeler version 17 and lowerthe R integration of non-terminal nodes (ie transform and nuggets) will by default be done inbatches of 1000 The reason for this was to allow these R nodes to work on hadoop and otherclustered environments As a result of this it is very important to realize that any R code thatwould span over multiple lines of data would lead to false results For a workaround for this werefer to 511

Take as an example the z scores above If we would calculate the mean and the standard de-viation of the variable in a non-terminal node it would start with running this code for the rst1000 lines of data So that leads to a specic mean deviation and corresponding z-scores How-ever the next 1000 lines a new mean and deviation would be calculated and the z scores will bebased on these

As a solution the means and standard deviations are calculated in the R model (ie a termi-nal node) over all the data and used in the R nugget to calculate the z scores

Note that this approach may lead to a very slow integration between SPSS and R in the caseof streaming R nodes in a local non-clustered environment However as from IBM SPSS Modelerversion 171 there is a default option not to use this approach of batch processing or to increasethe batch size For the lower versions there is a workaround possible if you still want to increasethis batch size or turn it o (see later)

4 Custom Dialog builder

The Custom Dialog Builder allow you to create and manage R nodes with prefilled R code to use insideIBM SPSS Modeler streams In this way users can create their own nodes You can start the CustomDialog builder in the Tools menu under rdquoCustom Dialog Builderrdquo

When opening a custom dialog builder you will see a 2 windows One of them is the custom dialogitself the other is the toolset to populate the dialog

Page 11 of 20

IBM SPSS Modeler and R

41 Tools

The tools window is a list of items you can place within your dialog This include among others the fieldchooser Check and combo boxes Text and number controls and tabs You can select any and drag themonto the dialog itselfOnce you have any item in the dialog you can select it and you will see the item properties These are theproperties of this specific item and will change dependent on the type of item it is The most importantare the identifier (the way it will be referenced within the script) and the Title (the one that will be visualin the dialog)

42 Custom dialog

The big gray window is the dialog itself For the moment it is empty as it should be populated with itemsfrom the Tools listClicking on this gray dialog will show you the dialog properties below As main items this includes thename and title of the dialog the script itself and the type and position of the created nodeWith regards to the script to be written The global rule is that you reference to the items within thedialog using their identifier between double percentages (rdquoltidentifiergtrdquoOnce you finished creating the custom node you can install it by pressing the green arrow in the toolbarYou can also save intermediate versions to the disk

43 Simple example

Let us create a custom dialog for the randomForest model created earlier in section 33 Below you willfind a step by step approachThe most important thing we have to wonder is what within this code we want flexible for the user Inthe case of this model there might be 3 things that we want flexibel the input variables the target andthe number of trees in our forest

First fill in the Custom dialog properties as indicated

Page 12 of 20

IBM SPSS Modeler and R

In our fixed example the target is tenure but a user might chooseany other field As a result we will place a field chooser on the dialogChange the properties like shown The variable filter properties allowsyou to select only categorical variables

In our fixed example the inputs are age income but a user mightchoose any other field As a result we will place a field chooser on thedialog The biggest difference is that now we can select several variablesas there might be different inputs To make it easier we will separatethese values by a + Therefore change the properties like shown

As a third custom choice we would like to add the number of trees inour forest In the original script is was 50 so we will choose this as thedefault However users may choose any integer value between 1 and1000 Add a number control on the dialog and change the properties

So now the dialog is ready and we need to add the script to it Go tothe Edit options and choose rdquoScript Templaterdquo This will bring you toan empty window for the script In this case (as we selected we wanted2 scripts) there is a tab for the building code and one for the scoringscript If the coring script is greyed out you did set the dialog propertyrdquoScore from the Modelrdquo to True

Page 13 of 20

IBM SPSS Modeler and R

Let us start with the scoring script as this is easier The only thingwhich will need to be adapted for custom input is the variable namethat will be send back to SPSS So copy the scoring code and changethe name tenure to TARGET (this is the name of the identifier of thetarget)

Fill in the code for the building script and change in a similar wayas above the values for target and intput variables together with thenumber of trees Afterwards press OK to close the script window

Being back at your Custom Dialog builder save the dialog in any appro-priate location Also deploy the dialog by clicking on the green deployarrow in the toolbar Close the Dialog builder

Back to the stream you will now see the new node in the model paletteYou can use this node within your stream

You can find the resulted cfe file here (place this in the correct location see 521) and a stream

where it is deployed

5 Tips amp tricks Some more detailed

51 R code

511 ibmspsscf70 library

Let us now have a more detailled view about what actually happens with the code First of all it is worthto check what happens when you do the R installation correctly This will install the by IBM delivered Rpackage ibmspsscf70 in the library folder of your R installation This library contains several functionsto handle the data traffic between SPSS and R

Running any R node in SPSS will not only run the code you write but it will also run some extracode behind the scenes You can see this code in the rdquoConsole outputrdquo window of the R node Lookingfor example at this tab for an R nugget you will see that your code will be something like

1 modelerModel lt- ibmspsscfoutputGetModel()

2 while(ibmspsscfdataHasMoreData())

3 modelerDataModel lt- ibmspsscfdatamodelGetDataModel()

4 modelerData lt- ibmspsscfdataGetData(rowCount=1000 missing=NA rDate=None

logicalFields=FALSE)

5

Page 14 of 20

app-extensionxml

cdb_peerjar

comspsssharedcustom_guiui_builderpeersRStatsApplierPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsApplierPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsApplierPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                    
                

comspsssharedcustom_guiui_builderpeersCDExtensionPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class CDExtensionPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static String                         RBUILDER                            private                         static String                         SCORE_SYNTAX                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            public void CDExtensionPeer()                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private String                         replaceFieldSeparator(String String)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$1                         implements javaawteventActionListener     void ActionHandler$1(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$2class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$2                         implements javaawteventActionListener     void ActionHandler$2(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsUIPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsUIPeer                         implements compaswframeworkuiswingspiExtensionObjectSwingUIPeer compaswframeworkcommonextensionspiInteractorListener compaswcorecontainerContainerSetListener                             private ActionHandler                         actionHandler                            public                         static                         final String                         UPDATE_CONTAINER = updateContainer                            public void RStatsUIPeer()                            public void                         initSwingUI(compaswframeworkuiswingExtensionObjectSwingUI compaswframeworkcommoncoreProperties compaswframeworkuiswingExtensionObjectSwingUIContext)                            public void                         disposeSwingUI()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            public void                         handleAction(String String compaswframeworkcommoncoreProperties)                            public void                         containerChanged(compaswcorecontainerContainerSetEvent)                            public void                         updateContainer(compaswcorecontainerContainerSetEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsProcessPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsProcessPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsProcessPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersCopyrightclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         abstract                         interface Copyright                         extends annotationAnnotation                             public                         abstract String                         value()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxControl                         extends javaxswingJTextArea                         implements compaswframeworkuiswingspiPropertyControl                             private                         static                         final String                         editScoreSyntax = editScoreSyntax                            private                         static                         final String                         editScoreSyntaxTitle = editScoreSyntaxTitle                            private                         static                         final String                         editScoreSyntaxWarning = editScoreSyntaxWarning                            private                         static                         final String                         editScoreSyntaxContinueBtn = editScoreSyntax_Continue                            private                         static                         final String                         editScoreSyntaxCancelBtn = editScoreSyntax_Cancel                            private javaxswingJPanel                         containerPanel                            private javaxswingJButton                         editScoreSyntaxBtn                            private compaswframeworkuiswingPropertyControlContext                         propertyControlContext                            public void ManagedScoreSyntaxControl()                            public void                         disposeControl()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public javaxswingJComponent                         getUIComponent()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            public boolean                         isControlEnabled()                            public void                         setControlEnabled(boolean)                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxPanel                         implements compaswframeworkuiswingspiManagedPanel                             public void ManagedScoreSyntaxPanel()                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ROutputFileChooserControl                         implements compaswframeworkuiswingspiPropertyControl compaswuicommoncontrolControlListener                             public                         static                         final String                         OUTPUT_TYPE = output_type                            public                         static                         final String                         OUTPUT_TYPE_GRAPH = Graph                            public                         static                         final String                         GRAPH_OUTPUT_FILE_TYPE = graph_output_file_type                            public                         static                         final String                         TEXT_OUTPUT_FILE_TYPE = text_output_file_type                            public                         static                         final String                         OUTPUT_FILE_TYPE_HTML = HTML                            public                         static                         final String                         OUTPUT_FILE_TYPE_TEXT = Text                            public                         static                         final String                         OUTPUT_FILE_TYPE_COU = Cou                            public                         static                         final String                         FILENAME = OutputFilename                            public                         static                         final String                         OUTPUT_FILE_CHOOSER_BUTTON = OutputFileChooserButton                            public                         static                         final char                         BACK_SLASH_CHAR = 92                            public                         static                         final char                         FORWARD_SLASH_CHAR = 47                            private ActionHandler                         actionHandler                            private javaxswingJLabel                         fileNameLabel                            private javaxswingtextJTextComponent                         textComponent                            private javaxswingAbstractButton                         browseButton                            private compaswuiswingcontrolDefaultControlPanel                         mainPanel                            private String                         propertyName                            private compaswframeworkuiswingPropertyControlContext                         context                            public void ROutputFileChooserControl()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            private void                         initComponent()                            private compaswframeworkuiswingSwingResourceProvider                         getSwingResourceProvider()                            private String                         getLabelFromSwingResourceProvider(String)                            private Object                         getSwingUiProperty(String)                            protected String                         getTextComponentValue()                            public javaxswingJComponent                         getUIComponent()                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public void                         setControlEnabled(boolean)                            public boolean                         isControlEnabled()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public void                         disposeControl()                            public void                         controlValueChanged(compaswuicommoncontrolControlEvent)                            private String                         removeExtension(String)                            public void                         controlSelectionChanged(compaswuicommoncontrolControlEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandlerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ActionHandler                             private compaswframeworkuiswingExtensionObjectSwingUI                         extensionObjectSwingUI                            private boolean                         bPrintAll                            private int                         printScope    String                         actionErrorMessage                            public void ActionHandler(compaswframeworkuiswingExtensionObjectSwingUI)                            public void                         handleAction(String String)                            private void                         handleExportHTML(String)                            private void                         handleExportText(String)                            private void                         handlePrint(String String)                            private void                         handlePrintView()                            private void                         handlePageSetup(String)                            private void                         handleSaveNode(String)                            private void                         handlePreviewTable(String)                            private void                         handleSaveToProject(String)                            private void                         handleHelpButton()                            private void                         exportHTMLFile(String)                            private void                         saveTextFile(String String)                            private javaxswingJPanel                         createPrintViewPanel(javautilList)                            public String                         launchSaveFileChooser(String String[])                            private void                         configureSaveFileChooser(javaxswingJFileChooser)                            private void                         configureSaveChooserCallback(compaswcoreapplicationcallbackFileSystemChooserRequest)                            private String                         getSaveDialogTitle()                            private String                         getApproveButtonText()                            private boolean                         useApplicationFileChoosers()                            private compaswframeworkuicommonUISession                         getUISession()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            private String                         getTextContainerData(String)                            private boolean                         isExportText(String)                            private boolean                         isExportHTML()                            private boolean                         isSelectNode()                            private compaswuiswingcontrolModelViewerPanel                         getModelViewerPanel()                            private String                         getDefaultFileName()                            private void                         handleUpdateContainer(String)                    
                

comspsssharedcustom_guiui_builderpeersManagedBasePanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedBasePanel                         extends javaxswingJPanel                         implements javaawtprintPrintable     void ManagedBasePanel(javaawtLayoutManager)                            public int                         print(javaawtGraphics javaawtprintPageFormat int)                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$1                     
                

comspsssharedcustom_guiui_builderpeersUIUtilitiesclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class UIUtilities                             public void UIUtilities()                            public                         static javaxswingJButton                         createButton(compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComboBox                         createComBox(compaswframeworkuiswingSwingResourceProvider String int javaxswingDefaultComboBoxModel)                            public                         static javaxswingDefaultComboBoxModel                         createEnumComboBoxModel(compaswframeworkcommoncoreEnumerationValueDescriptor)                            public                         static javaxswingAbstractButton                         addAccessKey(javaxswingAbstractButton compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingtextJTextComponent                         createTextFiled(compaswframeworkuiswingSwingResourceProvider String int)                            public                         static javaxswingJComponent                         addTooltip(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComponent                         addDescription(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static                         transient void                         fitAllComponents(javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxPreferSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setPreferSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMaxSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMaxSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMinSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMinSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                    
                

comspsssharedcustom_guiui_builderpeersManagedHTMLOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedHTMLOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedHTMLOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedModelOutputPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedModelOutputPanel                         implements compaswframeworkuiswingspiManagedPanel compaswframeworkcommonextensionspiInteractorListener                             public                         static                         final String                         BUILD_TEXT_OUTPUT = buildTextOutputDisplay                            public                         static                         final String                         BUILD_CONSOLE_OUTPUT = buildConsoleOutputDisplay                            public                         static                         final String                         BUILD_HTML_OUTPUT = buildHTMLOutputDisplay                            public                         static                         final String                         BUILD_MV_OUTPUT = buildModelViewerDisplay                            public                         static                         final String                         APPLIER_CONSOLE_OUTPUT = applierConsoleOutputDisplay                            public                         static                         final String                         BUILD_TEXT_CONTAINER_NAME = TextOutput                            public                         static                         final String                         BUILD_CONSOLE_CONTAINER_NAME = ConsoleOutput                            public                         static                         final String                         BUILD_HTML_CONTAINER_NAME = HTMLOutput                            public                         static                         final String                         BUILD_PMML_CONTAINER_NAME = PMML                            public                         static                         final String                         BUILD_STAT_XML_CONTAINER_NAME = StatXML                            public                         static                         final String                         APPLIER_CONSOLE_CONTAINER_NAME = ApplierConsoleOutput                            private ManagedTextOutputArea                         textOutput                            private ManagedTextOutputArea                         consoleOutput                            private ManagedHTMLOutputArea                         htmlOutput                            private ManagedEmptyOutputArea                         emptyOutput                            private javautilHashMap                         componentMap                            private compaswframeworkuiswingManagedPanelContext                         managedPanelContext                            public void ManagedModelOutputPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private compaswframeworkuiswingspiManagedUIElement                         setupTextOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupConsoleOutputControl(String)                            public void                         updateConsoleOutputControl(String)                            private compaswframeworkuiswingspiManagedUIElement                         setupHTMLOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupModelViewerControl()                            public String                         getTextContainerData(String)                            public                         static boolean                         exportToHTML(compaswframeworkcommoncontainerContainer String)                            public                         static javaioFile                         getHTMLFileTest(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractHTMLCompoundContainer(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractZip(String String boolean)                         throws javaioIOException                            public                         static javaioFile                         getHTMLFile(String String boolean)                         throws javaioIOException                    
                

comspsssharedcustom_guiui_builderpeersManagedTextOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedTextOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedTextOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$ButtonsActionclass

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$ButtonsAction                         implements javaawteventActionListener                             private void ManagedScoreSyntaxControl$ButtonsAction(ManagedScoreSyntaxControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersManagedEmptyOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedEmptyOutputArea                         extends javaxswingJPanel                         implements compaswframeworkuiswingspiManagedUIElement                             public void ManagedEmptyOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public javaxswingJComponent                         getComponent()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ROutputFileChooserControl$1                         implements javaawteventActionListener     void ROutputFileChooserControl$1(ROutputFileChooserControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

extensionxml

imagescopygif

imagesdeletegif

imagesfileMenugif

imagesgenerateMenugif

imageshelp_buttongif

imageslg_gm_rpng

imageslg_r_cdb_gmgif

imageslg_r_cdb_modgif

imageslg_r_cdb_outputgif

imageslg_r_cdb_processgif

imagespreview_tablegif

randomForestxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

IBM SPSS Modeler and R

6 Your R code7

8 ibmspsscfdatamodelSetDataModel(modelerDataModel)

9 ibmspsscfdataSetData(modelerData)

10

All the functions starting with ibmspsscf are functions within this library It is this part of the codethat is responsible for the transfer of the data and the metadata to R Also you will see a while loopindicating that the data will go to R in batches of 1000 The other options are the values of the data readoptions within the nodeThe last lines of the code prepares the data to be send back to SPSS Also note the closing brace whichis the end of the while loop

Now since this library will be loaded with every interaction between SPSS and R you are free to use thesefunctions within your code as well So if you would like to avoid these runs in batches of 1K and you donot have version 171 available you can start your R code with another loop to first continue filling themodelerData and only than start your actual code In practice that means that in your R node you startwith

1 while(ibmspsscfdataHasMoreData())

2

3 modelerData lt-rbind(modelerDataibmspsscfdataGetData(rowCount=100000missing=NArDate=None

logicalFields=FALSE))

4

5

6 Some more code using modelerData which is now the complete dataset

512 Some useful parts of R code

Make sure a package is installed Whenever you have created any R node using packages you have tomake sure that anyone using this node has this package on his machine installed if needed withouttoo much user interference Therefore you can use this part of code

1 packages lt- function(x)

2 x lt- ascharacter(matchcall()[[2]])

3 if (require(xcharacteronly=TRUE))

4 installpackages(pkgs=xrepos=httpcranr-projectorg)

5 require(xcharacteronly=TRUE)

6

7

8

9 packages(rpart)

It will firstly verify if the library is installed on the system if not it will silently install it fromthe given CRAN mirror (you can change this to another mirror or a local repository if needed)Installation will only happen the first time the node will be used

Create the metadata corresponding to the R data frame Sometimes your data is very much trans-formed compared to the original data that it is difficult to build your metadata starting from theoriginal one So you might want to change the metadata to link to the data in R independent ofthe original This is in particularly useful if you want to use R as a sort node Below is a functionthat asks for a data frame and creates modelerData and modelerDataModel accordingly

1 sendToModeler lt- function (dataFrame)

2 if(isnull(dim(dataFrame)))

3 stop(Invalid data received not a dataframe)

4 if (dim(dataFrame)[1]lt=0)

5 print(Warning modelerData has no line all fieldStorage fields set to strings)

6 getStorage lt- function(x)return(string)

7 else

8 getStorage lt- function(x)

9 x lt- unlist(x)

10 res lt- NULL

11 if x is a factor typeof will return rsquointegerrsquo so we handle this case first

Page 15 of 20

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 4: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

This is the R console ready for commands to run You might often hear the term RStudio whichis nothing more than a development environment on top of this R gui Installation of RStudio isnot required for this introduction but might be handy for further useWe will start the R introduction by stating R is a powerful programming language and environmentfor statistical computing and graphics An important part within that last phrase is that R is aprogramming language unlike IBM SPSS Modeler That means it is built on objects that aredened by the user As an example assume the following R code (feel free to type it within theR console to see the R outputs)

1 x lt- 1+1

2 y lt- 2x

3 xyVector lt- c(xy)

4 z lt- mean(xyVector)

5 print(z)

Here x is an object This statement will ll the object x with the value of the evaluated formula1 + 1 being 2 So whenever the program refers to x it will be interpreted as 2 In the secondline we will dene y as twice the value of x In the third line we create a vector containing thecontent of x and y to calculate the mean of these 2 objects and place it in an object z

The operator lt- could also be replaced by = but for various reasons lots of R users pre-fer this way of writing (actually it is not exactly the same but that could be ignored for thepurpose of this document) If you feel more comfortable in using = please do so

Like we lled x y and z with some numbers any R object can be lled with a variety of typesHere is a list of the most important for our purposes

Vector is a sequence of data elements of the same type (eg numeric or character) This includesvectors of length 1 which can be interpreted as just being numbers You can create a vectorwith the R function c() So in the example code above all the values of x y and z arevectors of length one xyVector is a vector of length 2 containing the values of (the vector)x followed by (the vector) y Trying to link it back to SPSS you can interpret a vector asthe values of a single data column

Data frame is a list of vectors of equal length If you look at a vector as the values of a variablea data frame could be interpreted as a 2-dimensional dataset with columns (the number ofvectors) and lines (the size of each vector)

1 n lt- c(2 3 5 3 9) A first vector of 5 numeric values

2 n2 lt- c(1 3 2 5 4) A second vector of 5 numeric values

3 s lt- c(aa bb cc aa zz) A third vector of 5 string values

4 b lt- c(TRUE FALSE TRUE TRUE TRUE) A fourth vector of 5 flag values

5 Data lt- dataframe(n s b New = n+n2) A data frame containing 4 vectors

Page 4 of 20

IBM SPSS Modeler and R

6 Note n+n2 will be a new vector called New with the sum of the n + n2 c(3 6 7 8 13)

7

8 dim(Data) Will show you it is a 5x4 dataset

9 Data[24] Will give back the value on the 2nd line the 3rd column

10 colnames(Data) Will give the column names as a vector (nsb New)

11 Data$n[1] Will give back the first value of the vector n within the data frame

12

13 iris predefined data frame

There are also several pre-dened data frames installed within R One of them is called irisSometimes this document will refer back to iris

Model class which is actually a specic list containing predened objects dening a statisticalmodel For example a linear model class will be a list containing among others the coecientsof the regression model

List is an ordered collection of objects As an example you can have a list where the rst elementis a vector the second is a data frame and the third is a model Note that a data frame isa special type of a list where all the elements are vectors of equal sizes

3 The basics of R nodes in IBM SPSS Modeler

31 The R nodes

Once the installation for the R essentials are done you will see 3 new nodes in your node palettesThere is also a 4th R node which is the R nugget The dierence between and understanding ofthese 4 objects are essential

Output with this node data will be sent to R but it will never go back to SPSS (as it is aterminal node) The only thing that can go back to SPSS is the outputs generated by Rthatwill be presented within an SPSS output window

Transform data will go from SPSS to R but will also go back to SPSS after which the SPSSprocess can be continued

Model like the output node this is a terminal node so data will not go back to SPSS Howeverthere will be a reusable R object created within a nugget

Nugget similar to Transform node with the dierence that there is a reusable R object that canbe used in the R code

Node

Name R output node R transform node R model node R syntax nodePalette tab Output Record Ops Model NAData back to SPSS No Yes No YesReusable R object No No Create Use

32 Simple R code example

Let us start with saying that all the examples in this section are intentionally kept very simple soas to explain the interaction in a functional and structured way and be simple enough for non Rprogrammers We are certainly aware that most of the R code snippets we show in this chapter

Page 5 of 20

IBM SPSS Modeler and R

could also easily be implemented using standard SPSS Modeler functionality

There are 3 very important and reserved R objects that you should keep in mind when youuse the SPSS Modeler R integration Here is a brief description of these 3 after which we will gointo more detail for each of them

modelerData This is an R data frame that will be lled by the data entering in this R node Thisdata frame can be used and changed within your R code Eventually it will also be the dataframe that will be sent back to SPSS Modeler as a dataset Note that it will only containthe content of the data not (necessarily) the data column names and other metadata items

modelerDataModel This is also an R data frame containing the metadata of the data that is sentto R and back to SPSS Modeler It contains most of the information that you may expectwithin an SPSS Modeler type node This will be the object that will be most strange forexperienced R users

modelerModel this is an R object that can be lled by the user by any type of object you wantIt does not need to have a certain structure It will be calculated in the R model node afterwhich it will be saved within the R nugget where it can be used in the R-syntax

Note that R code is case-sensitive and therefore so are these object names In the following sectionswe will explain the usage of these objects

321 modelerData

modelerData is the R data frame that will be lled by the dataset entering the SPSS Modeler nodeit comes from So you can use this data frame to perform the desired calculations transformationsand outputs in R

Place the following code in an R output node

1 Print the first 6 lines of the data

2 head(modelerData)

3

4 Give a summary of the data

5 summary(modelerData)

6

7 create a histogram of the variable tenure

8 hist(modelerData$tenure xlab = years main = Tenure histogram)

9

10 change the tenure unit from months to years

11 modelerData$tenure lt- modelerData$tenure1212

13 recreate the histogram now in months

14 hist(modelerData$tenure xlab = months main = Tenure histogram)

Execution of this node will result in an SPSS Modeler output window in which all the R outputswill be assembled These will always be divided in 2 tabs Text output and Graph output

Page 6 of 20

IBM SPSS Modeler and R

In this case the text output is linked to the code on line 2 and 5 rst it prints the rst 6 lines(head) of the data next it will give summary statistics for each column

The graph output are two histograms One for the tenure in months the other for the samecolumn but after redening it by dividing the original value by twelve to give the tenure in years(note the X-axis scale)

As shown in the example stream Explain modelerDatastr you can also copy exactly this samecode into a transform node and attach a table node to it After running this table node you willnot see any R output (as none is expected) That means that even though the output code hasrun no outputs will be given However the data frame of modelerData will be send back to SPSSModeler In this case you will see the value of tenure being divided by 12

322 modelerDataModel

Metadata is very important in SPSS Modeler Let us for simplicity say that within modeler meta-data is represented by the type node With metadata we mean the type of each of the variablesin the dataset (numeric 13ag String storage ) At all times modeler will know exactly all themetadata at every step in the stream

R does not handle the metadata in a similar way as SPSS Modeler We already explainedmodelerDataModel taking over the role of the type node This is done by a data frame(=dataset) of the following structure

X1 X2 Xn

fieldName region tenure agefieldLabel Geographic indicator Months with service Age in years

fieldStorage real real realfieldMeasure nominal continuous continuousfieldFormat standard standard standardfieldRole input input input

So this means that this dataset will always have 6 lines with xed names (yes in R also thelines have names) The thing with this dataset is that it is completely the responsibility of theuser to align this metadata with the appropriate data So that means if we would like to add avariable with R the user must also manually add a column in modelerDataModel to make suremodelerData correctly goes back to SPSS Modeler In the earlier example above we did not makeany changes to the modelerDataModel and it was also not needed as the metadata did not change(dividing a number by 12 will not change the metadata) Now let us continue on the previousexample But now rather than changing the value of tenure in the same data variable we willcreate another one As a result we would have to update the metadata

1 Create the vector of tenure in years

2 Rcolumn lt- modelerData$tenure123

4 Paste this vector to the right of the dataset

5 modelerData lt- cbind(modelerDataRcolumn)

6

7 create the metadata for the column to add

8 newVar lt- c(fieldName=tenureYears fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

9

10 paste the new column metadata to the existing metadata

11 modelerDataModel lt- cbind(modelerDataModelnewVar)

Running a table node downstream of this transform node will show you the new variable withthe name tenureYears There are some important things to realize in this

Page 7 of 20

IBM SPSS Modeler and R

bull fieldName and fieldStorage are the only 2 required rows that needs to be lled in for anynew column In the code we left all the other lines empty meaning they will be lled in bythe stream default For a list of available values we refer to the user guide

bull As modelerDataModel is only useful when you go back to SPSS Modeler you will generallyonly usechange this object in non-terminal R-nodes It might still be handy to use it interminal nodes if the value of the modelerDataModel is important for your output (egrun a histogram of all continuous variables)

bull When data will go back to SPSS modeler it will be the content of the data frame excludingthe column- and row names That means that even though the column in the modelerData

will be called Rcolumn the name of the column in SPSS will only be dened by the metadatawithin the row fieldName In this case it is called tenureYears

bull The only link between modelerData and modelerDataModel is the order of the columns Itwill not look by name The rst column in the data will be given the metadata of the rstcolumn of modelerDataModel In case the metadata (modelerDataModel ) does not matchthe modelerData an error is thrown The table below shows schematic how this works

modelerData modelerDataModel

R

RName1 RNamenx11 x1n

x21 x2n

xm11 xm1n

xm1 xmn

X1 Xn

fieldName Name1 NamenfieldLabel

fieldStorage xxx xxxfieldMeasure fieldFormat

fieldRole

SPSS

Name1 Namenx11 x1n

x21 x2n

xm1 xmn

Note that only the names of the modelerDataModel are used

Since this concept is very strange to standard R users We found this part the most dicult toexplain To people who know SPSS you can summarize it as modelerDataModel taking over therole of the type node

You can nd all streams and R scripts explaining modelerDataModel here

323 modelerModel

modelerModel is the R object that is stored within the R nugget This object will be populatedwithin the R model node after which you could use modelerModel within the R nugget for scoringThis very much works the same way as IBM Modeler works You ask a model node to calculatea formula after which that formula will be stored within the nugget together with the way itshould be used to calculate a scoring

You will only use this object within the R model node and nugget Note that within the Rmodel node there are 2 syntax window

Page 8 of 20

Explain modelerDataModelxml

data0001dat

data0002dat

IBM SPSS Modeler and R

R model building is to calculate whatever you want to store within modelerModel that couldbe used within your nugget calculations to score your data This can be any object withinR As any SPSS Model builder node this will be a terminal code meaning that from thiscode no data will go back to SPSS modeler unless some outputs and things that are storedwithin the modelerModel object

R model scoring is the syntax to dene how you will use the object modelerModel containingthe content you stored to it in the R model building syntax to derive the new data Apartfrom the use of modelerModel this is very similar to the R transform node

Let us start with a simple example where we would like to create a basic linear model for thevariable tenure The formula of this model should be saved in the modelerModel after which itcan be used in the scoring

1 Create the model and save it in modelerModel

2 modelerModel lt- lm(tenure ~ age + region + ed + income data= modelerData)

3

4 Add some summary of the model in the nugget

5 summary(modelerModel)

6

7 together with a histogram

8 hist(modelerModel$residuals main = residual histogram)

9

10 and the residual vs actuals scatterplot

11 plot(modelerData$tenure modelerModel$fittedvalues xlab = actual ylab = predicted )

12

13 All of these output will be stored in the modeler nugget tabs

1 Use the model to make a prediction and add it to the existing data

2 pred lt- predict(modelerModel modelerData)

3 modelerData lt- cbind(modelerDatapred)

4

5 Take care of the metadata

6 newVar lt-c(fieldName=$L-tenure fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

It is important to note that modelerModel can be lled with any type of object but will veryoften be of a model class but does not have to be In the previous example the object stored toit was clearly a (statistical) model In the next example we will just save 2 numbers within themodelerModel object Imagine we want to calculate the z-values of a certain variable In orderto create the z-values we need the mean and the standard deviation of the column We will storeboth of these within modelerModel after which we will use them in the scoring syntax1 Thisexample shows you do not need to store a statistical model within your modelerModel objectbut it really can be any R object

1 calculate mean and standard deviation

2 M lt- mean(modelerData$tenure)3 SD lt- sd(modelerData$tenure)4

5 and save it in a list called modelerModel

6 modelerModel lt- list(avg = M sDev = SD)

1 calculate z scores using the elements in modelerModel

2 zTenure lt- (modelerData$tenure - modelerModel$avg)modelerModel$sDev3 modelerData lt- cbind(modelerDatazTenure)

4

5 define new metadata column and add it

6 newVar lt- c(fieldName=zTenure fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

1Note that there is a very good reason this is not combined into an R Transform node explained in 34

Page 9 of 20

IBM SPSS Modeler and R

You can nd all streams and R scripts explaining modelerModel here

33 Some general remarks

bull Although it might seems this way you are not required to build modelerData from theexisting data within that frame modelerData will be lled with the dataset you have inModeler however nothing stops you to throw that data away in R and dene some new datacoming from another data source in R As an example imagine this link from the WeatherCompany website This will give the weather history in Brussels Belgium in the monthNovember 2015 Now we can use R code as a source node by just overwriting modelerData

and redening modelerDataModel

1 Define the link

2 linkPath lt- httpwwwwundergroundcomhistoryairportEBBR20151101

MonthlyHistoryhtmlreq_city=Brusselsampformat=1

3 Read the data as csv

4 modelerData lt- readcsv(linkPath)

5 modelerData[1] lt- asDate(modelerData[1])

6

7 Redefining modelerDataModel all are real numbers except the first column is the date

8 modelerDataModel lt- asdataframe(t(dataframe(fieldName = colnames(modelerData)

fieldLabel = fieldStorage = c(daterep(realncol(modelerData)-1))

fieldMeasure = fieldFormat = fieldRole = )))

As you can see this code does not use the old denition of the dened R objects butcompletely redenes them Placing this in a R transform node will give back this newdataset to modeler So in this way you can use this approach to create an R input node

You can nd an example here

bull Within an R model node there is place for 2 scripts The building script will be the scriptthat will be populated within the R nugget It will not be run when you run the model nodeAs a result of this these 2 scripts are independent The only thing they share is the valueof the object modelerModel which is saved within the nugget when running the buildingsyntax and picked up within the R scoring syntaxThis also means that eventual R-libraries that are required should be loaded in both scriptsTake for example a model for a random forest

1 Load the library

2 library(randomForest)

3 Create the model and save it in modelerModel

4 modelerModel lt- randomForest(tenure ~ age + region + ed + income data= modelerDatantree

=50)

1 Load the library

2 library(randomForest)

3

4 Use the model to make a prediction and add it to the existing data

5 predlt- predict(modelerModel modelerData)

6 modelerData lt- cbind(modelerDatapred)

7

8 Take care of the metadata

9 newVar lt- c(fieldName=$RF-tenurefieldLabel=fieldStorage=realfieldMeasure=fieldFormat=fieldRole=)

10 modelerDataModel lt- cbind(modelerDataModelnewVar)

bull Talking about libraries and package A package is a collection of R objects dened for acertain purpose These often are specic statistical functionalities like randomForest in theexample above A basic R installation comes with the standard packages however there aremany more packages available made available by the R community on CRAN

Page 10 of 20

Explain modelerModelxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

R as sourcexml

data0001dat

data0002dat

IBM SPSS Modeler and R

Packages needs to be installed and made locally available in libraries Once the package isinstalled on the system as a library you can load this library in any R session by the codelibrary(ltnamegt)To install a package you have several options The easiest is probably to write a codelike installpackages(randomForest) within R You will have to select a CRAN mirrorwhere this library will be downloaded from and the download will go automatically Nor-mally you will only have to do this onceAlthough possible it is not recommended to run this package installation command fromwithin SPSS The reason is that these libraries will than be saved in a temporary folder andafterwards be deleted If you still want to this through SPSS you will have to hard codethe installation path

34 Read data options

Something we ignored until now are the settings within the node under Read data options Thebasics of the R integration with Modeler can be done without the knowledge of this as it requiressome more advanced R knowledge The user guide still has a good explanation about these items

However there is one more thing that might be important For modeler version 17 and lowerthe R integration of non-terminal nodes (ie transform and nuggets) will by default be done inbatches of 1000 The reason for this was to allow these R nodes to work on hadoop and otherclustered environments As a result of this it is very important to realize that any R code thatwould span over multiple lines of data would lead to false results For a workaround for this werefer to 511

Take as an example the z scores above If we would calculate the mean and the standard de-viation of the variable in a non-terminal node it would start with running this code for the rst1000 lines of data So that leads to a specic mean deviation and corresponding z-scores How-ever the next 1000 lines a new mean and deviation would be calculated and the z scores will bebased on these

As a solution the means and standard deviations are calculated in the R model (ie a termi-nal node) over all the data and used in the R nugget to calculate the z scores

Note that this approach may lead to a very slow integration between SPSS and R in the caseof streaming R nodes in a local non-clustered environment However as from IBM SPSS Modelerversion 171 there is a default option not to use this approach of batch processing or to increasethe batch size For the lower versions there is a workaround possible if you still want to increasethis batch size or turn it o (see later)

4 Custom Dialog builder

The Custom Dialog Builder allow you to create and manage R nodes with prefilled R code to use insideIBM SPSS Modeler streams In this way users can create their own nodes You can start the CustomDialog builder in the Tools menu under rdquoCustom Dialog Builderrdquo

When opening a custom dialog builder you will see a 2 windows One of them is the custom dialogitself the other is the toolset to populate the dialog

Page 11 of 20

IBM SPSS Modeler and R

41 Tools

The tools window is a list of items you can place within your dialog This include among others the fieldchooser Check and combo boxes Text and number controls and tabs You can select any and drag themonto the dialog itselfOnce you have any item in the dialog you can select it and you will see the item properties These are theproperties of this specific item and will change dependent on the type of item it is The most importantare the identifier (the way it will be referenced within the script) and the Title (the one that will be visualin the dialog)

42 Custom dialog

The big gray window is the dialog itself For the moment it is empty as it should be populated with itemsfrom the Tools listClicking on this gray dialog will show you the dialog properties below As main items this includes thename and title of the dialog the script itself and the type and position of the created nodeWith regards to the script to be written The global rule is that you reference to the items within thedialog using their identifier between double percentages (rdquoltidentifiergtrdquoOnce you finished creating the custom node you can install it by pressing the green arrow in the toolbarYou can also save intermediate versions to the disk

43 Simple example

Let us create a custom dialog for the randomForest model created earlier in section 33 Below you willfind a step by step approachThe most important thing we have to wonder is what within this code we want flexible for the user Inthe case of this model there might be 3 things that we want flexibel the input variables the target andthe number of trees in our forest

First fill in the Custom dialog properties as indicated

Page 12 of 20

IBM SPSS Modeler and R

In our fixed example the target is tenure but a user might chooseany other field As a result we will place a field chooser on the dialogChange the properties like shown The variable filter properties allowsyou to select only categorical variables

In our fixed example the inputs are age income but a user mightchoose any other field As a result we will place a field chooser on thedialog The biggest difference is that now we can select several variablesas there might be different inputs To make it easier we will separatethese values by a + Therefore change the properties like shown

As a third custom choice we would like to add the number of trees inour forest In the original script is was 50 so we will choose this as thedefault However users may choose any integer value between 1 and1000 Add a number control on the dialog and change the properties

So now the dialog is ready and we need to add the script to it Go tothe Edit options and choose rdquoScript Templaterdquo This will bring you toan empty window for the script In this case (as we selected we wanted2 scripts) there is a tab for the building code and one for the scoringscript If the coring script is greyed out you did set the dialog propertyrdquoScore from the Modelrdquo to True

Page 13 of 20

IBM SPSS Modeler and R

Let us start with the scoring script as this is easier The only thingwhich will need to be adapted for custom input is the variable namethat will be send back to SPSS So copy the scoring code and changethe name tenure to TARGET (this is the name of the identifier of thetarget)

Fill in the code for the building script and change in a similar wayas above the values for target and intput variables together with thenumber of trees Afterwards press OK to close the script window

Being back at your Custom Dialog builder save the dialog in any appro-priate location Also deploy the dialog by clicking on the green deployarrow in the toolbar Close the Dialog builder

Back to the stream you will now see the new node in the model paletteYou can use this node within your stream

You can find the resulted cfe file here (place this in the correct location see 521) and a stream

where it is deployed

5 Tips amp tricks Some more detailed

51 R code

511 ibmspsscf70 library

Let us now have a more detailled view about what actually happens with the code First of all it is worthto check what happens when you do the R installation correctly This will install the by IBM delivered Rpackage ibmspsscf70 in the library folder of your R installation This library contains several functionsto handle the data traffic between SPSS and R

Running any R node in SPSS will not only run the code you write but it will also run some extracode behind the scenes You can see this code in the rdquoConsole outputrdquo window of the R node Lookingfor example at this tab for an R nugget you will see that your code will be something like

1 modelerModel lt- ibmspsscfoutputGetModel()

2 while(ibmspsscfdataHasMoreData())

3 modelerDataModel lt- ibmspsscfdatamodelGetDataModel()

4 modelerData lt- ibmspsscfdataGetData(rowCount=1000 missing=NA rDate=None

logicalFields=FALSE)

5

Page 14 of 20

app-extensionxml

cdb_peerjar

comspsssharedcustom_guiui_builderpeersRStatsApplierPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsApplierPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsApplierPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                    
                

comspsssharedcustom_guiui_builderpeersCDExtensionPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class CDExtensionPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static String                         RBUILDER                            private                         static String                         SCORE_SYNTAX                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            public void CDExtensionPeer()                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private String                         replaceFieldSeparator(String String)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$1                         implements javaawteventActionListener     void ActionHandler$1(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$2class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$2                         implements javaawteventActionListener     void ActionHandler$2(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsUIPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsUIPeer                         implements compaswframeworkuiswingspiExtensionObjectSwingUIPeer compaswframeworkcommonextensionspiInteractorListener compaswcorecontainerContainerSetListener                             private ActionHandler                         actionHandler                            public                         static                         final String                         UPDATE_CONTAINER = updateContainer                            public void RStatsUIPeer()                            public void                         initSwingUI(compaswframeworkuiswingExtensionObjectSwingUI compaswframeworkcommoncoreProperties compaswframeworkuiswingExtensionObjectSwingUIContext)                            public void                         disposeSwingUI()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            public void                         handleAction(String String compaswframeworkcommoncoreProperties)                            public void                         containerChanged(compaswcorecontainerContainerSetEvent)                            public void                         updateContainer(compaswcorecontainerContainerSetEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsProcessPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsProcessPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsProcessPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersCopyrightclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         abstract                         interface Copyright                         extends annotationAnnotation                             public                         abstract String                         value()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxControl                         extends javaxswingJTextArea                         implements compaswframeworkuiswingspiPropertyControl                             private                         static                         final String                         editScoreSyntax = editScoreSyntax                            private                         static                         final String                         editScoreSyntaxTitle = editScoreSyntaxTitle                            private                         static                         final String                         editScoreSyntaxWarning = editScoreSyntaxWarning                            private                         static                         final String                         editScoreSyntaxContinueBtn = editScoreSyntax_Continue                            private                         static                         final String                         editScoreSyntaxCancelBtn = editScoreSyntax_Cancel                            private javaxswingJPanel                         containerPanel                            private javaxswingJButton                         editScoreSyntaxBtn                            private compaswframeworkuiswingPropertyControlContext                         propertyControlContext                            public void ManagedScoreSyntaxControl()                            public void                         disposeControl()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public javaxswingJComponent                         getUIComponent()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            public boolean                         isControlEnabled()                            public void                         setControlEnabled(boolean)                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxPanel                         implements compaswframeworkuiswingspiManagedPanel                             public void ManagedScoreSyntaxPanel()                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ROutputFileChooserControl                         implements compaswframeworkuiswingspiPropertyControl compaswuicommoncontrolControlListener                             public                         static                         final String                         OUTPUT_TYPE = output_type                            public                         static                         final String                         OUTPUT_TYPE_GRAPH = Graph                            public                         static                         final String                         GRAPH_OUTPUT_FILE_TYPE = graph_output_file_type                            public                         static                         final String                         TEXT_OUTPUT_FILE_TYPE = text_output_file_type                            public                         static                         final String                         OUTPUT_FILE_TYPE_HTML = HTML                            public                         static                         final String                         OUTPUT_FILE_TYPE_TEXT = Text                            public                         static                         final String                         OUTPUT_FILE_TYPE_COU = Cou                            public                         static                         final String                         FILENAME = OutputFilename                            public                         static                         final String                         OUTPUT_FILE_CHOOSER_BUTTON = OutputFileChooserButton                            public                         static                         final char                         BACK_SLASH_CHAR = 92                            public                         static                         final char                         FORWARD_SLASH_CHAR = 47                            private ActionHandler                         actionHandler                            private javaxswingJLabel                         fileNameLabel                            private javaxswingtextJTextComponent                         textComponent                            private javaxswingAbstractButton                         browseButton                            private compaswuiswingcontrolDefaultControlPanel                         mainPanel                            private String                         propertyName                            private compaswframeworkuiswingPropertyControlContext                         context                            public void ROutputFileChooserControl()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            private void                         initComponent()                            private compaswframeworkuiswingSwingResourceProvider                         getSwingResourceProvider()                            private String                         getLabelFromSwingResourceProvider(String)                            private Object                         getSwingUiProperty(String)                            protected String                         getTextComponentValue()                            public javaxswingJComponent                         getUIComponent()                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public void                         setControlEnabled(boolean)                            public boolean                         isControlEnabled()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public void                         disposeControl()                            public void                         controlValueChanged(compaswuicommoncontrolControlEvent)                            private String                         removeExtension(String)                            public void                         controlSelectionChanged(compaswuicommoncontrolControlEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandlerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ActionHandler                             private compaswframeworkuiswingExtensionObjectSwingUI                         extensionObjectSwingUI                            private boolean                         bPrintAll                            private int                         printScope    String                         actionErrorMessage                            public void ActionHandler(compaswframeworkuiswingExtensionObjectSwingUI)                            public void                         handleAction(String String)                            private void                         handleExportHTML(String)                            private void                         handleExportText(String)                            private void                         handlePrint(String String)                            private void                         handlePrintView()                            private void                         handlePageSetup(String)                            private void                         handleSaveNode(String)                            private void                         handlePreviewTable(String)                            private void                         handleSaveToProject(String)                            private void                         handleHelpButton()                            private void                         exportHTMLFile(String)                            private void                         saveTextFile(String String)                            private javaxswingJPanel                         createPrintViewPanel(javautilList)                            public String                         launchSaveFileChooser(String String[])                            private void                         configureSaveFileChooser(javaxswingJFileChooser)                            private void                         configureSaveChooserCallback(compaswcoreapplicationcallbackFileSystemChooserRequest)                            private String                         getSaveDialogTitle()                            private String                         getApproveButtonText()                            private boolean                         useApplicationFileChoosers()                            private compaswframeworkuicommonUISession                         getUISession()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            private String                         getTextContainerData(String)                            private boolean                         isExportText(String)                            private boolean                         isExportHTML()                            private boolean                         isSelectNode()                            private compaswuiswingcontrolModelViewerPanel                         getModelViewerPanel()                            private String                         getDefaultFileName()                            private void                         handleUpdateContainer(String)                    
                

comspsssharedcustom_guiui_builderpeersManagedBasePanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedBasePanel                         extends javaxswingJPanel                         implements javaawtprintPrintable     void ManagedBasePanel(javaawtLayoutManager)                            public int                         print(javaawtGraphics javaawtprintPageFormat int)                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$1                     
                

comspsssharedcustom_guiui_builderpeersUIUtilitiesclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class UIUtilities                             public void UIUtilities()                            public                         static javaxswingJButton                         createButton(compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComboBox                         createComBox(compaswframeworkuiswingSwingResourceProvider String int javaxswingDefaultComboBoxModel)                            public                         static javaxswingDefaultComboBoxModel                         createEnumComboBoxModel(compaswframeworkcommoncoreEnumerationValueDescriptor)                            public                         static javaxswingAbstractButton                         addAccessKey(javaxswingAbstractButton compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingtextJTextComponent                         createTextFiled(compaswframeworkuiswingSwingResourceProvider String int)                            public                         static javaxswingJComponent                         addTooltip(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComponent                         addDescription(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static                         transient void                         fitAllComponents(javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxPreferSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setPreferSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMaxSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMaxSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMinSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMinSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                    
                

comspsssharedcustom_guiui_builderpeersManagedHTMLOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedHTMLOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedHTMLOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedModelOutputPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedModelOutputPanel                         implements compaswframeworkuiswingspiManagedPanel compaswframeworkcommonextensionspiInteractorListener                             public                         static                         final String                         BUILD_TEXT_OUTPUT = buildTextOutputDisplay                            public                         static                         final String                         BUILD_CONSOLE_OUTPUT = buildConsoleOutputDisplay                            public                         static                         final String                         BUILD_HTML_OUTPUT = buildHTMLOutputDisplay                            public                         static                         final String                         BUILD_MV_OUTPUT = buildModelViewerDisplay                            public                         static                         final String                         APPLIER_CONSOLE_OUTPUT = applierConsoleOutputDisplay                            public                         static                         final String                         BUILD_TEXT_CONTAINER_NAME = TextOutput                            public                         static                         final String                         BUILD_CONSOLE_CONTAINER_NAME = ConsoleOutput                            public                         static                         final String                         BUILD_HTML_CONTAINER_NAME = HTMLOutput                            public                         static                         final String                         BUILD_PMML_CONTAINER_NAME = PMML                            public                         static                         final String                         BUILD_STAT_XML_CONTAINER_NAME = StatXML                            public                         static                         final String                         APPLIER_CONSOLE_CONTAINER_NAME = ApplierConsoleOutput                            private ManagedTextOutputArea                         textOutput                            private ManagedTextOutputArea                         consoleOutput                            private ManagedHTMLOutputArea                         htmlOutput                            private ManagedEmptyOutputArea                         emptyOutput                            private javautilHashMap                         componentMap                            private compaswframeworkuiswingManagedPanelContext                         managedPanelContext                            public void ManagedModelOutputPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private compaswframeworkuiswingspiManagedUIElement                         setupTextOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupConsoleOutputControl(String)                            public void                         updateConsoleOutputControl(String)                            private compaswframeworkuiswingspiManagedUIElement                         setupHTMLOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupModelViewerControl()                            public String                         getTextContainerData(String)                            public                         static boolean                         exportToHTML(compaswframeworkcommoncontainerContainer String)                            public                         static javaioFile                         getHTMLFileTest(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractHTMLCompoundContainer(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractZip(String String boolean)                         throws javaioIOException                            public                         static javaioFile                         getHTMLFile(String String boolean)                         throws javaioIOException                    
                

comspsssharedcustom_guiui_builderpeersManagedTextOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedTextOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedTextOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$ButtonsActionclass

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$ButtonsAction                         implements javaawteventActionListener                             private void ManagedScoreSyntaxControl$ButtonsAction(ManagedScoreSyntaxControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersManagedEmptyOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedEmptyOutputArea                         extends javaxswingJPanel                         implements compaswframeworkuiswingspiManagedUIElement                             public void ManagedEmptyOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public javaxswingJComponent                         getComponent()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ROutputFileChooserControl$1                         implements javaawteventActionListener     void ROutputFileChooserControl$1(ROutputFileChooserControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

extensionxml

imagescopygif

imagesdeletegif

imagesfileMenugif

imagesgenerateMenugif

imageshelp_buttongif

imageslg_gm_rpng

imageslg_r_cdb_gmgif

imageslg_r_cdb_modgif

imageslg_r_cdb_outputgif

imageslg_r_cdb_processgif

imagespreview_tablegif

randomForestxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

IBM SPSS Modeler and R

6 Your R code7

8 ibmspsscfdatamodelSetDataModel(modelerDataModel)

9 ibmspsscfdataSetData(modelerData)

10

All the functions starting with ibmspsscf are functions within this library It is this part of the codethat is responsible for the transfer of the data and the metadata to R Also you will see a while loopindicating that the data will go to R in batches of 1000 The other options are the values of the data readoptions within the nodeThe last lines of the code prepares the data to be send back to SPSS Also note the closing brace whichis the end of the while loop

Now since this library will be loaded with every interaction between SPSS and R you are free to use thesefunctions within your code as well So if you would like to avoid these runs in batches of 1K and you donot have version 171 available you can start your R code with another loop to first continue filling themodelerData and only than start your actual code In practice that means that in your R node you startwith

1 while(ibmspsscfdataHasMoreData())

2

3 modelerData lt-rbind(modelerDataibmspsscfdataGetData(rowCount=100000missing=NArDate=None

logicalFields=FALSE))

4

5

6 Some more code using modelerData which is now the complete dataset

512 Some useful parts of R code

Make sure a package is installed Whenever you have created any R node using packages you have tomake sure that anyone using this node has this package on his machine installed if needed withouttoo much user interference Therefore you can use this part of code

1 packages lt- function(x)

2 x lt- ascharacter(matchcall()[[2]])

3 if (require(xcharacteronly=TRUE))

4 installpackages(pkgs=xrepos=httpcranr-projectorg)

5 require(xcharacteronly=TRUE)

6

7

8

9 packages(rpart)

It will firstly verify if the library is installed on the system if not it will silently install it fromthe given CRAN mirror (you can change this to another mirror or a local repository if needed)Installation will only happen the first time the node will be used

Create the metadata corresponding to the R data frame Sometimes your data is very much trans-formed compared to the original data that it is difficult to build your metadata starting from theoriginal one So you might want to change the metadata to link to the data in R independent ofthe original This is in particularly useful if you want to use R as a sort node Below is a functionthat asks for a data frame and creates modelerData and modelerDataModel accordingly

1 sendToModeler lt- function (dataFrame)

2 if(isnull(dim(dataFrame)))

3 stop(Invalid data received not a dataframe)

4 if (dim(dataFrame)[1]lt=0)

5 print(Warning modelerData has no line all fieldStorage fields set to strings)

6 getStorage lt- function(x)return(string)

7 else

8 getStorage lt- function(x)

9 x lt- unlist(x)

10 res lt- NULL

11 if x is a factor typeof will return rsquointegerrsquo so we handle this case first

Page 15 of 20

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 5: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

6 Note n+n2 will be a new vector called New with the sum of the n + n2 c(3 6 7 8 13)

7

8 dim(Data) Will show you it is a 5x4 dataset

9 Data[24] Will give back the value on the 2nd line the 3rd column

10 colnames(Data) Will give the column names as a vector (nsb New)

11 Data$n[1] Will give back the first value of the vector n within the data frame

12

13 iris predefined data frame

There are also several pre-dened data frames installed within R One of them is called irisSometimes this document will refer back to iris

Model class which is actually a specic list containing predened objects dening a statisticalmodel For example a linear model class will be a list containing among others the coecientsof the regression model

List is an ordered collection of objects As an example you can have a list where the rst elementis a vector the second is a data frame and the third is a model Note that a data frame isa special type of a list where all the elements are vectors of equal sizes

3 The basics of R nodes in IBM SPSS Modeler

31 The R nodes

Once the installation for the R essentials are done you will see 3 new nodes in your node palettesThere is also a 4th R node which is the R nugget The dierence between and understanding ofthese 4 objects are essential

Output with this node data will be sent to R but it will never go back to SPSS (as it is aterminal node) The only thing that can go back to SPSS is the outputs generated by Rthatwill be presented within an SPSS output window

Transform data will go from SPSS to R but will also go back to SPSS after which the SPSSprocess can be continued

Model like the output node this is a terminal node so data will not go back to SPSS Howeverthere will be a reusable R object created within a nugget

Nugget similar to Transform node with the dierence that there is a reusable R object that canbe used in the R code

Node

Name R output node R transform node R model node R syntax nodePalette tab Output Record Ops Model NAData back to SPSS No Yes No YesReusable R object No No Create Use

32 Simple R code example

Let us start with saying that all the examples in this section are intentionally kept very simple soas to explain the interaction in a functional and structured way and be simple enough for non Rprogrammers We are certainly aware that most of the R code snippets we show in this chapter

Page 5 of 20

IBM SPSS Modeler and R

could also easily be implemented using standard SPSS Modeler functionality

There are 3 very important and reserved R objects that you should keep in mind when youuse the SPSS Modeler R integration Here is a brief description of these 3 after which we will gointo more detail for each of them

modelerData This is an R data frame that will be lled by the data entering in this R node Thisdata frame can be used and changed within your R code Eventually it will also be the dataframe that will be sent back to SPSS Modeler as a dataset Note that it will only containthe content of the data not (necessarily) the data column names and other metadata items

modelerDataModel This is also an R data frame containing the metadata of the data that is sentto R and back to SPSS Modeler It contains most of the information that you may expectwithin an SPSS Modeler type node This will be the object that will be most strange forexperienced R users

modelerModel this is an R object that can be lled by the user by any type of object you wantIt does not need to have a certain structure It will be calculated in the R model node afterwhich it will be saved within the R nugget where it can be used in the R-syntax

Note that R code is case-sensitive and therefore so are these object names In the following sectionswe will explain the usage of these objects

321 modelerData

modelerData is the R data frame that will be lled by the dataset entering the SPSS Modeler nodeit comes from So you can use this data frame to perform the desired calculations transformationsand outputs in R

Place the following code in an R output node

1 Print the first 6 lines of the data

2 head(modelerData)

3

4 Give a summary of the data

5 summary(modelerData)

6

7 create a histogram of the variable tenure

8 hist(modelerData$tenure xlab = years main = Tenure histogram)

9

10 change the tenure unit from months to years

11 modelerData$tenure lt- modelerData$tenure1212

13 recreate the histogram now in months

14 hist(modelerData$tenure xlab = months main = Tenure histogram)

Execution of this node will result in an SPSS Modeler output window in which all the R outputswill be assembled These will always be divided in 2 tabs Text output and Graph output

Page 6 of 20

IBM SPSS Modeler and R

In this case the text output is linked to the code on line 2 and 5 rst it prints the rst 6 lines(head) of the data next it will give summary statistics for each column

The graph output are two histograms One for the tenure in months the other for the samecolumn but after redening it by dividing the original value by twelve to give the tenure in years(note the X-axis scale)

As shown in the example stream Explain modelerDatastr you can also copy exactly this samecode into a transform node and attach a table node to it After running this table node you willnot see any R output (as none is expected) That means that even though the output code hasrun no outputs will be given However the data frame of modelerData will be send back to SPSSModeler In this case you will see the value of tenure being divided by 12

322 modelerDataModel

Metadata is very important in SPSS Modeler Let us for simplicity say that within modeler meta-data is represented by the type node With metadata we mean the type of each of the variablesin the dataset (numeric 13ag String storage ) At all times modeler will know exactly all themetadata at every step in the stream

R does not handle the metadata in a similar way as SPSS Modeler We already explainedmodelerDataModel taking over the role of the type node This is done by a data frame(=dataset) of the following structure

X1 X2 Xn

fieldName region tenure agefieldLabel Geographic indicator Months with service Age in years

fieldStorage real real realfieldMeasure nominal continuous continuousfieldFormat standard standard standardfieldRole input input input

So this means that this dataset will always have 6 lines with xed names (yes in R also thelines have names) The thing with this dataset is that it is completely the responsibility of theuser to align this metadata with the appropriate data So that means if we would like to add avariable with R the user must also manually add a column in modelerDataModel to make suremodelerData correctly goes back to SPSS Modeler In the earlier example above we did not makeany changes to the modelerDataModel and it was also not needed as the metadata did not change(dividing a number by 12 will not change the metadata) Now let us continue on the previousexample But now rather than changing the value of tenure in the same data variable we willcreate another one As a result we would have to update the metadata

1 Create the vector of tenure in years

2 Rcolumn lt- modelerData$tenure123

4 Paste this vector to the right of the dataset

5 modelerData lt- cbind(modelerDataRcolumn)

6

7 create the metadata for the column to add

8 newVar lt- c(fieldName=tenureYears fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

9

10 paste the new column metadata to the existing metadata

11 modelerDataModel lt- cbind(modelerDataModelnewVar)

Running a table node downstream of this transform node will show you the new variable withthe name tenureYears There are some important things to realize in this

Page 7 of 20

IBM SPSS Modeler and R

bull fieldName and fieldStorage are the only 2 required rows that needs to be lled in for anynew column In the code we left all the other lines empty meaning they will be lled in bythe stream default For a list of available values we refer to the user guide

bull As modelerDataModel is only useful when you go back to SPSS Modeler you will generallyonly usechange this object in non-terminal R-nodes It might still be handy to use it interminal nodes if the value of the modelerDataModel is important for your output (egrun a histogram of all continuous variables)

bull When data will go back to SPSS modeler it will be the content of the data frame excludingthe column- and row names That means that even though the column in the modelerData

will be called Rcolumn the name of the column in SPSS will only be dened by the metadatawithin the row fieldName In this case it is called tenureYears

bull The only link between modelerData and modelerDataModel is the order of the columns Itwill not look by name The rst column in the data will be given the metadata of the rstcolumn of modelerDataModel In case the metadata (modelerDataModel ) does not matchthe modelerData an error is thrown The table below shows schematic how this works

modelerData modelerDataModel

R

RName1 RNamenx11 x1n

x21 x2n

xm11 xm1n

xm1 xmn

X1 Xn

fieldName Name1 NamenfieldLabel

fieldStorage xxx xxxfieldMeasure fieldFormat

fieldRole

SPSS

Name1 Namenx11 x1n

x21 x2n

xm1 xmn

Note that only the names of the modelerDataModel are used

Since this concept is very strange to standard R users We found this part the most dicult toexplain To people who know SPSS you can summarize it as modelerDataModel taking over therole of the type node

You can nd all streams and R scripts explaining modelerDataModel here

323 modelerModel

modelerModel is the R object that is stored within the R nugget This object will be populatedwithin the R model node after which you could use modelerModel within the R nugget for scoringThis very much works the same way as IBM Modeler works You ask a model node to calculatea formula after which that formula will be stored within the nugget together with the way itshould be used to calculate a scoring

You will only use this object within the R model node and nugget Note that within the Rmodel node there are 2 syntax window

Page 8 of 20

Explain modelerDataModelxml

data0001dat

data0002dat

IBM SPSS Modeler and R

R model building is to calculate whatever you want to store within modelerModel that couldbe used within your nugget calculations to score your data This can be any object withinR As any SPSS Model builder node this will be a terminal code meaning that from thiscode no data will go back to SPSS modeler unless some outputs and things that are storedwithin the modelerModel object

R model scoring is the syntax to dene how you will use the object modelerModel containingthe content you stored to it in the R model building syntax to derive the new data Apartfrom the use of modelerModel this is very similar to the R transform node

Let us start with a simple example where we would like to create a basic linear model for thevariable tenure The formula of this model should be saved in the modelerModel after which itcan be used in the scoring

1 Create the model and save it in modelerModel

2 modelerModel lt- lm(tenure ~ age + region + ed + income data= modelerData)

3

4 Add some summary of the model in the nugget

5 summary(modelerModel)

6

7 together with a histogram

8 hist(modelerModel$residuals main = residual histogram)

9

10 and the residual vs actuals scatterplot

11 plot(modelerData$tenure modelerModel$fittedvalues xlab = actual ylab = predicted )

12

13 All of these output will be stored in the modeler nugget tabs

1 Use the model to make a prediction and add it to the existing data

2 pred lt- predict(modelerModel modelerData)

3 modelerData lt- cbind(modelerDatapred)

4

5 Take care of the metadata

6 newVar lt-c(fieldName=$L-tenure fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

It is important to note that modelerModel can be lled with any type of object but will veryoften be of a model class but does not have to be In the previous example the object stored toit was clearly a (statistical) model In the next example we will just save 2 numbers within themodelerModel object Imagine we want to calculate the z-values of a certain variable In orderto create the z-values we need the mean and the standard deviation of the column We will storeboth of these within modelerModel after which we will use them in the scoring syntax1 Thisexample shows you do not need to store a statistical model within your modelerModel objectbut it really can be any R object

1 calculate mean and standard deviation

2 M lt- mean(modelerData$tenure)3 SD lt- sd(modelerData$tenure)4

5 and save it in a list called modelerModel

6 modelerModel lt- list(avg = M sDev = SD)

1 calculate z scores using the elements in modelerModel

2 zTenure lt- (modelerData$tenure - modelerModel$avg)modelerModel$sDev3 modelerData lt- cbind(modelerDatazTenure)

4

5 define new metadata column and add it

6 newVar lt- c(fieldName=zTenure fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

1Note that there is a very good reason this is not combined into an R Transform node explained in 34

Page 9 of 20

IBM SPSS Modeler and R

You can nd all streams and R scripts explaining modelerModel here

33 Some general remarks

bull Although it might seems this way you are not required to build modelerData from theexisting data within that frame modelerData will be lled with the dataset you have inModeler however nothing stops you to throw that data away in R and dene some new datacoming from another data source in R As an example imagine this link from the WeatherCompany website This will give the weather history in Brussels Belgium in the monthNovember 2015 Now we can use R code as a source node by just overwriting modelerData

and redening modelerDataModel

1 Define the link

2 linkPath lt- httpwwwwundergroundcomhistoryairportEBBR20151101

MonthlyHistoryhtmlreq_city=Brusselsampformat=1

3 Read the data as csv

4 modelerData lt- readcsv(linkPath)

5 modelerData[1] lt- asDate(modelerData[1])

6

7 Redefining modelerDataModel all are real numbers except the first column is the date

8 modelerDataModel lt- asdataframe(t(dataframe(fieldName = colnames(modelerData)

fieldLabel = fieldStorage = c(daterep(realncol(modelerData)-1))

fieldMeasure = fieldFormat = fieldRole = )))

As you can see this code does not use the old denition of the dened R objects butcompletely redenes them Placing this in a R transform node will give back this newdataset to modeler So in this way you can use this approach to create an R input node

You can nd an example here

bull Within an R model node there is place for 2 scripts The building script will be the scriptthat will be populated within the R nugget It will not be run when you run the model nodeAs a result of this these 2 scripts are independent The only thing they share is the valueof the object modelerModel which is saved within the nugget when running the buildingsyntax and picked up within the R scoring syntaxThis also means that eventual R-libraries that are required should be loaded in both scriptsTake for example a model for a random forest

1 Load the library

2 library(randomForest)

3 Create the model and save it in modelerModel

4 modelerModel lt- randomForest(tenure ~ age + region + ed + income data= modelerDatantree

=50)

1 Load the library

2 library(randomForest)

3

4 Use the model to make a prediction and add it to the existing data

5 predlt- predict(modelerModel modelerData)

6 modelerData lt- cbind(modelerDatapred)

7

8 Take care of the metadata

9 newVar lt- c(fieldName=$RF-tenurefieldLabel=fieldStorage=realfieldMeasure=fieldFormat=fieldRole=)

10 modelerDataModel lt- cbind(modelerDataModelnewVar)

bull Talking about libraries and package A package is a collection of R objects dened for acertain purpose These often are specic statistical functionalities like randomForest in theexample above A basic R installation comes with the standard packages however there aremany more packages available made available by the R community on CRAN

Page 10 of 20

Explain modelerModelxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

R as sourcexml

data0001dat

data0002dat

IBM SPSS Modeler and R

Packages needs to be installed and made locally available in libraries Once the package isinstalled on the system as a library you can load this library in any R session by the codelibrary(ltnamegt)To install a package you have several options The easiest is probably to write a codelike installpackages(randomForest) within R You will have to select a CRAN mirrorwhere this library will be downloaded from and the download will go automatically Nor-mally you will only have to do this onceAlthough possible it is not recommended to run this package installation command fromwithin SPSS The reason is that these libraries will than be saved in a temporary folder andafterwards be deleted If you still want to this through SPSS you will have to hard codethe installation path

34 Read data options

Something we ignored until now are the settings within the node under Read data options Thebasics of the R integration with Modeler can be done without the knowledge of this as it requiressome more advanced R knowledge The user guide still has a good explanation about these items

However there is one more thing that might be important For modeler version 17 and lowerthe R integration of non-terminal nodes (ie transform and nuggets) will by default be done inbatches of 1000 The reason for this was to allow these R nodes to work on hadoop and otherclustered environments As a result of this it is very important to realize that any R code thatwould span over multiple lines of data would lead to false results For a workaround for this werefer to 511

Take as an example the z scores above If we would calculate the mean and the standard de-viation of the variable in a non-terminal node it would start with running this code for the rst1000 lines of data So that leads to a specic mean deviation and corresponding z-scores How-ever the next 1000 lines a new mean and deviation would be calculated and the z scores will bebased on these

As a solution the means and standard deviations are calculated in the R model (ie a termi-nal node) over all the data and used in the R nugget to calculate the z scores

Note that this approach may lead to a very slow integration between SPSS and R in the caseof streaming R nodes in a local non-clustered environment However as from IBM SPSS Modelerversion 171 there is a default option not to use this approach of batch processing or to increasethe batch size For the lower versions there is a workaround possible if you still want to increasethis batch size or turn it o (see later)

4 Custom Dialog builder

The Custom Dialog Builder allow you to create and manage R nodes with prefilled R code to use insideIBM SPSS Modeler streams In this way users can create their own nodes You can start the CustomDialog builder in the Tools menu under rdquoCustom Dialog Builderrdquo

When opening a custom dialog builder you will see a 2 windows One of them is the custom dialogitself the other is the toolset to populate the dialog

Page 11 of 20

IBM SPSS Modeler and R

41 Tools

The tools window is a list of items you can place within your dialog This include among others the fieldchooser Check and combo boxes Text and number controls and tabs You can select any and drag themonto the dialog itselfOnce you have any item in the dialog you can select it and you will see the item properties These are theproperties of this specific item and will change dependent on the type of item it is The most importantare the identifier (the way it will be referenced within the script) and the Title (the one that will be visualin the dialog)

42 Custom dialog

The big gray window is the dialog itself For the moment it is empty as it should be populated with itemsfrom the Tools listClicking on this gray dialog will show you the dialog properties below As main items this includes thename and title of the dialog the script itself and the type and position of the created nodeWith regards to the script to be written The global rule is that you reference to the items within thedialog using their identifier between double percentages (rdquoltidentifiergtrdquoOnce you finished creating the custom node you can install it by pressing the green arrow in the toolbarYou can also save intermediate versions to the disk

43 Simple example

Let us create a custom dialog for the randomForest model created earlier in section 33 Below you willfind a step by step approachThe most important thing we have to wonder is what within this code we want flexible for the user Inthe case of this model there might be 3 things that we want flexibel the input variables the target andthe number of trees in our forest

First fill in the Custom dialog properties as indicated

Page 12 of 20

IBM SPSS Modeler and R

In our fixed example the target is tenure but a user might chooseany other field As a result we will place a field chooser on the dialogChange the properties like shown The variable filter properties allowsyou to select only categorical variables

In our fixed example the inputs are age income but a user mightchoose any other field As a result we will place a field chooser on thedialog The biggest difference is that now we can select several variablesas there might be different inputs To make it easier we will separatethese values by a + Therefore change the properties like shown

As a third custom choice we would like to add the number of trees inour forest In the original script is was 50 so we will choose this as thedefault However users may choose any integer value between 1 and1000 Add a number control on the dialog and change the properties

So now the dialog is ready and we need to add the script to it Go tothe Edit options and choose rdquoScript Templaterdquo This will bring you toan empty window for the script In this case (as we selected we wanted2 scripts) there is a tab for the building code and one for the scoringscript If the coring script is greyed out you did set the dialog propertyrdquoScore from the Modelrdquo to True

Page 13 of 20

IBM SPSS Modeler and R

Let us start with the scoring script as this is easier The only thingwhich will need to be adapted for custom input is the variable namethat will be send back to SPSS So copy the scoring code and changethe name tenure to TARGET (this is the name of the identifier of thetarget)

Fill in the code for the building script and change in a similar wayas above the values for target and intput variables together with thenumber of trees Afterwards press OK to close the script window

Being back at your Custom Dialog builder save the dialog in any appro-priate location Also deploy the dialog by clicking on the green deployarrow in the toolbar Close the Dialog builder

Back to the stream you will now see the new node in the model paletteYou can use this node within your stream

You can find the resulted cfe file here (place this in the correct location see 521) and a stream

where it is deployed

5 Tips amp tricks Some more detailed

51 R code

511 ibmspsscf70 library

Let us now have a more detailled view about what actually happens with the code First of all it is worthto check what happens when you do the R installation correctly This will install the by IBM delivered Rpackage ibmspsscf70 in the library folder of your R installation This library contains several functionsto handle the data traffic between SPSS and R

Running any R node in SPSS will not only run the code you write but it will also run some extracode behind the scenes You can see this code in the rdquoConsole outputrdquo window of the R node Lookingfor example at this tab for an R nugget you will see that your code will be something like

1 modelerModel lt- ibmspsscfoutputGetModel()

2 while(ibmspsscfdataHasMoreData())

3 modelerDataModel lt- ibmspsscfdatamodelGetDataModel()

4 modelerData lt- ibmspsscfdataGetData(rowCount=1000 missing=NA rDate=None

logicalFields=FALSE)

5

Page 14 of 20

app-extensionxml

cdb_peerjar

comspsssharedcustom_guiui_builderpeersRStatsApplierPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsApplierPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsApplierPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                    
                

comspsssharedcustom_guiui_builderpeersCDExtensionPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class CDExtensionPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static String                         RBUILDER                            private                         static String                         SCORE_SYNTAX                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            public void CDExtensionPeer()                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private String                         replaceFieldSeparator(String String)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$1                         implements javaawteventActionListener     void ActionHandler$1(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$2class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$2                         implements javaawteventActionListener     void ActionHandler$2(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsUIPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsUIPeer                         implements compaswframeworkuiswingspiExtensionObjectSwingUIPeer compaswframeworkcommonextensionspiInteractorListener compaswcorecontainerContainerSetListener                             private ActionHandler                         actionHandler                            public                         static                         final String                         UPDATE_CONTAINER = updateContainer                            public void RStatsUIPeer()                            public void                         initSwingUI(compaswframeworkuiswingExtensionObjectSwingUI compaswframeworkcommoncoreProperties compaswframeworkuiswingExtensionObjectSwingUIContext)                            public void                         disposeSwingUI()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            public void                         handleAction(String String compaswframeworkcommoncoreProperties)                            public void                         containerChanged(compaswcorecontainerContainerSetEvent)                            public void                         updateContainer(compaswcorecontainerContainerSetEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsProcessPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsProcessPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsProcessPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersCopyrightclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         abstract                         interface Copyright                         extends annotationAnnotation                             public                         abstract String                         value()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxControl                         extends javaxswingJTextArea                         implements compaswframeworkuiswingspiPropertyControl                             private                         static                         final String                         editScoreSyntax = editScoreSyntax                            private                         static                         final String                         editScoreSyntaxTitle = editScoreSyntaxTitle                            private                         static                         final String                         editScoreSyntaxWarning = editScoreSyntaxWarning                            private                         static                         final String                         editScoreSyntaxContinueBtn = editScoreSyntax_Continue                            private                         static                         final String                         editScoreSyntaxCancelBtn = editScoreSyntax_Cancel                            private javaxswingJPanel                         containerPanel                            private javaxswingJButton                         editScoreSyntaxBtn                            private compaswframeworkuiswingPropertyControlContext                         propertyControlContext                            public void ManagedScoreSyntaxControl()                            public void                         disposeControl()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public javaxswingJComponent                         getUIComponent()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            public boolean                         isControlEnabled()                            public void                         setControlEnabled(boolean)                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxPanel                         implements compaswframeworkuiswingspiManagedPanel                             public void ManagedScoreSyntaxPanel()                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ROutputFileChooserControl                         implements compaswframeworkuiswingspiPropertyControl compaswuicommoncontrolControlListener                             public                         static                         final String                         OUTPUT_TYPE = output_type                            public                         static                         final String                         OUTPUT_TYPE_GRAPH = Graph                            public                         static                         final String                         GRAPH_OUTPUT_FILE_TYPE = graph_output_file_type                            public                         static                         final String                         TEXT_OUTPUT_FILE_TYPE = text_output_file_type                            public                         static                         final String                         OUTPUT_FILE_TYPE_HTML = HTML                            public                         static                         final String                         OUTPUT_FILE_TYPE_TEXT = Text                            public                         static                         final String                         OUTPUT_FILE_TYPE_COU = Cou                            public                         static                         final String                         FILENAME = OutputFilename                            public                         static                         final String                         OUTPUT_FILE_CHOOSER_BUTTON = OutputFileChooserButton                            public                         static                         final char                         BACK_SLASH_CHAR = 92                            public                         static                         final char                         FORWARD_SLASH_CHAR = 47                            private ActionHandler                         actionHandler                            private javaxswingJLabel                         fileNameLabel                            private javaxswingtextJTextComponent                         textComponent                            private javaxswingAbstractButton                         browseButton                            private compaswuiswingcontrolDefaultControlPanel                         mainPanel                            private String                         propertyName                            private compaswframeworkuiswingPropertyControlContext                         context                            public void ROutputFileChooserControl()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            private void                         initComponent()                            private compaswframeworkuiswingSwingResourceProvider                         getSwingResourceProvider()                            private String                         getLabelFromSwingResourceProvider(String)                            private Object                         getSwingUiProperty(String)                            protected String                         getTextComponentValue()                            public javaxswingJComponent                         getUIComponent()                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public void                         setControlEnabled(boolean)                            public boolean                         isControlEnabled()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public void                         disposeControl()                            public void                         controlValueChanged(compaswuicommoncontrolControlEvent)                            private String                         removeExtension(String)                            public void                         controlSelectionChanged(compaswuicommoncontrolControlEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandlerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ActionHandler                             private compaswframeworkuiswingExtensionObjectSwingUI                         extensionObjectSwingUI                            private boolean                         bPrintAll                            private int                         printScope    String                         actionErrorMessage                            public void ActionHandler(compaswframeworkuiswingExtensionObjectSwingUI)                            public void                         handleAction(String String)                            private void                         handleExportHTML(String)                            private void                         handleExportText(String)                            private void                         handlePrint(String String)                            private void                         handlePrintView()                            private void                         handlePageSetup(String)                            private void                         handleSaveNode(String)                            private void                         handlePreviewTable(String)                            private void                         handleSaveToProject(String)                            private void                         handleHelpButton()                            private void                         exportHTMLFile(String)                            private void                         saveTextFile(String String)                            private javaxswingJPanel                         createPrintViewPanel(javautilList)                            public String                         launchSaveFileChooser(String String[])                            private void                         configureSaveFileChooser(javaxswingJFileChooser)                            private void                         configureSaveChooserCallback(compaswcoreapplicationcallbackFileSystemChooserRequest)                            private String                         getSaveDialogTitle()                            private String                         getApproveButtonText()                            private boolean                         useApplicationFileChoosers()                            private compaswframeworkuicommonUISession                         getUISession()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            private String                         getTextContainerData(String)                            private boolean                         isExportText(String)                            private boolean                         isExportHTML()                            private boolean                         isSelectNode()                            private compaswuiswingcontrolModelViewerPanel                         getModelViewerPanel()                            private String                         getDefaultFileName()                            private void                         handleUpdateContainer(String)                    
                

comspsssharedcustom_guiui_builderpeersManagedBasePanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedBasePanel                         extends javaxswingJPanel                         implements javaawtprintPrintable     void ManagedBasePanel(javaawtLayoutManager)                            public int                         print(javaawtGraphics javaawtprintPageFormat int)                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$1                     
                

comspsssharedcustom_guiui_builderpeersUIUtilitiesclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class UIUtilities                             public void UIUtilities()                            public                         static javaxswingJButton                         createButton(compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComboBox                         createComBox(compaswframeworkuiswingSwingResourceProvider String int javaxswingDefaultComboBoxModel)                            public                         static javaxswingDefaultComboBoxModel                         createEnumComboBoxModel(compaswframeworkcommoncoreEnumerationValueDescriptor)                            public                         static javaxswingAbstractButton                         addAccessKey(javaxswingAbstractButton compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingtextJTextComponent                         createTextFiled(compaswframeworkuiswingSwingResourceProvider String int)                            public                         static javaxswingJComponent                         addTooltip(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComponent                         addDescription(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static                         transient void                         fitAllComponents(javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxPreferSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setPreferSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMaxSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMaxSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMinSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMinSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                    
                

comspsssharedcustom_guiui_builderpeersManagedHTMLOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedHTMLOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedHTMLOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedModelOutputPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedModelOutputPanel                         implements compaswframeworkuiswingspiManagedPanel compaswframeworkcommonextensionspiInteractorListener                             public                         static                         final String                         BUILD_TEXT_OUTPUT = buildTextOutputDisplay                            public                         static                         final String                         BUILD_CONSOLE_OUTPUT = buildConsoleOutputDisplay                            public                         static                         final String                         BUILD_HTML_OUTPUT = buildHTMLOutputDisplay                            public                         static                         final String                         BUILD_MV_OUTPUT = buildModelViewerDisplay                            public                         static                         final String                         APPLIER_CONSOLE_OUTPUT = applierConsoleOutputDisplay                            public                         static                         final String                         BUILD_TEXT_CONTAINER_NAME = TextOutput                            public                         static                         final String                         BUILD_CONSOLE_CONTAINER_NAME = ConsoleOutput                            public                         static                         final String                         BUILD_HTML_CONTAINER_NAME = HTMLOutput                            public                         static                         final String                         BUILD_PMML_CONTAINER_NAME = PMML                            public                         static                         final String                         BUILD_STAT_XML_CONTAINER_NAME = StatXML                            public                         static                         final String                         APPLIER_CONSOLE_CONTAINER_NAME = ApplierConsoleOutput                            private ManagedTextOutputArea                         textOutput                            private ManagedTextOutputArea                         consoleOutput                            private ManagedHTMLOutputArea                         htmlOutput                            private ManagedEmptyOutputArea                         emptyOutput                            private javautilHashMap                         componentMap                            private compaswframeworkuiswingManagedPanelContext                         managedPanelContext                            public void ManagedModelOutputPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private compaswframeworkuiswingspiManagedUIElement                         setupTextOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupConsoleOutputControl(String)                            public void                         updateConsoleOutputControl(String)                            private compaswframeworkuiswingspiManagedUIElement                         setupHTMLOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupModelViewerControl()                            public String                         getTextContainerData(String)                            public                         static boolean                         exportToHTML(compaswframeworkcommoncontainerContainer String)                            public                         static javaioFile                         getHTMLFileTest(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractHTMLCompoundContainer(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractZip(String String boolean)                         throws javaioIOException                            public                         static javaioFile                         getHTMLFile(String String boolean)                         throws javaioIOException                    
                

comspsssharedcustom_guiui_builderpeersManagedTextOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedTextOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedTextOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$ButtonsActionclass

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$ButtonsAction                         implements javaawteventActionListener                             private void ManagedScoreSyntaxControl$ButtonsAction(ManagedScoreSyntaxControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersManagedEmptyOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedEmptyOutputArea                         extends javaxswingJPanel                         implements compaswframeworkuiswingspiManagedUIElement                             public void ManagedEmptyOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public javaxswingJComponent                         getComponent()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ROutputFileChooserControl$1                         implements javaawteventActionListener     void ROutputFileChooserControl$1(ROutputFileChooserControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

extensionxml

imagescopygif

imagesdeletegif

imagesfileMenugif

imagesgenerateMenugif

imageshelp_buttongif

imageslg_gm_rpng

imageslg_r_cdb_gmgif

imageslg_r_cdb_modgif

imageslg_r_cdb_outputgif

imageslg_r_cdb_processgif

imagespreview_tablegif

randomForestxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

IBM SPSS Modeler and R

6 Your R code7

8 ibmspsscfdatamodelSetDataModel(modelerDataModel)

9 ibmspsscfdataSetData(modelerData)

10

All the functions starting with ibmspsscf are functions within this library It is this part of the codethat is responsible for the transfer of the data and the metadata to R Also you will see a while loopindicating that the data will go to R in batches of 1000 The other options are the values of the data readoptions within the nodeThe last lines of the code prepares the data to be send back to SPSS Also note the closing brace whichis the end of the while loop

Now since this library will be loaded with every interaction between SPSS and R you are free to use thesefunctions within your code as well So if you would like to avoid these runs in batches of 1K and you donot have version 171 available you can start your R code with another loop to first continue filling themodelerData and only than start your actual code In practice that means that in your R node you startwith

1 while(ibmspsscfdataHasMoreData())

2

3 modelerData lt-rbind(modelerDataibmspsscfdataGetData(rowCount=100000missing=NArDate=None

logicalFields=FALSE))

4

5

6 Some more code using modelerData which is now the complete dataset

512 Some useful parts of R code

Make sure a package is installed Whenever you have created any R node using packages you have tomake sure that anyone using this node has this package on his machine installed if needed withouttoo much user interference Therefore you can use this part of code

1 packages lt- function(x)

2 x lt- ascharacter(matchcall()[[2]])

3 if (require(xcharacteronly=TRUE))

4 installpackages(pkgs=xrepos=httpcranr-projectorg)

5 require(xcharacteronly=TRUE)

6

7

8

9 packages(rpart)

It will firstly verify if the library is installed on the system if not it will silently install it fromthe given CRAN mirror (you can change this to another mirror or a local repository if needed)Installation will only happen the first time the node will be used

Create the metadata corresponding to the R data frame Sometimes your data is very much trans-formed compared to the original data that it is difficult to build your metadata starting from theoriginal one So you might want to change the metadata to link to the data in R independent ofthe original This is in particularly useful if you want to use R as a sort node Below is a functionthat asks for a data frame and creates modelerData and modelerDataModel accordingly

1 sendToModeler lt- function (dataFrame)

2 if(isnull(dim(dataFrame)))

3 stop(Invalid data received not a dataframe)

4 if (dim(dataFrame)[1]lt=0)

5 print(Warning modelerData has no line all fieldStorage fields set to strings)

6 getStorage lt- function(x)return(string)

7 else

8 getStorage lt- function(x)

9 x lt- unlist(x)

10 res lt- NULL

11 if x is a factor typeof will return rsquointegerrsquo so we handle this case first

Page 15 of 20

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 6: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

could also easily be implemented using standard SPSS Modeler functionality

There are 3 very important and reserved R objects that you should keep in mind when youuse the SPSS Modeler R integration Here is a brief description of these 3 after which we will gointo more detail for each of them

modelerData This is an R data frame that will be lled by the data entering in this R node Thisdata frame can be used and changed within your R code Eventually it will also be the dataframe that will be sent back to SPSS Modeler as a dataset Note that it will only containthe content of the data not (necessarily) the data column names and other metadata items

modelerDataModel This is also an R data frame containing the metadata of the data that is sentto R and back to SPSS Modeler It contains most of the information that you may expectwithin an SPSS Modeler type node This will be the object that will be most strange forexperienced R users

modelerModel this is an R object that can be lled by the user by any type of object you wantIt does not need to have a certain structure It will be calculated in the R model node afterwhich it will be saved within the R nugget where it can be used in the R-syntax

Note that R code is case-sensitive and therefore so are these object names In the following sectionswe will explain the usage of these objects

321 modelerData

modelerData is the R data frame that will be lled by the dataset entering the SPSS Modeler nodeit comes from So you can use this data frame to perform the desired calculations transformationsand outputs in R

Place the following code in an R output node

1 Print the first 6 lines of the data

2 head(modelerData)

3

4 Give a summary of the data

5 summary(modelerData)

6

7 create a histogram of the variable tenure

8 hist(modelerData$tenure xlab = years main = Tenure histogram)

9

10 change the tenure unit from months to years

11 modelerData$tenure lt- modelerData$tenure1212

13 recreate the histogram now in months

14 hist(modelerData$tenure xlab = months main = Tenure histogram)

Execution of this node will result in an SPSS Modeler output window in which all the R outputswill be assembled These will always be divided in 2 tabs Text output and Graph output

Page 6 of 20

IBM SPSS Modeler and R

In this case the text output is linked to the code on line 2 and 5 rst it prints the rst 6 lines(head) of the data next it will give summary statistics for each column

The graph output are two histograms One for the tenure in months the other for the samecolumn but after redening it by dividing the original value by twelve to give the tenure in years(note the X-axis scale)

As shown in the example stream Explain modelerDatastr you can also copy exactly this samecode into a transform node and attach a table node to it After running this table node you willnot see any R output (as none is expected) That means that even though the output code hasrun no outputs will be given However the data frame of modelerData will be send back to SPSSModeler In this case you will see the value of tenure being divided by 12

322 modelerDataModel

Metadata is very important in SPSS Modeler Let us for simplicity say that within modeler meta-data is represented by the type node With metadata we mean the type of each of the variablesin the dataset (numeric 13ag String storage ) At all times modeler will know exactly all themetadata at every step in the stream

R does not handle the metadata in a similar way as SPSS Modeler We already explainedmodelerDataModel taking over the role of the type node This is done by a data frame(=dataset) of the following structure

X1 X2 Xn

fieldName region tenure agefieldLabel Geographic indicator Months with service Age in years

fieldStorage real real realfieldMeasure nominal continuous continuousfieldFormat standard standard standardfieldRole input input input

So this means that this dataset will always have 6 lines with xed names (yes in R also thelines have names) The thing with this dataset is that it is completely the responsibility of theuser to align this metadata with the appropriate data So that means if we would like to add avariable with R the user must also manually add a column in modelerDataModel to make suremodelerData correctly goes back to SPSS Modeler In the earlier example above we did not makeany changes to the modelerDataModel and it was also not needed as the metadata did not change(dividing a number by 12 will not change the metadata) Now let us continue on the previousexample But now rather than changing the value of tenure in the same data variable we willcreate another one As a result we would have to update the metadata

1 Create the vector of tenure in years

2 Rcolumn lt- modelerData$tenure123

4 Paste this vector to the right of the dataset

5 modelerData lt- cbind(modelerDataRcolumn)

6

7 create the metadata for the column to add

8 newVar lt- c(fieldName=tenureYears fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

9

10 paste the new column metadata to the existing metadata

11 modelerDataModel lt- cbind(modelerDataModelnewVar)

Running a table node downstream of this transform node will show you the new variable withthe name tenureYears There are some important things to realize in this

Page 7 of 20

IBM SPSS Modeler and R

bull fieldName and fieldStorage are the only 2 required rows that needs to be lled in for anynew column In the code we left all the other lines empty meaning they will be lled in bythe stream default For a list of available values we refer to the user guide

bull As modelerDataModel is only useful when you go back to SPSS Modeler you will generallyonly usechange this object in non-terminal R-nodes It might still be handy to use it interminal nodes if the value of the modelerDataModel is important for your output (egrun a histogram of all continuous variables)

bull When data will go back to SPSS modeler it will be the content of the data frame excludingthe column- and row names That means that even though the column in the modelerData

will be called Rcolumn the name of the column in SPSS will only be dened by the metadatawithin the row fieldName In this case it is called tenureYears

bull The only link between modelerData and modelerDataModel is the order of the columns Itwill not look by name The rst column in the data will be given the metadata of the rstcolumn of modelerDataModel In case the metadata (modelerDataModel ) does not matchthe modelerData an error is thrown The table below shows schematic how this works

modelerData modelerDataModel

R

RName1 RNamenx11 x1n

x21 x2n

xm11 xm1n

xm1 xmn

X1 Xn

fieldName Name1 NamenfieldLabel

fieldStorage xxx xxxfieldMeasure fieldFormat

fieldRole

SPSS

Name1 Namenx11 x1n

x21 x2n

xm1 xmn

Note that only the names of the modelerDataModel are used

Since this concept is very strange to standard R users We found this part the most dicult toexplain To people who know SPSS you can summarize it as modelerDataModel taking over therole of the type node

You can nd all streams and R scripts explaining modelerDataModel here

323 modelerModel

modelerModel is the R object that is stored within the R nugget This object will be populatedwithin the R model node after which you could use modelerModel within the R nugget for scoringThis very much works the same way as IBM Modeler works You ask a model node to calculatea formula after which that formula will be stored within the nugget together with the way itshould be used to calculate a scoring

You will only use this object within the R model node and nugget Note that within the Rmodel node there are 2 syntax window

Page 8 of 20

Explain modelerDataModelxml

data0001dat

data0002dat

IBM SPSS Modeler and R

R model building is to calculate whatever you want to store within modelerModel that couldbe used within your nugget calculations to score your data This can be any object withinR As any SPSS Model builder node this will be a terminal code meaning that from thiscode no data will go back to SPSS modeler unless some outputs and things that are storedwithin the modelerModel object

R model scoring is the syntax to dene how you will use the object modelerModel containingthe content you stored to it in the R model building syntax to derive the new data Apartfrom the use of modelerModel this is very similar to the R transform node

Let us start with a simple example where we would like to create a basic linear model for thevariable tenure The formula of this model should be saved in the modelerModel after which itcan be used in the scoring

1 Create the model and save it in modelerModel

2 modelerModel lt- lm(tenure ~ age + region + ed + income data= modelerData)

3

4 Add some summary of the model in the nugget

5 summary(modelerModel)

6

7 together with a histogram

8 hist(modelerModel$residuals main = residual histogram)

9

10 and the residual vs actuals scatterplot

11 plot(modelerData$tenure modelerModel$fittedvalues xlab = actual ylab = predicted )

12

13 All of these output will be stored in the modeler nugget tabs

1 Use the model to make a prediction and add it to the existing data

2 pred lt- predict(modelerModel modelerData)

3 modelerData lt- cbind(modelerDatapred)

4

5 Take care of the metadata

6 newVar lt-c(fieldName=$L-tenure fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

It is important to note that modelerModel can be lled with any type of object but will veryoften be of a model class but does not have to be In the previous example the object stored toit was clearly a (statistical) model In the next example we will just save 2 numbers within themodelerModel object Imagine we want to calculate the z-values of a certain variable In orderto create the z-values we need the mean and the standard deviation of the column We will storeboth of these within modelerModel after which we will use them in the scoring syntax1 Thisexample shows you do not need to store a statistical model within your modelerModel objectbut it really can be any R object

1 calculate mean and standard deviation

2 M lt- mean(modelerData$tenure)3 SD lt- sd(modelerData$tenure)4

5 and save it in a list called modelerModel

6 modelerModel lt- list(avg = M sDev = SD)

1 calculate z scores using the elements in modelerModel

2 zTenure lt- (modelerData$tenure - modelerModel$avg)modelerModel$sDev3 modelerData lt- cbind(modelerDatazTenure)

4

5 define new metadata column and add it

6 newVar lt- c(fieldName=zTenure fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

1Note that there is a very good reason this is not combined into an R Transform node explained in 34

Page 9 of 20

IBM SPSS Modeler and R

You can nd all streams and R scripts explaining modelerModel here

33 Some general remarks

bull Although it might seems this way you are not required to build modelerData from theexisting data within that frame modelerData will be lled with the dataset you have inModeler however nothing stops you to throw that data away in R and dene some new datacoming from another data source in R As an example imagine this link from the WeatherCompany website This will give the weather history in Brussels Belgium in the monthNovember 2015 Now we can use R code as a source node by just overwriting modelerData

and redening modelerDataModel

1 Define the link

2 linkPath lt- httpwwwwundergroundcomhistoryairportEBBR20151101

MonthlyHistoryhtmlreq_city=Brusselsampformat=1

3 Read the data as csv

4 modelerData lt- readcsv(linkPath)

5 modelerData[1] lt- asDate(modelerData[1])

6

7 Redefining modelerDataModel all are real numbers except the first column is the date

8 modelerDataModel lt- asdataframe(t(dataframe(fieldName = colnames(modelerData)

fieldLabel = fieldStorage = c(daterep(realncol(modelerData)-1))

fieldMeasure = fieldFormat = fieldRole = )))

As you can see this code does not use the old denition of the dened R objects butcompletely redenes them Placing this in a R transform node will give back this newdataset to modeler So in this way you can use this approach to create an R input node

You can nd an example here

bull Within an R model node there is place for 2 scripts The building script will be the scriptthat will be populated within the R nugget It will not be run when you run the model nodeAs a result of this these 2 scripts are independent The only thing they share is the valueof the object modelerModel which is saved within the nugget when running the buildingsyntax and picked up within the R scoring syntaxThis also means that eventual R-libraries that are required should be loaded in both scriptsTake for example a model for a random forest

1 Load the library

2 library(randomForest)

3 Create the model and save it in modelerModel

4 modelerModel lt- randomForest(tenure ~ age + region + ed + income data= modelerDatantree

=50)

1 Load the library

2 library(randomForest)

3

4 Use the model to make a prediction and add it to the existing data

5 predlt- predict(modelerModel modelerData)

6 modelerData lt- cbind(modelerDatapred)

7

8 Take care of the metadata

9 newVar lt- c(fieldName=$RF-tenurefieldLabel=fieldStorage=realfieldMeasure=fieldFormat=fieldRole=)

10 modelerDataModel lt- cbind(modelerDataModelnewVar)

bull Talking about libraries and package A package is a collection of R objects dened for acertain purpose These often are specic statistical functionalities like randomForest in theexample above A basic R installation comes with the standard packages however there aremany more packages available made available by the R community on CRAN

Page 10 of 20

Explain modelerModelxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

R as sourcexml

data0001dat

data0002dat

IBM SPSS Modeler and R

Packages needs to be installed and made locally available in libraries Once the package isinstalled on the system as a library you can load this library in any R session by the codelibrary(ltnamegt)To install a package you have several options The easiest is probably to write a codelike installpackages(randomForest) within R You will have to select a CRAN mirrorwhere this library will be downloaded from and the download will go automatically Nor-mally you will only have to do this onceAlthough possible it is not recommended to run this package installation command fromwithin SPSS The reason is that these libraries will than be saved in a temporary folder andafterwards be deleted If you still want to this through SPSS you will have to hard codethe installation path

34 Read data options

Something we ignored until now are the settings within the node under Read data options Thebasics of the R integration with Modeler can be done without the knowledge of this as it requiressome more advanced R knowledge The user guide still has a good explanation about these items

However there is one more thing that might be important For modeler version 17 and lowerthe R integration of non-terminal nodes (ie transform and nuggets) will by default be done inbatches of 1000 The reason for this was to allow these R nodes to work on hadoop and otherclustered environments As a result of this it is very important to realize that any R code thatwould span over multiple lines of data would lead to false results For a workaround for this werefer to 511

Take as an example the z scores above If we would calculate the mean and the standard de-viation of the variable in a non-terminal node it would start with running this code for the rst1000 lines of data So that leads to a specic mean deviation and corresponding z-scores How-ever the next 1000 lines a new mean and deviation would be calculated and the z scores will bebased on these

As a solution the means and standard deviations are calculated in the R model (ie a termi-nal node) over all the data and used in the R nugget to calculate the z scores

Note that this approach may lead to a very slow integration between SPSS and R in the caseof streaming R nodes in a local non-clustered environment However as from IBM SPSS Modelerversion 171 there is a default option not to use this approach of batch processing or to increasethe batch size For the lower versions there is a workaround possible if you still want to increasethis batch size or turn it o (see later)

4 Custom Dialog builder

The Custom Dialog Builder allow you to create and manage R nodes with prefilled R code to use insideIBM SPSS Modeler streams In this way users can create their own nodes You can start the CustomDialog builder in the Tools menu under rdquoCustom Dialog Builderrdquo

When opening a custom dialog builder you will see a 2 windows One of them is the custom dialogitself the other is the toolset to populate the dialog

Page 11 of 20

IBM SPSS Modeler and R

41 Tools

The tools window is a list of items you can place within your dialog This include among others the fieldchooser Check and combo boxes Text and number controls and tabs You can select any and drag themonto the dialog itselfOnce you have any item in the dialog you can select it and you will see the item properties These are theproperties of this specific item and will change dependent on the type of item it is The most importantare the identifier (the way it will be referenced within the script) and the Title (the one that will be visualin the dialog)

42 Custom dialog

The big gray window is the dialog itself For the moment it is empty as it should be populated with itemsfrom the Tools listClicking on this gray dialog will show you the dialog properties below As main items this includes thename and title of the dialog the script itself and the type and position of the created nodeWith regards to the script to be written The global rule is that you reference to the items within thedialog using their identifier between double percentages (rdquoltidentifiergtrdquoOnce you finished creating the custom node you can install it by pressing the green arrow in the toolbarYou can also save intermediate versions to the disk

43 Simple example

Let us create a custom dialog for the randomForest model created earlier in section 33 Below you willfind a step by step approachThe most important thing we have to wonder is what within this code we want flexible for the user Inthe case of this model there might be 3 things that we want flexibel the input variables the target andthe number of trees in our forest

First fill in the Custom dialog properties as indicated

Page 12 of 20

IBM SPSS Modeler and R

In our fixed example the target is tenure but a user might chooseany other field As a result we will place a field chooser on the dialogChange the properties like shown The variable filter properties allowsyou to select only categorical variables

In our fixed example the inputs are age income but a user mightchoose any other field As a result we will place a field chooser on thedialog The biggest difference is that now we can select several variablesas there might be different inputs To make it easier we will separatethese values by a + Therefore change the properties like shown

As a third custom choice we would like to add the number of trees inour forest In the original script is was 50 so we will choose this as thedefault However users may choose any integer value between 1 and1000 Add a number control on the dialog and change the properties

So now the dialog is ready and we need to add the script to it Go tothe Edit options and choose rdquoScript Templaterdquo This will bring you toan empty window for the script In this case (as we selected we wanted2 scripts) there is a tab for the building code and one for the scoringscript If the coring script is greyed out you did set the dialog propertyrdquoScore from the Modelrdquo to True

Page 13 of 20

IBM SPSS Modeler and R

Let us start with the scoring script as this is easier The only thingwhich will need to be adapted for custom input is the variable namethat will be send back to SPSS So copy the scoring code and changethe name tenure to TARGET (this is the name of the identifier of thetarget)

Fill in the code for the building script and change in a similar wayas above the values for target and intput variables together with thenumber of trees Afterwards press OK to close the script window

Being back at your Custom Dialog builder save the dialog in any appro-priate location Also deploy the dialog by clicking on the green deployarrow in the toolbar Close the Dialog builder

Back to the stream you will now see the new node in the model paletteYou can use this node within your stream

You can find the resulted cfe file here (place this in the correct location see 521) and a stream

where it is deployed

5 Tips amp tricks Some more detailed

51 R code

511 ibmspsscf70 library

Let us now have a more detailled view about what actually happens with the code First of all it is worthto check what happens when you do the R installation correctly This will install the by IBM delivered Rpackage ibmspsscf70 in the library folder of your R installation This library contains several functionsto handle the data traffic between SPSS and R

Running any R node in SPSS will not only run the code you write but it will also run some extracode behind the scenes You can see this code in the rdquoConsole outputrdquo window of the R node Lookingfor example at this tab for an R nugget you will see that your code will be something like

1 modelerModel lt- ibmspsscfoutputGetModel()

2 while(ibmspsscfdataHasMoreData())

3 modelerDataModel lt- ibmspsscfdatamodelGetDataModel()

4 modelerData lt- ibmspsscfdataGetData(rowCount=1000 missing=NA rDate=None

logicalFields=FALSE)

5

Page 14 of 20

app-extensionxml

cdb_peerjar

comspsssharedcustom_guiui_builderpeersRStatsApplierPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsApplierPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsApplierPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                    
                

comspsssharedcustom_guiui_builderpeersCDExtensionPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class CDExtensionPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static String                         RBUILDER                            private                         static String                         SCORE_SYNTAX                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            public void CDExtensionPeer()                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private String                         replaceFieldSeparator(String String)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$1                         implements javaawteventActionListener     void ActionHandler$1(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$2class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$2                         implements javaawteventActionListener     void ActionHandler$2(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsUIPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsUIPeer                         implements compaswframeworkuiswingspiExtensionObjectSwingUIPeer compaswframeworkcommonextensionspiInteractorListener compaswcorecontainerContainerSetListener                             private ActionHandler                         actionHandler                            public                         static                         final String                         UPDATE_CONTAINER = updateContainer                            public void RStatsUIPeer()                            public void                         initSwingUI(compaswframeworkuiswingExtensionObjectSwingUI compaswframeworkcommoncoreProperties compaswframeworkuiswingExtensionObjectSwingUIContext)                            public void                         disposeSwingUI()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            public void                         handleAction(String String compaswframeworkcommoncoreProperties)                            public void                         containerChanged(compaswcorecontainerContainerSetEvent)                            public void                         updateContainer(compaswcorecontainerContainerSetEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsProcessPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsProcessPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsProcessPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersCopyrightclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         abstract                         interface Copyright                         extends annotationAnnotation                             public                         abstract String                         value()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxControl                         extends javaxswingJTextArea                         implements compaswframeworkuiswingspiPropertyControl                             private                         static                         final String                         editScoreSyntax = editScoreSyntax                            private                         static                         final String                         editScoreSyntaxTitle = editScoreSyntaxTitle                            private                         static                         final String                         editScoreSyntaxWarning = editScoreSyntaxWarning                            private                         static                         final String                         editScoreSyntaxContinueBtn = editScoreSyntax_Continue                            private                         static                         final String                         editScoreSyntaxCancelBtn = editScoreSyntax_Cancel                            private javaxswingJPanel                         containerPanel                            private javaxswingJButton                         editScoreSyntaxBtn                            private compaswframeworkuiswingPropertyControlContext                         propertyControlContext                            public void ManagedScoreSyntaxControl()                            public void                         disposeControl()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public javaxswingJComponent                         getUIComponent()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            public boolean                         isControlEnabled()                            public void                         setControlEnabled(boolean)                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxPanel                         implements compaswframeworkuiswingspiManagedPanel                             public void ManagedScoreSyntaxPanel()                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ROutputFileChooserControl                         implements compaswframeworkuiswingspiPropertyControl compaswuicommoncontrolControlListener                             public                         static                         final String                         OUTPUT_TYPE = output_type                            public                         static                         final String                         OUTPUT_TYPE_GRAPH = Graph                            public                         static                         final String                         GRAPH_OUTPUT_FILE_TYPE = graph_output_file_type                            public                         static                         final String                         TEXT_OUTPUT_FILE_TYPE = text_output_file_type                            public                         static                         final String                         OUTPUT_FILE_TYPE_HTML = HTML                            public                         static                         final String                         OUTPUT_FILE_TYPE_TEXT = Text                            public                         static                         final String                         OUTPUT_FILE_TYPE_COU = Cou                            public                         static                         final String                         FILENAME = OutputFilename                            public                         static                         final String                         OUTPUT_FILE_CHOOSER_BUTTON = OutputFileChooserButton                            public                         static                         final char                         BACK_SLASH_CHAR = 92                            public                         static                         final char                         FORWARD_SLASH_CHAR = 47                            private ActionHandler                         actionHandler                            private javaxswingJLabel                         fileNameLabel                            private javaxswingtextJTextComponent                         textComponent                            private javaxswingAbstractButton                         browseButton                            private compaswuiswingcontrolDefaultControlPanel                         mainPanel                            private String                         propertyName                            private compaswframeworkuiswingPropertyControlContext                         context                            public void ROutputFileChooserControl()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            private void                         initComponent()                            private compaswframeworkuiswingSwingResourceProvider                         getSwingResourceProvider()                            private String                         getLabelFromSwingResourceProvider(String)                            private Object                         getSwingUiProperty(String)                            protected String                         getTextComponentValue()                            public javaxswingJComponent                         getUIComponent()                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public void                         setControlEnabled(boolean)                            public boolean                         isControlEnabled()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public void                         disposeControl()                            public void                         controlValueChanged(compaswuicommoncontrolControlEvent)                            private String                         removeExtension(String)                            public void                         controlSelectionChanged(compaswuicommoncontrolControlEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandlerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ActionHandler                             private compaswframeworkuiswingExtensionObjectSwingUI                         extensionObjectSwingUI                            private boolean                         bPrintAll                            private int                         printScope    String                         actionErrorMessage                            public void ActionHandler(compaswframeworkuiswingExtensionObjectSwingUI)                            public void                         handleAction(String String)                            private void                         handleExportHTML(String)                            private void                         handleExportText(String)                            private void                         handlePrint(String String)                            private void                         handlePrintView()                            private void                         handlePageSetup(String)                            private void                         handleSaveNode(String)                            private void                         handlePreviewTable(String)                            private void                         handleSaveToProject(String)                            private void                         handleHelpButton()                            private void                         exportHTMLFile(String)                            private void                         saveTextFile(String String)                            private javaxswingJPanel                         createPrintViewPanel(javautilList)                            public String                         launchSaveFileChooser(String String[])                            private void                         configureSaveFileChooser(javaxswingJFileChooser)                            private void                         configureSaveChooserCallback(compaswcoreapplicationcallbackFileSystemChooserRequest)                            private String                         getSaveDialogTitle()                            private String                         getApproveButtonText()                            private boolean                         useApplicationFileChoosers()                            private compaswframeworkuicommonUISession                         getUISession()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            private String                         getTextContainerData(String)                            private boolean                         isExportText(String)                            private boolean                         isExportHTML()                            private boolean                         isSelectNode()                            private compaswuiswingcontrolModelViewerPanel                         getModelViewerPanel()                            private String                         getDefaultFileName()                            private void                         handleUpdateContainer(String)                    
                

comspsssharedcustom_guiui_builderpeersManagedBasePanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedBasePanel                         extends javaxswingJPanel                         implements javaawtprintPrintable     void ManagedBasePanel(javaawtLayoutManager)                            public int                         print(javaawtGraphics javaawtprintPageFormat int)                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$1                     
                

comspsssharedcustom_guiui_builderpeersUIUtilitiesclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class UIUtilities                             public void UIUtilities()                            public                         static javaxswingJButton                         createButton(compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComboBox                         createComBox(compaswframeworkuiswingSwingResourceProvider String int javaxswingDefaultComboBoxModel)                            public                         static javaxswingDefaultComboBoxModel                         createEnumComboBoxModel(compaswframeworkcommoncoreEnumerationValueDescriptor)                            public                         static javaxswingAbstractButton                         addAccessKey(javaxswingAbstractButton compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingtextJTextComponent                         createTextFiled(compaswframeworkuiswingSwingResourceProvider String int)                            public                         static javaxswingJComponent                         addTooltip(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComponent                         addDescription(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static                         transient void                         fitAllComponents(javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxPreferSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setPreferSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMaxSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMaxSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMinSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMinSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                    
                

comspsssharedcustom_guiui_builderpeersManagedHTMLOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedHTMLOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedHTMLOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedModelOutputPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedModelOutputPanel                         implements compaswframeworkuiswingspiManagedPanel compaswframeworkcommonextensionspiInteractorListener                             public                         static                         final String                         BUILD_TEXT_OUTPUT = buildTextOutputDisplay                            public                         static                         final String                         BUILD_CONSOLE_OUTPUT = buildConsoleOutputDisplay                            public                         static                         final String                         BUILD_HTML_OUTPUT = buildHTMLOutputDisplay                            public                         static                         final String                         BUILD_MV_OUTPUT = buildModelViewerDisplay                            public                         static                         final String                         APPLIER_CONSOLE_OUTPUT = applierConsoleOutputDisplay                            public                         static                         final String                         BUILD_TEXT_CONTAINER_NAME = TextOutput                            public                         static                         final String                         BUILD_CONSOLE_CONTAINER_NAME = ConsoleOutput                            public                         static                         final String                         BUILD_HTML_CONTAINER_NAME = HTMLOutput                            public                         static                         final String                         BUILD_PMML_CONTAINER_NAME = PMML                            public                         static                         final String                         BUILD_STAT_XML_CONTAINER_NAME = StatXML                            public                         static                         final String                         APPLIER_CONSOLE_CONTAINER_NAME = ApplierConsoleOutput                            private ManagedTextOutputArea                         textOutput                            private ManagedTextOutputArea                         consoleOutput                            private ManagedHTMLOutputArea                         htmlOutput                            private ManagedEmptyOutputArea                         emptyOutput                            private javautilHashMap                         componentMap                            private compaswframeworkuiswingManagedPanelContext                         managedPanelContext                            public void ManagedModelOutputPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private compaswframeworkuiswingspiManagedUIElement                         setupTextOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupConsoleOutputControl(String)                            public void                         updateConsoleOutputControl(String)                            private compaswframeworkuiswingspiManagedUIElement                         setupHTMLOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupModelViewerControl()                            public String                         getTextContainerData(String)                            public                         static boolean                         exportToHTML(compaswframeworkcommoncontainerContainer String)                            public                         static javaioFile                         getHTMLFileTest(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractHTMLCompoundContainer(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractZip(String String boolean)                         throws javaioIOException                            public                         static javaioFile                         getHTMLFile(String String boolean)                         throws javaioIOException                    
                

comspsssharedcustom_guiui_builderpeersManagedTextOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedTextOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedTextOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$ButtonsActionclass

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$ButtonsAction                         implements javaawteventActionListener                             private void ManagedScoreSyntaxControl$ButtonsAction(ManagedScoreSyntaxControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersManagedEmptyOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedEmptyOutputArea                         extends javaxswingJPanel                         implements compaswframeworkuiswingspiManagedUIElement                             public void ManagedEmptyOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public javaxswingJComponent                         getComponent()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ROutputFileChooserControl$1                         implements javaawteventActionListener     void ROutputFileChooserControl$1(ROutputFileChooserControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

extensionxml

imagescopygif

imagesdeletegif

imagesfileMenugif

imagesgenerateMenugif

imageshelp_buttongif

imageslg_gm_rpng

imageslg_r_cdb_gmgif

imageslg_r_cdb_modgif

imageslg_r_cdb_outputgif

imageslg_r_cdb_processgif

imagespreview_tablegif

randomForestxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

IBM SPSS Modeler and R

6 Your R code7

8 ibmspsscfdatamodelSetDataModel(modelerDataModel)

9 ibmspsscfdataSetData(modelerData)

10

All the functions starting with ibmspsscf are functions within this library It is this part of the codethat is responsible for the transfer of the data and the metadata to R Also you will see a while loopindicating that the data will go to R in batches of 1000 The other options are the values of the data readoptions within the nodeThe last lines of the code prepares the data to be send back to SPSS Also note the closing brace whichis the end of the while loop

Now since this library will be loaded with every interaction between SPSS and R you are free to use thesefunctions within your code as well So if you would like to avoid these runs in batches of 1K and you donot have version 171 available you can start your R code with another loop to first continue filling themodelerData and only than start your actual code In practice that means that in your R node you startwith

1 while(ibmspsscfdataHasMoreData())

2

3 modelerData lt-rbind(modelerDataibmspsscfdataGetData(rowCount=100000missing=NArDate=None

logicalFields=FALSE))

4

5

6 Some more code using modelerData which is now the complete dataset

512 Some useful parts of R code

Make sure a package is installed Whenever you have created any R node using packages you have tomake sure that anyone using this node has this package on his machine installed if needed withouttoo much user interference Therefore you can use this part of code

1 packages lt- function(x)

2 x lt- ascharacter(matchcall()[[2]])

3 if (require(xcharacteronly=TRUE))

4 installpackages(pkgs=xrepos=httpcranr-projectorg)

5 require(xcharacteronly=TRUE)

6

7

8

9 packages(rpart)

It will firstly verify if the library is installed on the system if not it will silently install it fromthe given CRAN mirror (you can change this to another mirror or a local repository if needed)Installation will only happen the first time the node will be used

Create the metadata corresponding to the R data frame Sometimes your data is very much trans-formed compared to the original data that it is difficult to build your metadata starting from theoriginal one So you might want to change the metadata to link to the data in R independent ofthe original This is in particularly useful if you want to use R as a sort node Below is a functionthat asks for a data frame and creates modelerData and modelerDataModel accordingly

1 sendToModeler lt- function (dataFrame)

2 if(isnull(dim(dataFrame)))

3 stop(Invalid data received not a dataframe)

4 if (dim(dataFrame)[1]lt=0)

5 print(Warning modelerData has no line all fieldStorage fields set to strings)

6 getStorage lt- function(x)return(string)

7 else

8 getStorage lt- function(x)

9 x lt- unlist(x)

10 res lt- NULL

11 if x is a factor typeof will return rsquointegerrsquo so we handle this case first

Page 15 of 20

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 7: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

In this case the text output is linked to the code on line 2 and 5 rst it prints the rst 6 lines(head) of the data next it will give summary statistics for each column

The graph output are two histograms One for the tenure in months the other for the samecolumn but after redening it by dividing the original value by twelve to give the tenure in years(note the X-axis scale)

As shown in the example stream Explain modelerDatastr you can also copy exactly this samecode into a transform node and attach a table node to it After running this table node you willnot see any R output (as none is expected) That means that even though the output code hasrun no outputs will be given However the data frame of modelerData will be send back to SPSSModeler In this case you will see the value of tenure being divided by 12

322 modelerDataModel

Metadata is very important in SPSS Modeler Let us for simplicity say that within modeler meta-data is represented by the type node With metadata we mean the type of each of the variablesin the dataset (numeric 13ag String storage ) At all times modeler will know exactly all themetadata at every step in the stream

R does not handle the metadata in a similar way as SPSS Modeler We already explainedmodelerDataModel taking over the role of the type node This is done by a data frame(=dataset) of the following structure

X1 X2 Xn

fieldName region tenure agefieldLabel Geographic indicator Months with service Age in years

fieldStorage real real realfieldMeasure nominal continuous continuousfieldFormat standard standard standardfieldRole input input input

So this means that this dataset will always have 6 lines with xed names (yes in R also thelines have names) The thing with this dataset is that it is completely the responsibility of theuser to align this metadata with the appropriate data So that means if we would like to add avariable with R the user must also manually add a column in modelerDataModel to make suremodelerData correctly goes back to SPSS Modeler In the earlier example above we did not makeany changes to the modelerDataModel and it was also not needed as the metadata did not change(dividing a number by 12 will not change the metadata) Now let us continue on the previousexample But now rather than changing the value of tenure in the same data variable we willcreate another one As a result we would have to update the metadata

1 Create the vector of tenure in years

2 Rcolumn lt- modelerData$tenure123

4 Paste this vector to the right of the dataset

5 modelerData lt- cbind(modelerDataRcolumn)

6

7 create the metadata for the column to add

8 newVar lt- c(fieldName=tenureYears fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

9

10 paste the new column metadata to the existing metadata

11 modelerDataModel lt- cbind(modelerDataModelnewVar)

Running a table node downstream of this transform node will show you the new variable withthe name tenureYears There are some important things to realize in this

Page 7 of 20

IBM SPSS Modeler and R

bull fieldName and fieldStorage are the only 2 required rows that needs to be lled in for anynew column In the code we left all the other lines empty meaning they will be lled in bythe stream default For a list of available values we refer to the user guide

bull As modelerDataModel is only useful when you go back to SPSS Modeler you will generallyonly usechange this object in non-terminal R-nodes It might still be handy to use it interminal nodes if the value of the modelerDataModel is important for your output (egrun a histogram of all continuous variables)

bull When data will go back to SPSS modeler it will be the content of the data frame excludingthe column- and row names That means that even though the column in the modelerData

will be called Rcolumn the name of the column in SPSS will only be dened by the metadatawithin the row fieldName In this case it is called tenureYears

bull The only link between modelerData and modelerDataModel is the order of the columns Itwill not look by name The rst column in the data will be given the metadata of the rstcolumn of modelerDataModel In case the metadata (modelerDataModel ) does not matchthe modelerData an error is thrown The table below shows schematic how this works

modelerData modelerDataModel

R

RName1 RNamenx11 x1n

x21 x2n

xm11 xm1n

xm1 xmn

X1 Xn

fieldName Name1 NamenfieldLabel

fieldStorage xxx xxxfieldMeasure fieldFormat

fieldRole

SPSS

Name1 Namenx11 x1n

x21 x2n

xm1 xmn

Note that only the names of the modelerDataModel are used

Since this concept is very strange to standard R users We found this part the most dicult toexplain To people who know SPSS you can summarize it as modelerDataModel taking over therole of the type node

You can nd all streams and R scripts explaining modelerDataModel here

323 modelerModel

modelerModel is the R object that is stored within the R nugget This object will be populatedwithin the R model node after which you could use modelerModel within the R nugget for scoringThis very much works the same way as IBM Modeler works You ask a model node to calculatea formula after which that formula will be stored within the nugget together with the way itshould be used to calculate a scoring

You will only use this object within the R model node and nugget Note that within the Rmodel node there are 2 syntax window

Page 8 of 20

Explain modelerDataModelxml

data0001dat

data0002dat

IBM SPSS Modeler and R

R model building is to calculate whatever you want to store within modelerModel that couldbe used within your nugget calculations to score your data This can be any object withinR As any SPSS Model builder node this will be a terminal code meaning that from thiscode no data will go back to SPSS modeler unless some outputs and things that are storedwithin the modelerModel object

R model scoring is the syntax to dene how you will use the object modelerModel containingthe content you stored to it in the R model building syntax to derive the new data Apartfrom the use of modelerModel this is very similar to the R transform node

Let us start with a simple example where we would like to create a basic linear model for thevariable tenure The formula of this model should be saved in the modelerModel after which itcan be used in the scoring

1 Create the model and save it in modelerModel

2 modelerModel lt- lm(tenure ~ age + region + ed + income data= modelerData)

3

4 Add some summary of the model in the nugget

5 summary(modelerModel)

6

7 together with a histogram

8 hist(modelerModel$residuals main = residual histogram)

9

10 and the residual vs actuals scatterplot

11 plot(modelerData$tenure modelerModel$fittedvalues xlab = actual ylab = predicted )

12

13 All of these output will be stored in the modeler nugget tabs

1 Use the model to make a prediction and add it to the existing data

2 pred lt- predict(modelerModel modelerData)

3 modelerData lt- cbind(modelerDatapred)

4

5 Take care of the metadata

6 newVar lt-c(fieldName=$L-tenure fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

It is important to note that modelerModel can be lled with any type of object but will veryoften be of a model class but does not have to be In the previous example the object stored toit was clearly a (statistical) model In the next example we will just save 2 numbers within themodelerModel object Imagine we want to calculate the z-values of a certain variable In orderto create the z-values we need the mean and the standard deviation of the column We will storeboth of these within modelerModel after which we will use them in the scoring syntax1 Thisexample shows you do not need to store a statistical model within your modelerModel objectbut it really can be any R object

1 calculate mean and standard deviation

2 M lt- mean(modelerData$tenure)3 SD lt- sd(modelerData$tenure)4

5 and save it in a list called modelerModel

6 modelerModel lt- list(avg = M sDev = SD)

1 calculate z scores using the elements in modelerModel

2 zTenure lt- (modelerData$tenure - modelerModel$avg)modelerModel$sDev3 modelerData lt- cbind(modelerDatazTenure)

4

5 define new metadata column and add it

6 newVar lt- c(fieldName=zTenure fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

1Note that there is a very good reason this is not combined into an R Transform node explained in 34

Page 9 of 20

IBM SPSS Modeler and R

You can nd all streams and R scripts explaining modelerModel here

33 Some general remarks

bull Although it might seems this way you are not required to build modelerData from theexisting data within that frame modelerData will be lled with the dataset you have inModeler however nothing stops you to throw that data away in R and dene some new datacoming from another data source in R As an example imagine this link from the WeatherCompany website This will give the weather history in Brussels Belgium in the monthNovember 2015 Now we can use R code as a source node by just overwriting modelerData

and redening modelerDataModel

1 Define the link

2 linkPath lt- httpwwwwundergroundcomhistoryairportEBBR20151101

MonthlyHistoryhtmlreq_city=Brusselsampformat=1

3 Read the data as csv

4 modelerData lt- readcsv(linkPath)

5 modelerData[1] lt- asDate(modelerData[1])

6

7 Redefining modelerDataModel all are real numbers except the first column is the date

8 modelerDataModel lt- asdataframe(t(dataframe(fieldName = colnames(modelerData)

fieldLabel = fieldStorage = c(daterep(realncol(modelerData)-1))

fieldMeasure = fieldFormat = fieldRole = )))

As you can see this code does not use the old denition of the dened R objects butcompletely redenes them Placing this in a R transform node will give back this newdataset to modeler So in this way you can use this approach to create an R input node

You can nd an example here

bull Within an R model node there is place for 2 scripts The building script will be the scriptthat will be populated within the R nugget It will not be run when you run the model nodeAs a result of this these 2 scripts are independent The only thing they share is the valueof the object modelerModel which is saved within the nugget when running the buildingsyntax and picked up within the R scoring syntaxThis also means that eventual R-libraries that are required should be loaded in both scriptsTake for example a model for a random forest

1 Load the library

2 library(randomForest)

3 Create the model and save it in modelerModel

4 modelerModel lt- randomForest(tenure ~ age + region + ed + income data= modelerDatantree

=50)

1 Load the library

2 library(randomForest)

3

4 Use the model to make a prediction and add it to the existing data

5 predlt- predict(modelerModel modelerData)

6 modelerData lt- cbind(modelerDatapred)

7

8 Take care of the metadata

9 newVar lt- c(fieldName=$RF-tenurefieldLabel=fieldStorage=realfieldMeasure=fieldFormat=fieldRole=)

10 modelerDataModel lt- cbind(modelerDataModelnewVar)

bull Talking about libraries and package A package is a collection of R objects dened for acertain purpose These often are specic statistical functionalities like randomForest in theexample above A basic R installation comes with the standard packages however there aremany more packages available made available by the R community on CRAN

Page 10 of 20

Explain modelerModelxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

R as sourcexml

data0001dat

data0002dat

IBM SPSS Modeler and R

Packages needs to be installed and made locally available in libraries Once the package isinstalled on the system as a library you can load this library in any R session by the codelibrary(ltnamegt)To install a package you have several options The easiest is probably to write a codelike installpackages(randomForest) within R You will have to select a CRAN mirrorwhere this library will be downloaded from and the download will go automatically Nor-mally you will only have to do this onceAlthough possible it is not recommended to run this package installation command fromwithin SPSS The reason is that these libraries will than be saved in a temporary folder andafterwards be deleted If you still want to this through SPSS you will have to hard codethe installation path

34 Read data options

Something we ignored until now are the settings within the node under Read data options Thebasics of the R integration with Modeler can be done without the knowledge of this as it requiressome more advanced R knowledge The user guide still has a good explanation about these items

However there is one more thing that might be important For modeler version 17 and lowerthe R integration of non-terminal nodes (ie transform and nuggets) will by default be done inbatches of 1000 The reason for this was to allow these R nodes to work on hadoop and otherclustered environments As a result of this it is very important to realize that any R code thatwould span over multiple lines of data would lead to false results For a workaround for this werefer to 511

Take as an example the z scores above If we would calculate the mean and the standard de-viation of the variable in a non-terminal node it would start with running this code for the rst1000 lines of data So that leads to a specic mean deviation and corresponding z-scores How-ever the next 1000 lines a new mean and deviation would be calculated and the z scores will bebased on these

As a solution the means and standard deviations are calculated in the R model (ie a termi-nal node) over all the data and used in the R nugget to calculate the z scores

Note that this approach may lead to a very slow integration between SPSS and R in the caseof streaming R nodes in a local non-clustered environment However as from IBM SPSS Modelerversion 171 there is a default option not to use this approach of batch processing or to increasethe batch size For the lower versions there is a workaround possible if you still want to increasethis batch size or turn it o (see later)

4 Custom Dialog builder

The Custom Dialog Builder allow you to create and manage R nodes with prefilled R code to use insideIBM SPSS Modeler streams In this way users can create their own nodes You can start the CustomDialog builder in the Tools menu under rdquoCustom Dialog Builderrdquo

When opening a custom dialog builder you will see a 2 windows One of them is the custom dialogitself the other is the toolset to populate the dialog

Page 11 of 20

IBM SPSS Modeler and R

41 Tools

The tools window is a list of items you can place within your dialog This include among others the fieldchooser Check and combo boxes Text and number controls and tabs You can select any and drag themonto the dialog itselfOnce you have any item in the dialog you can select it and you will see the item properties These are theproperties of this specific item and will change dependent on the type of item it is The most importantare the identifier (the way it will be referenced within the script) and the Title (the one that will be visualin the dialog)

42 Custom dialog

The big gray window is the dialog itself For the moment it is empty as it should be populated with itemsfrom the Tools listClicking on this gray dialog will show you the dialog properties below As main items this includes thename and title of the dialog the script itself and the type and position of the created nodeWith regards to the script to be written The global rule is that you reference to the items within thedialog using their identifier between double percentages (rdquoltidentifiergtrdquoOnce you finished creating the custom node you can install it by pressing the green arrow in the toolbarYou can also save intermediate versions to the disk

43 Simple example

Let us create a custom dialog for the randomForest model created earlier in section 33 Below you willfind a step by step approachThe most important thing we have to wonder is what within this code we want flexible for the user Inthe case of this model there might be 3 things that we want flexibel the input variables the target andthe number of trees in our forest

First fill in the Custom dialog properties as indicated

Page 12 of 20

IBM SPSS Modeler and R

In our fixed example the target is tenure but a user might chooseany other field As a result we will place a field chooser on the dialogChange the properties like shown The variable filter properties allowsyou to select only categorical variables

In our fixed example the inputs are age income but a user mightchoose any other field As a result we will place a field chooser on thedialog The biggest difference is that now we can select several variablesas there might be different inputs To make it easier we will separatethese values by a + Therefore change the properties like shown

As a third custom choice we would like to add the number of trees inour forest In the original script is was 50 so we will choose this as thedefault However users may choose any integer value between 1 and1000 Add a number control on the dialog and change the properties

So now the dialog is ready and we need to add the script to it Go tothe Edit options and choose rdquoScript Templaterdquo This will bring you toan empty window for the script In this case (as we selected we wanted2 scripts) there is a tab for the building code and one for the scoringscript If the coring script is greyed out you did set the dialog propertyrdquoScore from the Modelrdquo to True

Page 13 of 20

IBM SPSS Modeler and R

Let us start with the scoring script as this is easier The only thingwhich will need to be adapted for custom input is the variable namethat will be send back to SPSS So copy the scoring code and changethe name tenure to TARGET (this is the name of the identifier of thetarget)

Fill in the code for the building script and change in a similar wayas above the values for target and intput variables together with thenumber of trees Afterwards press OK to close the script window

Being back at your Custom Dialog builder save the dialog in any appro-priate location Also deploy the dialog by clicking on the green deployarrow in the toolbar Close the Dialog builder

Back to the stream you will now see the new node in the model paletteYou can use this node within your stream

You can find the resulted cfe file here (place this in the correct location see 521) and a stream

where it is deployed

5 Tips amp tricks Some more detailed

51 R code

511 ibmspsscf70 library

Let us now have a more detailled view about what actually happens with the code First of all it is worthto check what happens when you do the R installation correctly This will install the by IBM delivered Rpackage ibmspsscf70 in the library folder of your R installation This library contains several functionsto handle the data traffic between SPSS and R

Running any R node in SPSS will not only run the code you write but it will also run some extracode behind the scenes You can see this code in the rdquoConsole outputrdquo window of the R node Lookingfor example at this tab for an R nugget you will see that your code will be something like

1 modelerModel lt- ibmspsscfoutputGetModel()

2 while(ibmspsscfdataHasMoreData())

3 modelerDataModel lt- ibmspsscfdatamodelGetDataModel()

4 modelerData lt- ibmspsscfdataGetData(rowCount=1000 missing=NA rDate=None

logicalFields=FALSE)

5

Page 14 of 20

app-extensionxml

cdb_peerjar

comspsssharedcustom_guiui_builderpeersRStatsApplierPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsApplierPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsApplierPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                    
                

comspsssharedcustom_guiui_builderpeersCDExtensionPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class CDExtensionPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static String                         RBUILDER                            private                         static String                         SCORE_SYNTAX                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            public void CDExtensionPeer()                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private String                         replaceFieldSeparator(String String)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$1                         implements javaawteventActionListener     void ActionHandler$1(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$2class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$2                         implements javaawteventActionListener     void ActionHandler$2(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsUIPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsUIPeer                         implements compaswframeworkuiswingspiExtensionObjectSwingUIPeer compaswframeworkcommonextensionspiInteractorListener compaswcorecontainerContainerSetListener                             private ActionHandler                         actionHandler                            public                         static                         final String                         UPDATE_CONTAINER = updateContainer                            public void RStatsUIPeer()                            public void                         initSwingUI(compaswframeworkuiswingExtensionObjectSwingUI compaswframeworkcommoncoreProperties compaswframeworkuiswingExtensionObjectSwingUIContext)                            public void                         disposeSwingUI()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            public void                         handleAction(String String compaswframeworkcommoncoreProperties)                            public void                         containerChanged(compaswcorecontainerContainerSetEvent)                            public void                         updateContainer(compaswcorecontainerContainerSetEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsProcessPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsProcessPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsProcessPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersCopyrightclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         abstract                         interface Copyright                         extends annotationAnnotation                             public                         abstract String                         value()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxControl                         extends javaxswingJTextArea                         implements compaswframeworkuiswingspiPropertyControl                             private                         static                         final String                         editScoreSyntax = editScoreSyntax                            private                         static                         final String                         editScoreSyntaxTitle = editScoreSyntaxTitle                            private                         static                         final String                         editScoreSyntaxWarning = editScoreSyntaxWarning                            private                         static                         final String                         editScoreSyntaxContinueBtn = editScoreSyntax_Continue                            private                         static                         final String                         editScoreSyntaxCancelBtn = editScoreSyntax_Cancel                            private javaxswingJPanel                         containerPanel                            private javaxswingJButton                         editScoreSyntaxBtn                            private compaswframeworkuiswingPropertyControlContext                         propertyControlContext                            public void ManagedScoreSyntaxControl()                            public void                         disposeControl()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public javaxswingJComponent                         getUIComponent()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            public boolean                         isControlEnabled()                            public void                         setControlEnabled(boolean)                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxPanel                         implements compaswframeworkuiswingspiManagedPanel                             public void ManagedScoreSyntaxPanel()                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ROutputFileChooserControl                         implements compaswframeworkuiswingspiPropertyControl compaswuicommoncontrolControlListener                             public                         static                         final String                         OUTPUT_TYPE = output_type                            public                         static                         final String                         OUTPUT_TYPE_GRAPH = Graph                            public                         static                         final String                         GRAPH_OUTPUT_FILE_TYPE = graph_output_file_type                            public                         static                         final String                         TEXT_OUTPUT_FILE_TYPE = text_output_file_type                            public                         static                         final String                         OUTPUT_FILE_TYPE_HTML = HTML                            public                         static                         final String                         OUTPUT_FILE_TYPE_TEXT = Text                            public                         static                         final String                         OUTPUT_FILE_TYPE_COU = Cou                            public                         static                         final String                         FILENAME = OutputFilename                            public                         static                         final String                         OUTPUT_FILE_CHOOSER_BUTTON = OutputFileChooserButton                            public                         static                         final char                         BACK_SLASH_CHAR = 92                            public                         static                         final char                         FORWARD_SLASH_CHAR = 47                            private ActionHandler                         actionHandler                            private javaxswingJLabel                         fileNameLabel                            private javaxswingtextJTextComponent                         textComponent                            private javaxswingAbstractButton                         browseButton                            private compaswuiswingcontrolDefaultControlPanel                         mainPanel                            private String                         propertyName                            private compaswframeworkuiswingPropertyControlContext                         context                            public void ROutputFileChooserControl()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            private void                         initComponent()                            private compaswframeworkuiswingSwingResourceProvider                         getSwingResourceProvider()                            private String                         getLabelFromSwingResourceProvider(String)                            private Object                         getSwingUiProperty(String)                            protected String                         getTextComponentValue()                            public javaxswingJComponent                         getUIComponent()                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public void                         setControlEnabled(boolean)                            public boolean                         isControlEnabled()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public void                         disposeControl()                            public void                         controlValueChanged(compaswuicommoncontrolControlEvent)                            private String                         removeExtension(String)                            public void                         controlSelectionChanged(compaswuicommoncontrolControlEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandlerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ActionHandler                             private compaswframeworkuiswingExtensionObjectSwingUI                         extensionObjectSwingUI                            private boolean                         bPrintAll                            private int                         printScope    String                         actionErrorMessage                            public void ActionHandler(compaswframeworkuiswingExtensionObjectSwingUI)                            public void                         handleAction(String String)                            private void                         handleExportHTML(String)                            private void                         handleExportText(String)                            private void                         handlePrint(String String)                            private void                         handlePrintView()                            private void                         handlePageSetup(String)                            private void                         handleSaveNode(String)                            private void                         handlePreviewTable(String)                            private void                         handleSaveToProject(String)                            private void                         handleHelpButton()                            private void                         exportHTMLFile(String)                            private void                         saveTextFile(String String)                            private javaxswingJPanel                         createPrintViewPanel(javautilList)                            public String                         launchSaveFileChooser(String String[])                            private void                         configureSaveFileChooser(javaxswingJFileChooser)                            private void                         configureSaveChooserCallback(compaswcoreapplicationcallbackFileSystemChooserRequest)                            private String                         getSaveDialogTitle()                            private String                         getApproveButtonText()                            private boolean                         useApplicationFileChoosers()                            private compaswframeworkuicommonUISession                         getUISession()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            private String                         getTextContainerData(String)                            private boolean                         isExportText(String)                            private boolean                         isExportHTML()                            private boolean                         isSelectNode()                            private compaswuiswingcontrolModelViewerPanel                         getModelViewerPanel()                            private String                         getDefaultFileName()                            private void                         handleUpdateContainer(String)                    
                

comspsssharedcustom_guiui_builderpeersManagedBasePanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedBasePanel                         extends javaxswingJPanel                         implements javaawtprintPrintable     void ManagedBasePanel(javaawtLayoutManager)                            public int                         print(javaawtGraphics javaawtprintPageFormat int)                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$1                     
                

comspsssharedcustom_guiui_builderpeersUIUtilitiesclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class UIUtilities                             public void UIUtilities()                            public                         static javaxswingJButton                         createButton(compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComboBox                         createComBox(compaswframeworkuiswingSwingResourceProvider String int javaxswingDefaultComboBoxModel)                            public                         static javaxswingDefaultComboBoxModel                         createEnumComboBoxModel(compaswframeworkcommoncoreEnumerationValueDescriptor)                            public                         static javaxswingAbstractButton                         addAccessKey(javaxswingAbstractButton compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingtextJTextComponent                         createTextFiled(compaswframeworkuiswingSwingResourceProvider String int)                            public                         static javaxswingJComponent                         addTooltip(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComponent                         addDescription(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static                         transient void                         fitAllComponents(javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxPreferSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setPreferSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMaxSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMaxSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMinSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMinSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                    
                

comspsssharedcustom_guiui_builderpeersManagedHTMLOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedHTMLOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedHTMLOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedModelOutputPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedModelOutputPanel                         implements compaswframeworkuiswingspiManagedPanel compaswframeworkcommonextensionspiInteractorListener                             public                         static                         final String                         BUILD_TEXT_OUTPUT = buildTextOutputDisplay                            public                         static                         final String                         BUILD_CONSOLE_OUTPUT = buildConsoleOutputDisplay                            public                         static                         final String                         BUILD_HTML_OUTPUT = buildHTMLOutputDisplay                            public                         static                         final String                         BUILD_MV_OUTPUT = buildModelViewerDisplay                            public                         static                         final String                         APPLIER_CONSOLE_OUTPUT = applierConsoleOutputDisplay                            public                         static                         final String                         BUILD_TEXT_CONTAINER_NAME = TextOutput                            public                         static                         final String                         BUILD_CONSOLE_CONTAINER_NAME = ConsoleOutput                            public                         static                         final String                         BUILD_HTML_CONTAINER_NAME = HTMLOutput                            public                         static                         final String                         BUILD_PMML_CONTAINER_NAME = PMML                            public                         static                         final String                         BUILD_STAT_XML_CONTAINER_NAME = StatXML                            public                         static                         final String                         APPLIER_CONSOLE_CONTAINER_NAME = ApplierConsoleOutput                            private ManagedTextOutputArea                         textOutput                            private ManagedTextOutputArea                         consoleOutput                            private ManagedHTMLOutputArea                         htmlOutput                            private ManagedEmptyOutputArea                         emptyOutput                            private javautilHashMap                         componentMap                            private compaswframeworkuiswingManagedPanelContext                         managedPanelContext                            public void ManagedModelOutputPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private compaswframeworkuiswingspiManagedUIElement                         setupTextOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupConsoleOutputControl(String)                            public void                         updateConsoleOutputControl(String)                            private compaswframeworkuiswingspiManagedUIElement                         setupHTMLOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupModelViewerControl()                            public String                         getTextContainerData(String)                            public                         static boolean                         exportToHTML(compaswframeworkcommoncontainerContainer String)                            public                         static javaioFile                         getHTMLFileTest(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractHTMLCompoundContainer(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractZip(String String boolean)                         throws javaioIOException                            public                         static javaioFile                         getHTMLFile(String String boolean)                         throws javaioIOException                    
                

comspsssharedcustom_guiui_builderpeersManagedTextOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedTextOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedTextOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$ButtonsActionclass

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$ButtonsAction                         implements javaawteventActionListener                             private void ManagedScoreSyntaxControl$ButtonsAction(ManagedScoreSyntaxControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersManagedEmptyOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedEmptyOutputArea                         extends javaxswingJPanel                         implements compaswframeworkuiswingspiManagedUIElement                             public void ManagedEmptyOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public javaxswingJComponent                         getComponent()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ROutputFileChooserControl$1                         implements javaawteventActionListener     void ROutputFileChooserControl$1(ROutputFileChooserControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

extensionxml

imagescopygif

imagesdeletegif

imagesfileMenugif

imagesgenerateMenugif

imageshelp_buttongif

imageslg_gm_rpng

imageslg_r_cdb_gmgif

imageslg_r_cdb_modgif

imageslg_r_cdb_outputgif

imageslg_r_cdb_processgif

imagespreview_tablegif

randomForestxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

IBM SPSS Modeler and R

6 Your R code7

8 ibmspsscfdatamodelSetDataModel(modelerDataModel)

9 ibmspsscfdataSetData(modelerData)

10

All the functions starting with ibmspsscf are functions within this library It is this part of the codethat is responsible for the transfer of the data and the metadata to R Also you will see a while loopindicating that the data will go to R in batches of 1000 The other options are the values of the data readoptions within the nodeThe last lines of the code prepares the data to be send back to SPSS Also note the closing brace whichis the end of the while loop

Now since this library will be loaded with every interaction between SPSS and R you are free to use thesefunctions within your code as well So if you would like to avoid these runs in batches of 1K and you donot have version 171 available you can start your R code with another loop to first continue filling themodelerData and only than start your actual code In practice that means that in your R node you startwith

1 while(ibmspsscfdataHasMoreData())

2

3 modelerData lt-rbind(modelerDataibmspsscfdataGetData(rowCount=100000missing=NArDate=None

logicalFields=FALSE))

4

5

6 Some more code using modelerData which is now the complete dataset

512 Some useful parts of R code

Make sure a package is installed Whenever you have created any R node using packages you have tomake sure that anyone using this node has this package on his machine installed if needed withouttoo much user interference Therefore you can use this part of code

1 packages lt- function(x)

2 x lt- ascharacter(matchcall()[[2]])

3 if (require(xcharacteronly=TRUE))

4 installpackages(pkgs=xrepos=httpcranr-projectorg)

5 require(xcharacteronly=TRUE)

6

7

8

9 packages(rpart)

It will firstly verify if the library is installed on the system if not it will silently install it fromthe given CRAN mirror (you can change this to another mirror or a local repository if needed)Installation will only happen the first time the node will be used

Create the metadata corresponding to the R data frame Sometimes your data is very much trans-formed compared to the original data that it is difficult to build your metadata starting from theoriginal one So you might want to change the metadata to link to the data in R independent ofthe original This is in particularly useful if you want to use R as a sort node Below is a functionthat asks for a data frame and creates modelerData and modelerDataModel accordingly

1 sendToModeler lt- function (dataFrame)

2 if(isnull(dim(dataFrame)))

3 stop(Invalid data received not a dataframe)

4 if (dim(dataFrame)[1]lt=0)

5 print(Warning modelerData has no line all fieldStorage fields set to strings)

6 getStorage lt- function(x)return(string)

7 else

8 getStorage lt- function(x)

9 x lt- unlist(x)

10 res lt- NULL

11 if x is a factor typeof will return rsquointegerrsquo so we handle this case first

Page 15 of 20

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 8: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

bull fieldName and fieldStorage are the only 2 required rows that needs to be lled in for anynew column In the code we left all the other lines empty meaning they will be lled in bythe stream default For a list of available values we refer to the user guide

bull As modelerDataModel is only useful when you go back to SPSS Modeler you will generallyonly usechange this object in non-terminal R-nodes It might still be handy to use it interminal nodes if the value of the modelerDataModel is important for your output (egrun a histogram of all continuous variables)

bull When data will go back to SPSS modeler it will be the content of the data frame excludingthe column- and row names That means that even though the column in the modelerData

will be called Rcolumn the name of the column in SPSS will only be dened by the metadatawithin the row fieldName In this case it is called tenureYears

bull The only link between modelerData and modelerDataModel is the order of the columns Itwill not look by name The rst column in the data will be given the metadata of the rstcolumn of modelerDataModel In case the metadata (modelerDataModel ) does not matchthe modelerData an error is thrown The table below shows schematic how this works

modelerData modelerDataModel

R

RName1 RNamenx11 x1n

x21 x2n

xm11 xm1n

xm1 xmn

X1 Xn

fieldName Name1 NamenfieldLabel

fieldStorage xxx xxxfieldMeasure fieldFormat

fieldRole

SPSS

Name1 Namenx11 x1n

x21 x2n

xm1 xmn

Note that only the names of the modelerDataModel are used

Since this concept is very strange to standard R users We found this part the most dicult toexplain To people who know SPSS you can summarize it as modelerDataModel taking over therole of the type node

You can nd all streams and R scripts explaining modelerDataModel here

323 modelerModel

modelerModel is the R object that is stored within the R nugget This object will be populatedwithin the R model node after which you could use modelerModel within the R nugget for scoringThis very much works the same way as IBM Modeler works You ask a model node to calculatea formula after which that formula will be stored within the nugget together with the way itshould be used to calculate a scoring

You will only use this object within the R model node and nugget Note that within the Rmodel node there are 2 syntax window

Page 8 of 20

Explain modelerDataModelxml

data0001dat

data0002dat

IBM SPSS Modeler and R

R model building is to calculate whatever you want to store within modelerModel that couldbe used within your nugget calculations to score your data This can be any object withinR As any SPSS Model builder node this will be a terminal code meaning that from thiscode no data will go back to SPSS modeler unless some outputs and things that are storedwithin the modelerModel object

R model scoring is the syntax to dene how you will use the object modelerModel containingthe content you stored to it in the R model building syntax to derive the new data Apartfrom the use of modelerModel this is very similar to the R transform node

Let us start with a simple example where we would like to create a basic linear model for thevariable tenure The formula of this model should be saved in the modelerModel after which itcan be used in the scoring

1 Create the model and save it in modelerModel

2 modelerModel lt- lm(tenure ~ age + region + ed + income data= modelerData)

3

4 Add some summary of the model in the nugget

5 summary(modelerModel)

6

7 together with a histogram

8 hist(modelerModel$residuals main = residual histogram)

9

10 and the residual vs actuals scatterplot

11 plot(modelerData$tenure modelerModel$fittedvalues xlab = actual ylab = predicted )

12

13 All of these output will be stored in the modeler nugget tabs

1 Use the model to make a prediction and add it to the existing data

2 pred lt- predict(modelerModel modelerData)

3 modelerData lt- cbind(modelerDatapred)

4

5 Take care of the metadata

6 newVar lt-c(fieldName=$L-tenure fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

It is important to note that modelerModel can be lled with any type of object but will veryoften be of a model class but does not have to be In the previous example the object stored toit was clearly a (statistical) model In the next example we will just save 2 numbers within themodelerModel object Imagine we want to calculate the z-values of a certain variable In orderto create the z-values we need the mean and the standard deviation of the column We will storeboth of these within modelerModel after which we will use them in the scoring syntax1 Thisexample shows you do not need to store a statistical model within your modelerModel objectbut it really can be any R object

1 calculate mean and standard deviation

2 M lt- mean(modelerData$tenure)3 SD lt- sd(modelerData$tenure)4

5 and save it in a list called modelerModel

6 modelerModel lt- list(avg = M sDev = SD)

1 calculate z scores using the elements in modelerModel

2 zTenure lt- (modelerData$tenure - modelerModel$avg)modelerModel$sDev3 modelerData lt- cbind(modelerDatazTenure)

4

5 define new metadata column and add it

6 newVar lt- c(fieldName=zTenure fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

1Note that there is a very good reason this is not combined into an R Transform node explained in 34

Page 9 of 20

IBM SPSS Modeler and R

You can nd all streams and R scripts explaining modelerModel here

33 Some general remarks

bull Although it might seems this way you are not required to build modelerData from theexisting data within that frame modelerData will be lled with the dataset you have inModeler however nothing stops you to throw that data away in R and dene some new datacoming from another data source in R As an example imagine this link from the WeatherCompany website This will give the weather history in Brussels Belgium in the monthNovember 2015 Now we can use R code as a source node by just overwriting modelerData

and redening modelerDataModel

1 Define the link

2 linkPath lt- httpwwwwundergroundcomhistoryairportEBBR20151101

MonthlyHistoryhtmlreq_city=Brusselsampformat=1

3 Read the data as csv

4 modelerData lt- readcsv(linkPath)

5 modelerData[1] lt- asDate(modelerData[1])

6

7 Redefining modelerDataModel all are real numbers except the first column is the date

8 modelerDataModel lt- asdataframe(t(dataframe(fieldName = colnames(modelerData)

fieldLabel = fieldStorage = c(daterep(realncol(modelerData)-1))

fieldMeasure = fieldFormat = fieldRole = )))

As you can see this code does not use the old denition of the dened R objects butcompletely redenes them Placing this in a R transform node will give back this newdataset to modeler So in this way you can use this approach to create an R input node

You can nd an example here

bull Within an R model node there is place for 2 scripts The building script will be the scriptthat will be populated within the R nugget It will not be run when you run the model nodeAs a result of this these 2 scripts are independent The only thing they share is the valueof the object modelerModel which is saved within the nugget when running the buildingsyntax and picked up within the R scoring syntaxThis also means that eventual R-libraries that are required should be loaded in both scriptsTake for example a model for a random forest

1 Load the library

2 library(randomForest)

3 Create the model and save it in modelerModel

4 modelerModel lt- randomForest(tenure ~ age + region + ed + income data= modelerDatantree

=50)

1 Load the library

2 library(randomForest)

3

4 Use the model to make a prediction and add it to the existing data

5 predlt- predict(modelerModel modelerData)

6 modelerData lt- cbind(modelerDatapred)

7

8 Take care of the metadata

9 newVar lt- c(fieldName=$RF-tenurefieldLabel=fieldStorage=realfieldMeasure=fieldFormat=fieldRole=)

10 modelerDataModel lt- cbind(modelerDataModelnewVar)

bull Talking about libraries and package A package is a collection of R objects dened for acertain purpose These often are specic statistical functionalities like randomForest in theexample above A basic R installation comes with the standard packages however there aremany more packages available made available by the R community on CRAN

Page 10 of 20

Explain modelerModelxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

R as sourcexml

data0001dat

data0002dat

IBM SPSS Modeler and R

Packages needs to be installed and made locally available in libraries Once the package isinstalled on the system as a library you can load this library in any R session by the codelibrary(ltnamegt)To install a package you have several options The easiest is probably to write a codelike installpackages(randomForest) within R You will have to select a CRAN mirrorwhere this library will be downloaded from and the download will go automatically Nor-mally you will only have to do this onceAlthough possible it is not recommended to run this package installation command fromwithin SPSS The reason is that these libraries will than be saved in a temporary folder andafterwards be deleted If you still want to this through SPSS you will have to hard codethe installation path

34 Read data options

Something we ignored until now are the settings within the node under Read data options Thebasics of the R integration with Modeler can be done without the knowledge of this as it requiressome more advanced R knowledge The user guide still has a good explanation about these items

However there is one more thing that might be important For modeler version 17 and lowerthe R integration of non-terminal nodes (ie transform and nuggets) will by default be done inbatches of 1000 The reason for this was to allow these R nodes to work on hadoop and otherclustered environments As a result of this it is very important to realize that any R code thatwould span over multiple lines of data would lead to false results For a workaround for this werefer to 511

Take as an example the z scores above If we would calculate the mean and the standard de-viation of the variable in a non-terminal node it would start with running this code for the rst1000 lines of data So that leads to a specic mean deviation and corresponding z-scores How-ever the next 1000 lines a new mean and deviation would be calculated and the z scores will bebased on these

As a solution the means and standard deviations are calculated in the R model (ie a termi-nal node) over all the data and used in the R nugget to calculate the z scores

Note that this approach may lead to a very slow integration between SPSS and R in the caseof streaming R nodes in a local non-clustered environment However as from IBM SPSS Modelerversion 171 there is a default option not to use this approach of batch processing or to increasethe batch size For the lower versions there is a workaround possible if you still want to increasethis batch size or turn it o (see later)

4 Custom Dialog builder

The Custom Dialog Builder allow you to create and manage R nodes with prefilled R code to use insideIBM SPSS Modeler streams In this way users can create their own nodes You can start the CustomDialog builder in the Tools menu under rdquoCustom Dialog Builderrdquo

When opening a custom dialog builder you will see a 2 windows One of them is the custom dialogitself the other is the toolset to populate the dialog

Page 11 of 20

IBM SPSS Modeler and R

41 Tools

The tools window is a list of items you can place within your dialog This include among others the fieldchooser Check and combo boxes Text and number controls and tabs You can select any and drag themonto the dialog itselfOnce you have any item in the dialog you can select it and you will see the item properties These are theproperties of this specific item and will change dependent on the type of item it is The most importantare the identifier (the way it will be referenced within the script) and the Title (the one that will be visualin the dialog)

42 Custom dialog

The big gray window is the dialog itself For the moment it is empty as it should be populated with itemsfrom the Tools listClicking on this gray dialog will show you the dialog properties below As main items this includes thename and title of the dialog the script itself and the type and position of the created nodeWith regards to the script to be written The global rule is that you reference to the items within thedialog using their identifier between double percentages (rdquoltidentifiergtrdquoOnce you finished creating the custom node you can install it by pressing the green arrow in the toolbarYou can also save intermediate versions to the disk

43 Simple example

Let us create a custom dialog for the randomForest model created earlier in section 33 Below you willfind a step by step approachThe most important thing we have to wonder is what within this code we want flexible for the user Inthe case of this model there might be 3 things that we want flexibel the input variables the target andthe number of trees in our forest

First fill in the Custom dialog properties as indicated

Page 12 of 20

IBM SPSS Modeler and R

In our fixed example the target is tenure but a user might chooseany other field As a result we will place a field chooser on the dialogChange the properties like shown The variable filter properties allowsyou to select only categorical variables

In our fixed example the inputs are age income but a user mightchoose any other field As a result we will place a field chooser on thedialog The biggest difference is that now we can select several variablesas there might be different inputs To make it easier we will separatethese values by a + Therefore change the properties like shown

As a third custom choice we would like to add the number of trees inour forest In the original script is was 50 so we will choose this as thedefault However users may choose any integer value between 1 and1000 Add a number control on the dialog and change the properties

So now the dialog is ready and we need to add the script to it Go tothe Edit options and choose rdquoScript Templaterdquo This will bring you toan empty window for the script In this case (as we selected we wanted2 scripts) there is a tab for the building code and one for the scoringscript If the coring script is greyed out you did set the dialog propertyrdquoScore from the Modelrdquo to True

Page 13 of 20

IBM SPSS Modeler and R

Let us start with the scoring script as this is easier The only thingwhich will need to be adapted for custom input is the variable namethat will be send back to SPSS So copy the scoring code and changethe name tenure to TARGET (this is the name of the identifier of thetarget)

Fill in the code for the building script and change in a similar wayas above the values for target and intput variables together with thenumber of trees Afterwards press OK to close the script window

Being back at your Custom Dialog builder save the dialog in any appro-priate location Also deploy the dialog by clicking on the green deployarrow in the toolbar Close the Dialog builder

Back to the stream you will now see the new node in the model paletteYou can use this node within your stream

You can find the resulted cfe file here (place this in the correct location see 521) and a stream

where it is deployed

5 Tips amp tricks Some more detailed

51 R code

511 ibmspsscf70 library

Let us now have a more detailled view about what actually happens with the code First of all it is worthto check what happens when you do the R installation correctly This will install the by IBM delivered Rpackage ibmspsscf70 in the library folder of your R installation This library contains several functionsto handle the data traffic between SPSS and R

Running any R node in SPSS will not only run the code you write but it will also run some extracode behind the scenes You can see this code in the rdquoConsole outputrdquo window of the R node Lookingfor example at this tab for an R nugget you will see that your code will be something like

1 modelerModel lt- ibmspsscfoutputGetModel()

2 while(ibmspsscfdataHasMoreData())

3 modelerDataModel lt- ibmspsscfdatamodelGetDataModel()

4 modelerData lt- ibmspsscfdataGetData(rowCount=1000 missing=NA rDate=None

logicalFields=FALSE)

5

Page 14 of 20

app-extensionxml

cdb_peerjar

comspsssharedcustom_guiui_builderpeersRStatsApplierPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsApplierPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsApplierPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                    
                

comspsssharedcustom_guiui_builderpeersCDExtensionPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class CDExtensionPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static String                         RBUILDER                            private                         static String                         SCORE_SYNTAX                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            public void CDExtensionPeer()                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private String                         replaceFieldSeparator(String String)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$1                         implements javaawteventActionListener     void ActionHandler$1(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$2class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$2                         implements javaawteventActionListener     void ActionHandler$2(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsUIPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsUIPeer                         implements compaswframeworkuiswingspiExtensionObjectSwingUIPeer compaswframeworkcommonextensionspiInteractorListener compaswcorecontainerContainerSetListener                             private ActionHandler                         actionHandler                            public                         static                         final String                         UPDATE_CONTAINER = updateContainer                            public void RStatsUIPeer()                            public void                         initSwingUI(compaswframeworkuiswingExtensionObjectSwingUI compaswframeworkcommoncoreProperties compaswframeworkuiswingExtensionObjectSwingUIContext)                            public void                         disposeSwingUI()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            public void                         handleAction(String String compaswframeworkcommoncoreProperties)                            public void                         containerChanged(compaswcorecontainerContainerSetEvent)                            public void                         updateContainer(compaswcorecontainerContainerSetEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsProcessPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsProcessPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsProcessPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersCopyrightclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         abstract                         interface Copyright                         extends annotationAnnotation                             public                         abstract String                         value()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxControl                         extends javaxswingJTextArea                         implements compaswframeworkuiswingspiPropertyControl                             private                         static                         final String                         editScoreSyntax = editScoreSyntax                            private                         static                         final String                         editScoreSyntaxTitle = editScoreSyntaxTitle                            private                         static                         final String                         editScoreSyntaxWarning = editScoreSyntaxWarning                            private                         static                         final String                         editScoreSyntaxContinueBtn = editScoreSyntax_Continue                            private                         static                         final String                         editScoreSyntaxCancelBtn = editScoreSyntax_Cancel                            private javaxswingJPanel                         containerPanel                            private javaxswingJButton                         editScoreSyntaxBtn                            private compaswframeworkuiswingPropertyControlContext                         propertyControlContext                            public void ManagedScoreSyntaxControl()                            public void                         disposeControl()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public javaxswingJComponent                         getUIComponent()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            public boolean                         isControlEnabled()                            public void                         setControlEnabled(boolean)                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxPanel                         implements compaswframeworkuiswingspiManagedPanel                             public void ManagedScoreSyntaxPanel()                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ROutputFileChooserControl                         implements compaswframeworkuiswingspiPropertyControl compaswuicommoncontrolControlListener                             public                         static                         final String                         OUTPUT_TYPE = output_type                            public                         static                         final String                         OUTPUT_TYPE_GRAPH = Graph                            public                         static                         final String                         GRAPH_OUTPUT_FILE_TYPE = graph_output_file_type                            public                         static                         final String                         TEXT_OUTPUT_FILE_TYPE = text_output_file_type                            public                         static                         final String                         OUTPUT_FILE_TYPE_HTML = HTML                            public                         static                         final String                         OUTPUT_FILE_TYPE_TEXT = Text                            public                         static                         final String                         OUTPUT_FILE_TYPE_COU = Cou                            public                         static                         final String                         FILENAME = OutputFilename                            public                         static                         final String                         OUTPUT_FILE_CHOOSER_BUTTON = OutputFileChooserButton                            public                         static                         final char                         BACK_SLASH_CHAR = 92                            public                         static                         final char                         FORWARD_SLASH_CHAR = 47                            private ActionHandler                         actionHandler                            private javaxswingJLabel                         fileNameLabel                            private javaxswingtextJTextComponent                         textComponent                            private javaxswingAbstractButton                         browseButton                            private compaswuiswingcontrolDefaultControlPanel                         mainPanel                            private String                         propertyName                            private compaswframeworkuiswingPropertyControlContext                         context                            public void ROutputFileChooserControl()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            private void                         initComponent()                            private compaswframeworkuiswingSwingResourceProvider                         getSwingResourceProvider()                            private String                         getLabelFromSwingResourceProvider(String)                            private Object                         getSwingUiProperty(String)                            protected String                         getTextComponentValue()                            public javaxswingJComponent                         getUIComponent()                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public void                         setControlEnabled(boolean)                            public boolean                         isControlEnabled()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public void                         disposeControl()                            public void                         controlValueChanged(compaswuicommoncontrolControlEvent)                            private String                         removeExtension(String)                            public void                         controlSelectionChanged(compaswuicommoncontrolControlEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandlerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ActionHandler                             private compaswframeworkuiswingExtensionObjectSwingUI                         extensionObjectSwingUI                            private boolean                         bPrintAll                            private int                         printScope    String                         actionErrorMessage                            public void ActionHandler(compaswframeworkuiswingExtensionObjectSwingUI)                            public void                         handleAction(String String)                            private void                         handleExportHTML(String)                            private void                         handleExportText(String)                            private void                         handlePrint(String String)                            private void                         handlePrintView()                            private void                         handlePageSetup(String)                            private void                         handleSaveNode(String)                            private void                         handlePreviewTable(String)                            private void                         handleSaveToProject(String)                            private void                         handleHelpButton()                            private void                         exportHTMLFile(String)                            private void                         saveTextFile(String String)                            private javaxswingJPanel                         createPrintViewPanel(javautilList)                            public String                         launchSaveFileChooser(String String[])                            private void                         configureSaveFileChooser(javaxswingJFileChooser)                            private void                         configureSaveChooserCallback(compaswcoreapplicationcallbackFileSystemChooserRequest)                            private String                         getSaveDialogTitle()                            private String                         getApproveButtonText()                            private boolean                         useApplicationFileChoosers()                            private compaswframeworkuicommonUISession                         getUISession()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            private String                         getTextContainerData(String)                            private boolean                         isExportText(String)                            private boolean                         isExportHTML()                            private boolean                         isSelectNode()                            private compaswuiswingcontrolModelViewerPanel                         getModelViewerPanel()                            private String                         getDefaultFileName()                            private void                         handleUpdateContainer(String)                    
                

comspsssharedcustom_guiui_builderpeersManagedBasePanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedBasePanel                         extends javaxswingJPanel                         implements javaawtprintPrintable     void ManagedBasePanel(javaawtLayoutManager)                            public int                         print(javaawtGraphics javaawtprintPageFormat int)                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$1                     
                

comspsssharedcustom_guiui_builderpeersUIUtilitiesclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class UIUtilities                             public void UIUtilities()                            public                         static javaxswingJButton                         createButton(compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComboBox                         createComBox(compaswframeworkuiswingSwingResourceProvider String int javaxswingDefaultComboBoxModel)                            public                         static javaxswingDefaultComboBoxModel                         createEnumComboBoxModel(compaswframeworkcommoncoreEnumerationValueDescriptor)                            public                         static javaxswingAbstractButton                         addAccessKey(javaxswingAbstractButton compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingtextJTextComponent                         createTextFiled(compaswframeworkuiswingSwingResourceProvider String int)                            public                         static javaxswingJComponent                         addTooltip(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComponent                         addDescription(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static                         transient void                         fitAllComponents(javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxPreferSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setPreferSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMaxSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMaxSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMinSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMinSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                    
                

comspsssharedcustom_guiui_builderpeersManagedHTMLOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedHTMLOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedHTMLOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedModelOutputPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedModelOutputPanel                         implements compaswframeworkuiswingspiManagedPanel compaswframeworkcommonextensionspiInteractorListener                             public                         static                         final String                         BUILD_TEXT_OUTPUT = buildTextOutputDisplay                            public                         static                         final String                         BUILD_CONSOLE_OUTPUT = buildConsoleOutputDisplay                            public                         static                         final String                         BUILD_HTML_OUTPUT = buildHTMLOutputDisplay                            public                         static                         final String                         BUILD_MV_OUTPUT = buildModelViewerDisplay                            public                         static                         final String                         APPLIER_CONSOLE_OUTPUT = applierConsoleOutputDisplay                            public                         static                         final String                         BUILD_TEXT_CONTAINER_NAME = TextOutput                            public                         static                         final String                         BUILD_CONSOLE_CONTAINER_NAME = ConsoleOutput                            public                         static                         final String                         BUILD_HTML_CONTAINER_NAME = HTMLOutput                            public                         static                         final String                         BUILD_PMML_CONTAINER_NAME = PMML                            public                         static                         final String                         BUILD_STAT_XML_CONTAINER_NAME = StatXML                            public                         static                         final String                         APPLIER_CONSOLE_CONTAINER_NAME = ApplierConsoleOutput                            private ManagedTextOutputArea                         textOutput                            private ManagedTextOutputArea                         consoleOutput                            private ManagedHTMLOutputArea                         htmlOutput                            private ManagedEmptyOutputArea                         emptyOutput                            private javautilHashMap                         componentMap                            private compaswframeworkuiswingManagedPanelContext                         managedPanelContext                            public void ManagedModelOutputPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private compaswframeworkuiswingspiManagedUIElement                         setupTextOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupConsoleOutputControl(String)                            public void                         updateConsoleOutputControl(String)                            private compaswframeworkuiswingspiManagedUIElement                         setupHTMLOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupModelViewerControl()                            public String                         getTextContainerData(String)                            public                         static boolean                         exportToHTML(compaswframeworkcommoncontainerContainer String)                            public                         static javaioFile                         getHTMLFileTest(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractHTMLCompoundContainer(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractZip(String String boolean)                         throws javaioIOException                            public                         static javaioFile                         getHTMLFile(String String boolean)                         throws javaioIOException                    
                

comspsssharedcustom_guiui_builderpeersManagedTextOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedTextOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedTextOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$ButtonsActionclass

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$ButtonsAction                         implements javaawteventActionListener                             private void ManagedScoreSyntaxControl$ButtonsAction(ManagedScoreSyntaxControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersManagedEmptyOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedEmptyOutputArea                         extends javaxswingJPanel                         implements compaswframeworkuiswingspiManagedUIElement                             public void ManagedEmptyOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public javaxswingJComponent                         getComponent()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ROutputFileChooserControl$1                         implements javaawteventActionListener     void ROutputFileChooserControl$1(ROutputFileChooserControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

extensionxml

imagescopygif

imagesdeletegif

imagesfileMenugif

imagesgenerateMenugif

imageshelp_buttongif

imageslg_gm_rpng

imageslg_r_cdb_gmgif

imageslg_r_cdb_modgif

imageslg_r_cdb_outputgif

imageslg_r_cdb_processgif

imagespreview_tablegif

randomForestxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

IBM SPSS Modeler and R

6 Your R code7

8 ibmspsscfdatamodelSetDataModel(modelerDataModel)

9 ibmspsscfdataSetData(modelerData)

10

All the functions starting with ibmspsscf are functions within this library It is this part of the codethat is responsible for the transfer of the data and the metadata to R Also you will see a while loopindicating that the data will go to R in batches of 1000 The other options are the values of the data readoptions within the nodeThe last lines of the code prepares the data to be send back to SPSS Also note the closing brace whichis the end of the while loop

Now since this library will be loaded with every interaction between SPSS and R you are free to use thesefunctions within your code as well So if you would like to avoid these runs in batches of 1K and you donot have version 171 available you can start your R code with another loop to first continue filling themodelerData and only than start your actual code In practice that means that in your R node you startwith

1 while(ibmspsscfdataHasMoreData())

2

3 modelerData lt-rbind(modelerDataibmspsscfdataGetData(rowCount=100000missing=NArDate=None

logicalFields=FALSE))

4

5

6 Some more code using modelerData which is now the complete dataset

512 Some useful parts of R code

Make sure a package is installed Whenever you have created any R node using packages you have tomake sure that anyone using this node has this package on his machine installed if needed withouttoo much user interference Therefore you can use this part of code

1 packages lt- function(x)

2 x lt- ascharacter(matchcall()[[2]])

3 if (require(xcharacteronly=TRUE))

4 installpackages(pkgs=xrepos=httpcranr-projectorg)

5 require(xcharacteronly=TRUE)

6

7

8

9 packages(rpart)

It will firstly verify if the library is installed on the system if not it will silently install it fromthe given CRAN mirror (you can change this to another mirror or a local repository if needed)Installation will only happen the first time the node will be used

Create the metadata corresponding to the R data frame Sometimes your data is very much trans-formed compared to the original data that it is difficult to build your metadata starting from theoriginal one So you might want to change the metadata to link to the data in R independent ofthe original This is in particularly useful if you want to use R as a sort node Below is a functionthat asks for a data frame and creates modelerData and modelerDataModel accordingly

1 sendToModeler lt- function (dataFrame)

2 if(isnull(dim(dataFrame)))

3 stop(Invalid data received not a dataframe)

4 if (dim(dataFrame)[1]lt=0)

5 print(Warning modelerData has no line all fieldStorage fields set to strings)

6 getStorage lt- function(x)return(string)

7 else

8 getStorage lt- function(x)

9 x lt- unlist(x)

10 res lt- NULL

11 if x is a factor typeof will return rsquointegerrsquo so we handle this case first

Page 15 of 20

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 9: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

R model building is to calculate whatever you want to store within modelerModel that couldbe used within your nugget calculations to score your data This can be any object withinR As any SPSS Model builder node this will be a terminal code meaning that from thiscode no data will go back to SPSS modeler unless some outputs and things that are storedwithin the modelerModel object

R model scoring is the syntax to dene how you will use the object modelerModel containingthe content you stored to it in the R model building syntax to derive the new data Apartfrom the use of modelerModel this is very similar to the R transform node

Let us start with a simple example where we would like to create a basic linear model for thevariable tenure The formula of this model should be saved in the modelerModel after which itcan be used in the scoring

1 Create the model and save it in modelerModel

2 modelerModel lt- lm(tenure ~ age + region + ed + income data= modelerData)

3

4 Add some summary of the model in the nugget

5 summary(modelerModel)

6

7 together with a histogram

8 hist(modelerModel$residuals main = residual histogram)

9

10 and the residual vs actuals scatterplot

11 plot(modelerData$tenure modelerModel$fittedvalues xlab = actual ylab = predicted )

12

13 All of these output will be stored in the modeler nugget tabs

1 Use the model to make a prediction and add it to the existing data

2 pred lt- predict(modelerModel modelerData)

3 modelerData lt- cbind(modelerDatapred)

4

5 Take care of the metadata

6 newVar lt-c(fieldName=$L-tenure fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

It is important to note that modelerModel can be lled with any type of object but will veryoften be of a model class but does not have to be In the previous example the object stored toit was clearly a (statistical) model In the next example we will just save 2 numbers within themodelerModel object Imagine we want to calculate the z-values of a certain variable In orderto create the z-values we need the mean and the standard deviation of the column We will storeboth of these within modelerModel after which we will use them in the scoring syntax1 Thisexample shows you do not need to store a statistical model within your modelerModel objectbut it really can be any R object

1 calculate mean and standard deviation

2 M lt- mean(modelerData$tenure)3 SD lt- sd(modelerData$tenure)4

5 and save it in a list called modelerModel

6 modelerModel lt- list(avg = M sDev = SD)

1 calculate z scores using the elements in modelerModel

2 zTenure lt- (modelerData$tenure - modelerModel$avg)modelerModel$sDev3 modelerData lt- cbind(modelerDatazTenure)

4

5 define new metadata column and add it

6 newVar lt- c(fieldName=zTenure fieldLabel=fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

7 modelerDataModel lt- cbind(modelerDataModelnewVar)

1Note that there is a very good reason this is not combined into an R Transform node explained in 34

Page 9 of 20

IBM SPSS Modeler and R

You can nd all streams and R scripts explaining modelerModel here

33 Some general remarks

bull Although it might seems this way you are not required to build modelerData from theexisting data within that frame modelerData will be lled with the dataset you have inModeler however nothing stops you to throw that data away in R and dene some new datacoming from another data source in R As an example imagine this link from the WeatherCompany website This will give the weather history in Brussels Belgium in the monthNovember 2015 Now we can use R code as a source node by just overwriting modelerData

and redening modelerDataModel

1 Define the link

2 linkPath lt- httpwwwwundergroundcomhistoryairportEBBR20151101

MonthlyHistoryhtmlreq_city=Brusselsampformat=1

3 Read the data as csv

4 modelerData lt- readcsv(linkPath)

5 modelerData[1] lt- asDate(modelerData[1])

6

7 Redefining modelerDataModel all are real numbers except the first column is the date

8 modelerDataModel lt- asdataframe(t(dataframe(fieldName = colnames(modelerData)

fieldLabel = fieldStorage = c(daterep(realncol(modelerData)-1))

fieldMeasure = fieldFormat = fieldRole = )))

As you can see this code does not use the old denition of the dened R objects butcompletely redenes them Placing this in a R transform node will give back this newdataset to modeler So in this way you can use this approach to create an R input node

You can nd an example here

bull Within an R model node there is place for 2 scripts The building script will be the scriptthat will be populated within the R nugget It will not be run when you run the model nodeAs a result of this these 2 scripts are independent The only thing they share is the valueof the object modelerModel which is saved within the nugget when running the buildingsyntax and picked up within the R scoring syntaxThis also means that eventual R-libraries that are required should be loaded in both scriptsTake for example a model for a random forest

1 Load the library

2 library(randomForest)

3 Create the model and save it in modelerModel

4 modelerModel lt- randomForest(tenure ~ age + region + ed + income data= modelerDatantree

=50)

1 Load the library

2 library(randomForest)

3

4 Use the model to make a prediction and add it to the existing data

5 predlt- predict(modelerModel modelerData)

6 modelerData lt- cbind(modelerDatapred)

7

8 Take care of the metadata

9 newVar lt- c(fieldName=$RF-tenurefieldLabel=fieldStorage=realfieldMeasure=fieldFormat=fieldRole=)

10 modelerDataModel lt- cbind(modelerDataModelnewVar)

bull Talking about libraries and package A package is a collection of R objects dened for acertain purpose These often are specic statistical functionalities like randomForest in theexample above A basic R installation comes with the standard packages however there aremany more packages available made available by the R community on CRAN

Page 10 of 20

Explain modelerModelxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

R as sourcexml

data0001dat

data0002dat

IBM SPSS Modeler and R

Packages needs to be installed and made locally available in libraries Once the package isinstalled on the system as a library you can load this library in any R session by the codelibrary(ltnamegt)To install a package you have several options The easiest is probably to write a codelike installpackages(randomForest) within R You will have to select a CRAN mirrorwhere this library will be downloaded from and the download will go automatically Nor-mally you will only have to do this onceAlthough possible it is not recommended to run this package installation command fromwithin SPSS The reason is that these libraries will than be saved in a temporary folder andafterwards be deleted If you still want to this through SPSS you will have to hard codethe installation path

34 Read data options

Something we ignored until now are the settings within the node under Read data options Thebasics of the R integration with Modeler can be done without the knowledge of this as it requiressome more advanced R knowledge The user guide still has a good explanation about these items

However there is one more thing that might be important For modeler version 17 and lowerthe R integration of non-terminal nodes (ie transform and nuggets) will by default be done inbatches of 1000 The reason for this was to allow these R nodes to work on hadoop and otherclustered environments As a result of this it is very important to realize that any R code thatwould span over multiple lines of data would lead to false results For a workaround for this werefer to 511

Take as an example the z scores above If we would calculate the mean and the standard de-viation of the variable in a non-terminal node it would start with running this code for the rst1000 lines of data So that leads to a specic mean deviation and corresponding z-scores How-ever the next 1000 lines a new mean and deviation would be calculated and the z scores will bebased on these

As a solution the means and standard deviations are calculated in the R model (ie a termi-nal node) over all the data and used in the R nugget to calculate the z scores

Note that this approach may lead to a very slow integration between SPSS and R in the caseof streaming R nodes in a local non-clustered environment However as from IBM SPSS Modelerversion 171 there is a default option not to use this approach of batch processing or to increasethe batch size For the lower versions there is a workaround possible if you still want to increasethis batch size or turn it o (see later)

4 Custom Dialog builder

The Custom Dialog Builder allow you to create and manage R nodes with prefilled R code to use insideIBM SPSS Modeler streams In this way users can create their own nodes You can start the CustomDialog builder in the Tools menu under rdquoCustom Dialog Builderrdquo

When opening a custom dialog builder you will see a 2 windows One of them is the custom dialogitself the other is the toolset to populate the dialog

Page 11 of 20

IBM SPSS Modeler and R

41 Tools

The tools window is a list of items you can place within your dialog This include among others the fieldchooser Check and combo boxes Text and number controls and tabs You can select any and drag themonto the dialog itselfOnce you have any item in the dialog you can select it and you will see the item properties These are theproperties of this specific item and will change dependent on the type of item it is The most importantare the identifier (the way it will be referenced within the script) and the Title (the one that will be visualin the dialog)

42 Custom dialog

The big gray window is the dialog itself For the moment it is empty as it should be populated with itemsfrom the Tools listClicking on this gray dialog will show you the dialog properties below As main items this includes thename and title of the dialog the script itself and the type and position of the created nodeWith regards to the script to be written The global rule is that you reference to the items within thedialog using their identifier between double percentages (rdquoltidentifiergtrdquoOnce you finished creating the custom node you can install it by pressing the green arrow in the toolbarYou can also save intermediate versions to the disk

43 Simple example

Let us create a custom dialog for the randomForest model created earlier in section 33 Below you willfind a step by step approachThe most important thing we have to wonder is what within this code we want flexible for the user Inthe case of this model there might be 3 things that we want flexibel the input variables the target andthe number of trees in our forest

First fill in the Custom dialog properties as indicated

Page 12 of 20

IBM SPSS Modeler and R

In our fixed example the target is tenure but a user might chooseany other field As a result we will place a field chooser on the dialogChange the properties like shown The variable filter properties allowsyou to select only categorical variables

In our fixed example the inputs are age income but a user mightchoose any other field As a result we will place a field chooser on thedialog The biggest difference is that now we can select several variablesas there might be different inputs To make it easier we will separatethese values by a + Therefore change the properties like shown

As a third custom choice we would like to add the number of trees inour forest In the original script is was 50 so we will choose this as thedefault However users may choose any integer value between 1 and1000 Add a number control on the dialog and change the properties

So now the dialog is ready and we need to add the script to it Go tothe Edit options and choose rdquoScript Templaterdquo This will bring you toan empty window for the script In this case (as we selected we wanted2 scripts) there is a tab for the building code and one for the scoringscript If the coring script is greyed out you did set the dialog propertyrdquoScore from the Modelrdquo to True

Page 13 of 20

IBM SPSS Modeler and R

Let us start with the scoring script as this is easier The only thingwhich will need to be adapted for custom input is the variable namethat will be send back to SPSS So copy the scoring code and changethe name tenure to TARGET (this is the name of the identifier of thetarget)

Fill in the code for the building script and change in a similar wayas above the values for target and intput variables together with thenumber of trees Afterwards press OK to close the script window

Being back at your Custom Dialog builder save the dialog in any appro-priate location Also deploy the dialog by clicking on the green deployarrow in the toolbar Close the Dialog builder

Back to the stream you will now see the new node in the model paletteYou can use this node within your stream

You can find the resulted cfe file here (place this in the correct location see 521) and a stream

where it is deployed

5 Tips amp tricks Some more detailed

51 R code

511 ibmspsscf70 library

Let us now have a more detailled view about what actually happens with the code First of all it is worthto check what happens when you do the R installation correctly This will install the by IBM delivered Rpackage ibmspsscf70 in the library folder of your R installation This library contains several functionsto handle the data traffic between SPSS and R

Running any R node in SPSS will not only run the code you write but it will also run some extracode behind the scenes You can see this code in the rdquoConsole outputrdquo window of the R node Lookingfor example at this tab for an R nugget you will see that your code will be something like

1 modelerModel lt- ibmspsscfoutputGetModel()

2 while(ibmspsscfdataHasMoreData())

3 modelerDataModel lt- ibmspsscfdatamodelGetDataModel()

4 modelerData lt- ibmspsscfdataGetData(rowCount=1000 missing=NA rDate=None

logicalFields=FALSE)

5

Page 14 of 20

app-extensionxml

cdb_peerjar

comspsssharedcustom_guiui_builderpeersRStatsApplierPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsApplierPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsApplierPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                    
                

comspsssharedcustom_guiui_builderpeersCDExtensionPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class CDExtensionPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static String                         RBUILDER                            private                         static String                         SCORE_SYNTAX                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            public void CDExtensionPeer()                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private String                         replaceFieldSeparator(String String)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$1                         implements javaawteventActionListener     void ActionHandler$1(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$2class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$2                         implements javaawteventActionListener     void ActionHandler$2(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsUIPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsUIPeer                         implements compaswframeworkuiswingspiExtensionObjectSwingUIPeer compaswframeworkcommonextensionspiInteractorListener compaswcorecontainerContainerSetListener                             private ActionHandler                         actionHandler                            public                         static                         final String                         UPDATE_CONTAINER = updateContainer                            public void RStatsUIPeer()                            public void                         initSwingUI(compaswframeworkuiswingExtensionObjectSwingUI compaswframeworkcommoncoreProperties compaswframeworkuiswingExtensionObjectSwingUIContext)                            public void                         disposeSwingUI()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            public void                         handleAction(String String compaswframeworkcommoncoreProperties)                            public void                         containerChanged(compaswcorecontainerContainerSetEvent)                            public void                         updateContainer(compaswcorecontainerContainerSetEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsProcessPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsProcessPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsProcessPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersCopyrightclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         abstract                         interface Copyright                         extends annotationAnnotation                             public                         abstract String                         value()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxControl                         extends javaxswingJTextArea                         implements compaswframeworkuiswingspiPropertyControl                             private                         static                         final String                         editScoreSyntax = editScoreSyntax                            private                         static                         final String                         editScoreSyntaxTitle = editScoreSyntaxTitle                            private                         static                         final String                         editScoreSyntaxWarning = editScoreSyntaxWarning                            private                         static                         final String                         editScoreSyntaxContinueBtn = editScoreSyntax_Continue                            private                         static                         final String                         editScoreSyntaxCancelBtn = editScoreSyntax_Cancel                            private javaxswingJPanel                         containerPanel                            private javaxswingJButton                         editScoreSyntaxBtn                            private compaswframeworkuiswingPropertyControlContext                         propertyControlContext                            public void ManagedScoreSyntaxControl()                            public void                         disposeControl()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public javaxswingJComponent                         getUIComponent()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            public boolean                         isControlEnabled()                            public void                         setControlEnabled(boolean)                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxPanel                         implements compaswframeworkuiswingspiManagedPanel                             public void ManagedScoreSyntaxPanel()                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ROutputFileChooserControl                         implements compaswframeworkuiswingspiPropertyControl compaswuicommoncontrolControlListener                             public                         static                         final String                         OUTPUT_TYPE = output_type                            public                         static                         final String                         OUTPUT_TYPE_GRAPH = Graph                            public                         static                         final String                         GRAPH_OUTPUT_FILE_TYPE = graph_output_file_type                            public                         static                         final String                         TEXT_OUTPUT_FILE_TYPE = text_output_file_type                            public                         static                         final String                         OUTPUT_FILE_TYPE_HTML = HTML                            public                         static                         final String                         OUTPUT_FILE_TYPE_TEXT = Text                            public                         static                         final String                         OUTPUT_FILE_TYPE_COU = Cou                            public                         static                         final String                         FILENAME = OutputFilename                            public                         static                         final String                         OUTPUT_FILE_CHOOSER_BUTTON = OutputFileChooserButton                            public                         static                         final char                         BACK_SLASH_CHAR = 92                            public                         static                         final char                         FORWARD_SLASH_CHAR = 47                            private ActionHandler                         actionHandler                            private javaxswingJLabel                         fileNameLabel                            private javaxswingtextJTextComponent                         textComponent                            private javaxswingAbstractButton                         browseButton                            private compaswuiswingcontrolDefaultControlPanel                         mainPanel                            private String                         propertyName                            private compaswframeworkuiswingPropertyControlContext                         context                            public void ROutputFileChooserControl()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            private void                         initComponent()                            private compaswframeworkuiswingSwingResourceProvider                         getSwingResourceProvider()                            private String                         getLabelFromSwingResourceProvider(String)                            private Object                         getSwingUiProperty(String)                            protected String                         getTextComponentValue()                            public javaxswingJComponent                         getUIComponent()                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public void                         setControlEnabled(boolean)                            public boolean                         isControlEnabled()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public void                         disposeControl()                            public void                         controlValueChanged(compaswuicommoncontrolControlEvent)                            private String                         removeExtension(String)                            public void                         controlSelectionChanged(compaswuicommoncontrolControlEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandlerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ActionHandler                             private compaswframeworkuiswingExtensionObjectSwingUI                         extensionObjectSwingUI                            private boolean                         bPrintAll                            private int                         printScope    String                         actionErrorMessage                            public void ActionHandler(compaswframeworkuiswingExtensionObjectSwingUI)                            public void                         handleAction(String String)                            private void                         handleExportHTML(String)                            private void                         handleExportText(String)                            private void                         handlePrint(String String)                            private void                         handlePrintView()                            private void                         handlePageSetup(String)                            private void                         handleSaveNode(String)                            private void                         handlePreviewTable(String)                            private void                         handleSaveToProject(String)                            private void                         handleHelpButton()                            private void                         exportHTMLFile(String)                            private void                         saveTextFile(String String)                            private javaxswingJPanel                         createPrintViewPanel(javautilList)                            public String                         launchSaveFileChooser(String String[])                            private void                         configureSaveFileChooser(javaxswingJFileChooser)                            private void                         configureSaveChooserCallback(compaswcoreapplicationcallbackFileSystemChooserRequest)                            private String                         getSaveDialogTitle()                            private String                         getApproveButtonText()                            private boolean                         useApplicationFileChoosers()                            private compaswframeworkuicommonUISession                         getUISession()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            private String                         getTextContainerData(String)                            private boolean                         isExportText(String)                            private boolean                         isExportHTML()                            private boolean                         isSelectNode()                            private compaswuiswingcontrolModelViewerPanel                         getModelViewerPanel()                            private String                         getDefaultFileName()                            private void                         handleUpdateContainer(String)                    
                

comspsssharedcustom_guiui_builderpeersManagedBasePanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedBasePanel                         extends javaxswingJPanel                         implements javaawtprintPrintable     void ManagedBasePanel(javaawtLayoutManager)                            public int                         print(javaawtGraphics javaawtprintPageFormat int)                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$1                     
                

comspsssharedcustom_guiui_builderpeersUIUtilitiesclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class UIUtilities                             public void UIUtilities()                            public                         static javaxswingJButton                         createButton(compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComboBox                         createComBox(compaswframeworkuiswingSwingResourceProvider String int javaxswingDefaultComboBoxModel)                            public                         static javaxswingDefaultComboBoxModel                         createEnumComboBoxModel(compaswframeworkcommoncoreEnumerationValueDescriptor)                            public                         static javaxswingAbstractButton                         addAccessKey(javaxswingAbstractButton compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingtextJTextComponent                         createTextFiled(compaswframeworkuiswingSwingResourceProvider String int)                            public                         static javaxswingJComponent                         addTooltip(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComponent                         addDescription(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static                         transient void                         fitAllComponents(javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxPreferSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setPreferSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMaxSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMaxSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMinSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMinSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                    
                

comspsssharedcustom_guiui_builderpeersManagedHTMLOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedHTMLOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedHTMLOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedModelOutputPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedModelOutputPanel                         implements compaswframeworkuiswingspiManagedPanel compaswframeworkcommonextensionspiInteractorListener                             public                         static                         final String                         BUILD_TEXT_OUTPUT = buildTextOutputDisplay                            public                         static                         final String                         BUILD_CONSOLE_OUTPUT = buildConsoleOutputDisplay                            public                         static                         final String                         BUILD_HTML_OUTPUT = buildHTMLOutputDisplay                            public                         static                         final String                         BUILD_MV_OUTPUT = buildModelViewerDisplay                            public                         static                         final String                         APPLIER_CONSOLE_OUTPUT = applierConsoleOutputDisplay                            public                         static                         final String                         BUILD_TEXT_CONTAINER_NAME = TextOutput                            public                         static                         final String                         BUILD_CONSOLE_CONTAINER_NAME = ConsoleOutput                            public                         static                         final String                         BUILD_HTML_CONTAINER_NAME = HTMLOutput                            public                         static                         final String                         BUILD_PMML_CONTAINER_NAME = PMML                            public                         static                         final String                         BUILD_STAT_XML_CONTAINER_NAME = StatXML                            public                         static                         final String                         APPLIER_CONSOLE_CONTAINER_NAME = ApplierConsoleOutput                            private ManagedTextOutputArea                         textOutput                            private ManagedTextOutputArea                         consoleOutput                            private ManagedHTMLOutputArea                         htmlOutput                            private ManagedEmptyOutputArea                         emptyOutput                            private javautilHashMap                         componentMap                            private compaswframeworkuiswingManagedPanelContext                         managedPanelContext                            public void ManagedModelOutputPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private compaswframeworkuiswingspiManagedUIElement                         setupTextOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupConsoleOutputControl(String)                            public void                         updateConsoleOutputControl(String)                            private compaswframeworkuiswingspiManagedUIElement                         setupHTMLOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupModelViewerControl()                            public String                         getTextContainerData(String)                            public                         static boolean                         exportToHTML(compaswframeworkcommoncontainerContainer String)                            public                         static javaioFile                         getHTMLFileTest(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractHTMLCompoundContainer(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractZip(String String boolean)                         throws javaioIOException                            public                         static javaioFile                         getHTMLFile(String String boolean)                         throws javaioIOException                    
                

comspsssharedcustom_guiui_builderpeersManagedTextOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedTextOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedTextOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$ButtonsActionclass

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$ButtonsAction                         implements javaawteventActionListener                             private void ManagedScoreSyntaxControl$ButtonsAction(ManagedScoreSyntaxControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersManagedEmptyOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedEmptyOutputArea                         extends javaxswingJPanel                         implements compaswframeworkuiswingspiManagedUIElement                             public void ManagedEmptyOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public javaxswingJComponent                         getComponent()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ROutputFileChooserControl$1                         implements javaawteventActionListener     void ROutputFileChooserControl$1(ROutputFileChooserControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

extensionxml

imagescopygif

imagesdeletegif

imagesfileMenugif

imagesgenerateMenugif

imageshelp_buttongif

imageslg_gm_rpng

imageslg_r_cdb_gmgif

imageslg_r_cdb_modgif

imageslg_r_cdb_outputgif

imageslg_r_cdb_processgif

imagespreview_tablegif

randomForestxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

IBM SPSS Modeler and R

6 Your R code7

8 ibmspsscfdatamodelSetDataModel(modelerDataModel)

9 ibmspsscfdataSetData(modelerData)

10

All the functions starting with ibmspsscf are functions within this library It is this part of the codethat is responsible for the transfer of the data and the metadata to R Also you will see a while loopindicating that the data will go to R in batches of 1000 The other options are the values of the data readoptions within the nodeThe last lines of the code prepares the data to be send back to SPSS Also note the closing brace whichis the end of the while loop

Now since this library will be loaded with every interaction between SPSS and R you are free to use thesefunctions within your code as well So if you would like to avoid these runs in batches of 1K and you donot have version 171 available you can start your R code with another loop to first continue filling themodelerData and only than start your actual code In practice that means that in your R node you startwith

1 while(ibmspsscfdataHasMoreData())

2

3 modelerData lt-rbind(modelerDataibmspsscfdataGetData(rowCount=100000missing=NArDate=None

logicalFields=FALSE))

4

5

6 Some more code using modelerData which is now the complete dataset

512 Some useful parts of R code

Make sure a package is installed Whenever you have created any R node using packages you have tomake sure that anyone using this node has this package on his machine installed if needed withouttoo much user interference Therefore you can use this part of code

1 packages lt- function(x)

2 x lt- ascharacter(matchcall()[[2]])

3 if (require(xcharacteronly=TRUE))

4 installpackages(pkgs=xrepos=httpcranr-projectorg)

5 require(xcharacteronly=TRUE)

6

7

8

9 packages(rpart)

It will firstly verify if the library is installed on the system if not it will silently install it fromthe given CRAN mirror (you can change this to another mirror or a local repository if needed)Installation will only happen the first time the node will be used

Create the metadata corresponding to the R data frame Sometimes your data is very much trans-formed compared to the original data that it is difficult to build your metadata starting from theoriginal one So you might want to change the metadata to link to the data in R independent ofthe original This is in particularly useful if you want to use R as a sort node Below is a functionthat asks for a data frame and creates modelerData and modelerDataModel accordingly

1 sendToModeler lt- function (dataFrame)

2 if(isnull(dim(dataFrame)))

3 stop(Invalid data received not a dataframe)

4 if (dim(dataFrame)[1]lt=0)

5 print(Warning modelerData has no line all fieldStorage fields set to strings)

6 getStorage lt- function(x)return(string)

7 else

8 getStorage lt- function(x)

9 x lt- unlist(x)

10 res lt- NULL

11 if x is a factor typeof will return rsquointegerrsquo so we handle this case first

Page 15 of 20

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 10: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

You can nd all streams and R scripts explaining modelerModel here

33 Some general remarks

bull Although it might seems this way you are not required to build modelerData from theexisting data within that frame modelerData will be lled with the dataset you have inModeler however nothing stops you to throw that data away in R and dene some new datacoming from another data source in R As an example imagine this link from the WeatherCompany website This will give the weather history in Brussels Belgium in the monthNovember 2015 Now we can use R code as a source node by just overwriting modelerData

and redening modelerDataModel

1 Define the link

2 linkPath lt- httpwwwwundergroundcomhistoryairportEBBR20151101

MonthlyHistoryhtmlreq_city=Brusselsampformat=1

3 Read the data as csv

4 modelerData lt- readcsv(linkPath)

5 modelerData[1] lt- asDate(modelerData[1])

6

7 Redefining modelerDataModel all are real numbers except the first column is the date

8 modelerDataModel lt- asdataframe(t(dataframe(fieldName = colnames(modelerData)

fieldLabel = fieldStorage = c(daterep(realncol(modelerData)-1))

fieldMeasure = fieldFormat = fieldRole = )))

As you can see this code does not use the old denition of the dened R objects butcompletely redenes them Placing this in a R transform node will give back this newdataset to modeler So in this way you can use this approach to create an R input node

You can nd an example here

bull Within an R model node there is place for 2 scripts The building script will be the scriptthat will be populated within the R nugget It will not be run when you run the model nodeAs a result of this these 2 scripts are independent The only thing they share is the valueof the object modelerModel which is saved within the nugget when running the buildingsyntax and picked up within the R scoring syntaxThis also means that eventual R-libraries that are required should be loaded in both scriptsTake for example a model for a random forest

1 Load the library

2 library(randomForest)

3 Create the model and save it in modelerModel

4 modelerModel lt- randomForest(tenure ~ age + region + ed + income data= modelerDatantree

=50)

1 Load the library

2 library(randomForest)

3

4 Use the model to make a prediction and add it to the existing data

5 predlt- predict(modelerModel modelerData)

6 modelerData lt- cbind(modelerDatapred)

7

8 Take care of the metadata

9 newVar lt- c(fieldName=$RF-tenurefieldLabel=fieldStorage=realfieldMeasure=fieldFormat=fieldRole=)

10 modelerDataModel lt- cbind(modelerDataModelnewVar)

bull Talking about libraries and package A package is a collection of R objects dened for acertain purpose These often are specic statistical functionalities like randomForest in theexample above A basic R installation comes with the standard packages however there aremany more packages available made available by the R community on CRAN

Page 10 of 20

Explain modelerModelxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

R as sourcexml

data0001dat

data0002dat

IBM SPSS Modeler and R

Packages needs to be installed and made locally available in libraries Once the package isinstalled on the system as a library you can load this library in any R session by the codelibrary(ltnamegt)To install a package you have several options The easiest is probably to write a codelike installpackages(randomForest) within R You will have to select a CRAN mirrorwhere this library will be downloaded from and the download will go automatically Nor-mally you will only have to do this onceAlthough possible it is not recommended to run this package installation command fromwithin SPSS The reason is that these libraries will than be saved in a temporary folder andafterwards be deleted If you still want to this through SPSS you will have to hard codethe installation path

34 Read data options

Something we ignored until now are the settings within the node under Read data options Thebasics of the R integration with Modeler can be done without the knowledge of this as it requiressome more advanced R knowledge The user guide still has a good explanation about these items

However there is one more thing that might be important For modeler version 17 and lowerthe R integration of non-terminal nodes (ie transform and nuggets) will by default be done inbatches of 1000 The reason for this was to allow these R nodes to work on hadoop and otherclustered environments As a result of this it is very important to realize that any R code thatwould span over multiple lines of data would lead to false results For a workaround for this werefer to 511

Take as an example the z scores above If we would calculate the mean and the standard de-viation of the variable in a non-terminal node it would start with running this code for the rst1000 lines of data So that leads to a specic mean deviation and corresponding z-scores How-ever the next 1000 lines a new mean and deviation would be calculated and the z scores will bebased on these

As a solution the means and standard deviations are calculated in the R model (ie a termi-nal node) over all the data and used in the R nugget to calculate the z scores

Note that this approach may lead to a very slow integration between SPSS and R in the caseof streaming R nodes in a local non-clustered environment However as from IBM SPSS Modelerversion 171 there is a default option not to use this approach of batch processing or to increasethe batch size For the lower versions there is a workaround possible if you still want to increasethis batch size or turn it o (see later)

4 Custom Dialog builder

The Custom Dialog Builder allow you to create and manage R nodes with prefilled R code to use insideIBM SPSS Modeler streams In this way users can create their own nodes You can start the CustomDialog builder in the Tools menu under rdquoCustom Dialog Builderrdquo

When opening a custom dialog builder you will see a 2 windows One of them is the custom dialogitself the other is the toolset to populate the dialog

Page 11 of 20

IBM SPSS Modeler and R

41 Tools

The tools window is a list of items you can place within your dialog This include among others the fieldchooser Check and combo boxes Text and number controls and tabs You can select any and drag themonto the dialog itselfOnce you have any item in the dialog you can select it and you will see the item properties These are theproperties of this specific item and will change dependent on the type of item it is The most importantare the identifier (the way it will be referenced within the script) and the Title (the one that will be visualin the dialog)

42 Custom dialog

The big gray window is the dialog itself For the moment it is empty as it should be populated with itemsfrom the Tools listClicking on this gray dialog will show you the dialog properties below As main items this includes thename and title of the dialog the script itself and the type and position of the created nodeWith regards to the script to be written The global rule is that you reference to the items within thedialog using their identifier between double percentages (rdquoltidentifiergtrdquoOnce you finished creating the custom node you can install it by pressing the green arrow in the toolbarYou can also save intermediate versions to the disk

43 Simple example

Let us create a custom dialog for the randomForest model created earlier in section 33 Below you willfind a step by step approachThe most important thing we have to wonder is what within this code we want flexible for the user Inthe case of this model there might be 3 things that we want flexibel the input variables the target andthe number of trees in our forest

First fill in the Custom dialog properties as indicated

Page 12 of 20

IBM SPSS Modeler and R

In our fixed example the target is tenure but a user might chooseany other field As a result we will place a field chooser on the dialogChange the properties like shown The variable filter properties allowsyou to select only categorical variables

In our fixed example the inputs are age income but a user mightchoose any other field As a result we will place a field chooser on thedialog The biggest difference is that now we can select several variablesas there might be different inputs To make it easier we will separatethese values by a + Therefore change the properties like shown

As a third custom choice we would like to add the number of trees inour forest In the original script is was 50 so we will choose this as thedefault However users may choose any integer value between 1 and1000 Add a number control on the dialog and change the properties

So now the dialog is ready and we need to add the script to it Go tothe Edit options and choose rdquoScript Templaterdquo This will bring you toan empty window for the script In this case (as we selected we wanted2 scripts) there is a tab for the building code and one for the scoringscript If the coring script is greyed out you did set the dialog propertyrdquoScore from the Modelrdquo to True

Page 13 of 20

IBM SPSS Modeler and R

Let us start with the scoring script as this is easier The only thingwhich will need to be adapted for custom input is the variable namethat will be send back to SPSS So copy the scoring code and changethe name tenure to TARGET (this is the name of the identifier of thetarget)

Fill in the code for the building script and change in a similar wayas above the values for target and intput variables together with thenumber of trees Afterwards press OK to close the script window

Being back at your Custom Dialog builder save the dialog in any appro-priate location Also deploy the dialog by clicking on the green deployarrow in the toolbar Close the Dialog builder

Back to the stream you will now see the new node in the model paletteYou can use this node within your stream

You can find the resulted cfe file here (place this in the correct location see 521) and a stream

where it is deployed

5 Tips amp tricks Some more detailed

51 R code

511 ibmspsscf70 library

Let us now have a more detailled view about what actually happens with the code First of all it is worthto check what happens when you do the R installation correctly This will install the by IBM delivered Rpackage ibmspsscf70 in the library folder of your R installation This library contains several functionsto handle the data traffic between SPSS and R

Running any R node in SPSS will not only run the code you write but it will also run some extracode behind the scenes You can see this code in the rdquoConsole outputrdquo window of the R node Lookingfor example at this tab for an R nugget you will see that your code will be something like

1 modelerModel lt- ibmspsscfoutputGetModel()

2 while(ibmspsscfdataHasMoreData())

3 modelerDataModel lt- ibmspsscfdatamodelGetDataModel()

4 modelerData lt- ibmspsscfdataGetData(rowCount=1000 missing=NA rDate=None

logicalFields=FALSE)

5

Page 14 of 20

app-extensionxml

cdb_peerjar

comspsssharedcustom_guiui_builderpeersRStatsApplierPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsApplierPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsApplierPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                    
                

comspsssharedcustom_guiui_builderpeersCDExtensionPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class CDExtensionPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static String                         RBUILDER                            private                         static String                         SCORE_SYNTAX                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            public void CDExtensionPeer()                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private String                         replaceFieldSeparator(String String)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$1                         implements javaawteventActionListener     void ActionHandler$1(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$2class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$2                         implements javaawteventActionListener     void ActionHandler$2(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsUIPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsUIPeer                         implements compaswframeworkuiswingspiExtensionObjectSwingUIPeer compaswframeworkcommonextensionspiInteractorListener compaswcorecontainerContainerSetListener                             private ActionHandler                         actionHandler                            public                         static                         final String                         UPDATE_CONTAINER = updateContainer                            public void RStatsUIPeer()                            public void                         initSwingUI(compaswframeworkuiswingExtensionObjectSwingUI compaswframeworkcommoncoreProperties compaswframeworkuiswingExtensionObjectSwingUIContext)                            public void                         disposeSwingUI()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            public void                         handleAction(String String compaswframeworkcommoncoreProperties)                            public void                         containerChanged(compaswcorecontainerContainerSetEvent)                            public void                         updateContainer(compaswcorecontainerContainerSetEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsProcessPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsProcessPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsProcessPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersCopyrightclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         abstract                         interface Copyright                         extends annotationAnnotation                             public                         abstract String                         value()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxControl                         extends javaxswingJTextArea                         implements compaswframeworkuiswingspiPropertyControl                             private                         static                         final String                         editScoreSyntax = editScoreSyntax                            private                         static                         final String                         editScoreSyntaxTitle = editScoreSyntaxTitle                            private                         static                         final String                         editScoreSyntaxWarning = editScoreSyntaxWarning                            private                         static                         final String                         editScoreSyntaxContinueBtn = editScoreSyntax_Continue                            private                         static                         final String                         editScoreSyntaxCancelBtn = editScoreSyntax_Cancel                            private javaxswingJPanel                         containerPanel                            private javaxswingJButton                         editScoreSyntaxBtn                            private compaswframeworkuiswingPropertyControlContext                         propertyControlContext                            public void ManagedScoreSyntaxControl()                            public void                         disposeControl()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public javaxswingJComponent                         getUIComponent()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            public boolean                         isControlEnabled()                            public void                         setControlEnabled(boolean)                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxPanel                         implements compaswframeworkuiswingspiManagedPanel                             public void ManagedScoreSyntaxPanel()                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ROutputFileChooserControl                         implements compaswframeworkuiswingspiPropertyControl compaswuicommoncontrolControlListener                             public                         static                         final String                         OUTPUT_TYPE = output_type                            public                         static                         final String                         OUTPUT_TYPE_GRAPH = Graph                            public                         static                         final String                         GRAPH_OUTPUT_FILE_TYPE = graph_output_file_type                            public                         static                         final String                         TEXT_OUTPUT_FILE_TYPE = text_output_file_type                            public                         static                         final String                         OUTPUT_FILE_TYPE_HTML = HTML                            public                         static                         final String                         OUTPUT_FILE_TYPE_TEXT = Text                            public                         static                         final String                         OUTPUT_FILE_TYPE_COU = Cou                            public                         static                         final String                         FILENAME = OutputFilename                            public                         static                         final String                         OUTPUT_FILE_CHOOSER_BUTTON = OutputFileChooserButton                            public                         static                         final char                         BACK_SLASH_CHAR = 92                            public                         static                         final char                         FORWARD_SLASH_CHAR = 47                            private ActionHandler                         actionHandler                            private javaxswingJLabel                         fileNameLabel                            private javaxswingtextJTextComponent                         textComponent                            private javaxswingAbstractButton                         browseButton                            private compaswuiswingcontrolDefaultControlPanel                         mainPanel                            private String                         propertyName                            private compaswframeworkuiswingPropertyControlContext                         context                            public void ROutputFileChooserControl()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            private void                         initComponent()                            private compaswframeworkuiswingSwingResourceProvider                         getSwingResourceProvider()                            private String                         getLabelFromSwingResourceProvider(String)                            private Object                         getSwingUiProperty(String)                            protected String                         getTextComponentValue()                            public javaxswingJComponent                         getUIComponent()                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public void                         setControlEnabled(boolean)                            public boolean                         isControlEnabled()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public void                         disposeControl()                            public void                         controlValueChanged(compaswuicommoncontrolControlEvent)                            private String                         removeExtension(String)                            public void                         controlSelectionChanged(compaswuicommoncontrolControlEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandlerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ActionHandler                             private compaswframeworkuiswingExtensionObjectSwingUI                         extensionObjectSwingUI                            private boolean                         bPrintAll                            private int                         printScope    String                         actionErrorMessage                            public void ActionHandler(compaswframeworkuiswingExtensionObjectSwingUI)                            public void                         handleAction(String String)                            private void                         handleExportHTML(String)                            private void                         handleExportText(String)                            private void                         handlePrint(String String)                            private void                         handlePrintView()                            private void                         handlePageSetup(String)                            private void                         handleSaveNode(String)                            private void                         handlePreviewTable(String)                            private void                         handleSaveToProject(String)                            private void                         handleHelpButton()                            private void                         exportHTMLFile(String)                            private void                         saveTextFile(String String)                            private javaxswingJPanel                         createPrintViewPanel(javautilList)                            public String                         launchSaveFileChooser(String String[])                            private void                         configureSaveFileChooser(javaxswingJFileChooser)                            private void                         configureSaveChooserCallback(compaswcoreapplicationcallbackFileSystemChooserRequest)                            private String                         getSaveDialogTitle()                            private String                         getApproveButtonText()                            private boolean                         useApplicationFileChoosers()                            private compaswframeworkuicommonUISession                         getUISession()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            private String                         getTextContainerData(String)                            private boolean                         isExportText(String)                            private boolean                         isExportHTML()                            private boolean                         isSelectNode()                            private compaswuiswingcontrolModelViewerPanel                         getModelViewerPanel()                            private String                         getDefaultFileName()                            private void                         handleUpdateContainer(String)                    
                

comspsssharedcustom_guiui_builderpeersManagedBasePanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedBasePanel                         extends javaxswingJPanel                         implements javaawtprintPrintable     void ManagedBasePanel(javaawtLayoutManager)                            public int                         print(javaawtGraphics javaawtprintPageFormat int)                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$1                     
                

comspsssharedcustom_guiui_builderpeersUIUtilitiesclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class UIUtilities                             public void UIUtilities()                            public                         static javaxswingJButton                         createButton(compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComboBox                         createComBox(compaswframeworkuiswingSwingResourceProvider String int javaxswingDefaultComboBoxModel)                            public                         static javaxswingDefaultComboBoxModel                         createEnumComboBoxModel(compaswframeworkcommoncoreEnumerationValueDescriptor)                            public                         static javaxswingAbstractButton                         addAccessKey(javaxswingAbstractButton compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingtextJTextComponent                         createTextFiled(compaswframeworkuiswingSwingResourceProvider String int)                            public                         static javaxswingJComponent                         addTooltip(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComponent                         addDescription(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static                         transient void                         fitAllComponents(javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxPreferSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setPreferSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMaxSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMaxSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMinSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMinSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                    
                

comspsssharedcustom_guiui_builderpeersManagedHTMLOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedHTMLOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedHTMLOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedModelOutputPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedModelOutputPanel                         implements compaswframeworkuiswingspiManagedPanel compaswframeworkcommonextensionspiInteractorListener                             public                         static                         final String                         BUILD_TEXT_OUTPUT = buildTextOutputDisplay                            public                         static                         final String                         BUILD_CONSOLE_OUTPUT = buildConsoleOutputDisplay                            public                         static                         final String                         BUILD_HTML_OUTPUT = buildHTMLOutputDisplay                            public                         static                         final String                         BUILD_MV_OUTPUT = buildModelViewerDisplay                            public                         static                         final String                         APPLIER_CONSOLE_OUTPUT = applierConsoleOutputDisplay                            public                         static                         final String                         BUILD_TEXT_CONTAINER_NAME = TextOutput                            public                         static                         final String                         BUILD_CONSOLE_CONTAINER_NAME = ConsoleOutput                            public                         static                         final String                         BUILD_HTML_CONTAINER_NAME = HTMLOutput                            public                         static                         final String                         BUILD_PMML_CONTAINER_NAME = PMML                            public                         static                         final String                         BUILD_STAT_XML_CONTAINER_NAME = StatXML                            public                         static                         final String                         APPLIER_CONSOLE_CONTAINER_NAME = ApplierConsoleOutput                            private ManagedTextOutputArea                         textOutput                            private ManagedTextOutputArea                         consoleOutput                            private ManagedHTMLOutputArea                         htmlOutput                            private ManagedEmptyOutputArea                         emptyOutput                            private javautilHashMap                         componentMap                            private compaswframeworkuiswingManagedPanelContext                         managedPanelContext                            public void ManagedModelOutputPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private compaswframeworkuiswingspiManagedUIElement                         setupTextOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupConsoleOutputControl(String)                            public void                         updateConsoleOutputControl(String)                            private compaswframeworkuiswingspiManagedUIElement                         setupHTMLOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupModelViewerControl()                            public String                         getTextContainerData(String)                            public                         static boolean                         exportToHTML(compaswframeworkcommoncontainerContainer String)                            public                         static javaioFile                         getHTMLFileTest(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractHTMLCompoundContainer(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractZip(String String boolean)                         throws javaioIOException                            public                         static javaioFile                         getHTMLFile(String String boolean)                         throws javaioIOException                    
                

comspsssharedcustom_guiui_builderpeersManagedTextOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedTextOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedTextOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$ButtonsActionclass

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$ButtonsAction                         implements javaawteventActionListener                             private void ManagedScoreSyntaxControl$ButtonsAction(ManagedScoreSyntaxControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersManagedEmptyOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedEmptyOutputArea                         extends javaxswingJPanel                         implements compaswframeworkuiswingspiManagedUIElement                             public void ManagedEmptyOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public javaxswingJComponent                         getComponent()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ROutputFileChooserControl$1                         implements javaawteventActionListener     void ROutputFileChooserControl$1(ROutputFileChooserControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

extensionxml

imagescopygif

imagesdeletegif

imagesfileMenugif

imagesgenerateMenugif

imageshelp_buttongif

imageslg_gm_rpng

imageslg_r_cdb_gmgif

imageslg_r_cdb_modgif

imageslg_r_cdb_outputgif

imageslg_r_cdb_processgif

imagespreview_tablegif

randomForestxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

IBM SPSS Modeler and R

6 Your R code7

8 ibmspsscfdatamodelSetDataModel(modelerDataModel)

9 ibmspsscfdataSetData(modelerData)

10

All the functions starting with ibmspsscf are functions within this library It is this part of the codethat is responsible for the transfer of the data and the metadata to R Also you will see a while loopindicating that the data will go to R in batches of 1000 The other options are the values of the data readoptions within the nodeThe last lines of the code prepares the data to be send back to SPSS Also note the closing brace whichis the end of the while loop

Now since this library will be loaded with every interaction between SPSS and R you are free to use thesefunctions within your code as well So if you would like to avoid these runs in batches of 1K and you donot have version 171 available you can start your R code with another loop to first continue filling themodelerData and only than start your actual code In practice that means that in your R node you startwith

1 while(ibmspsscfdataHasMoreData())

2

3 modelerData lt-rbind(modelerDataibmspsscfdataGetData(rowCount=100000missing=NArDate=None

logicalFields=FALSE))

4

5

6 Some more code using modelerData which is now the complete dataset

512 Some useful parts of R code

Make sure a package is installed Whenever you have created any R node using packages you have tomake sure that anyone using this node has this package on his machine installed if needed withouttoo much user interference Therefore you can use this part of code

1 packages lt- function(x)

2 x lt- ascharacter(matchcall()[[2]])

3 if (require(xcharacteronly=TRUE))

4 installpackages(pkgs=xrepos=httpcranr-projectorg)

5 require(xcharacteronly=TRUE)

6

7

8

9 packages(rpart)

It will firstly verify if the library is installed on the system if not it will silently install it fromthe given CRAN mirror (you can change this to another mirror or a local repository if needed)Installation will only happen the first time the node will be used

Create the metadata corresponding to the R data frame Sometimes your data is very much trans-formed compared to the original data that it is difficult to build your metadata starting from theoriginal one So you might want to change the metadata to link to the data in R independent ofthe original This is in particularly useful if you want to use R as a sort node Below is a functionthat asks for a data frame and creates modelerData and modelerDataModel accordingly

1 sendToModeler lt- function (dataFrame)

2 if(isnull(dim(dataFrame)))

3 stop(Invalid data received not a dataframe)

4 if (dim(dataFrame)[1]lt=0)

5 print(Warning modelerData has no line all fieldStorage fields set to strings)

6 getStorage lt- function(x)return(string)

7 else

8 getStorage lt- function(x)

9 x lt- unlist(x)

10 res lt- NULL

11 if x is a factor typeof will return rsquointegerrsquo so we handle this case first

Page 15 of 20

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 11: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

Packages needs to be installed and made locally available in libraries Once the package isinstalled on the system as a library you can load this library in any R session by the codelibrary(ltnamegt)To install a package you have several options The easiest is probably to write a codelike installpackages(randomForest) within R You will have to select a CRAN mirrorwhere this library will be downloaded from and the download will go automatically Nor-mally you will only have to do this onceAlthough possible it is not recommended to run this package installation command fromwithin SPSS The reason is that these libraries will than be saved in a temporary folder andafterwards be deleted If you still want to this through SPSS you will have to hard codethe installation path

34 Read data options

Something we ignored until now are the settings within the node under Read data options Thebasics of the R integration with Modeler can be done without the knowledge of this as it requiressome more advanced R knowledge The user guide still has a good explanation about these items

However there is one more thing that might be important For modeler version 17 and lowerthe R integration of non-terminal nodes (ie transform and nuggets) will by default be done inbatches of 1000 The reason for this was to allow these R nodes to work on hadoop and otherclustered environments As a result of this it is very important to realize that any R code thatwould span over multiple lines of data would lead to false results For a workaround for this werefer to 511

Take as an example the z scores above If we would calculate the mean and the standard de-viation of the variable in a non-terminal node it would start with running this code for the rst1000 lines of data So that leads to a specic mean deviation and corresponding z-scores How-ever the next 1000 lines a new mean and deviation would be calculated and the z scores will bebased on these

As a solution the means and standard deviations are calculated in the R model (ie a termi-nal node) over all the data and used in the R nugget to calculate the z scores

Note that this approach may lead to a very slow integration between SPSS and R in the caseof streaming R nodes in a local non-clustered environment However as from IBM SPSS Modelerversion 171 there is a default option not to use this approach of batch processing or to increasethe batch size For the lower versions there is a workaround possible if you still want to increasethis batch size or turn it o (see later)

4 Custom Dialog builder

The Custom Dialog Builder allow you to create and manage R nodes with prefilled R code to use insideIBM SPSS Modeler streams In this way users can create their own nodes You can start the CustomDialog builder in the Tools menu under rdquoCustom Dialog Builderrdquo

When opening a custom dialog builder you will see a 2 windows One of them is the custom dialogitself the other is the toolset to populate the dialog

Page 11 of 20

IBM SPSS Modeler and R

41 Tools

The tools window is a list of items you can place within your dialog This include among others the fieldchooser Check and combo boxes Text and number controls and tabs You can select any and drag themonto the dialog itselfOnce you have any item in the dialog you can select it and you will see the item properties These are theproperties of this specific item and will change dependent on the type of item it is The most importantare the identifier (the way it will be referenced within the script) and the Title (the one that will be visualin the dialog)

42 Custom dialog

The big gray window is the dialog itself For the moment it is empty as it should be populated with itemsfrom the Tools listClicking on this gray dialog will show you the dialog properties below As main items this includes thename and title of the dialog the script itself and the type and position of the created nodeWith regards to the script to be written The global rule is that you reference to the items within thedialog using their identifier between double percentages (rdquoltidentifiergtrdquoOnce you finished creating the custom node you can install it by pressing the green arrow in the toolbarYou can also save intermediate versions to the disk

43 Simple example

Let us create a custom dialog for the randomForest model created earlier in section 33 Below you willfind a step by step approachThe most important thing we have to wonder is what within this code we want flexible for the user Inthe case of this model there might be 3 things that we want flexibel the input variables the target andthe number of trees in our forest

First fill in the Custom dialog properties as indicated

Page 12 of 20

IBM SPSS Modeler and R

In our fixed example the target is tenure but a user might chooseany other field As a result we will place a field chooser on the dialogChange the properties like shown The variable filter properties allowsyou to select only categorical variables

In our fixed example the inputs are age income but a user mightchoose any other field As a result we will place a field chooser on thedialog The biggest difference is that now we can select several variablesas there might be different inputs To make it easier we will separatethese values by a + Therefore change the properties like shown

As a third custom choice we would like to add the number of trees inour forest In the original script is was 50 so we will choose this as thedefault However users may choose any integer value between 1 and1000 Add a number control on the dialog and change the properties

So now the dialog is ready and we need to add the script to it Go tothe Edit options and choose rdquoScript Templaterdquo This will bring you toan empty window for the script In this case (as we selected we wanted2 scripts) there is a tab for the building code and one for the scoringscript If the coring script is greyed out you did set the dialog propertyrdquoScore from the Modelrdquo to True

Page 13 of 20

IBM SPSS Modeler and R

Let us start with the scoring script as this is easier The only thingwhich will need to be adapted for custom input is the variable namethat will be send back to SPSS So copy the scoring code and changethe name tenure to TARGET (this is the name of the identifier of thetarget)

Fill in the code for the building script and change in a similar wayas above the values for target and intput variables together with thenumber of trees Afterwards press OK to close the script window

Being back at your Custom Dialog builder save the dialog in any appro-priate location Also deploy the dialog by clicking on the green deployarrow in the toolbar Close the Dialog builder

Back to the stream you will now see the new node in the model paletteYou can use this node within your stream

You can find the resulted cfe file here (place this in the correct location see 521) and a stream

where it is deployed

5 Tips amp tricks Some more detailed

51 R code

511 ibmspsscf70 library

Let us now have a more detailled view about what actually happens with the code First of all it is worthto check what happens when you do the R installation correctly This will install the by IBM delivered Rpackage ibmspsscf70 in the library folder of your R installation This library contains several functionsto handle the data traffic between SPSS and R

Running any R node in SPSS will not only run the code you write but it will also run some extracode behind the scenes You can see this code in the rdquoConsole outputrdquo window of the R node Lookingfor example at this tab for an R nugget you will see that your code will be something like

1 modelerModel lt- ibmspsscfoutputGetModel()

2 while(ibmspsscfdataHasMoreData())

3 modelerDataModel lt- ibmspsscfdatamodelGetDataModel()

4 modelerData lt- ibmspsscfdataGetData(rowCount=1000 missing=NA rDate=None

logicalFields=FALSE)

5

Page 14 of 20

app-extensionxml

cdb_peerjar

comspsssharedcustom_guiui_builderpeersRStatsApplierPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsApplierPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsApplierPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                    
                

comspsssharedcustom_guiui_builderpeersCDExtensionPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class CDExtensionPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static String                         RBUILDER                            private                         static String                         SCORE_SYNTAX                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            public void CDExtensionPeer()                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private String                         replaceFieldSeparator(String String)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$1                         implements javaawteventActionListener     void ActionHandler$1(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$2class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$2                         implements javaawteventActionListener     void ActionHandler$2(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsUIPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsUIPeer                         implements compaswframeworkuiswingspiExtensionObjectSwingUIPeer compaswframeworkcommonextensionspiInteractorListener compaswcorecontainerContainerSetListener                             private ActionHandler                         actionHandler                            public                         static                         final String                         UPDATE_CONTAINER = updateContainer                            public void RStatsUIPeer()                            public void                         initSwingUI(compaswframeworkuiswingExtensionObjectSwingUI compaswframeworkcommoncoreProperties compaswframeworkuiswingExtensionObjectSwingUIContext)                            public void                         disposeSwingUI()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            public void                         handleAction(String String compaswframeworkcommoncoreProperties)                            public void                         containerChanged(compaswcorecontainerContainerSetEvent)                            public void                         updateContainer(compaswcorecontainerContainerSetEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsProcessPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsProcessPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsProcessPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersCopyrightclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         abstract                         interface Copyright                         extends annotationAnnotation                             public                         abstract String                         value()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxControl                         extends javaxswingJTextArea                         implements compaswframeworkuiswingspiPropertyControl                             private                         static                         final String                         editScoreSyntax = editScoreSyntax                            private                         static                         final String                         editScoreSyntaxTitle = editScoreSyntaxTitle                            private                         static                         final String                         editScoreSyntaxWarning = editScoreSyntaxWarning                            private                         static                         final String                         editScoreSyntaxContinueBtn = editScoreSyntax_Continue                            private                         static                         final String                         editScoreSyntaxCancelBtn = editScoreSyntax_Cancel                            private javaxswingJPanel                         containerPanel                            private javaxswingJButton                         editScoreSyntaxBtn                            private compaswframeworkuiswingPropertyControlContext                         propertyControlContext                            public void ManagedScoreSyntaxControl()                            public void                         disposeControl()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public javaxswingJComponent                         getUIComponent()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            public boolean                         isControlEnabled()                            public void                         setControlEnabled(boolean)                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxPanel                         implements compaswframeworkuiswingspiManagedPanel                             public void ManagedScoreSyntaxPanel()                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ROutputFileChooserControl                         implements compaswframeworkuiswingspiPropertyControl compaswuicommoncontrolControlListener                             public                         static                         final String                         OUTPUT_TYPE = output_type                            public                         static                         final String                         OUTPUT_TYPE_GRAPH = Graph                            public                         static                         final String                         GRAPH_OUTPUT_FILE_TYPE = graph_output_file_type                            public                         static                         final String                         TEXT_OUTPUT_FILE_TYPE = text_output_file_type                            public                         static                         final String                         OUTPUT_FILE_TYPE_HTML = HTML                            public                         static                         final String                         OUTPUT_FILE_TYPE_TEXT = Text                            public                         static                         final String                         OUTPUT_FILE_TYPE_COU = Cou                            public                         static                         final String                         FILENAME = OutputFilename                            public                         static                         final String                         OUTPUT_FILE_CHOOSER_BUTTON = OutputFileChooserButton                            public                         static                         final char                         BACK_SLASH_CHAR = 92                            public                         static                         final char                         FORWARD_SLASH_CHAR = 47                            private ActionHandler                         actionHandler                            private javaxswingJLabel                         fileNameLabel                            private javaxswingtextJTextComponent                         textComponent                            private javaxswingAbstractButton                         browseButton                            private compaswuiswingcontrolDefaultControlPanel                         mainPanel                            private String                         propertyName                            private compaswframeworkuiswingPropertyControlContext                         context                            public void ROutputFileChooserControl()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            private void                         initComponent()                            private compaswframeworkuiswingSwingResourceProvider                         getSwingResourceProvider()                            private String                         getLabelFromSwingResourceProvider(String)                            private Object                         getSwingUiProperty(String)                            protected String                         getTextComponentValue()                            public javaxswingJComponent                         getUIComponent()                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public void                         setControlEnabled(boolean)                            public boolean                         isControlEnabled()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public void                         disposeControl()                            public void                         controlValueChanged(compaswuicommoncontrolControlEvent)                            private String                         removeExtension(String)                            public void                         controlSelectionChanged(compaswuicommoncontrolControlEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandlerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ActionHandler                             private compaswframeworkuiswingExtensionObjectSwingUI                         extensionObjectSwingUI                            private boolean                         bPrintAll                            private int                         printScope    String                         actionErrorMessage                            public void ActionHandler(compaswframeworkuiswingExtensionObjectSwingUI)                            public void                         handleAction(String String)                            private void                         handleExportHTML(String)                            private void                         handleExportText(String)                            private void                         handlePrint(String String)                            private void                         handlePrintView()                            private void                         handlePageSetup(String)                            private void                         handleSaveNode(String)                            private void                         handlePreviewTable(String)                            private void                         handleSaveToProject(String)                            private void                         handleHelpButton()                            private void                         exportHTMLFile(String)                            private void                         saveTextFile(String String)                            private javaxswingJPanel                         createPrintViewPanel(javautilList)                            public String                         launchSaveFileChooser(String String[])                            private void                         configureSaveFileChooser(javaxswingJFileChooser)                            private void                         configureSaveChooserCallback(compaswcoreapplicationcallbackFileSystemChooserRequest)                            private String                         getSaveDialogTitle()                            private String                         getApproveButtonText()                            private boolean                         useApplicationFileChoosers()                            private compaswframeworkuicommonUISession                         getUISession()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            private String                         getTextContainerData(String)                            private boolean                         isExportText(String)                            private boolean                         isExportHTML()                            private boolean                         isSelectNode()                            private compaswuiswingcontrolModelViewerPanel                         getModelViewerPanel()                            private String                         getDefaultFileName()                            private void                         handleUpdateContainer(String)                    
                

comspsssharedcustom_guiui_builderpeersManagedBasePanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedBasePanel                         extends javaxswingJPanel                         implements javaawtprintPrintable     void ManagedBasePanel(javaawtLayoutManager)                            public int                         print(javaawtGraphics javaawtprintPageFormat int)                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$1                     
                

comspsssharedcustom_guiui_builderpeersUIUtilitiesclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class UIUtilities                             public void UIUtilities()                            public                         static javaxswingJButton                         createButton(compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComboBox                         createComBox(compaswframeworkuiswingSwingResourceProvider String int javaxswingDefaultComboBoxModel)                            public                         static javaxswingDefaultComboBoxModel                         createEnumComboBoxModel(compaswframeworkcommoncoreEnumerationValueDescriptor)                            public                         static javaxswingAbstractButton                         addAccessKey(javaxswingAbstractButton compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingtextJTextComponent                         createTextFiled(compaswframeworkuiswingSwingResourceProvider String int)                            public                         static javaxswingJComponent                         addTooltip(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComponent                         addDescription(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static                         transient void                         fitAllComponents(javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxPreferSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setPreferSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMaxSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMaxSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMinSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMinSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                    
                

comspsssharedcustom_guiui_builderpeersManagedHTMLOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedHTMLOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedHTMLOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedModelOutputPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedModelOutputPanel                         implements compaswframeworkuiswingspiManagedPanel compaswframeworkcommonextensionspiInteractorListener                             public                         static                         final String                         BUILD_TEXT_OUTPUT = buildTextOutputDisplay                            public                         static                         final String                         BUILD_CONSOLE_OUTPUT = buildConsoleOutputDisplay                            public                         static                         final String                         BUILD_HTML_OUTPUT = buildHTMLOutputDisplay                            public                         static                         final String                         BUILD_MV_OUTPUT = buildModelViewerDisplay                            public                         static                         final String                         APPLIER_CONSOLE_OUTPUT = applierConsoleOutputDisplay                            public                         static                         final String                         BUILD_TEXT_CONTAINER_NAME = TextOutput                            public                         static                         final String                         BUILD_CONSOLE_CONTAINER_NAME = ConsoleOutput                            public                         static                         final String                         BUILD_HTML_CONTAINER_NAME = HTMLOutput                            public                         static                         final String                         BUILD_PMML_CONTAINER_NAME = PMML                            public                         static                         final String                         BUILD_STAT_XML_CONTAINER_NAME = StatXML                            public                         static                         final String                         APPLIER_CONSOLE_CONTAINER_NAME = ApplierConsoleOutput                            private ManagedTextOutputArea                         textOutput                            private ManagedTextOutputArea                         consoleOutput                            private ManagedHTMLOutputArea                         htmlOutput                            private ManagedEmptyOutputArea                         emptyOutput                            private javautilHashMap                         componentMap                            private compaswframeworkuiswingManagedPanelContext                         managedPanelContext                            public void ManagedModelOutputPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private compaswframeworkuiswingspiManagedUIElement                         setupTextOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupConsoleOutputControl(String)                            public void                         updateConsoleOutputControl(String)                            private compaswframeworkuiswingspiManagedUIElement                         setupHTMLOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupModelViewerControl()                            public String                         getTextContainerData(String)                            public                         static boolean                         exportToHTML(compaswframeworkcommoncontainerContainer String)                            public                         static javaioFile                         getHTMLFileTest(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractHTMLCompoundContainer(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractZip(String String boolean)                         throws javaioIOException                            public                         static javaioFile                         getHTMLFile(String String boolean)                         throws javaioIOException                    
                

comspsssharedcustom_guiui_builderpeersManagedTextOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedTextOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedTextOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$ButtonsActionclass

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$ButtonsAction                         implements javaawteventActionListener                             private void ManagedScoreSyntaxControl$ButtonsAction(ManagedScoreSyntaxControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersManagedEmptyOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedEmptyOutputArea                         extends javaxswingJPanel                         implements compaswframeworkuiswingspiManagedUIElement                             public void ManagedEmptyOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public javaxswingJComponent                         getComponent()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ROutputFileChooserControl$1                         implements javaawteventActionListener     void ROutputFileChooserControl$1(ROutputFileChooserControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

extensionxml

imagescopygif

imagesdeletegif

imagesfileMenugif

imagesgenerateMenugif

imageshelp_buttongif

imageslg_gm_rpng

imageslg_r_cdb_gmgif

imageslg_r_cdb_modgif

imageslg_r_cdb_outputgif

imageslg_r_cdb_processgif

imagespreview_tablegif

randomForestxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

IBM SPSS Modeler and R

6 Your R code7

8 ibmspsscfdatamodelSetDataModel(modelerDataModel)

9 ibmspsscfdataSetData(modelerData)

10

All the functions starting with ibmspsscf are functions within this library It is this part of the codethat is responsible for the transfer of the data and the metadata to R Also you will see a while loopindicating that the data will go to R in batches of 1000 The other options are the values of the data readoptions within the nodeThe last lines of the code prepares the data to be send back to SPSS Also note the closing brace whichis the end of the while loop

Now since this library will be loaded with every interaction between SPSS and R you are free to use thesefunctions within your code as well So if you would like to avoid these runs in batches of 1K and you donot have version 171 available you can start your R code with another loop to first continue filling themodelerData and only than start your actual code In practice that means that in your R node you startwith

1 while(ibmspsscfdataHasMoreData())

2

3 modelerData lt-rbind(modelerDataibmspsscfdataGetData(rowCount=100000missing=NArDate=None

logicalFields=FALSE))

4

5

6 Some more code using modelerData which is now the complete dataset

512 Some useful parts of R code

Make sure a package is installed Whenever you have created any R node using packages you have tomake sure that anyone using this node has this package on his machine installed if needed withouttoo much user interference Therefore you can use this part of code

1 packages lt- function(x)

2 x lt- ascharacter(matchcall()[[2]])

3 if (require(xcharacteronly=TRUE))

4 installpackages(pkgs=xrepos=httpcranr-projectorg)

5 require(xcharacteronly=TRUE)

6

7

8

9 packages(rpart)

It will firstly verify if the library is installed on the system if not it will silently install it fromthe given CRAN mirror (you can change this to another mirror or a local repository if needed)Installation will only happen the first time the node will be used

Create the metadata corresponding to the R data frame Sometimes your data is very much trans-formed compared to the original data that it is difficult to build your metadata starting from theoriginal one So you might want to change the metadata to link to the data in R independent ofthe original This is in particularly useful if you want to use R as a sort node Below is a functionthat asks for a data frame and creates modelerData and modelerDataModel accordingly

1 sendToModeler lt- function (dataFrame)

2 if(isnull(dim(dataFrame)))

3 stop(Invalid data received not a dataframe)

4 if (dim(dataFrame)[1]lt=0)

5 print(Warning modelerData has no line all fieldStorage fields set to strings)

6 getStorage lt- function(x)return(string)

7 else

8 getStorage lt- function(x)

9 x lt- unlist(x)

10 res lt- NULL

11 if x is a factor typeof will return rsquointegerrsquo so we handle this case first

Page 15 of 20

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 12: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

41 Tools

The tools window is a list of items you can place within your dialog This include among others the fieldchooser Check and combo boxes Text and number controls and tabs You can select any and drag themonto the dialog itselfOnce you have any item in the dialog you can select it and you will see the item properties These are theproperties of this specific item and will change dependent on the type of item it is The most importantare the identifier (the way it will be referenced within the script) and the Title (the one that will be visualin the dialog)

42 Custom dialog

The big gray window is the dialog itself For the moment it is empty as it should be populated with itemsfrom the Tools listClicking on this gray dialog will show you the dialog properties below As main items this includes thename and title of the dialog the script itself and the type and position of the created nodeWith regards to the script to be written The global rule is that you reference to the items within thedialog using their identifier between double percentages (rdquoltidentifiergtrdquoOnce you finished creating the custom node you can install it by pressing the green arrow in the toolbarYou can also save intermediate versions to the disk

43 Simple example

Let us create a custom dialog for the randomForest model created earlier in section 33 Below you willfind a step by step approachThe most important thing we have to wonder is what within this code we want flexible for the user Inthe case of this model there might be 3 things that we want flexibel the input variables the target andthe number of trees in our forest

First fill in the Custom dialog properties as indicated

Page 12 of 20

IBM SPSS Modeler and R

In our fixed example the target is tenure but a user might chooseany other field As a result we will place a field chooser on the dialogChange the properties like shown The variable filter properties allowsyou to select only categorical variables

In our fixed example the inputs are age income but a user mightchoose any other field As a result we will place a field chooser on thedialog The biggest difference is that now we can select several variablesas there might be different inputs To make it easier we will separatethese values by a + Therefore change the properties like shown

As a third custom choice we would like to add the number of trees inour forest In the original script is was 50 so we will choose this as thedefault However users may choose any integer value between 1 and1000 Add a number control on the dialog and change the properties

So now the dialog is ready and we need to add the script to it Go tothe Edit options and choose rdquoScript Templaterdquo This will bring you toan empty window for the script In this case (as we selected we wanted2 scripts) there is a tab for the building code and one for the scoringscript If the coring script is greyed out you did set the dialog propertyrdquoScore from the Modelrdquo to True

Page 13 of 20

IBM SPSS Modeler and R

Let us start with the scoring script as this is easier The only thingwhich will need to be adapted for custom input is the variable namethat will be send back to SPSS So copy the scoring code and changethe name tenure to TARGET (this is the name of the identifier of thetarget)

Fill in the code for the building script and change in a similar wayas above the values for target and intput variables together with thenumber of trees Afterwards press OK to close the script window

Being back at your Custom Dialog builder save the dialog in any appro-priate location Also deploy the dialog by clicking on the green deployarrow in the toolbar Close the Dialog builder

Back to the stream you will now see the new node in the model paletteYou can use this node within your stream

You can find the resulted cfe file here (place this in the correct location see 521) and a stream

where it is deployed

5 Tips amp tricks Some more detailed

51 R code

511 ibmspsscf70 library

Let us now have a more detailled view about what actually happens with the code First of all it is worthto check what happens when you do the R installation correctly This will install the by IBM delivered Rpackage ibmspsscf70 in the library folder of your R installation This library contains several functionsto handle the data traffic between SPSS and R

Running any R node in SPSS will not only run the code you write but it will also run some extracode behind the scenes You can see this code in the rdquoConsole outputrdquo window of the R node Lookingfor example at this tab for an R nugget you will see that your code will be something like

1 modelerModel lt- ibmspsscfoutputGetModel()

2 while(ibmspsscfdataHasMoreData())

3 modelerDataModel lt- ibmspsscfdatamodelGetDataModel()

4 modelerData lt- ibmspsscfdataGetData(rowCount=1000 missing=NA rDate=None

logicalFields=FALSE)

5

Page 14 of 20

app-extensionxml

cdb_peerjar

comspsssharedcustom_guiui_builderpeersRStatsApplierPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsApplierPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsApplierPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                    
                

comspsssharedcustom_guiui_builderpeersCDExtensionPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class CDExtensionPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static String                         RBUILDER                            private                         static String                         SCORE_SYNTAX                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            public void CDExtensionPeer()                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private String                         replaceFieldSeparator(String String)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$1                         implements javaawteventActionListener     void ActionHandler$1(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$2class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$2                         implements javaawteventActionListener     void ActionHandler$2(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsUIPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsUIPeer                         implements compaswframeworkuiswingspiExtensionObjectSwingUIPeer compaswframeworkcommonextensionspiInteractorListener compaswcorecontainerContainerSetListener                             private ActionHandler                         actionHandler                            public                         static                         final String                         UPDATE_CONTAINER = updateContainer                            public void RStatsUIPeer()                            public void                         initSwingUI(compaswframeworkuiswingExtensionObjectSwingUI compaswframeworkcommoncoreProperties compaswframeworkuiswingExtensionObjectSwingUIContext)                            public void                         disposeSwingUI()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            public void                         handleAction(String String compaswframeworkcommoncoreProperties)                            public void                         containerChanged(compaswcorecontainerContainerSetEvent)                            public void                         updateContainer(compaswcorecontainerContainerSetEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsProcessPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsProcessPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsProcessPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersCopyrightclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         abstract                         interface Copyright                         extends annotationAnnotation                             public                         abstract String                         value()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxControl                         extends javaxswingJTextArea                         implements compaswframeworkuiswingspiPropertyControl                             private                         static                         final String                         editScoreSyntax = editScoreSyntax                            private                         static                         final String                         editScoreSyntaxTitle = editScoreSyntaxTitle                            private                         static                         final String                         editScoreSyntaxWarning = editScoreSyntaxWarning                            private                         static                         final String                         editScoreSyntaxContinueBtn = editScoreSyntax_Continue                            private                         static                         final String                         editScoreSyntaxCancelBtn = editScoreSyntax_Cancel                            private javaxswingJPanel                         containerPanel                            private javaxswingJButton                         editScoreSyntaxBtn                            private compaswframeworkuiswingPropertyControlContext                         propertyControlContext                            public void ManagedScoreSyntaxControl()                            public void                         disposeControl()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public javaxswingJComponent                         getUIComponent()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            public boolean                         isControlEnabled()                            public void                         setControlEnabled(boolean)                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxPanel                         implements compaswframeworkuiswingspiManagedPanel                             public void ManagedScoreSyntaxPanel()                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ROutputFileChooserControl                         implements compaswframeworkuiswingspiPropertyControl compaswuicommoncontrolControlListener                             public                         static                         final String                         OUTPUT_TYPE = output_type                            public                         static                         final String                         OUTPUT_TYPE_GRAPH = Graph                            public                         static                         final String                         GRAPH_OUTPUT_FILE_TYPE = graph_output_file_type                            public                         static                         final String                         TEXT_OUTPUT_FILE_TYPE = text_output_file_type                            public                         static                         final String                         OUTPUT_FILE_TYPE_HTML = HTML                            public                         static                         final String                         OUTPUT_FILE_TYPE_TEXT = Text                            public                         static                         final String                         OUTPUT_FILE_TYPE_COU = Cou                            public                         static                         final String                         FILENAME = OutputFilename                            public                         static                         final String                         OUTPUT_FILE_CHOOSER_BUTTON = OutputFileChooserButton                            public                         static                         final char                         BACK_SLASH_CHAR = 92                            public                         static                         final char                         FORWARD_SLASH_CHAR = 47                            private ActionHandler                         actionHandler                            private javaxswingJLabel                         fileNameLabel                            private javaxswingtextJTextComponent                         textComponent                            private javaxswingAbstractButton                         browseButton                            private compaswuiswingcontrolDefaultControlPanel                         mainPanel                            private String                         propertyName                            private compaswframeworkuiswingPropertyControlContext                         context                            public void ROutputFileChooserControl()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            private void                         initComponent()                            private compaswframeworkuiswingSwingResourceProvider                         getSwingResourceProvider()                            private String                         getLabelFromSwingResourceProvider(String)                            private Object                         getSwingUiProperty(String)                            protected String                         getTextComponentValue()                            public javaxswingJComponent                         getUIComponent()                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public void                         setControlEnabled(boolean)                            public boolean                         isControlEnabled()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public void                         disposeControl()                            public void                         controlValueChanged(compaswuicommoncontrolControlEvent)                            private String                         removeExtension(String)                            public void                         controlSelectionChanged(compaswuicommoncontrolControlEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandlerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ActionHandler                             private compaswframeworkuiswingExtensionObjectSwingUI                         extensionObjectSwingUI                            private boolean                         bPrintAll                            private int                         printScope    String                         actionErrorMessage                            public void ActionHandler(compaswframeworkuiswingExtensionObjectSwingUI)                            public void                         handleAction(String String)                            private void                         handleExportHTML(String)                            private void                         handleExportText(String)                            private void                         handlePrint(String String)                            private void                         handlePrintView()                            private void                         handlePageSetup(String)                            private void                         handleSaveNode(String)                            private void                         handlePreviewTable(String)                            private void                         handleSaveToProject(String)                            private void                         handleHelpButton()                            private void                         exportHTMLFile(String)                            private void                         saveTextFile(String String)                            private javaxswingJPanel                         createPrintViewPanel(javautilList)                            public String                         launchSaveFileChooser(String String[])                            private void                         configureSaveFileChooser(javaxswingJFileChooser)                            private void                         configureSaveChooserCallback(compaswcoreapplicationcallbackFileSystemChooserRequest)                            private String                         getSaveDialogTitle()                            private String                         getApproveButtonText()                            private boolean                         useApplicationFileChoosers()                            private compaswframeworkuicommonUISession                         getUISession()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            private String                         getTextContainerData(String)                            private boolean                         isExportText(String)                            private boolean                         isExportHTML()                            private boolean                         isSelectNode()                            private compaswuiswingcontrolModelViewerPanel                         getModelViewerPanel()                            private String                         getDefaultFileName()                            private void                         handleUpdateContainer(String)                    
                

comspsssharedcustom_guiui_builderpeersManagedBasePanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedBasePanel                         extends javaxswingJPanel                         implements javaawtprintPrintable     void ManagedBasePanel(javaawtLayoutManager)                            public int                         print(javaawtGraphics javaawtprintPageFormat int)                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$1                     
                

comspsssharedcustom_guiui_builderpeersUIUtilitiesclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class UIUtilities                             public void UIUtilities()                            public                         static javaxswingJButton                         createButton(compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComboBox                         createComBox(compaswframeworkuiswingSwingResourceProvider String int javaxswingDefaultComboBoxModel)                            public                         static javaxswingDefaultComboBoxModel                         createEnumComboBoxModel(compaswframeworkcommoncoreEnumerationValueDescriptor)                            public                         static javaxswingAbstractButton                         addAccessKey(javaxswingAbstractButton compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingtextJTextComponent                         createTextFiled(compaswframeworkuiswingSwingResourceProvider String int)                            public                         static javaxswingJComponent                         addTooltip(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComponent                         addDescription(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static                         transient void                         fitAllComponents(javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxPreferSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setPreferSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMaxSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMaxSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMinSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMinSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                    
                

comspsssharedcustom_guiui_builderpeersManagedHTMLOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedHTMLOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedHTMLOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedModelOutputPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedModelOutputPanel                         implements compaswframeworkuiswingspiManagedPanel compaswframeworkcommonextensionspiInteractorListener                             public                         static                         final String                         BUILD_TEXT_OUTPUT = buildTextOutputDisplay                            public                         static                         final String                         BUILD_CONSOLE_OUTPUT = buildConsoleOutputDisplay                            public                         static                         final String                         BUILD_HTML_OUTPUT = buildHTMLOutputDisplay                            public                         static                         final String                         BUILD_MV_OUTPUT = buildModelViewerDisplay                            public                         static                         final String                         APPLIER_CONSOLE_OUTPUT = applierConsoleOutputDisplay                            public                         static                         final String                         BUILD_TEXT_CONTAINER_NAME = TextOutput                            public                         static                         final String                         BUILD_CONSOLE_CONTAINER_NAME = ConsoleOutput                            public                         static                         final String                         BUILD_HTML_CONTAINER_NAME = HTMLOutput                            public                         static                         final String                         BUILD_PMML_CONTAINER_NAME = PMML                            public                         static                         final String                         BUILD_STAT_XML_CONTAINER_NAME = StatXML                            public                         static                         final String                         APPLIER_CONSOLE_CONTAINER_NAME = ApplierConsoleOutput                            private ManagedTextOutputArea                         textOutput                            private ManagedTextOutputArea                         consoleOutput                            private ManagedHTMLOutputArea                         htmlOutput                            private ManagedEmptyOutputArea                         emptyOutput                            private javautilHashMap                         componentMap                            private compaswframeworkuiswingManagedPanelContext                         managedPanelContext                            public void ManagedModelOutputPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private compaswframeworkuiswingspiManagedUIElement                         setupTextOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupConsoleOutputControl(String)                            public void                         updateConsoleOutputControl(String)                            private compaswframeworkuiswingspiManagedUIElement                         setupHTMLOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupModelViewerControl()                            public String                         getTextContainerData(String)                            public                         static boolean                         exportToHTML(compaswframeworkcommoncontainerContainer String)                            public                         static javaioFile                         getHTMLFileTest(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractHTMLCompoundContainer(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractZip(String String boolean)                         throws javaioIOException                            public                         static javaioFile                         getHTMLFile(String String boolean)                         throws javaioIOException                    
                

comspsssharedcustom_guiui_builderpeersManagedTextOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedTextOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedTextOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$ButtonsActionclass

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$ButtonsAction                         implements javaawteventActionListener                             private void ManagedScoreSyntaxControl$ButtonsAction(ManagedScoreSyntaxControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersManagedEmptyOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedEmptyOutputArea                         extends javaxswingJPanel                         implements compaswframeworkuiswingspiManagedUIElement                             public void ManagedEmptyOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public javaxswingJComponent                         getComponent()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ROutputFileChooserControl$1                         implements javaawteventActionListener     void ROutputFileChooserControl$1(ROutputFileChooserControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

extensionxml

imagescopygif

imagesdeletegif

imagesfileMenugif

imagesgenerateMenugif

imageshelp_buttongif

imageslg_gm_rpng

imageslg_r_cdb_gmgif

imageslg_r_cdb_modgif

imageslg_r_cdb_outputgif

imageslg_r_cdb_processgif

imagespreview_tablegif

randomForestxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

IBM SPSS Modeler and R

6 Your R code7

8 ibmspsscfdatamodelSetDataModel(modelerDataModel)

9 ibmspsscfdataSetData(modelerData)

10

All the functions starting with ibmspsscf are functions within this library It is this part of the codethat is responsible for the transfer of the data and the metadata to R Also you will see a while loopindicating that the data will go to R in batches of 1000 The other options are the values of the data readoptions within the nodeThe last lines of the code prepares the data to be send back to SPSS Also note the closing brace whichis the end of the while loop

Now since this library will be loaded with every interaction between SPSS and R you are free to use thesefunctions within your code as well So if you would like to avoid these runs in batches of 1K and you donot have version 171 available you can start your R code with another loop to first continue filling themodelerData and only than start your actual code In practice that means that in your R node you startwith

1 while(ibmspsscfdataHasMoreData())

2

3 modelerData lt-rbind(modelerDataibmspsscfdataGetData(rowCount=100000missing=NArDate=None

logicalFields=FALSE))

4

5

6 Some more code using modelerData which is now the complete dataset

512 Some useful parts of R code

Make sure a package is installed Whenever you have created any R node using packages you have tomake sure that anyone using this node has this package on his machine installed if needed withouttoo much user interference Therefore you can use this part of code

1 packages lt- function(x)

2 x lt- ascharacter(matchcall()[[2]])

3 if (require(xcharacteronly=TRUE))

4 installpackages(pkgs=xrepos=httpcranr-projectorg)

5 require(xcharacteronly=TRUE)

6

7

8

9 packages(rpart)

It will firstly verify if the library is installed on the system if not it will silently install it fromthe given CRAN mirror (you can change this to another mirror or a local repository if needed)Installation will only happen the first time the node will be used

Create the metadata corresponding to the R data frame Sometimes your data is very much trans-formed compared to the original data that it is difficult to build your metadata starting from theoriginal one So you might want to change the metadata to link to the data in R independent ofthe original This is in particularly useful if you want to use R as a sort node Below is a functionthat asks for a data frame and creates modelerData and modelerDataModel accordingly

1 sendToModeler lt- function (dataFrame)

2 if(isnull(dim(dataFrame)))

3 stop(Invalid data received not a dataframe)

4 if (dim(dataFrame)[1]lt=0)

5 print(Warning modelerData has no line all fieldStorage fields set to strings)

6 getStorage lt- function(x)return(string)

7 else

8 getStorage lt- function(x)

9 x lt- unlist(x)

10 res lt- NULL

11 if x is a factor typeof will return rsquointegerrsquo so we handle this case first

Page 15 of 20

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 13: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

In our fixed example the target is tenure but a user might chooseany other field As a result we will place a field chooser on the dialogChange the properties like shown The variable filter properties allowsyou to select only categorical variables

In our fixed example the inputs are age income but a user mightchoose any other field As a result we will place a field chooser on thedialog The biggest difference is that now we can select several variablesas there might be different inputs To make it easier we will separatethese values by a + Therefore change the properties like shown

As a third custom choice we would like to add the number of trees inour forest In the original script is was 50 so we will choose this as thedefault However users may choose any integer value between 1 and1000 Add a number control on the dialog and change the properties

So now the dialog is ready and we need to add the script to it Go tothe Edit options and choose rdquoScript Templaterdquo This will bring you toan empty window for the script In this case (as we selected we wanted2 scripts) there is a tab for the building code and one for the scoringscript If the coring script is greyed out you did set the dialog propertyrdquoScore from the Modelrdquo to True

Page 13 of 20

IBM SPSS Modeler and R

Let us start with the scoring script as this is easier The only thingwhich will need to be adapted for custom input is the variable namethat will be send back to SPSS So copy the scoring code and changethe name tenure to TARGET (this is the name of the identifier of thetarget)

Fill in the code for the building script and change in a similar wayas above the values for target and intput variables together with thenumber of trees Afterwards press OK to close the script window

Being back at your Custom Dialog builder save the dialog in any appro-priate location Also deploy the dialog by clicking on the green deployarrow in the toolbar Close the Dialog builder

Back to the stream you will now see the new node in the model paletteYou can use this node within your stream

You can find the resulted cfe file here (place this in the correct location see 521) and a stream

where it is deployed

5 Tips amp tricks Some more detailed

51 R code

511 ibmspsscf70 library

Let us now have a more detailled view about what actually happens with the code First of all it is worthto check what happens when you do the R installation correctly This will install the by IBM delivered Rpackage ibmspsscf70 in the library folder of your R installation This library contains several functionsto handle the data traffic between SPSS and R

Running any R node in SPSS will not only run the code you write but it will also run some extracode behind the scenes You can see this code in the rdquoConsole outputrdquo window of the R node Lookingfor example at this tab for an R nugget you will see that your code will be something like

1 modelerModel lt- ibmspsscfoutputGetModel()

2 while(ibmspsscfdataHasMoreData())

3 modelerDataModel lt- ibmspsscfdatamodelGetDataModel()

4 modelerData lt- ibmspsscfdataGetData(rowCount=1000 missing=NA rDate=None

logicalFields=FALSE)

5

Page 14 of 20

app-extensionxml

cdb_peerjar

comspsssharedcustom_guiui_builderpeersRStatsApplierPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsApplierPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsApplierPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                    
                

comspsssharedcustom_guiui_builderpeersCDExtensionPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class CDExtensionPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static String                         RBUILDER                            private                         static String                         SCORE_SYNTAX                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            public void CDExtensionPeer()                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private String                         replaceFieldSeparator(String String)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$1                         implements javaawteventActionListener     void ActionHandler$1(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$2class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$2                         implements javaawteventActionListener     void ActionHandler$2(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsUIPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsUIPeer                         implements compaswframeworkuiswingspiExtensionObjectSwingUIPeer compaswframeworkcommonextensionspiInteractorListener compaswcorecontainerContainerSetListener                             private ActionHandler                         actionHandler                            public                         static                         final String                         UPDATE_CONTAINER = updateContainer                            public void RStatsUIPeer()                            public void                         initSwingUI(compaswframeworkuiswingExtensionObjectSwingUI compaswframeworkcommoncoreProperties compaswframeworkuiswingExtensionObjectSwingUIContext)                            public void                         disposeSwingUI()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            public void                         handleAction(String String compaswframeworkcommoncoreProperties)                            public void                         containerChanged(compaswcorecontainerContainerSetEvent)                            public void                         updateContainer(compaswcorecontainerContainerSetEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsProcessPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsProcessPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsProcessPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersCopyrightclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         abstract                         interface Copyright                         extends annotationAnnotation                             public                         abstract String                         value()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxControl                         extends javaxswingJTextArea                         implements compaswframeworkuiswingspiPropertyControl                             private                         static                         final String                         editScoreSyntax = editScoreSyntax                            private                         static                         final String                         editScoreSyntaxTitle = editScoreSyntaxTitle                            private                         static                         final String                         editScoreSyntaxWarning = editScoreSyntaxWarning                            private                         static                         final String                         editScoreSyntaxContinueBtn = editScoreSyntax_Continue                            private                         static                         final String                         editScoreSyntaxCancelBtn = editScoreSyntax_Cancel                            private javaxswingJPanel                         containerPanel                            private javaxswingJButton                         editScoreSyntaxBtn                            private compaswframeworkuiswingPropertyControlContext                         propertyControlContext                            public void ManagedScoreSyntaxControl()                            public void                         disposeControl()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public javaxswingJComponent                         getUIComponent()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            public boolean                         isControlEnabled()                            public void                         setControlEnabled(boolean)                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxPanel                         implements compaswframeworkuiswingspiManagedPanel                             public void ManagedScoreSyntaxPanel()                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ROutputFileChooserControl                         implements compaswframeworkuiswingspiPropertyControl compaswuicommoncontrolControlListener                             public                         static                         final String                         OUTPUT_TYPE = output_type                            public                         static                         final String                         OUTPUT_TYPE_GRAPH = Graph                            public                         static                         final String                         GRAPH_OUTPUT_FILE_TYPE = graph_output_file_type                            public                         static                         final String                         TEXT_OUTPUT_FILE_TYPE = text_output_file_type                            public                         static                         final String                         OUTPUT_FILE_TYPE_HTML = HTML                            public                         static                         final String                         OUTPUT_FILE_TYPE_TEXT = Text                            public                         static                         final String                         OUTPUT_FILE_TYPE_COU = Cou                            public                         static                         final String                         FILENAME = OutputFilename                            public                         static                         final String                         OUTPUT_FILE_CHOOSER_BUTTON = OutputFileChooserButton                            public                         static                         final char                         BACK_SLASH_CHAR = 92                            public                         static                         final char                         FORWARD_SLASH_CHAR = 47                            private ActionHandler                         actionHandler                            private javaxswingJLabel                         fileNameLabel                            private javaxswingtextJTextComponent                         textComponent                            private javaxswingAbstractButton                         browseButton                            private compaswuiswingcontrolDefaultControlPanel                         mainPanel                            private String                         propertyName                            private compaswframeworkuiswingPropertyControlContext                         context                            public void ROutputFileChooserControl()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            private void                         initComponent()                            private compaswframeworkuiswingSwingResourceProvider                         getSwingResourceProvider()                            private String                         getLabelFromSwingResourceProvider(String)                            private Object                         getSwingUiProperty(String)                            protected String                         getTextComponentValue()                            public javaxswingJComponent                         getUIComponent()                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public void                         setControlEnabled(boolean)                            public boolean                         isControlEnabled()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public void                         disposeControl()                            public void                         controlValueChanged(compaswuicommoncontrolControlEvent)                            private String                         removeExtension(String)                            public void                         controlSelectionChanged(compaswuicommoncontrolControlEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandlerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ActionHandler                             private compaswframeworkuiswingExtensionObjectSwingUI                         extensionObjectSwingUI                            private boolean                         bPrintAll                            private int                         printScope    String                         actionErrorMessage                            public void ActionHandler(compaswframeworkuiswingExtensionObjectSwingUI)                            public void                         handleAction(String String)                            private void                         handleExportHTML(String)                            private void                         handleExportText(String)                            private void                         handlePrint(String String)                            private void                         handlePrintView()                            private void                         handlePageSetup(String)                            private void                         handleSaveNode(String)                            private void                         handlePreviewTable(String)                            private void                         handleSaveToProject(String)                            private void                         handleHelpButton()                            private void                         exportHTMLFile(String)                            private void                         saveTextFile(String String)                            private javaxswingJPanel                         createPrintViewPanel(javautilList)                            public String                         launchSaveFileChooser(String String[])                            private void                         configureSaveFileChooser(javaxswingJFileChooser)                            private void                         configureSaveChooserCallback(compaswcoreapplicationcallbackFileSystemChooserRequest)                            private String                         getSaveDialogTitle()                            private String                         getApproveButtonText()                            private boolean                         useApplicationFileChoosers()                            private compaswframeworkuicommonUISession                         getUISession()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            private String                         getTextContainerData(String)                            private boolean                         isExportText(String)                            private boolean                         isExportHTML()                            private boolean                         isSelectNode()                            private compaswuiswingcontrolModelViewerPanel                         getModelViewerPanel()                            private String                         getDefaultFileName()                            private void                         handleUpdateContainer(String)                    
                

comspsssharedcustom_guiui_builderpeersManagedBasePanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedBasePanel                         extends javaxswingJPanel                         implements javaawtprintPrintable     void ManagedBasePanel(javaawtLayoutManager)                            public int                         print(javaawtGraphics javaawtprintPageFormat int)                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$1                     
                

comspsssharedcustom_guiui_builderpeersUIUtilitiesclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class UIUtilities                             public void UIUtilities()                            public                         static javaxswingJButton                         createButton(compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComboBox                         createComBox(compaswframeworkuiswingSwingResourceProvider String int javaxswingDefaultComboBoxModel)                            public                         static javaxswingDefaultComboBoxModel                         createEnumComboBoxModel(compaswframeworkcommoncoreEnumerationValueDescriptor)                            public                         static javaxswingAbstractButton                         addAccessKey(javaxswingAbstractButton compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingtextJTextComponent                         createTextFiled(compaswframeworkuiswingSwingResourceProvider String int)                            public                         static javaxswingJComponent                         addTooltip(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComponent                         addDescription(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static                         transient void                         fitAllComponents(javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxPreferSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setPreferSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMaxSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMaxSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMinSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMinSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                    
                

comspsssharedcustom_guiui_builderpeersManagedHTMLOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedHTMLOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedHTMLOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedModelOutputPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedModelOutputPanel                         implements compaswframeworkuiswingspiManagedPanel compaswframeworkcommonextensionspiInteractorListener                             public                         static                         final String                         BUILD_TEXT_OUTPUT = buildTextOutputDisplay                            public                         static                         final String                         BUILD_CONSOLE_OUTPUT = buildConsoleOutputDisplay                            public                         static                         final String                         BUILD_HTML_OUTPUT = buildHTMLOutputDisplay                            public                         static                         final String                         BUILD_MV_OUTPUT = buildModelViewerDisplay                            public                         static                         final String                         APPLIER_CONSOLE_OUTPUT = applierConsoleOutputDisplay                            public                         static                         final String                         BUILD_TEXT_CONTAINER_NAME = TextOutput                            public                         static                         final String                         BUILD_CONSOLE_CONTAINER_NAME = ConsoleOutput                            public                         static                         final String                         BUILD_HTML_CONTAINER_NAME = HTMLOutput                            public                         static                         final String                         BUILD_PMML_CONTAINER_NAME = PMML                            public                         static                         final String                         BUILD_STAT_XML_CONTAINER_NAME = StatXML                            public                         static                         final String                         APPLIER_CONSOLE_CONTAINER_NAME = ApplierConsoleOutput                            private ManagedTextOutputArea                         textOutput                            private ManagedTextOutputArea                         consoleOutput                            private ManagedHTMLOutputArea                         htmlOutput                            private ManagedEmptyOutputArea                         emptyOutput                            private javautilHashMap                         componentMap                            private compaswframeworkuiswingManagedPanelContext                         managedPanelContext                            public void ManagedModelOutputPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private compaswframeworkuiswingspiManagedUIElement                         setupTextOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupConsoleOutputControl(String)                            public void                         updateConsoleOutputControl(String)                            private compaswframeworkuiswingspiManagedUIElement                         setupHTMLOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupModelViewerControl()                            public String                         getTextContainerData(String)                            public                         static boolean                         exportToHTML(compaswframeworkcommoncontainerContainer String)                            public                         static javaioFile                         getHTMLFileTest(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractHTMLCompoundContainer(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractZip(String String boolean)                         throws javaioIOException                            public                         static javaioFile                         getHTMLFile(String String boolean)                         throws javaioIOException                    
                

comspsssharedcustom_guiui_builderpeersManagedTextOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedTextOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedTextOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$ButtonsActionclass

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$ButtonsAction                         implements javaawteventActionListener                             private void ManagedScoreSyntaxControl$ButtonsAction(ManagedScoreSyntaxControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersManagedEmptyOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedEmptyOutputArea                         extends javaxswingJPanel                         implements compaswframeworkuiswingspiManagedUIElement                             public void ManagedEmptyOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public javaxswingJComponent                         getComponent()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ROutputFileChooserControl$1                         implements javaawteventActionListener     void ROutputFileChooserControl$1(ROutputFileChooserControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

extensionxml

imagescopygif

imagesdeletegif

imagesfileMenugif

imagesgenerateMenugif

imageshelp_buttongif

imageslg_gm_rpng

imageslg_r_cdb_gmgif

imageslg_r_cdb_modgif

imageslg_r_cdb_outputgif

imageslg_r_cdb_processgif

imagespreview_tablegif

randomForestxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

IBM SPSS Modeler and R

6 Your R code7

8 ibmspsscfdatamodelSetDataModel(modelerDataModel)

9 ibmspsscfdataSetData(modelerData)

10

All the functions starting with ibmspsscf are functions within this library It is this part of the codethat is responsible for the transfer of the data and the metadata to R Also you will see a while loopindicating that the data will go to R in batches of 1000 The other options are the values of the data readoptions within the nodeThe last lines of the code prepares the data to be send back to SPSS Also note the closing brace whichis the end of the while loop

Now since this library will be loaded with every interaction between SPSS and R you are free to use thesefunctions within your code as well So if you would like to avoid these runs in batches of 1K and you donot have version 171 available you can start your R code with another loop to first continue filling themodelerData and only than start your actual code In practice that means that in your R node you startwith

1 while(ibmspsscfdataHasMoreData())

2

3 modelerData lt-rbind(modelerDataibmspsscfdataGetData(rowCount=100000missing=NArDate=None

logicalFields=FALSE))

4

5

6 Some more code using modelerData which is now the complete dataset

512 Some useful parts of R code

Make sure a package is installed Whenever you have created any R node using packages you have tomake sure that anyone using this node has this package on his machine installed if needed withouttoo much user interference Therefore you can use this part of code

1 packages lt- function(x)

2 x lt- ascharacter(matchcall()[[2]])

3 if (require(xcharacteronly=TRUE))

4 installpackages(pkgs=xrepos=httpcranr-projectorg)

5 require(xcharacteronly=TRUE)

6

7

8

9 packages(rpart)

It will firstly verify if the library is installed on the system if not it will silently install it fromthe given CRAN mirror (you can change this to another mirror or a local repository if needed)Installation will only happen the first time the node will be used

Create the metadata corresponding to the R data frame Sometimes your data is very much trans-formed compared to the original data that it is difficult to build your metadata starting from theoriginal one So you might want to change the metadata to link to the data in R independent ofthe original This is in particularly useful if you want to use R as a sort node Below is a functionthat asks for a data frame and creates modelerData and modelerDataModel accordingly

1 sendToModeler lt- function (dataFrame)

2 if(isnull(dim(dataFrame)))

3 stop(Invalid data received not a dataframe)

4 if (dim(dataFrame)[1]lt=0)

5 print(Warning modelerData has no line all fieldStorage fields set to strings)

6 getStorage lt- function(x)return(string)

7 else

8 getStorage lt- function(x)

9 x lt- unlist(x)

10 res lt- NULL

11 if x is a factor typeof will return rsquointegerrsquo so we handle this case first

Page 15 of 20

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 14: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

Let us start with the scoring script as this is easier The only thingwhich will need to be adapted for custom input is the variable namethat will be send back to SPSS So copy the scoring code and changethe name tenure to TARGET (this is the name of the identifier of thetarget)

Fill in the code for the building script and change in a similar wayas above the values for target and intput variables together with thenumber of trees Afterwards press OK to close the script window

Being back at your Custom Dialog builder save the dialog in any appro-priate location Also deploy the dialog by clicking on the green deployarrow in the toolbar Close the Dialog builder

Back to the stream you will now see the new node in the model paletteYou can use this node within your stream

You can find the resulted cfe file here (place this in the correct location see 521) and a stream

where it is deployed

5 Tips amp tricks Some more detailed

51 R code

511 ibmspsscf70 library

Let us now have a more detailled view about what actually happens with the code First of all it is worthto check what happens when you do the R installation correctly This will install the by IBM delivered Rpackage ibmspsscf70 in the library folder of your R installation This library contains several functionsto handle the data traffic between SPSS and R

Running any R node in SPSS will not only run the code you write but it will also run some extracode behind the scenes You can see this code in the rdquoConsole outputrdquo window of the R node Lookingfor example at this tab for an R nugget you will see that your code will be something like

1 modelerModel lt- ibmspsscfoutputGetModel()

2 while(ibmspsscfdataHasMoreData())

3 modelerDataModel lt- ibmspsscfdatamodelGetDataModel()

4 modelerData lt- ibmspsscfdataGetData(rowCount=1000 missing=NA rDate=None

logicalFields=FALSE)

5

Page 14 of 20

app-extensionxml

cdb_peerjar

comspsssharedcustom_guiui_builderpeersRStatsApplierPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsApplierPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsApplierPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                    
                

comspsssharedcustom_guiui_builderpeersCDExtensionPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class CDExtensionPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static String                         RBUILDER                            private                         static String                         SCORE_SYNTAX                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            public void CDExtensionPeer()                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private String                         replaceFieldSeparator(String String)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$1                         implements javaawteventActionListener     void ActionHandler$1(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandler$2class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ActionHandler$2                         implements javaawteventActionListener     void ActionHandler$2(ActionHandler)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsUIPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsUIPeer                         implements compaswframeworkuiswingspiExtensionObjectSwingUIPeer compaswframeworkcommonextensionspiInteractorListener compaswcorecontainerContainerSetListener                             private ActionHandler                         actionHandler                            public                         static                         final String                         UPDATE_CONTAINER = updateContainer                            public void RStatsUIPeer()                            public void                         initSwingUI(compaswframeworkuiswingExtensionObjectSwingUI compaswframeworkcommoncoreProperties compaswframeworkuiswingExtensionObjectSwingUIContext)                            public void                         disposeSwingUI()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            public void                         handleAction(String String compaswframeworkcommoncoreProperties)                            public void                         containerChanged(compaswcorecontainerContainerSetEvent)                            public void                         updateContainer(compaswcorecontainerContainerSetEvent)                    
                

comspsssharedcustom_guiui_builderpeersRStatsProcessPeerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class RStatsProcessPeer                         implements compaswframeworkcommonextensionspiExtensionObjectPeer compaswframeworkcommonextensionspiOutputDataModelProvider compaswframeworkcommonextensionspiInteractorListener compaswcorepropertyPropertySetListener                             private compaswframeworkcommonextensionExtensionObject                         extensionObject                            private                         static                         final String                         RINTERACTOR = rinteractor                            private                         static                         final String                         RBUILDER = rbuilder                            private                         static                         final String                         RAPPLIER = rapplier                            private                         static                         final String                         ROUTPUT = routput                            private                         static                         final String                         RMODELAPPLIER = rmodel                            private                         static                         final String                         RPROCESS = rprocess                            private                         static                         final String                         SYNTAX = syntax                            private                         static                         final String                         SCORE_SYNTAX = score_syntax                            private                         static                         final String                         OUTPUT_DATAMODEL = output_datamodel                            private                         static                         final String                         INPUT_DATAMODEL = input_datamodel                            private                         static                         final String                         OUTPUT_MODE = output_mode                            private                         static                         final String                         FILE_MODE = File                            private                         static                         final String                         OUTPUT_TYPE = output_type                            private                         static                         final String                         GRAPH_TYPE = Graph                            private                         static                         final String                         TEXT_TYPE = Text                            private                         static                         final String                         OUTPUT_FILE_TYPE = output_file_type                            private                         static                         final String                         GRAPH_OUTPUT_TYPE = graph_output_file_type                            private                         static                         final String                         TEXT_OUTPUT_TYPE = text_output_file_type                            private                         static                         final String                         OUTPUT_CONTAINER_ID = output_container_id                            private                         static                         final String                         OUTPUT_CONTAINER_TYPE = output_container_type                            private                         static                         final String                         OUTPUT_CONTAINER_GRAPH = HTMLOutput                            private                         static                         final String                         OUTPUT_CONTAINER_TEXT = TextOutput                            private                         static                         final String                         CONTAINER_TYPE_HTML = html                            private                         static                         final String                         CONTAINER_TYPE_TEXT = txt                            private                         static String                         SCORESCRIPT                            private                         static String                         SCRIPT                            private                         static String                         BUILD_SYNTAX                            private                         static String                         ESCAPE                            private                         static String                         SELFVALUE                            private                         static                         final String                         SEPARATOR = plusmn                            private                         static                         final String                         FIELD_TAG = Field                            private                         static                         final String                         NAME_TAG = name                            private                         static                         final String                         DIS_LABEL_TAG = displayLabel                            private                         static                         final String                         STORAGE_TAG = storage                            private                         static                         final String                         MEASURE_TAG = measure                            private                         static                         final String                         ROLE_TAG = role                            public void RStatsProcessPeer()                            public javautilList                         getDataModelOperations(compaswframeworkcommondataDataModel compaswframeworkcommoncoreProperties)                            public void                         parseDataModel(String javautilList)                            public void                         disposeExtensionObject()                            public void                         initExtensionObject(compaswframeworkcommonextensionExtensionObject compaswframeworkcommoncoreProperties compaswframeworkcommonextensionExtensionObjectContext)                            public void                         onExtensionObjectEvent(compaswframeworkcommonextensionExtensionObjectEvent)                            public void                         propertySetChanged(compaswcorepropertyPropertySetEvent)                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private void                         setBuilderScirptProperty(compaswframeworkcommoncoreProperties)                            private String                         replaceControlEscapes(compaswframeworkcommoncoreProperties String)                            private String                         replaceSelfValue(compaswframeworkcommoncoreProperties String String)                            private String                         getCheckBoxSyntax(compaswframeworkcommoncoreProperties String String)                            private String                         replaceListValue(compaswframeworkcommoncoreProperties String)                            private orgw3cdomDocument                         parseDataModel(String)                            private orgw3cdomElement                         getMetaDataXML(compaswframeworkcommondataDataModel)                            private String                         toDataModelString(orgw3cdomElement)                            private boolean                         dataModelEquals(orgw3cdomElement orgw3cdomElement)                            static void                         ltclinitgt()                    
                

comspsssharedcustom_guiui_builderpeersCopyrightclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         abstract                         interface Copyright                         extends annotationAnnotation                             public                         abstract String                         value()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxControl                         extends javaxswingJTextArea                         implements compaswframeworkuiswingspiPropertyControl                             private                         static                         final String                         editScoreSyntax = editScoreSyntax                            private                         static                         final String                         editScoreSyntaxTitle = editScoreSyntaxTitle                            private                         static                         final String                         editScoreSyntaxWarning = editScoreSyntaxWarning                            private                         static                         final String                         editScoreSyntaxContinueBtn = editScoreSyntax_Continue                            private                         static                         final String                         editScoreSyntaxCancelBtn = editScoreSyntax_Cancel                            private javaxswingJPanel                         containerPanel                            private javaxswingJButton                         editScoreSyntaxBtn                            private compaswframeworkuiswingPropertyControlContext                         propertyControlContext                            public void ManagedScoreSyntaxControl()                            public void                         disposeControl()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public javaxswingJComponent                         getUIComponent()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            public boolean                         isControlEnabled()                            public void                         setControlEnabled(boolean)                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedScoreSyntaxPanel                         implements compaswframeworkuiswingspiManagedPanel                             public void ManagedScoreSyntaxPanel()                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControlclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ROutputFileChooserControl                         implements compaswframeworkuiswingspiPropertyControl compaswuicommoncontrolControlListener                             public                         static                         final String                         OUTPUT_TYPE = output_type                            public                         static                         final String                         OUTPUT_TYPE_GRAPH = Graph                            public                         static                         final String                         GRAPH_OUTPUT_FILE_TYPE = graph_output_file_type                            public                         static                         final String                         TEXT_OUTPUT_FILE_TYPE = text_output_file_type                            public                         static                         final String                         OUTPUT_FILE_TYPE_HTML = HTML                            public                         static                         final String                         OUTPUT_FILE_TYPE_TEXT = Text                            public                         static                         final String                         OUTPUT_FILE_TYPE_COU = Cou                            public                         static                         final String                         FILENAME = OutputFilename                            public                         static                         final String                         OUTPUT_FILE_CHOOSER_BUTTON = OutputFileChooserButton                            public                         static                         final char                         BACK_SLASH_CHAR = 92                            public                         static                         final char                         FORWARD_SLASH_CHAR = 47                            private ActionHandler                         actionHandler                            private javaxswingJLabel                         fileNameLabel                            private javaxswingtextJTextComponent                         textComponent                            private javaxswingAbstractButton                         browseButton                            private compaswuiswingcontrolDefaultControlPanel                         mainPanel                            private String                         propertyName                            private compaswframeworkuiswingPropertyControlContext                         context                            public void ROutputFileChooserControl()                            public void                         initControl(String compaswframeworkcommoncoreProperties compaswframeworkuiswingPropertyControlContext)                            private void                         initComponent()                            private compaswframeworkuiswingSwingResourceProvider                         getSwingResourceProvider()                            private String                         getLabelFromSwingResourceProvider(String)                            private Object                         getSwingUiProperty(String)                            protected String                         getTextComponentValue()                            public javaxswingJComponent                         getUIComponent()                            public Object                         getControlValue()                            public void                         setControlValue(Object)                            public void                         setControlEnabled(boolean)                            public boolean                         isControlEnabled()                            public void                         dataModelUpdated(compaswframeworkcommondataDataModel)                            public void                         disposeControl()                            public void                         controlValueChanged(compaswuicommoncontrolControlEvent)                            private String                         removeExtension(String)                            public void                         controlSelectionChanged(compaswuicommoncontrolControlEvent)                    
                

comspsssharedcustom_guiui_builderpeersActionHandlerclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ActionHandler                             private compaswframeworkuiswingExtensionObjectSwingUI                         extensionObjectSwingUI                            private boolean                         bPrintAll                            private int                         printScope    String                         actionErrorMessage                            public void ActionHandler(compaswframeworkuiswingExtensionObjectSwingUI)                            public void                         handleAction(String String)                            private void                         handleExportHTML(String)                            private void                         handleExportText(String)                            private void                         handlePrint(String String)                            private void                         handlePrintView()                            private void                         handlePageSetup(String)                            private void                         handleSaveNode(String)                            private void                         handlePreviewTable(String)                            private void                         handleSaveToProject(String)                            private void                         handleHelpButton()                            private void                         exportHTMLFile(String)                            private void                         saveTextFile(String String)                            private javaxswingJPanel                         createPrintViewPanel(javautilList)                            public String                         launchSaveFileChooser(String String[])                            private void                         configureSaveFileChooser(javaxswingJFileChooser)                            private void                         configureSaveChooserCallback(compaswcoreapplicationcallbackFileSystemChooserRequest)                            private String                         getSaveDialogTitle()                            private String                         getApproveButtonText()                            private boolean                         useApplicationFileChoosers()                            private compaswframeworkuicommonUISession                         getUISession()                            public void                         onSwingUIEvent(compaswframeworkuiswingSwingUIEvent)                            private String                         getTextContainerData(String)                            private boolean                         isExportText(String)                            private boolean                         isExportHTML()                            private boolean                         isSelectNode()                            private compaswuiswingcontrolModelViewerPanel                         getModelViewerPanel()                            private String                         getDefaultFileName()                            private void                         handleUpdateContainer(String)                    
                

comspsssharedcustom_guiui_builderpeersManagedBasePanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedBasePanel                         extends javaxswingJPanel                         implements javaawtprintPrintable     void ManagedBasePanel(javaawtLayoutManager)                            public int                         print(javaawtGraphics javaawtprintPageFormat int)                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$1                     
                

comspsssharedcustom_guiui_builderpeersUIUtilitiesclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class UIUtilities                             public void UIUtilities()                            public                         static javaxswingJButton                         createButton(compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComboBox                         createComBox(compaswframeworkuiswingSwingResourceProvider String int javaxswingDefaultComboBoxModel)                            public                         static javaxswingDefaultComboBoxModel                         createEnumComboBoxModel(compaswframeworkcommoncoreEnumerationValueDescriptor)                            public                         static javaxswingAbstractButton                         addAccessKey(javaxswingAbstractButton compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingtextJTextComponent                         createTextFiled(compaswframeworkuiswingSwingResourceProvider String int)                            public                         static javaxswingJComponent                         addTooltip(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static javaxswingJComponent                         addDescription(javaxswingJComponent compaswframeworkuiswingSwingResourceProvider String)                            public                         static                         transient void                         fitAllComponents(javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxPreferSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setPreferSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMaxSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMaxSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                            public                         static                         transient javaawtDimension                         getMaxMinSizeOfAllComponents(javaxswingJComponent[])                            public                         static                         transient void                         setMinSizeOfAllComponents(javaawtDimension javaxswingJComponent[])                    
                

comspsssharedcustom_guiui_builderpeersManagedHTMLOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedHTMLOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedHTMLOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedModelOutputPanelclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedModelOutputPanel                         implements compaswframeworkuiswingspiManagedPanel compaswframeworkcommonextensionspiInteractorListener                             public                         static                         final String                         BUILD_TEXT_OUTPUT = buildTextOutputDisplay                            public                         static                         final String                         BUILD_CONSOLE_OUTPUT = buildConsoleOutputDisplay                            public                         static                         final String                         BUILD_HTML_OUTPUT = buildHTMLOutputDisplay                            public                         static                         final String                         BUILD_MV_OUTPUT = buildModelViewerDisplay                            public                         static                         final String                         APPLIER_CONSOLE_OUTPUT = applierConsoleOutputDisplay                            public                         static                         final String                         BUILD_TEXT_CONTAINER_NAME = TextOutput                            public                         static                         final String                         BUILD_CONSOLE_CONTAINER_NAME = ConsoleOutput                            public                         static                         final String                         BUILD_HTML_CONTAINER_NAME = HTMLOutput                            public                         static                         final String                         BUILD_PMML_CONTAINER_NAME = PMML                            public                         static                         final String                         BUILD_STAT_XML_CONTAINER_NAME = StatXML                            public                         static                         final String                         APPLIER_CONSOLE_CONTAINER_NAME = ApplierConsoleOutput                            private ManagedTextOutputArea                         textOutput                            private ManagedTextOutputArea                         consoleOutput                            private ManagedHTMLOutputArea                         htmlOutput                            private ManagedEmptyOutputArea                         emptyOutput                            private javautilHashMap                         componentMap                            private compaswframeworkuiswingManagedPanelContext                         managedPanelContext                            public void ManagedModelOutputPanel()                            public void                         initManagedPanel(String compaswframeworkcommoncoreProperties compaswframeworkuiswingManagedPanelContext)                            public compaswframeworkuiswingspiManagedUIElement                         createManagedUIElement(String)                            public void                         disposeManagedPanel()                            public void                         onInteractorEvent(compaswframeworkcommonextensioninteractionInteractorEvent)                            private compaswframeworkuiswingspiManagedUIElement                         setupTextOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupConsoleOutputControl(String)                            public void                         updateConsoleOutputControl(String)                            private compaswframeworkuiswingspiManagedUIElement                         setupHTMLOutputControl()                            private compaswframeworkuiswingspiManagedUIElement                         setupModelViewerControl()                            public String                         getTextContainerData(String)                            public                         static boolean                         exportToHTML(compaswframeworkcommoncontainerContainer String)                            public                         static javaioFile                         getHTMLFileTest(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractHTMLCompoundContainer(compaswframeworkcommoncontainerContainer String boolean)                         throws javaioIOException                            private                         static void                         extractZip(String String boolean)                         throws javaioIOException                            public                         static javaioFile                         getHTMLFile(String String boolean)                         throws javaioIOException                    
                

comspsssharedcustom_guiui_builderpeersManagedTextOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedTextOutputArea                         extends javaxswingJEditorPane                         implements compaswframeworkuiswingspiManagedUIElement compaswframeworkuiswingcomponentPrintableProvider                             private ManagedBasePanel                         containerPanel                            private compaswframeworkuiswingSwingResourceProvider                         resources                            public void ManagedTextOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public void                         setData(Object)                            public javaxswingJComponent                         getComponent()                            public compaswframeworkuiswingSwingResourceProvider                         getResources()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                            public javautilList                         getPrintables(int javaawtprintPageFormat)                            public String                         getID()                    
                

comspsssharedcustom_guiui_builderpeersManagedScoreSyntaxControl$ButtonsActionclass

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ManagedScoreSyntaxControl$ButtonsAction                         implements javaawteventActionListener                             private void ManagedScoreSyntaxControl$ButtonsAction(ManagedScoreSyntaxControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

comspsssharedcustom_guiui_builderpeersManagedEmptyOutputAreaclass

                        package comspsssharedcustom_guiui_builderpeers                        public                         synchronized                         class ManagedEmptyOutputArea                         extends javaxswingJPanel                         implements compaswframeworkuiswingspiManagedUIElement                             public void ManagedEmptyOutputArea()                            public void                         initUIElement(String compaswframeworkuiswingSwingResourceProvider)                            public javaxswingJComponent                         getLayoutComponent()                            public javautilList                         getFocusComponents()                            public javaxswingJComponent                         getComponent()                            public boolean                         isCopy()                            public boolean                         hasSelection()                            public boolean                         canTransfer(javautilList)                            public javautilList                         getTransferData(String boolean)                            public void                         transferData(javautilList)                    
                

comspsssharedcustom_guiui_builderpeersROutputFileChooserControl$1class

                        package comspsssharedcustom_guiui_builderpeers                        synchronized                         class ROutputFileChooserControl$1                         implements javaawteventActionListener     void ROutputFileChooserControl$1(ROutputFileChooserControl)                            public void                         actionPerformed(javaawteventActionEvent)                    
                

extensionxml

imagescopygif

imagesdeletegif

imagesfileMenugif

imagesgenerateMenugif

imageshelp_buttongif

imageslg_gm_rpng

imageslg_r_cdb_gmgif

imageslg_r_cdb_modgif

imageslg_r_cdb_outputgif

imageslg_r_cdb_processgif

imagespreview_tablegif

randomForestxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

IBM SPSS Modeler and R

6 Your R code7

8 ibmspsscfdatamodelSetDataModel(modelerDataModel)

9 ibmspsscfdataSetData(modelerData)

10

All the functions starting with ibmspsscf are functions within this library It is this part of the codethat is responsible for the transfer of the data and the metadata to R Also you will see a while loopindicating that the data will go to R in batches of 1000 The other options are the values of the data readoptions within the nodeThe last lines of the code prepares the data to be send back to SPSS Also note the closing brace whichis the end of the while loop

Now since this library will be loaded with every interaction between SPSS and R you are free to use thesefunctions within your code as well So if you would like to avoid these runs in batches of 1K and you donot have version 171 available you can start your R code with another loop to first continue filling themodelerData and only than start your actual code In practice that means that in your R node you startwith

1 while(ibmspsscfdataHasMoreData())

2

3 modelerData lt-rbind(modelerDataibmspsscfdataGetData(rowCount=100000missing=NArDate=None

logicalFields=FALSE))

4

5

6 Some more code using modelerData which is now the complete dataset

512 Some useful parts of R code

Make sure a package is installed Whenever you have created any R node using packages you have tomake sure that anyone using this node has this package on his machine installed if needed withouttoo much user interference Therefore you can use this part of code

1 packages lt- function(x)

2 x lt- ascharacter(matchcall()[[2]])

3 if (require(xcharacteronly=TRUE))

4 installpackages(pkgs=xrepos=httpcranr-projectorg)

5 require(xcharacteronly=TRUE)

6

7

8

9 packages(rpart)

It will firstly verify if the library is installed on the system if not it will silently install it fromthe given CRAN mirror (you can change this to another mirror or a local repository if needed)Installation will only happen the first time the node will be used

Create the metadata corresponding to the R data frame Sometimes your data is very much trans-formed compared to the original data that it is difficult to build your metadata starting from theoriginal one So you might want to change the metadata to link to the data in R independent ofthe original This is in particularly useful if you want to use R as a sort node Below is a functionthat asks for a data frame and creates modelerData and modelerDataModel accordingly

1 sendToModeler lt- function (dataFrame)

2 if(isnull(dim(dataFrame)))

3 stop(Invalid data received not a dataframe)

4 if (dim(dataFrame)[1]lt=0)

5 print(Warning modelerData has no line all fieldStorage fields set to strings)

6 getStorage lt- function(x)return(string)

7 else

8 getStorage lt- function(x)

9 x lt- unlist(x)

10 res lt- NULL

11 if x is a factor typeof will return rsquointegerrsquo so we handle this case first

Page 15 of 20

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 15: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

6 Your R code7

8 ibmspsscfdatamodelSetDataModel(modelerDataModel)

9 ibmspsscfdataSetData(modelerData)

10

All the functions starting with ibmspsscf are functions within this library It is this part of the codethat is responsible for the transfer of the data and the metadata to R Also you will see a while loopindicating that the data will go to R in batches of 1000 The other options are the values of the data readoptions within the nodeThe last lines of the code prepares the data to be send back to SPSS Also note the closing brace whichis the end of the while loop

Now since this library will be loaded with every interaction between SPSS and R you are free to use thesefunctions within your code as well So if you would like to avoid these runs in batches of 1K and you donot have version 171 available you can start your R code with another loop to first continue filling themodelerData and only than start your actual code In practice that means that in your R node you startwith

1 while(ibmspsscfdataHasMoreData())

2

3 modelerData lt-rbind(modelerDataibmspsscfdataGetData(rowCount=100000missing=NArDate=None

logicalFields=FALSE))

4

5

6 Some more code using modelerData which is now the complete dataset

512 Some useful parts of R code

Make sure a package is installed Whenever you have created any R node using packages you have tomake sure that anyone using this node has this package on his machine installed if needed withouttoo much user interference Therefore you can use this part of code

1 packages lt- function(x)

2 x lt- ascharacter(matchcall()[[2]])

3 if (require(xcharacteronly=TRUE))

4 installpackages(pkgs=xrepos=httpcranr-projectorg)

5 require(xcharacteronly=TRUE)

6

7

8

9 packages(rpart)

It will firstly verify if the library is installed on the system if not it will silently install it fromthe given CRAN mirror (you can change this to another mirror or a local repository if needed)Installation will only happen the first time the node will be used

Create the metadata corresponding to the R data frame Sometimes your data is very much trans-formed compared to the original data that it is difficult to build your metadata starting from theoriginal one So you might want to change the metadata to link to the data in R independent ofthe original This is in particularly useful if you want to use R as a sort node Below is a functionthat asks for a data frame and creates modelerData and modelerDataModel accordingly

1 sendToModeler lt- function (dataFrame)

2 if(isnull(dim(dataFrame)))

3 stop(Invalid data received not a dataframe)

4 if (dim(dataFrame)[1]lt=0)

5 print(Warning modelerData has no line all fieldStorage fields set to strings)

6 getStorage lt- function(x)return(string)

7 else

8 getStorage lt- function(x)

9 x lt- unlist(x)

10 res lt- NULL

11 if x is a factor typeof will return rsquointegerrsquo so we handle this case first

Page 15 of 20

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 16: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

12 if(isfactor(x))

13 res lt- string

14 else

15 res lt- switch(typeof(x) integer=integer double = real string)

16

17 return (res)

18

19

20 col = vector(list dim(dataFrame)[2])

21 for (i in 1dim(dataFrame)[2])

22 col[[i]] lt- c(fieldName= names(dataFrame[i]) fieldLabel = fieldStorage=

getStorage(dataFrame[i]) fieldMeasure = fieldFormat = fieldRole = )

23

24 mdmlt-docall(cbindcol)

25

26 modelerDataModelltlt-dataframe(mdm)

27 modelerData ltlt- dataFrame

28

29

30 sendToModeler(iris)

If you use this code you should make sure you only use this on data frames which are not verymuch dependent on the content of the original modelerData If ever results are not as expected youmight find an answer in 55

Looping through several variables is a very common functionality which is relatively easy in R How-ever with the custom dialog builder it might be more difficult as the string INPUTS will beexactly replaced by the string age + income + gender or something similar (you can change therdquo+rdquo sign by commas or spaces depending on the separator chosen)Now the problem is that in order to loop in R we need to transform this string into the R vectorc(age income gender) You can do that using this code

1 Create a function to remove trailing spaces

2 trim lt- function(x) gsub(^s+|s+$ x)

3

4 Create the vector of using the strsplit functions

5 InputsAsVector lt- trim(strsplit(INPUTS+)[[1]])

6

7 for (input in inputAsVector)

8 Some more code to run for every field defined9

One important remark is that this method will never work if you have variable names containingtrailing spaces or + symbols As this code will recognize every + as the symbol to separate thevariables and remove the trailing spaces It will be difficult in this way to distinguish between a +coming from a variable name or a + being a separator

Use predefined roles this is an option SPSS modeler users are used to and you might want to extendto R usage The idea is to distinguish between inputs and targets (and others) merely in the Type-node Once this is defined all the modeling nodes by default will use these settings and variablesSo you might want to use some R code to distinguish between inputs and targets As an examplethe below code will look for the flag target(s) and for all the input variables

1 TARGET lt- modelerDataModel[1(modelerDataModel[6] == target amp modelerDataModel[4] ==

flag)]

2 INPUTS lt- asvector(t(modelerDataModel[1(modelerDataModel[6] == input]))

3

4 Some more code

Removing columns Sometimes you just want to remove a column in both modelerData and modelerDataModel

You should make sure to delete the appropriate column as the link between data and metadatais merely the order Therefore you can use this R code to remove the column ttenure

1 define the remove function

2 removeColumn lt- function(name)

3 modelerDataModel[modelerDataModel[1]==name]ltlt-NULL

Page 16 of 20

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 17: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

4 modelerData[colnames(modelerData)== name] ltlt- NULL

5

6

7 apply the function to the tenure variable

8 removeColumn(tenure)

9

10 Some more code

52 Custom Dialog builder

521 How to save and share a custom dialog

Specification of a custom dialog can be saved to an external file with the extension cfd This can besaved and reopened through the general save buttons on the custom dialog toolbar

Once you deploy the dialog to your palette this will also be saved as a local file under a slightly differentextension cfe You can find this cfe file back in the path CProgramDataIBMSPSSModelerXXCDB)2

In order to share this node with others this node needs to be copied within the same folder on theother SPSS Modeler instance

522 Link to dialog and script

Regarding the Custom Dialog builder there are several more things to say We earlier stated that in thecode of the custom dialog builder That means that if you have an identifier called TARGET and you willfill in a variable churn in these dialog All the references of TARGET will be replaced in your code bychurn If you have multiple variables selected (say age and income in the identifier INPUTS and you se-lect rdquo+rdquo as the separator then within the code INPUTS will be replaced by the verbatim age + income

Although this is valid in default cases it is not entirely true There is still another level It will also

depend on the tool property in called rdquoR scriptrdquo Within this line the value ThisValue will be replaced by whatever you fill in the dialog describedbefore And it is the value of this R script property that will verbatim replace your identifier in the mainR code Now because often the value of R script is just ThisValue there is no use there

However this will start to be useful when you will work with radio buttons in which case you wouldlike to run different codes for each button In general there are 2 ways to go

harddirty way You create your R script using if statements

If rsquochoicersquo == rsquoArsquo then else if rsquochoicersquo == rsquoBrsquo then

easyclean way within your radio buttons write the full code that has to be run when that bullet isselected

Imagine the following scenario The user should select some variables some computations are done oneach of them Depending on the outcome of the computations and defined cutoffs The user can choosebetween any of these 3 actions

bull to remove the columnsbull to keep them as data but automatically set the role to rdquoNonerdquobull do nothing and just let the data flow back to modeler without changes

So now there are different levels you can work with You will probably first add a radio button group inthe dialog and call it WHATTODO

2Replace rsquoXXrsquo by the version of your IBM SPSS Modeler installation

Page 17 of 20

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 18: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

Now you see that the value of the R script variable is not changed Clicking through to the radio buttonitself you want it to look something like this

So there are the 3 option each with its corresponding R code

1 removeColumn(input) (refering to a function written to remove columns

2 modelerDataModel[6modelerDataModel[1]==input] lt- none this changes the role of thatfield to rdquoNonerdquo

3 Basically an empty string

With these settings your code WHATTODO will be replaced by the corresponding R code depending onwhich radiobutton the user prefers So the R script behind the dialog will be something like

1 trim lt- function (x) gsub(^s+|s+$ x)

2 inputAsVector lt- trim(strsplit(INPUTS)[[1]])

3

4 for (input in inputAsVector)

5 WHATTODO

6

7

8 Some more code

This approach simplifies the possibility of running different R codes depending on the end-users choice

53 What about SQL Pushback Hadoop pushback

SQL pushback supports R nodes for Pure Data for Analytics (PDA) SAP Hana and Oracle by utilizingtheir R support Databases need to have the appropriate vendor-provided R extensions installed Offcourse depending on vendor a subset of libraries or scripts are supported

We will show an example on R usage on PDA The first important thing to note is that only R nuggetsare available for SQL pushback The reason for this is that data will be divided onto the several spusof PDA R code will therefore independently be run on the different spus and never on the entire datatogether (this is the main reason for the issues discussed in 34)

1 This code cannot be pushed back to PDA as it is a model building node2

3 Create the model and save it in modelerModel

4 modelerModel lt- lm(tenure ~ data= modelerData)

1 This code can be pushed back to PDA as it is a model scoring node2

3 Make sure they are considered as factors as PDA will by default only have numerics

4 modelerData$marital lt- asfactor(modelerData$marital)5 modelerData$ed lt- asfactor(modelerData$ed)6 modelerData$region lt- asfactor(modelerData$region)7 modelerData$churn lt- asfactor(modelerData$churn)8

9 Use the model to make a prediction and add it to the existing data

10 pred lt- predict(modelerModel modelerData)

11 modelerData lt- cbind(modelerDatapred)

12

13 Take care of the metadata

14 newVar lt-c(fieldName=TenureSQLScore fieldLabel= fieldStorage=real fieldMeasure=

fieldFormat= fieldRole=)

15 modelerDataModel lt- cbind(modelerDataModelnewVar)

Page 18 of 20

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 19: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

One more thing to note here is that the modelerModel will always be a local object Within the R codethat is pushed back this object will have to be transferred to PDA behind the scenes This is not alwaysa problem however the size of modelerModel can often be quite big As an example a linear model forthe telco dataset will be approx 370kb (As a comparison the size where that model is run on will beapprox 50kb) So random forests or any other type of models can be huge This is something that needsto be considered when using this approach

Since everything runs on the different spus this also means that R and all the necessary libraries has tobe installed on both the host and every spu of PDA For the R libraries you can in generally not do thatin the same way as loading R libraries on a local system as PDA will generally be not connected to theinternet How to solve this is more a question of the R-PDA link rather than SPSS However here is asmall R script to load a package on the PDA from local R instance

1 Note this will be a local R script2

3 Load PDA local R libraries

4 library(nzr)

5 library(nza)

6

7 Connect to the appropriate DSN

8 nzConnectDSN(rsquoPDA-DSNrsquo)

9

10 install the library on PDA

11 nzInstallPackages(httpcranr-projectorgsrccontribrpart_41-10targz)

This will install the package rpart onto PDA (assuming you are logged on with appropriate credentials)You can choose to run this code in native R or within SPSS in a R output node Both will work Once itis installed on the PDA you can use the libraries as normal

A stream showing all of these scripts is attached here

54 What about real-time scoring and Solution Publisher

There is really not much to say about these things except for the fact that it is supported There is justone minor point to mention You should make sure the R as well as the R extensions are installed at somemore location

bull For solution publisher you should just install the R extensions in the extbin directory of yourSolution Publisher Install R on the machine where your solution publisher is installed

bull For real time scoring you should just ensure that the R extensions are installed on the componentsmodelerextbin directory of both your server as your scoring server Off course functionally onlyR transform and R nuggets are relevant for this part Install R on the machine where your applicationserver (websphere) is installed

55 Something more about the metadata in modeler and the conse-quences on R integration

Metadata in SPSS Modeler is something particular It is very important for the way SPSS Modeler worksSo important in fact that SPSS Modeler will know at all times all the metadata of the data at every nodewithin the stream You may have already noticed that when you add a new field (with a derive node)all the type nodes downstream will immediately take into account this extra field In order to do thisbehind the scenes modeler let some small dummy data flow around This data only has the purpose toverify the metadata in near real time

If you want to know what this dummy data looks like you can add an R transform node just afterthe source node and use the following syntax to write out the data that is passed by SPSS through R andback This code doesnrsquot do anything with the data it just writes it back into a file

1 path lt- Ctesttxt

2 sink(path)

3 writeLines(ascharacter(Systime()))

Page 19 of 20

SQLxml

data0001dat

data0002dat

data0003dat

data0004dat

data0005dat

data0006dat

data0007dat

data0008dat

data0009dat

data0010dat

data0011dat

data0012dat

data0013dat

data0014dat

data0015dat

data0016dat

data0017dat

data0018dat

data0019dat

data0020dat

data0021dat

data0022dat

data0023dat

data0024dat

data0025dat

data0026dat

data0027dat

data0028dat

data0029dat

data0030dat

data0031dat

data0032dat

data0033dat

data0034dat

data0035dat

data0036dat

data0037dat

data0038dat

data0039dat

data0040dat

data0041dat

data0042dat

data0043dat

data0044dat

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration
Page 20: Introduction to R in IBM SPSS Modeler:

IBM SPSS Modeler and R

4 writeLines(Data)

5 print(modelerData)

Once these lines are added to the R transform node and we continue in creating the stream we will seethat this file will already be populated without even running anything You can see the data that is passedthrough the node only contains 5 lines of data with 1-2-3-4-5 and rdquoa-b-c-d-erdquo (some enclosed by quotesfor the string variables) depending on the metadata These 5 lines of dummy data modeler are the oncegoing around the stream every time SPSS Modeler needswants to check the metadataYou can even append this script with print(modelerModel) to see the value of modelerModel is notyet assigned That also means that modelerModel cannot place any role into the assigning of themodelerDataModel

Now this approach has some consequences with the R integration We placed it within this document asit will explain a lot of rdquostrangerdquo behavior in your projects where the reason is not always obviousAs an example let us assume the following exercise We have a multinomial logistic model with n differentcategories We want to have a column back for all of the categories A naive approach would be thefollowing

1 library(rpart)

2 modelerModel lt- rpart(custcat ~ tenure+age+income data = modelerData)

3 print (summary(modelerModel))

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in (((((((hhhhhhhcolnames(probs))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

Running this approach in R natively will produce the correct modelerData and modelerDataModel ob-jects However this approach will not work in SPSS because modelerModel is not assigned when modelerassesses the metadata Therefore this code will be run with an empty modelerModel and the 5 dummyrecords As a result probs will be empty so nothing within the for-loop will run

A workaround in this case is to derive the number of columns not from the modelerModel but in anotherway One approach can be

1 library(rpart)

2 probs lt- predict(modelerModelmodelerDatatype=prob)

3 modelerData lt- cbind(modelerDataprobs)

4

5 for (x in c(1234))

6 modelerDataModellt-cbind(modelerDataModel c(fieldName=paste($P-xsep = ) fieldLabel=

fieldStorage=real fieldMeasure= fieldFormat= fieldRole=))

7

However also with this approach there is a problem as it needs to hardcode the values of the for loop

The reason behind this approach of SPSS Modeler is that this metadata should be available in nearreal time However modelerData and modelerModel are objects that can be very big and therefore willlead to a big delay obtaining this metadata

Page 20 of 20

  • System Setup
    • Installing R
    • Enabling the R nodes
      • R basics
      • The basics of R nodes in IBM SPSS Modeler
        • The R nodes
        • Simple R code example
          • modelerData
          • modelerDataModel
          • modelerModel
            • Some general remarks
            • Read data options
              • Custom Dialog builder
                • Tools
                • Custom dialog
                • Simple example
                  • Tips amp tricks Some more detailed
                    • R code
                      • ibmspsscf70 library
                      • Some useful parts of R code
                        • Custom Dialog builder
                          • How to save and share a custom dialog
                          • Link to dialog and script
                            • What about SQL Pushback Hadoop pushback
                            • What about real-time scoring and Solution Publisher
                            • Something more about the metadata in modeler and the consequences on R integration