UDAP Developers Guide

download UDAP Developers Guide

of 361

Transcript of UDAP Developers Guide

  • 8/11/2019 UDAP Developers Guide

    1/360

    IBMNetezzaAnalytics

    Release 2.5

    User-Defined Analytic Process

    Developer's Guide

    Revised: October 26, 2012

    Part Number 00J2009-03 Rev. 2

  • 8/11/2019 UDAP Developers Guide

    2/360

    Note: Before using this information and the product that it supports, read the information inNotices and Trademarkson page357.

    Copyright IB Corporation !"##, !"#!.

    $% &o'ernment $sers (estricted (ights ) $se, dup*ication or disc*osure restricted +y &% - %chedu*e Contract /ith IB Corp.

  • 8/11/2019 UDAP Developers Guide

    3/360

    Contents

    Prefaceudience for This &uide.................................................................................................................0'ii

    urpose of This &uide....................................................................................................................0'ii

    %ym+o*s and Con'entions..............................................................................................................0'ii

    If 1ou Need 2e*p............................................................................................................................0'iii

    Comments on the -ocumentation...............................................................................................0'iii

    1 Introduction%amp*e -ataset Configuration...............................................................................................................#

    %4 6unctions.........................................................................................................................................#

    $ser -efined rocess 'er'ie/.............................................................................................................#

    $-8s................................................................................................................................................!"

    na*ytic 90ecuta+*es 9s;...............................................................................................................!#

    rogramming ode*..............................................................................................................................!>

    Code................................................................................................................................................>>

    Compi*ation.....................................................................................................................................>>

    (egistration.....................................................................................................................................>>

    v

  • 8/11/2019 UDAP Developers Guide

    6/360

    (unning...........................................................................................................................................>

    Ea'a anguage Con'ersion.....................................................................................................................>

    Code................................................................................................................................................>

    Compi*ation.....................................................................................................................................>

    (egistration.....................................................................................................................................>

    (unning...........................................................................................................................................>

    6ortran anguage Con'ersion................................................................................................................"

    Code................................................................................................................................................"

    Compi*ation....................................................................................................................................."

    (egistration....................................................................................................................................."

    (unning..........................................................................................................................................."

    ython anguage Con'ersion................................................................................................................."

    Code................................................................................................................................................"

    -ep*oyment.....................................................................................................................................#

    (egistration.....................................................................................................................................#

    (unning...........................................................................................................................................#

    er* anguage Con'ersion......................................................................................................................#

    Code................................................................................................................................................#-ep*oyment.....................................................................................................................................#

    (egistration.....................................................................................................................................#

    (unning...........................................................................................................................................!

    * )able Function +'i%ulated Ro# Function,

    Exa%plesC anguage %imu*ated (o/ 6unction.....................................................................................................3

    Code................................................................................................................................................3

    Compi*ation.....................................................................................................................................=

    (egistration.....................................................................................................................................7

    (unning...........................................................................................................................................7

    CDD anguage %imu*ated (o/ 6unction.................................................................................................7

    Code................................................................................................................................................7

    Compi*ation.....................................................................................................................................

    (egistration.....................................................................................................................................

    (unning...........................................................................................................................................

    Ea'a anguage %imu*ated (o/ 6unction................................................................................................

    Code................................................................................................................................................

    Compi*ation...................................................................................................................................#"!

    (egistration...................................................................................................................................#"!

    (unning.........................................................................................................................................#"!

    6ortran anguage %imu*ated (o/ 6unction.........................................................................................#"!

    Code..............................................................................................................................................#"!

    Compi*ation...................................................................................................................................#"3

    (egistration...................................................................................................................................#"3

    vi

  • 8/11/2019 UDAP Developers Guide

    7/360

    (unning.........................................................................................................................................#"3

    ython anguage %imu*ated (o/ 6unction.........................................................................................#"

    x

  • 8/11/2019 UDAP Developers Guide

    11/360

    Contro**ing a (unning (emote 9........................................................................................................!"

    %ing*e Connection oint Commands.............................................................................................!#"

    u*tip*e Connection oint Commands.........................................................................................!#"

    Ta+*e 6unction utput Co*umns....................................................................................................!##

    2ost 90amp*es...............................................................................................................................!#!

    %$ 90amp*es................................................................................................................................!#!

    %ing*e Co*umn 90amp*es on the 2ost /ith utput.......................................................................!#3

    -ata Connection anagement for (emote 9s..................................................................................!#5

    9s that are +oth oca* and (emote....................................................................................................!#7

    **Inne pproach......................................................................................................................!#7

    6orking pproach..........................................................................................................................!#>

    1& 'i%ple Re%ote ode Exa%plesC anguage %ca*ar 6unction (emote ode;.......................................................................................!!#

    Code..............................................................................................................................................!!#

    Compi*ation...................................................................................................................................!!=

    (egistration...................................................................................................................................!!=

    (unning.........................................................................................................................................!!7

    CDD anguage %ca*ar 6unction (emote ode;...................................................................................!!7

    Code..............................................................................................................................................!!7

    Compi*ation...................................................................................................................................!3"

    (egistration...................................................................................................................................!3"

    (unning.........................................................................................................................................!3#

    Ea'a anguage %ca*ar 6unction (emote ode;..................................................................................!3#

    Code..............................................................................................................................................!3#

    Compi*ation...................................................................................................................................!3

    -ep*oyment...................................................................................................................................!3>

    (egistration...................................................................................................................................!3>

    (unning.........................................................................................................................................!3>

    xi

  • 8/11/2019 UDAP Developers Guide

    12/360

    1( Advanced Re%ote ode Exa%plesC 90amp*e u*tip*e %imu*taneous Connections;................................................................................!

  • 8/11/2019 UDAP Developers Guide

    13/360

    Aa*grind..........................................................................................................................................!7>

    Ca**grind.........................................................................................................................................!>!

    $sing the Integrated Ea'a -e+ugger....................................................................................................!>5

    $sing the IB Netea 9c*ipse *ugIn.................................................................................................!>

    1- Debugging Exa%plesC anguage ogging and (untime Information....................................................................................!3

    Code..............................................................................................................................................!3

    Compi*ation...................................................................................................................................!7

    (egistration...................................................................................................................................!7

    (unning.........................................................................................................................................!7

    CDD anguage ogging and (untime Information................................................................................!>

    Code..............................................................................................................................................!>

    Compi*ation...................................................................................................................................3""

    (egistration...................................................................................................................................3"#

    (unning.........................................................................................................................................3"#

    Ea'a anguage ogging and (untime Information...............................................................................3"#

    Code..............................................................................................................................................3"#

    Compi*ation...................................................................................................................................3"

    (unning.........................................................................................................................................3">

    ython anguage ogging and (untime Information..........................................................................3">

    Code..............................................................................................................................................3"-ep*oyment...................................................................................................................................3"

    (egistration...................................................................................................................................3#"

    (unning.........................................................................................................................................3#"

    er* anguage ogging and (untime Information................................................................................3#"

    Code..............................................................................................................................................3##

    -ep*oyment...................................................................................................................................3#!

    (egistration...................................................................................................................................3#!

    (unning.........................................................................................................................................3#!

    1. Integrated Exa%plesEa'a anguage Integrated 90amp*e......................................................................................................3#5Concepts........................................................................................................................................3#5

    Test -ata........................................................................................................................................ 3#5

    pp*yperation.............................................................................................................................3#=

    pp*y-ri'er Aersion #....................................................................................................................3#>

    xiii

  • 8/11/2019 UDAP Developers Guide

    14/360

    pp*y(esu*t....................................................................................................................................3!"

    pp*y$ti*........................................................................................................................................3!3

    pp*y-ri'er Aersion !....................................................................................................................3!7

    C*one(o/s.....................................................................................................................................33!

    -ataConnection.............................................................................................................................335

    pp*y-ri'er Aersion 3....................................................................................................................337

    pp*y-ri'er Aersion

    Ta+*e 3: rder of 'aria+*e parsing........................................................................................................#77

    Ta+*e

  • 8/11/2019 UDAP Developers Guide

    15/360

    Ta+*e =: naeHo+contro* argument desciptions....................................................................................!"

    Ta+*e 7: naeHo+contro* returned co*umns..........................................................................................!##

    Ta+*e >: %amp*e ist*ogs 6unction utput............................................................................................!=3

    Ta+*e : %amp*e Aie/*og 6unction utput...........................................................................................!=, in the path shou*d match the data+ase for

    /hich the 9 is registered.

    4oteBThis does not app*y to the standard Netea *atform %oft/are N%; +ackup and restore

    2- 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    29/360

    Accessing t$e AE Export Directory )ree

    con'ention.

    Products Directory

    The products directory contains indi'idua**y /rapped 9 products. The products su+directory shou*d

    +e organied as:/n0/expo$t/ae/p$o4ucts/compan/p$o4uct/%e$sion

    !or"space Directory

    The /orkspace directory is used for temporary and semitemporary data that is created +y a specific

    product. The fo**o/ing path shou*d +e used:

    /n0/expo$t/ae/wo$2space/E.NDOR7DIR/PROD3CT8 C$oss8use$ 4ata 'o$ a gi%en p$o4uct/n0/expo$t/ae/wo$2space/use$s/use$name/E.NDOR7DIR/PROD3CT8 Single8use$ 4ata 'o$ a gi%en p$o4uct

    Adapters Directory

    The adapters directory, ne0portaeadapters, contains the functiona*ity to adapt a gi'en *anguage

    to the N% system so that it can +e ca**ed as an 9. These adapters define the interface and ena+*e

    programming of 9s in a particu*ar *anguage.

    The on*y su+directories that shou*d reside under ne0portaeadapters are those named for the

    *anguages supported +y the adapters. $nder the *anguage, there shou*d +e a 'ersion that app*ies to

    the specific adapter, for e0amp*e:

    /n0/expo$t/ae/a4apte$s/language8name/84igit8%e$sion/

    anguages DirectoryThe *anguages directory contains the *anguageon*y insta**ations for the %$ and the host.

    %u+directories typica**y found in this directory might inc*ude XHa'aX or Xpython.X

    These su+directories are organied as:

    /n0/expo$t/ae/languages/language8name/language8%e$sion/spu/8 The spu %e$sion o' the language/n0/expo$t/ae/languages/language8name/language8%e$sion/host/8 The host %e$sion o' the language

    Core Directory

    The core directory, *ocated at ne0portaecore, contains groups of Netea na*ytics functiona*itypro'ided as part of the core system. These functiona*ity too*s may inc*ude Inte* , Boost, and

    pen I. If there are thirdparty too*s needed +y the core functiona*ity, the too*s are insta**ed

    in ...coretoo*s. If the too* must +e used outside the XcoreX functiona*ity, then it is insta**ed into

    the ...products directory instead.

    The su+directory shou*d +e organied as fo**o/s:

    00?200.@03 Rev9 2 2.

  • 8/11/2019 UDAP Developers Guide

    30/360

    =ser@Defined Analytic Process Developers 5uide

    /n0/expo$t/ae/co$e/catego$/

  • 8/11/2019 UDAP Developers Guide

    31/360

    Co%pilation and Registration >vervie#

    Co%pilation and Registration >vervie#

    n 9 insta**ation uses a system /here code is compi*ed, dep*oyed, and then registered. T/o

    command *ine uti*ities are used to perform these tasks, co%pileae, /hich is used for compi*ation

    and dep*oyment, and registerae, /hich registers the 9 and connects it to the N%.

    Both uti*ities use a series of options, such as @@language, @@user, @@db, and @@te%plate, /hich define

    'arious configurations for the compi*ation and registration processes.

    =ser Definition

    The same user must +e used for +oth co%pileaeand registerae. If the @@useroption is not

    specified, the 'a*ue set +y the NV@$%9( en'ironment 'aria+*e is used +y defau*t. If the @@users/itch

    is supp*ied in on*y one command and NV@$%9( does not match the 'a*ue specified +y @@user,

    co%pileaedep*oys the fi*e to a different *ocation than that used +y registerae, causing an error.

    Extending co%pileae and registerae

    Both co%pileaeand registeraeare imp*emented using the Perl +e$plate +ool,it. *though not

    reMuired, e0tending the co%pileaeor registeraesystem may +e usefu*. In this e'ent, it is

    recommended that you create ne/ temp*ate fi*es instead of modifying the Netea temp*ates. 3

    ssuming the 9 e0port directory is *ocated at ne0portae, the e0isting temp*ates are *ocated at:

    /n0/expo$t/ae/a4apte$s/language/%e$sion/templates

    The @@ldiroption can +e used to point to an a*ternati'e adapters directory.

    The @@defineoption can +e used to set specific temp*ate 'aria+*es. 6or e0amp*e:

    884e'ine template%a$"%alue

    Co%piling and Deploying Analytic Executables

    n 9 insta**ation inc*udes the co%pileaecommand *ine uti*ity, /hich is used to compi*e a**

    *anguages and dep*oy the resu*t. %cripting *anguages such as ython a*so use co%pileaefor

    dep*oyment, as it is a/are of the 9 +ackup process used +y the N%. The uti*ity can a*so +e used

    indirect*y as a source of information /hen creating a custom compi*e process and su+seMuent

    dep*oyment. $sing co%pileae @@$elppro'ides he*p a+out usage and options.

    To see 'er+ose output from the co%pileaeuti*ity, use the @@ec$ooption. The output descri+es /hat

    is reMuired to compi*e 9 code and the information can then +e used in the custom +ui*d process.-epending on /hich programming *anguage is used, 9 compi*ation may reMuire shared *i+raries,

    inc*ude fi*es, and E( fi*es. 6or *anguages *ike CDD and 6ortran, /hich are compi*ed into nati'e

    3 The fo**o/ing resources may +e usefu* /hen e0tending co%pileaeor registeraeoptions: Perl +e$plate +ool,it+y

    -arren Cham+er*ain, -a'id Cross, and ndy ?ard*ey, U(ei**y, !""3, I%BN: "5=""

  • 8/11/2019 UDAP Developers Guide

    32/360

    =ser@Defined Analytic Process Developers 5uide

    machine code, output e0ecuta+*e o+Hects are produced for +oth the host and %$s.

    The +asic synta0 of co%pileaeis:

    compile7ae options sou$ce7'ile7names

    This section pro'ides a summary of the most important information a+out the co%pileaeuti*ity.

    4oteBost of the options +egin /ith t/o hyphens: FG /hich may +e difficu*t to 'ie/ in 'arious

    screen+ased documentation formats.

    =ser Configuration before Co%pilation

    The co%pileaeuti*ity can a*/ays +e run +y root or the n user. 2o/e'er, data+ase users other than

    root and n must +e added to the inu0 group n/+efore they can compi*e an 9. $sers can +e added

    to the group using the fo**o/ing command as user root, /here n specifies the group:

    gpassw4 8a use$ n0

    File ocationsThe *ocation of fi*es produced +y co%pileaeand input fi*es to registeraedefau*ts to:

    +. .xpo$t Di$ecto$/ae/applications/ae741/417use$name

    The *ocation of the 9 e0port directory is usua**y contained in the NV@98(T@-I( inu0

    en'ironment 'aria+*e. The defau*t *ocation of input fi*es can +e modified.

    Co%%on >ptions

    The fo**o/ing are common options of the co%pileaeuti*ity.

    @@language syste% cpp Gava fortran perl pyt$on rHIdentifies the programming *anguage used to /rite the 9. ccepted options are system for the +ase

    C I;, cpp for CDD;, Ha'a, fortran, per*, python, and r

  • 8/11/2019 UDAP Developers Guide

    33/360

    Co%piling and Deploying Analytic Executables

    6or e0amp*e:

    88template compileHa$

    @@version aeversion6%ets the 9 'ersion in use. This 'a*ue is tied to the current IB Netea ana*ytics 'ersion, and is used

    to track incompati+i*ity. 1ou can disp*ay the contents ofne0portaeadapterssystem@ae to *ist

    'ersions.; 6or e0amp*e:

    88%e$sion ptions

    @@sig function+arg6,6%pecifies the %4 function name and input argument signature. The signature must +e dou+*e

    Muoted to pre'ent the she** from interpreting the parentheses *itera**y. 6or e0amp*e:

    3( 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    37/360

    Registering Analytic Executables

    88sig JapplResult(%a$cha$()& 4ou1le)J

    It is possi+*e to define a function that takes a 'aria+*e num+er of arguments using the X'arargsX

    key/ord. 6or e0amp*e:

    88sig JapplOpe$ation(%a$a$gs)J

    6or a comp*ete *ist of supported data types, see the Netezza Data#ase User%s Guide. 6or more

    information a+out signatures, see the Netezza User-Defined Functions Developer%s Guide.

    @@return valuetype6%ets the return 'a*ue or return ta+*e of the function. 6or e0amp*e, for a sca*ar function or aggregate:

    88$etu$n 4ou1le

    6or a ta+*e function:

    88$etu$n Jta1le(total 4ou1le& la1el %a$cha$(

  • 8/11/2019 UDAP Developers Guide

    38/360

    =ser@Defined Analytic Process Developers 5uide

    88%e$sion ptions

    The fo**o/ing options are a'ai*a+*e for sca*ar functions on*y.

    @@nondet%ets that this sca*ar function is nondeterministic. In other /ords, it negates the defau*t optimiation,

    /here the N% system assumes that for the same input a sca*ar function a*/ays returns the same

    output. In this case, the sca*ar function may on*y +e in'oked once for constant input instead of once

    per input ro/. 6or e0amp*e:

    88non4et

    @@nullcallNegates the defau*t N$ ca** +eha'ior, a**o/ing a sca*ar function that recei'es a** N$ input to

    return nonN$ output. By defau*t, the N% system assumes that for any N$ input a sca*ar

    function a*/ays returns N$ output and thus does not in'oke the sca*ar function. 6or e0amp*e:

    88nullcall

    )able Function >ptions

    The fo**o/ing option is a'ai*a+*e for ta+*e functions on*y.

    @@noparallel(uns a** Muery data sing*estreamed through a sing*e 9, effecti'e*y causing the 9 torun on the host

    /hi*e processing a** data from the datas*ices. 6or e0amp*e:

    88nopa$allel

    Aggregate Function >ptionsThe fo**o/ing options are a'ai*a+*e for aggregate functions on*y.

    @@state%pecifies the signature of state 'aria+*es. 6or more information on state 'aria+*es, refer to the N%

    guide, IBM Netezza User-Defined Functions Developers Guide.; 6or e0amp*e:

    88state J(4ou1le)J

    @@aggtype ANYANALYTIC | GROUPED}%pecifies the aggregate type. 6or more information on aggregate types, refer to the N% guide, IBM

    Netezza User-Defined Functions Developers Guide.; 6or e0amp*e:

    88aggtpe +NB

    00?200.@03 Rev9 2 3

  • 8/11/2019 UDAP Developers Guide

    44/360

    =ser@Defined Analytic Process Developers 5uide

    4P' 'yste% '$ared ibrary Dependencies

    n 9 can dec*are dependencies on one or more of the N% system shared *i+raries. 6or information

    on shared *i+raries, see C(9T9 [( (9C9\ IB((1 in the Netezza User-Defined Functions

    Developer%s Guide. *so, see ccessing the oca* -ri'e %pace on the achine.; ?hi*e shared *i+raries

    for 9s may +e actua* inu0 o+Hect *i+^.so fi*es, they can +e any fi*e type.@@deps libNlibN999lib6-ec*ares shared *i+rary dependencies. i+rary names must +e commaseparated /ith no spaces. 6or

    e0amp*e:

    884eps mli1&mli1pti%i/ation

    This section descri+es options that pro'ide information to N% that can then +e used in p*anning and

    schedu*ing 9s.

    @@%e% value6Ob " % g%pecifies the amount of memory the 9 is e0pected to use. emory is typica**y entered as a num+er

    and, optiona**y, a unit. Aa*id units are b+ytes;, ki*o+ytes;, mmega+ytes;, !giga+ytes;. Thedefau*t unit is +ytes. The N% app*iance uses this 'a*ue for p*anning and schedu*ing Mueries.

    90amp*es:

    Exa%ple88mem ##88mem m88mem ##2

    Co%plete ist of registerae >ptions

    This section pro'ides a comp*ete *ist of options for "e!i#te"_ae. The synta0 for option usageoptions

    is:

    FN-7.GPORT7DIR/ae/utilities/1in/$egiste$7ae KOPTIONL

    To *ist the options:

    FN-7.GPORT7DIR/ae/utilities/1in/$egiste$7ae 88help

    5ene$al options 88echo .cho su1stitute4 template 1e'o$e $unning it as well as SQ, output

    00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    45/360

    Registering Analytic Executables

    88'ile 'ile 3se speci'ie4 template 'ile instea4 o' installe4 template

    881ase 1ase ?ase 4i$ecto$ 'o$ NPS instea4 o' en% N-7MIT7DIRo$ /n0/2it +lso a%aila1le as template %a$ia1le

    ae72it

    88l4i$ 1ase ?ase 4i$ecto$ 'o$ ,anguage con'igu$ations instea4 o' /n0/expo$t/ae/a4apte$s

    88language lang Speci' language con'igu$ation to use ,anguages a$e cpp& 'o$t$an& Ha%a& pe$l& pthon& $& sstem a%aila1le as template %a$ia1le ae7language

    88%e$sion %e$ Speci' language %e$sion to use +lso a%aila1le as template %a$ia1le ae7lang7%e$

    88template name Speci' the template to use Common ones a$e u4'& u4a& u4t'

    88use$ use$name NPS use$name (4e'aults to en% N-73S.R)

    +lso a%aila1le as template %a$ia1le ae7use$

    88pw passwo$4 NPS passwo$4 (4e'aults to en% N-7P+SS9ORD) +lso a%aila1le as template %a$ia1le ae7passwo$4

    8841 4ata1ase NPS 4ata1ase (4e'aults to en% N-7D+T+?+S.) +lso a%aila1le as template %a$ia1le ae741

    8h& 88help Show this comman4 usage

    5ene$al Template options P$e4e'ine4 template %a$ia1les inclu4e ae7expo$t74i$ ,ocation o' sha$e4 expo$t 4i$ecto$ ae7ss$oot7spu ,ocation o' sstem $oot on spu ae7ss$oot7host ,ocation o' sstem $oot on host

    ae7li1$a$7path7spu ,D7,I?R+RB7P+T@ 'o$ spu ae7li1$a$7path7host ,D7,I?R+RB7P+T@ 'o$ host 884e'ine %a$"%alue De'ine template %a$ia1le

    88path 4i$ ,ocation whe$e +. 'iles a$e sto$e4 De'aults to ae7expo$t74i$/ae/applications/ae741/ae7use$ +%aila1le as template %a$ia1le ae7application7path

    88wa$n 9a$n instea4 o' e$$o$ i' 88path 4oes not sta$t with ae7expo$t74i$/ae/applications/ae741

    88en%i$onment %al .n%i$onment ent$ ie 88en%i$onment J:name: " :%alue:J Can 1e speci'ie4 multiple times +%aila1le as template %a$ia1le

    ae7en%i$onment7ent$ies (a comma sepa$ate4 list)

    884eps li1s ,i1$a$ 4epen4encies ,i1s must 1e comma sepa$ate4 +%aila1le as template %a$ia1le ae74epen4encies

    88le%el le%el De1ug le%el 'o$ +. (4e'ault #) +%aila1le as template %a$ia1le ae74e1ug7le%el

    00?200.@03 Rev9 2 &

  • 8/11/2019 UDAP Developers Guide

    46/360

    =ser@Defined Analytic Process Developers 5uide

    88i4i$ 4i$ Install 4i$ 'o$ the +. language +%aila1le as template %a$ia1le ae7install74i$

    88u4xtpe tpe 3DG tpe *ust 1e one o' u4'& u4t' o$ u4a +%aila1le as template %a$ia1le ae7tpe

    88sig signatu$e +$gument signatu$e 'o$ 'unction o$ agg$egate

    3DGname(a$g& a$g

  • 8/11/2019 UDAP Developers Guide

    47/360

    Registering Analytic Executables

    as template %a$ia1le ae7nullcall

    Ta1le 6unction Template options 88nopa$allel Ta1le 'unction will 1e c$eate4 with pa$allel not allowe4 De'ault is pa$allel allowe4 +%aila1le as template %a$ia1le ae7pa$allel

    88lastcall a$gs Ta1le 'unction will 1e c$eate4 with a$gs +,,O9.D Possi1le %alues a$e :T+?,.:& :T+?,. 6IN+,: o$ :T+?,.& T+?,. 6IN+,: De'ault is :T+?,.& T+?,. 6IN+,: +%aila1le as template %a$ia1le ae7lastcall

    +gg$egate Template options 88state state State signatu$e 'o$ agg$egate (state& state

  • 8/11/2019 UDAP Developers Guide

    48/360

  • 8/11/2019 UDAP Developers Guide

    49/360

    C H A P T E R 3

    Running Function Analytic Executables

    Calling an AE fro% '

  • 8/11/2019 UDAP Developers Guide

    50/360

    =ser@Defined Analytic Process Developers 5uide

    Calling a )able Function

    ta+*e function can +e used any/here %4 a**o/s a ta+*e. The ta+*e function returns a ta+*e that

    may contain a num+er of ro/s *ess than, eMua* to, or greater than the num+er of input ro/s. 9s

    reMuire that the $IT% &INALc*ause +e used /ith a ta+*e function. 6o**o/ing is the synta0 for

    ca**ing a ta+*e function:T+?,. 9IT@ 6IN+,('unction7name(a$g& a$g 6RO* T+?,. 9IT@ 6IN+,(applOpe$ationET'(::& &

  • 8/11/2019 UDAP Developers Guide

    51/360

    =sing AEs to Exceed 4P' '

  • 8/11/2019 UDAP Developers Guide

    52/360

    =ser@Defined Analytic Process Developers 5uide

    'calar Functions

    sca*ar function that recei'es arguments that are a** *itera*s in a Muery /ith no user ta+*es runs on

    the host. sca*ar function that is deterministic and takes a** *itera*s runs on the host. ther/ise, the

    function may run on the host or %$s, depending on the decision of the Muery optimier. The sier,

    ho/e'er, a*/ays runs on the host e'en if the sca*ar function *ogic runs on the %$. 6or moreinformation on siers, seeIntroduction to %hapers and %iers.;

    )able Functions

    ta+*e function that recei'es arguments that are a** *itera*s runs on the host. ta+*e function

    registered as @@noparallelruns on the host. ** other ta+*e functions typica**y run on the %$s, +ut

    there are some situations /here the N% 4uery ptimier mo'es processing to the host.

    1our choice +et/een host and %$ /hether or not to use nopara**e*; for ta+*e functions has

    imp*ications.

    a ta+*e function 9 returning data at the end of the input may return more ro/s /hen running

    on the %$ than on the host, as it e0ecutes ntimes for ndatas*ices instead of a sing*e time on the

    host

    if the input is on the host e0terna* ta+*e, @'@dua*, or a temporary ta+*e in the p*an;, and the

    optimier chooses to e'a*uate it on the %$, it distri+utes the data to the %$s using random

    distri+ution.

    %hapers a*/ays run on the host, e'en if the ta+*e function *ogic runs on the %$. 6or more

    information on shapers, seeIntroduction to %hapers and %iers.;

    Aggregate Functions

    n aggregate function that recei'es arguments that are a** *itera*s in a Muery /ith no user ta+*es runson the host. grand aggregate /ith norma* ta+*es runs on +oth host and %$. grouped aggregate

    /ith norma* ta+*es runs on +oth host and %$s or on*y on the %$s. /indo/ed aggregate /ith no

    partition c*ause runs on the host in a** other cases, it runs on the %$s.

    Introduction to '$apers and 'i/ers

    ften /hen dea*ing /ith functions, you /ant the shape or sie of the output to +e dependent on the

    input. %hapers and siers ena+*e you to specify /hat the 9 is going to return that is, the output

    schema;.

    %haping and siing are +oth imp*emented using the same 9 I, ho/e'er the options are muchmore *imited for siing.

    '$apers

    %hapers used +y 9s are ca**ed from ta+*e functions to dynamica**y dec*are the return ta+*e

    &2 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    53/360

    Introduction to '$apers and 'i/ers

    metadata, specifica**y the num+er of co*umns, the co*umn names, types, and precisionsca*e.

    %hapers are in'oked /hen a ta+*e function is dec*ared as returning TB9N1;.

    6or *oca* 9s, a ne/ instance of the 9 process is created on the host to return metadata descri+ing

    the return ta+*e. The ne/ instance is ca**ed on*y to pro'ide this information and is not used for

    processing data. The 9 instance is ca**ed +efore any data is retrie'ed it recei'es metadata a+out the

    input arguments and can a*so retrie'e any arguments that are constant. 6or remote 9s, anotification is sent to an instance of the remote 9 running on the host. If no remote 9 instance

    e0ists on the host /ith an address that matches the %4 ta+*e function, the %4 ta+*e function

    returns an error.

    If an 9 is registered as a %4 ta+*e function that is dec*ared /ith return 'a*ue TB9N1;, the 9

    must +e /ritten to perform +oth XshapingX and data processing. The same process instance ne'er

    performs +oth, e0cept in the case of a remote 9. 9ach 9 *anguage has an I function that

    indicates /hether the 9 is +eing used as a shaper or as a norma* data processor.

    'i/ers

    %iers used +y 9s are ca**ed from sca*ar functions to dynamica**y dec*are the return sie of strings

    and numeric 'a*ues. 6or siing to occur, the %4 sca*ar function is dec*ared /ith return 'a*ue

    C2(N1;, A(C2(N1;, NC2(N1;, or NA(C2(N1; for strings and N$9(ICN1; for

    numeric 'a*ues. s /ith shapers, for *oca* 9s an instance for siing is created on the host, and for

    remote 9s, a notification is made on the host. The instance is ca**ed on*y to pro'ide this siing

    information and is not used for processing data. The 9 instance is ca**ed +efore any data is retrie'ed

    it recei'es metadata a+out the input arguments and can a*so retrie'e any arguments that are

    constant integers of type int< on*y not int#, int!, or int>;.

    %iers are more *imited than shapers, a**o/ing on*y specification of the sie of the string output or

    the precision and sca*e of a numeric output the type is specified in the registration process. %iers

    a**o/ access on*y to *itera* int3! fie*ds /ithin the input data.If an 9 is registered as a %4 sca*ar function, /hich is dec*ared /ith string return 'a*ue C2(N1;

    or N$9(ICN1;, the 9 must +e /ritten to perform +oth XsiingX and data processing. The same

    process instance ne'er performs +oth, e0cept in the case of a remote 9. 9ach 9 *anguage has an

    I function that indicates /hether the 9 is +eing used as a sier or as a norma* data processor.

    Introduction to AE anguage APIs

    9ach supported programming *anguage has its o/n specific I *i+rary. *though each I *i+rary is

    different, they fo**o/ simi*ar patterns and contain simi*ar I groupings. These groups consist of

    functions, methods, or c*asses, depending on the programming *anguage. This section pro'ides aconceptua* o'er'ie/ of the programming *anguage Is and offers a starting point for understanding

    the e0amp*es and using the *anguagespecific I references.

    o#@level API

    The *o/*e'e* I consists of a num+er of groups of functions responsi+*e for:

    00?200.@03 Rev9 2 &3

  • 8/11/2019 UDAP Developers Guide

    54/360

    =ser@Defined Analytic Process Developers 5uide

    hand*ing the connection +et/een the N% and 9s

    hand*ing the data f*o/, inc*uding data input and output, type and 'a*ues con'ersion, etc.

    Muerying the process en'ironment 'aria+*es e0ported +y the operating system as /e** as the N%

    defining the output signature, kno/n as FshapersG and FsiersG

    iterating through shared *i+raries associated /ith the 9 'ia the N% shared *i+rary mechanism. genera* usa+i*ity functions, inc*uding error reporting and gathering Neteare*ated runtime

    information :

    Initiali/ation API

    The initia*iations Is consist of the functions or c*asses used to create a data connection. These

    connections are reMuired +efore data can mo'e +et/een an 9 and the data+ase.

    90cept for the anguage -e'e*opment it -;, /hich is a C*anguage I, each *anguage supports

    initia*iation high*e'e* functiona*ity to create a data connection. They are:

    CDD: c*ass Naepi&enerator Ea'a: c*ass Naepi&enerator

    6ortran: su+routine nae(un;

    ython: deri'e from c*ass nae.e

    er*: deri'e from c*ass nae.e

    The C I contains initia*iation functions that operate on a *o/er *e'e*. %ome *anguages, such as CDD

    and Ea'a, a*so support these *o/er*e'e* I features. ?hen starting out /ith es other than /ith

    the -;, use the high*e'e* process to create a connection. The *o/er*e'e* I is most usefu* for

    remote 9 processes not *aunched +y N%.

    9ach of the *anguage reference guides inc*udes a section or page descri+ing initia*iation. ?hen using

    the higher *e'e* initia*iation, most of this documentation is not needed.

    The Ta+*e 6unction %imu*ated (o/ 6unction; 90amp*esand, in%imp*e (emote ode 90amp*es.

    %ca*ar 6unction (emote ode;G e0amp*es for the 'arious supported *anguages sho/ initia*iation in

    fu**, typica**y supporting +oth *oca* and remote 9. In these e0amp*es, for the - C I;, *o/ *e'e*

    initia*iation is sho/n. 6or the other *anguages, high*e'e* initia*iation is sho/n.

    Data Connections

    The resu*t of initia*iation is a data connection corresponding to one of the three 9 data connection

    types: function, aggregate, and shapersier. 9ach *anguage reference guide contains sections

    corresponding to these three data connection Is.

    The data connection Is are used to recei'e and return data /ith the data+ase. The 9 function I

    is used /ith %4 Ta+*e and %ca*ar 6unctions, the 9 ggregation I is used /ith %4 ggregate

    6unctions, and the 9 %haper%ier I is used /ith %4 Ta+*e6unction%hapers and %4 %ca*ar

    6unction%iers.

    The e0amp*es in %ca*ar 6unction 90amp*es,%imp*e Ta+*e 6unction 90amp*es,and Ta+*e 6unction

    & 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    55/360

    Introduction to AE anguage APIs

    %imu*ated (o/ 6unction; 90amp*esa** demonstrate the 9 6unction I. The %hapers and %iers

    90amp*esdemonstrate the 9 %haper%ier I. Theggregate 9 90amp*ese0amp*es demonstrate

    the 9 ggregate I.

    Record and Data )ype 'upport

    9ach 9 *anguage I imp*ementation contains common data structures used for a** three of the data

    connection Is. 6or e0amp*e, CDD and Ea'a ha'e an a+straction for input and output ro/s, a record

    c*ass containing co*umns ca**ed fie*ds. The c*ass for +oth is named Nae(ecord. The ython and er*

    9 I a+stract input co*umns as a *ist. anguages such as C and 6ortran treat co*umns indi'idua**y

    using inde0 function arguments and do not ha'e an e0p*icit record a+straction. 9ach *anguage *i+rary

    has an I for the N% data types it supports.

    ** of the e0amp*es demonstrate data type support.

    Advanced 4P' Features

    The programming *anguage Is contain functiona*ity for /orking /ith ad'anced N% features such

    as 9$-8 9n'ironment Aaria+*es and %hared i+raries, as demonstrated in the F9n'ironment _

    %hared i+rariesG e0amp*es.

    )$e API Road%ap

    n 9 first uses the initia*iation I to get a data connection. It then uses the I that corresponds

    to the connection type, either 6unction, ggregate, or %haper%ier. The 9 uses the data type

    support I to process co*umn and state 'a*ues.

    %tudy the *anguage e0amp*es in this guide to understand the +asic structure of an 9. 6urther

    information a+out the functions, methods, and c*asses found in the e0amp*es can +e found in the*anguagespecific I references.

    00?200.@03 Rev9 2 &&

  • 8/11/2019 UDAP Developers Guide

    56/360

  • 8/11/2019 UDAP Developers Guide

    57/360

    !or"ing #it$ Exa%ples

    C H A P T E R 4

    !or"ing #it$ Exa%ples

    The e0amp*es presented in this guide are intended to demonstrate functiona*ity of 'arious types of 9s

    in the *anguages supported +y the Netea soft/are.

    Assu%ptions

    Compi*ation and registration e0amp*es that specify the @@dboption assume that there is a 'a*id

    data+ase ca**ed Fde'G on the Netea app*iance.

    To take ad'antage of the e0amp*es as they are +ui*t on through this manua*, it is important to use

    the correct fi*e name. ?hen sa'ing the samp*e code, use the name supp*ied at the +eginning ofthe Code section. This is a*so the name used in the compi*e step.; The same fi*e name /i** +e

    ca**ed *ater in su+seMuent e0amp*es.

    Compi*ation and registration e0amp*es that specify the @@useroption assume that there is a 'a*id

    data+ase user named FnG. This user shou*d not +e confused /ith the $NI8 username.

    ** e0amp*es presented in this guide assume that the reader is /orking direct*y on an IB Netea

    host. In'oking scripts such as registerae, editing source fi*es, and other acti'ities must +e

    performed on the Netea app*iance. %ome of the presented commands may not e0ist on the *oca*

    desktop machine.

    Editor

    ?hen /orking direct*y on the Netea app*iance, the n*ugin for 9c*ipse I-9 cannot +e used for

    editing fi*es. If fi*es must +e edited, use one of the editors a'ai*a+*e in the standard inu0 distri+ution

    e.g., 'i; or edit the source fi*es *oca**y and then transfer them to the app*iance using a uti*ity such as

    scp, sftp, or 9c*ipse.

    &*

  • 8/11/2019 UDAP Developers Guide

    58/360

  • 8/11/2019 UDAP Developers Guide

    59/360

    C H A P T E R 5

    'calar Function Exa%ples

    The concepts co'ered in this section inc*ude:

    %ca*ar functions

    9rror hand*ing

    oca* mode

    C anguage 'calar Function

    The fo**o/ing e0amp*e sho/s a simp*e sca*ar function that sums a set of num+ers. This e0amp*e

    starts from the +eginning to construct a simp*e 9. It uses the fo**o/ing fi*e name:

    'uncc

    Code

    19 u** in the C interface. This can typica**y +e accomp*ished using:

    Vinclu4e Jn0aeapishJ

    29 ?rite a main. -etermine an I to use, in this case a function. %e*ect eithera *oca* 9

    connection or a remote 9 connection. This e0amp*e uses a *oca* 9 connection:

    Vinclu4e st4iohVinclu4e st4li1hVinclu4e Jn0aeapishJ

    int main(int a$gc& cha$ > a$g%KL){ i' (n0aeIs,ocal()) { N0aeInitiali0ation a$g; memset(Wa$g& #& si0eo'(a$g)); a$gl42Ee$sion " N-+.7,DM7E.RSION; i' (n0aeInitiali0e(Wa$g))

    00?200.@03 Rev9 2 &.

  • 8/11/2019 UDAP Developers Guide

    60/360

    =ser@Defined Analytic Process Developers 5uide

    { 'p$int'(st4e$$& Jinitiali0ation 'aile4XnJ); $etu$n 8; ! n0aeClose(a$ghan4le); ! else { 'p$int'(st4e$$& J.xpecting ,ocal +.XnJ); $etu$n 8; ! $etu$n #;!

    This sets up nae, /hich ena+*es the a+i*ity to /ork /ith 9 functions.

    39 dd a stu+ for ca**ing the function *ogic:

    Vinclu4e st4iohVinclu4e st4li1h

    Vinclu4e Jn0aeapishJstatic int $un(N-+.7@+ND,. h);int main(int a$gc& cha$ > a$g%KL){ i' (n0aeIs,ocal()) { N0aeInitiali0ation a$g; memset(Wa$g& #& si0eo'(a$g)); a$gl42Ee$sion " N-+.7,DM7E.RSION; i' (n0aeInitiali0e(Wa$g)) { 'p$int'(st4e$$& Jinitiali0ation 'aile4XnJ); $etu$n 8; ! $un(a$ghan4le); n0aeClose(a$ghan4le); ! else { 'p$int'(st4e$$& J.xpecting ,ocal +.XnJ); $etu$n 8; ! $etu$n #;!static int $un(N-+.7@+ND,. h){

    $etu$n #;

    !

    9 Imp*ement the fu** *ogic:

    Vinclu4e st4iohVinclu4e st4li1h

    (0 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    61/360

    C anguage 'calar Function

    Vinclu4e Jn0aeapishJstatic int $un(N-+.7@+ND,. h);int main(int a$gc& cha$ > a$g%KL){ i' (n0aeIs,ocal()) { N0aeInitiali0ation a$g; memset(Wa$g& #& si0eo'(a$g)); a$gl42Ee$sion " N-+.7,DM7E.RSION; i' (n0aeInitiali0e(Wa$g)) { 'p$int'(st4e$$& Jinitiali0ation 'aile4XnJ); $etu$n 8; ! $un(a$ghan4le); n0aeClose(a$ghan4le); ! else {

    'p$int'(st4e$$& J.xpecting ,ocal +.XnJ); $etu$n 8; ! $etu$n #;!static int $un(N-+.7@+ND,. h){ N0ae*eta4ata meta4ata; i' (n0ae5et*eta4ata(h& Wmeta4ata)) { 'p$int'(st4e$$& Jget meta4ata 'aile4XnJ); $etu$n 8; !

    V4e'ine [email protected](%alue) X

    { X N0aeRcCo4e $c " %alue; X i' ($c) X { X const cha$ > 'o$mat " JYs in Ys at Y4J; X 'p$int'(st4e$$& 'o$mat& X n0ae5et,ast.$$o$Text(h)& 776I,.77& 77,IN.77); X n0ae3se$.$$o$(h& 'o$mat& X n0ae5et,ast.$$o$Text(h)& 776I,.77& 77,IN.77); X exit(8); X ! X !

    i' (meta4ataoutputColumnCount Z" meta4ataoutputTpesK#L Z" N-3DS3DG7DO3?,.) { n0ae3se$.$$o$(h& Jexpecting one output column o' tpe 4ou1leJ); $etu$n 8; !

    00?200.@03 Rev9 2 (1

  • 8/11/2019 UDAP Developers Guide

    62/360

    =ser@Defined Analytic Process Developers 5uide

    i' (meta4atainputColumnCount ) { n0ae3se$.$$o$(h& Jexpecting at least one input columnJ); $etu$n 8; !

    i' (meta4atainputTpesK#L Z" N-3DS3DG76IG.D WW meta4atainputTpesK#L Z" N-3DS3DG7E+RI+?,.) { n0ae3se$.$$o$(h& J'i$st input column expecte4 to 1e a st$ingtpeJ); $etu$n 8; !

    'o$ (;;) { int i; 4ou1le $esult " #; N0aeRcCo4e $c " n0ae5etNext(h); i' ($c "" N-+.7RC7.ND) {

    1$ea2; !

    N0u4sData > input " N3,,; [email protected](n0ae5etInputColumn(h& #& Winput)); const cha$ > opSt$ing; i' (input8isNull) { n0ae3se$.$$o$(h& J'i$st input column ma not 1e nullJ); $etu$n 8; ! i' (input8tpe "" N-3DS3DG76IG.D) { opSt$ing " input84atap6ixe4St$ing;

    ! else i' (input8tpe "" N-3DS3DG7E+RI+?,.) { opSt$ing " input84atapEa$ia1leSt$ing; ! else { n0ae3se$.$$o$(h& J'i$st input column expecte4 to 1e a st$ingtpeJ); $etu$n 8; !

    enum Ope$ato$.num { OP7+DD " & OP7*3,T " J) "" #) { op " OP7*3,T;

    (2 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    63/360

  • 8/11/2019 UDAP Developers Guide

    64/360

    =ser@Defined Analytic Process Developers 5uide

    1$ea2; case N-3DS3DG7INT=" >input84atapInt=" >input84atapIntA; 1$ea2; case N-3DS3DG76,O+T $esult >" >input84atap6loat; 1$ea2; case N-3DS3DG7DO3?,. $esult >" >input84atapDou1le; 1$ea2;

    case N-3DS3DG7N3*.RIC=

  • 8/11/2019 UDAP Developers Guide

    65/360

    C anguage 'calar Function

    FN-7.GPORT7DIR/ae/utilities/1in/$egiste$7ae 88language sstem 88%e$sion a$g%KL){ N0ae+pi5ene$ato$ helpe$; N0ae+pi Wapi " helpe$get+pi(N0ae+pi63NCTION); $etu$n #;!

    This retrie'es an Naepi reference that contains a usa+*e Nae6unction o+Hect.

    39 No/ that the function is a'ai*a+*e, add a stu+ for ca**ing the function *ogic:

    00?200.@03 Rev9 2 (&

  • 8/11/2019 UDAP Developers Guide

    66/360

  • 8/11/2019 UDAP Developers Guide

    67/360

  • 8/11/2019 UDAP Developers Guide

    68/360

    =ser@Defined Analytic Process Developers 5uide

    then retrie'e fie*ds from the input record and use that to set the output fie*d. This code

    pro'ides a +asic, nonerrorchecking function that shou*d take as the first argument either the

    string 'or the string ( and for su+seMuent arguments int3!s. It then either adds or mu*tip*ies

    the int3!s, +ased on the specified argument 'a*ue, and returns the resu*t as a dou+*e.

    &9 The fina* code preparation step adds error checking and support for additiona* data types.

    Vinclu4e n0ae'acto$hppusing namespace n0ae;static int $un(n0aeN0ae6unction >ae6unc);int main(int a$gc& cha$ > a$g%KL){ N0ae+pi5ene$ato$ helpe$; N0ae+pi Wapi " helpe$get+pi(N0ae+pi63NCTION); $un(apiae6unction); $etu$n #;!class *@an4le$ pu1lic N0ae6unction*essage@an4le${pu1lic *@an4le$() { m7Once " 'alse; ! enum Ope$ato$.num { OP7+DD " & OP7*3,T " < ! op;

    %oi4 4oOnce(N0ae6unctionW api) { const N0ae*eta4ataW meta " apiget*eta4ata(); i' (metagetOutputColumnCount() Z"

    metagetOutputTpe(#) Z" N0aeDataTpesN-3DS3DG7DO3?,.) { th$ow N0ae.xception(Jexpecting one output column o' tpe4ou1leJ); ! i' (metagetInputColumnCount() ) { th$ow N0ae.xception(Jexpecting at least one input columnJ); ! i' (metagetInputTpe(#) Z" N0aeDataTpesN-3DS3DG76IG.D WW metagetInputTpe(#) Z" N0aeDataTpesN-3DS3DG7E+RI+?,.) { th$ow N0ae.xception(J'i$st input column expecte4 to 1e ast$ing X tpeJ); ! m7Once " t$ue;

    !

    %oi4 e%aluate(N0ae6unctionW api& N0aeReco$4 Winput& N0aeReco$4W$esult) {

    i' (Zm7Once) 4oOnce(api); 4ou1le $es " #;

    (- 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    69/360

  • 8/11/2019 UDAP Developers Guide

    70/360

    =ser@Defined Analytic Process Developers 5uide

    temp " (4ou1le)s'; 1$ea2; ! case N0aeDataTpesN-3DS3DG7N3*.RIC=ae6unc){

    ae6unc8$un(new *@an4le$()); $etu$n #;!

    This code ensures the correct num+er and types of input arguments and the correct num+er and

    type of output arguments. It supports a** of the integer typesSf*oat, dou+*e and numeric.

    nce the code is comp*ete, it must +e compi*ed and registered.

    *0 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    71/360

    CQQ anguage 'calar Function

    Co%pilation

    To compi*e, use the fo**o/ing:

    FN-7.GPORT7DIR/ae/utilities/1in/compile7ae 88language cpp 88template

    compile X 88exe applopcpp 88compa$gs J8g 89allJ 88lin2a$gs J8gJapplopcppcpp X 88%e$sion ;pu1lic class Testa%aInte$'ace {

    pu1lic static 'inal %oi4 main(St$ing KL a$gs) {

    t$ { mainImpl(a$gs); ! catch (Th$owa1le t) { Ssteme$$p$intln(ttoSt$ing()); N0ae3tillog.xception(t& JmainJ); ! !

    pu1lic static 'inal %oi4 mainImpl(St$ing KL a$gs) { N0ae+pi5ene$ato$ helpe$ " new N0ae+pi5ene$ato$(); 'inal N0ae+pi api " helpe$get+pi(N0ae+pi63NCTION); $un(apiae6unction); helpe$close(); !

    pu1lic static int $un(N0ae ae6unc) { $etu$n #; !!

    9 There are t/o /ays of imp*ementing a function. ne is to use the Nae o+Hect interface direct*y.

    The other is to imp*ement an Naeessage2and*erderi'ed c*ass that pro'ides a simp*er

    *2 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    73/360

  • 8/11/2019 UDAP Developers Guide

    74/360

    =ser@Defined Analytic Process Developers 5uide

    aeuse$.$$o$(Jin%ali4 ope$ato$ " J opSt$); ! continue; ! switch (4ataTpe) { case N0aeDataTpesN-3DS3DG7INT=

  • 8/11/2019 UDAP Developers Guide

    75/360

    ?ava anguage 'calar Function

    &9 The fina* code preparation step adds error checking and support for additiona* data types.

    impo$t o$gnete00aae>;pu1lic class Testa%aInte$'ace {

    pu1lic static 'inal %oi4 main(St$ing KL a$gs) {

    t$ { mainImpl(a$gs); ! catch (Th$owa1le t) { Ssteme$$p$intln(ttoSt$ing()); N0ae3tillog.xception(t& JmainJ); ! !

    pu1lic static 'inal %oi4 mainImpl(St$ing KL a$gs) { N0ae+pi5ene$ato$ helpe$ " new N0ae+pi5ene$ato$(); 'inal N0ae+pi api " helpe$get+pi(N0ae+pi63NCTION); $un(apiae6unction); helpe$close(); !

    pu1lic static class *@an4le$ implements N0ae*essage@an4le$

    { pu1lic %oi4 e%aluate(N0ae ae& N0aeReco$4 input& N0aeReco$4output) {

    'inal N0ae*eta4ata meta " aeget*eta4ata();

    int op " #; 4ou1le $esult " #;

    i' (metagetOutputColumnCount() Z" metagetOutputN0Tpe(#) Z" N0aeDataTpesN-3DS3DG7DO3?,.){

    th$ow new N0ae.xception(Jexpecting one output column o'tpe J J4ou1leJ); ! i' (metagetInputColumnCount() ) { th$ow new N0ae.xception(Jexpecting at least one inputcolumnJ); ! i' (metagetInputN0Tpe(#) Z" N0aeDataTpesN-3DS3DG76IG.D WW metagetInputN0Tpe(#) Z"N0aeDataTpesN-3DS3DG7E+RI+?,.) { th$ow new N0ae.xception(J'i$st input column expecte4 to1e J Ja st$ing tpeJ); !

    'o$ (int i " #; i inputsi0e(); i) { i' (inputget6iel4(i) "" null) { continue; ! int 4ataTpe " metagetInputN0Tpe(i); i' (i "" #) { i' (Z(4ataTpe "" N0aeDataTpesN-3DS3DG76IG.D

    00?200.@03 Rev9 2 *&

  • 8/11/2019 UDAP Developers Guide

    76/360

    =ser@Defined Analytic Process Developers 5uide

    4ataTpe ""N0aeDataTpesN-3DS3DG7E+RI+?,.)) {

    aeuse$.$$o$(J'i$st column must 1e a st$ingJ);

    ! St$ing opSt$ " inputget6iel4+sSt$ing(#); i' (opSt$euals(J>J)) { $esult " ; op " OP7*3,T; ! else i' (opSt$euals(JJ)) { $esult " #; op " OP7+DD; ! else { aeuse$.$$o$(Jin%ali4 ope$ato$ " J opSt$); ! continue; ! switch (4ataTpe) {

    case N0aeDataTpesN-3DS3DG7INT case N0aeDataTpesN-3DS3DG7INT case N0aeDataTpesN-3DS3DG7INT=

  • 8/11/2019 UDAP Developers Guide

    77/360

    ?ava anguage 'calar Function

    { ae$un(new *@an4le$()); $etu$n #; !!

    This code ensures the correct num+er and types of input arguments and the correct num+er and

    type of output arguments,. It supports a** of the integer types, f*oat, dou+*e and numeric.

    nce the code is comp*ete, it must +e compi*ed and registered.

    Co%pilation

    To compi*e use the fo**o/ing:

    FN-7.GPORT7DIR/ae/utilities/1in/compile7ae 88language Ha%a 88template X compile Testa%aInte$'aceHa%a 88%e$sion $ightInput

    V .RROR ON +,, [email protected] sel'use$.$$o$(J3nhan4le4 ope$ato$ to +pplOp :J ope$ato$ J:J)+pplOp34'+e$un()

    The function se*f.user9rror; reports an error to the %4 user. ?hate'er is returned +y

    @get6unction(esu*t; is the resu*t of the %4 operation. Because this is run as a $-6, there can on*y

    +e one co*umn in the resu*t. *ater e0amp*e demonstrates ho/ to use $-T6s to output mu*tip*e

    co*umns.

    nce the code is comp*ete, it must +e dep*oyed and registered.

    Co%pilation

    The ython 9s do not reMuire compi*ation.

    Deploy%ent

    *though ython 9s do not reMuire compi*ation, they do reMuire dep*oyment. The compile_ae

    command is sti** used /ith the @@te%plate deployoption to dep*oythe script to the default

    location on the shared export drive.

    FN-7.GPORT7DIR/ae/utilities/1in/compile7ae 88language pthon X 88template 4eplo /applopp 88%e$sion :& =& );

    ?hi*e the a+o'e e0amp*e on*y has one input, the N% system dea*s /ith streams of input.

    Therefore, the N% system is more *ike*y to use:

    S.,.CT applopPl(mta1leope$ato$& mta1le%alue& mta1le%alue F$ightInput; !

    V .RROR ON +,, [email protected]$oa2(n0ae.xceptions+eInte$nal.$$o$8new(J3nhan4le4 ope$ato$ to

    - 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    85/360

    Perl anguage 'calar Function

    +pplOp :J Fope$ato$ J:J));!;

    The croak function reports the error to the %4 user and, if *ogging is ena+*ed, returns a trace to the

    *ine of error.

    1ou can use the c*uck function in p*ace of croak for /riting /arnings to the *og fi*e. To catch any

    unhand*ed e0ceptions automatica**y, use autodie. -ie and /arn can a*so +e used to report a sing*e

    *ine error or /arning if *ogging is turned on. -ie errors out the e0ecution of the 9. %et *og *e'e* using

    the @@leveloption during registration. ?hat is returned +y @get6unction(esu*t; is the resu*t of the

    %4 operation +ecause this is run as a $-6, there can on*y +e one co*umn in the resu*t.

    In summary, to /rite a $-6 adding on*y the actua* functiona*ity /ithout customiing the methods for

    running it:

    19 Create a er* modu*e fi*e pp*yp.pm in the e0amp*e a+o'e;.

    29 Import and instantiate the nae::e c*ass in the fi*e.

    39 Import autodie to hand*e unhand*ed e0ceptions during e0ecution.9 Override_getFunctionResult to implement customized functionality for

    DF. !andle errors to be returned to the user using the croa"#$ method.

    &9 90ecute the run; method of the nae::e o+Hect.

    (9 %ince this is a er* modu*e, add a F#G at the end of the fi*e %pplyOp.pm.

    nce the code is comp*ete, it must +e dep*oyed and registered.

    Co%pilation

    er* 9s do not reMuire compi*ation.

    Deploy%ent

    %lthough &erl %'s do not re(uire compilation, they must +e dep*oyed. The co%pileaecommand is sti** used /ith the @@te%plate deployoption to dep*oy the script to the defau*t *ocation

    on the shared e0port dri'e.

    FN-7.GPORT7DIR/ae/utilities/1in/compile7ae 88language pe$l 88%e$sion < X88template 4eplo +pplOppm

    Registration

    Register the &erl file)

    FN-7.GPORT7DIR/ae/utilities/1in/$egiste$7ae 88language pe$l 88%e$sion < X 88template u4' 88exe +pplOppm 88sig JapplopPl(%a$cha$()& intA&intA)JX 88$etu$n intA

    00?200.@03 Rev9 2 -&

  • 8/11/2019 UDAP Developers Guide

    86/360

    =ser@Defined Analytic Process Developers 5uide

    Running

    The 9 can no/ +e run in %4 on the system data+ase:

    S.,.CT applopPl(::& A& #);+PP,BOPP,88888888888 A( $ow)

    Note that to 'a*idate types in @get6unction(esu*t;, the croak; method is ca**ed. The fo**o/ing

    e0amp*e triggers an error:

    S.,.CT applopPl(:8:& &

  • 8/11/2019 UDAP Developers Guide

    87/360

    C H A P T E R 6

    Converting to a 'i%ple )able Function

    Exa%ples

    C anguage Conversion

    The code from the C anguage %ca*ar 6unctioncan +e reused as a simp*e ta+*e function +y making

    minor modifications to the registration.

    Code

    The code is the same as found in the C anguage %ca*ar 6unctionsection.

    Co%pilation

    Compi*ation is the same as that performed in *oca* or remote modes.

    FN-7.GPORT7DIR/ae/utilities/1in/compile7ae 88language sstem 88%e$sion input " N3,,; [email protected](n0ae5etInputColumn(h& #& Winput)); const cha$ > opSt$ing; i' (input8isNull) { n0ae3se$.$$o$(h& J'i$st input column ma not 1e nullJ); n0aeDone(h); $etu$n 8; !

    i' (input8tpe "" N-3DS3DG76IG.D) { opSt$ing " input84atap6ixe4St$ing; ! else i' (input8tpe "" N-3DS3DG7E+RI+?,.) { opSt$ing " input84atapEa$ia1leSt$ing; ! else { n0ae3se$.$$o$(h& J'i$st input column expecte4 to 1e a st$ingtpeJ); n0aeDone(h); $etu$n 8; ! int hou$& min& sec;

    int n'oun4 " sscan'(opSt$ing& JY4Y4Y4J& Whou$& Wmin& Wsec); i' (n'oun4 Z" =) { n0ae3se$.$$o$(h& J?a4l 'o$matte4 time .xpecte4 hmsJ); n0aeDone(h); $etu$n 8; !

    [email protected](n0aeSetOutputInt=

  • 8/11/2019 UDAP Developers Guide

    97/360

    C anguage 'i%ulated Ro# Function

    88template compile splitc 88exe split

    Registration

    %ince there is no +ui*tin concept of a ro/ function in Netea %4, this function is registered as a

    ta+*e function, e'en though it outputs on*y one ro/ per input ro/:

    FN-7.GPORT7DIR/ae/utilities/1in/$egiste$7ae 88language sstem 88%e$sion input " N3,,; [email protected](n0ae5etInputColumn(h& #& Winput)); const cha$ > opSt$ing; i' (input8isNull) { n0ae3se$.$$o$(h& J'i$st input column ma not 1e nullJ); n0aeDone(h); $etu$n 8;

    ! i' (input8tpe "" N-3DS3DG76IG.D) { opSt$ing " input84atap6ixe4St$ing; ! else i' (input8tpe "" N-3DS3DG7E+RI+?,.) { opSt$ing " input84atapEa$ia1leSt$ing; ! else { n0ae3se$.$$o$(h& J'i$st input column expecte4 to 1e a st$ing X tpeJ); n0aeDone(h); $etu$n 8; !

    const cha$ >pt$ " opSt$ing; cha$ >to2en;

    while ((to2en " st$to2(pt$& J&J)) Z" N3,,) { pt$ " N3,,;

    [email protected](n0aeSetOutputSt$ing(h& #& to2en)); [email protected](n0aeOutputResult(h)); !

    ! n0aeDone(h); $etu$n #;!

    Co%pilation

    $se the standard compi*e:

    FN-7.GPORT7DIR/ae/utilities/1in/compile7ae 88language sstem 88%e$sion< X 88template compile sst$ingc 88exe sst$ing

    10.

  • 8/11/2019 UDAP Developers Guide

    110/360

  • 8/11/2019 UDAP Developers Guide

    111/360

  • 8/11/2019 UDAP Developers Guide

    112/360

    CQQ anguage )able Function

    ! i' (metagetInputColumnCount() Z" ) { th$ow N0ae.xception(Jexpecting one input columnJ); ! i' (metagetInputTpe(#) Z" N0aeDataTpesN-3DS3DG76IG.D WW

    metagetInputTpe(#) Z" N0aeDataTpesN-3DS3DG7E+RI+?,.) { th$ow N0ae.xception(J'i$st input column expecte4 to 1e a st$ingtpeJ); !

    'o$ (;;) { n0aeN0aeReco$4 >input " ae6unc8next(); i' (Zinput) 1$ea2; n0aeN0aeReco$4 >output " ae6unc8c$eateOutputReco$4();

    N0ae6iel4 W'iel4 " input8get(#); i' ('iel4isNull())

    th$ow N0ae.xception(J'i$st input column ma not 1e nullJ); N0aeSt$ing6iel4 Ws' " (N0aeSt$ing6iel4W) input8get(#); st4st$ing st$ " (st4st$ing)s'; %ecto$st$ing wo$4s; split7st$ing(st$&:&:& wo$4s); 'o$ (int i"#; i (int)wo$4ssi0e(); i) { st$ing tip " wo$4sKiL; N0aeSt$ing6iel4 Ws' " (N0aeSt$ing6iel4W) output8get(#); s' " tip; ae6unc8outputResult(>output); !

    4elete input;

    4elete output;!

    ae6unc84one(); $etu$n #;!

    Co%pilation

    $se the standard compi*e:

    FN-7.GPORT7DIR/ae/utilities/1in/compile7ae 88language cpp 88templatecompile X 88exe splitae 88compa$gs J8g 89allJ 88lin2a$gs J8gJ splitcpp88%e$sion a$g%KL){

    00?200.@03 Rev9 2 123

  • 8/11/2019 UDAP Developers Guide

    124/360

    =ser@Defined Analytic Process Developers 5uide

    i' (n0aeIs,ocal())

    { N0ae+pi $esult; cha$ e$$o$*essageK##L; i' (n0ae,ocp$ot5et+pi(W$esult& N-+.7,DM7E.RSION& e$$o$*essage& si0eo'(e$$o$*essage))) { 'p$int'(st4e$$& JYsXnJ& e$$o$*essage); $etu$n 8; ! i' ($esultapiTpe "" N-+.7+PI763NCTION) { $un($esulthan4le'unction); n0aeClose($esulthan4le'unction); ! else { $unShape$($esulthan4leshape$); n0aeShpClose($esulthan4leshape$); ! ! else {

    N-+.CONPT7@+ND,. hConpt " n0aeconptC$eate(); i' (ZhConpt) { 'p$int'(st4e$$& Je$$o$ c$eating connection pointXnJ); ''lush(st4e$$); $etu$n 8; ! const cha$ > conPtName " n0aeRemp$ot5etRemoteName(); i' (ZconPtName) { 'p$int'(st4e$$& Je$$o$ getting connection point nameXnJ); ''lush(st4e$$); exit(8); !

    i' (n0aeconptSetName(hConpt& conPtName)) { 'p$int'(st4e$$& Je$$o$ setting connection point nameXnJ); ''lush(st4e$$); n0aeconptClose(hConpt); $etu$n 8; ! N0ae$emp$otInitiali0ation a$gs; memset(Wa$gs& #& si0eo'(a$gs)); a$gsl42Ee$sion " N-+.7,DM7E.RSION; a$gshConpt " hConpt; i' (n0aeRemp$otC$eate,istene$(Wa$gs)) { 'p$int'(st4e$$& Juna1le to c$eate listene$ 8 YsXnJ& X

    a$gse$$o$*essage); ''lush(st4e$$); n0aeconptClose(hConpt); $etu$n 8; ! N-+.R.*PROT7@+ND,. hRemp$ot " a$gshan4le; N0ae+pi api; int i;

    12 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    125/360

    C anguage '$apers and 'i/ers

    'o$ (i " #; i ; i) { i' (n0aeRemp$ot+ccept+pi(hRemp$ot& Wapi)) { 'p$int'(st4e$$& Juna1le to accept +PI 8 YsXnJ& X n0aeRemp$ot5et,ast.$$o$Text(hRemp$ot)); ''lush(st4e$$); n0aeconptClose(hConpt); n0aeRemp$otClose(hRemp$ot); $etu$n 8; ! p$int'(Jtestcapi accepte4 a $emote $euestXnJ); ''lush(st4out); i' (apiapiTpe "" N-+.7+PI763NCTION) { $un(apihan4le'unction); n0aeClose(apihan4le'unction); ! else { $unShape$(apihan4leshape$); n0aeShpClose(apihan4leshape$); !

    ! n0aeRemp$otClose(hRemp$ot); n0aeconptClose(hConpt); ! $etu$n #;!static int $unShape$(N-+.S@P7@+ND,. h){ cha$ nameK

  • 8/11/2019 UDAP Developers Guide

    126/360

    =ser@Defined Analytic Process Developers 5uide

    metainputTpesK#L "" N-3DS3DG7E+RI+?,. metainputTpesK#L "" N-3DS3DG7N+TION+,76IG.D metainputTpesK#L "" N-3DS3DG7N+TION+,7E+RI+?,.) { [email protected] input " N3,,; [email protected](n0ae5etInputColumn(h& #& Winput)); const cha$ > opSt$ing;

    12( 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    127/360

  • 8/11/2019 UDAP Developers Guide

    128/360

  • 8/11/2019 UDAP Developers Guide

    129/360

    CQQ anguage '$apers and 'i/ers

    W$esult) { const N0ae*eta4ataW m " apiget*eta4ata(); n0aeN0ae6iel4 W'iel4 " inputget(#); i' (Z'iel4isNull() ) { n0aeN0ae6iel4 W' " $esultget(#); i' ('iel4tpe() "" N0aeDataTpesN-3DS3DG7N3*.RIC=< 'iel4tpe() "" N0aeDataTpesN-3DS3DG7N3*.RICA 'iel4tpe() "" N0aeDataTpesN-3DS3DG7N3*.RICnp); 4elete np; ! else 'assign('iel4); ! else { 'assign('iel4);

    ! !

    !!;class *@an4le$< pu1lic n0aeN0aeShape$*essage@an4le${pu1lic %oi4 shape$(N0aeShape$W api){ const N0ae*eta4ataW m " apiget*eta4ata(); cha$ nameK

  • 8/11/2019 UDAP Developers Guide

    130/360

    =ser@Defined Analytic Process Developers 5uide

    ! else { apia44OutputColumn(mgetInputTpe(#)& name); ! !!;static int 4oShape$(n0aeN0aeShape$ >aeShape$){ aeShape$8$un(new *@an4le$ae6unc){ ae6unc8$un(new *@an4le$()); $etu$n #;!

    Note in the main that an I o+Hect must +e retrie'ed in *oca* mode, /hich gets +oth a function I

    and a shaper I a*though not at the same time;. dditiona**y, the code modifies the *oop *ogic forremote mode so that the shaper is not incremented, a**o/ing the same instance of the program to

    hand*e +oth a shaper and a function ca** in remote mode +efore e0iting. 6ina**y, it adds the shaper

    message hand*er.

    Co%pilation

    $se the standard compi*e:

    FN-7.GPORT7DIR/ae/utilities/1in/compile7ae 88language cpp 88templatecompile X 88exe shape$ae 88compa$gs J8g 89allJ 88lin2a$gs J8gJ shape$cpp X

    88%e$sion inputData; N0u4sData > stateData; p8inputgetEalue(h& #& WinputData); p8stategetEalue(h& #& WstateData); i' (inputData8isNull) 1$ea2;

    i' (stateData8isNull >stateData84atapInt=< X >inputData84atapInt= p " (N0ae+gg*e$ge >) pTemp; N0u4sData > inputStateData; N0u4sData > stateData; p8inputStategetEalue(h& #& WinputStateData); p8stategetEalue(h& #& WstateData); i' (inputStateData8isNull)

    { 1$ea2; // no 4ata to me$ge ! i' (stateData8isNull >stateData84atapInt=< X >inputStateData84atapInt= p " (N0ae+gg6inalResult >) pTemp; N0u4sData > inputStateData;

    p8inputStategetEalue(h& #& WinputStateData); i' (inputStateData8isNull) { p8$esultsetNull(h& #); ! else {

    p8$esultsetEalue(h& #& inputStateData);

    !

    1$ea2; ! 4e'ault

    n0ae+gg3se$.$$o$(h& Junexpecte4 message tpeJ); ! n0ae+gg3p4ate(h); !

    $etu$n #;!

    1( 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    147/360

  • 8/11/2019 UDAP Developers Guide

    148/360

    =ser@Defined Analytic Process Developers 5uide

    CQQ anguage Aggregates

    This e0amp*e uses the fo**o/ing fi*e name:

    maxcpp

    Code

    The e0amp*e is s*ight*y *onger than the C *anguage aggregate e0amp*e +ecause the code is designed

    to hand*e a** possi+*e data types.

    Vinclu4e n0ae'acto$hppusing namespace n0ae;static int $un(n0aeN0ae+gg$egate >ae6unc);int main(int a$gc& cha$ > a$g%KL){ N0ae+pi5ene$ato$ helpe$; // The 'ollowing line is onl nee4e4 i' a launche$ is not use4

    helpe$setName(JtestcapiJ); 'o$ (int i"#; i

  • 8/11/2019 UDAP Developers Guide

    149/360

    CQQ anguage Aggregates

    case N0aeDataTpesN-3DS3DG7E+RI+?,. case N0aeDataTpesN-3DS3DG76IG.D case N0aeDataTpesN-3DS3DG7N+TION+,7E+RI+?,. case N0aeDataTpesN-3DS3DG7N+TION+,76IG.D { st4st$ing Ws " ((n0aeN0aeSt$ing6iel4W)state'iel4); st4st$ing W' " ((n0aeN0aeSt$ing6iel4W)'iel4); i' (s ') state'iel4 " 'iel4; 1$ea2; ! case N0aeDataTpesN-3DS3DG7?OO, { i' ((int=

  • 8/11/2019 UDAP Developers Guide

    150/360

  • 8/11/2019 UDAP Developers Guide

    151/360

    CQQ anguage Aggregates

    1$ea2; ! case N0aeDataTpesN-3DS3DG7INT.RE+, { i' (((n0aeN0aeInte$%al6iel4W)state'iel4) ((n0aeN0aeInte$%al6iel4W)'iel4)) { state'iel4 " 'iel4; ! 1$ea2; ! case N0aeDataTpesN-3DS3DG7TI*.ST+*P { i' ((intA7t)((n0aeN0aeTimestamp6iel4W)state'iel4) (intA7t)((n0aeN0aeTimestamp6iel4W)'iel4)) { state'iel4 " 'iel4; ! 1$ea2; ! 4e'ault

    th$ow n0aeN0ae.xception(J3nexpecte4 tpeJ);

    ! ! !

    %oi4 me$ge(n0aeN0ae+gg$egateW api& n0aeN0aeReco$4 WinputState& n0aeN0aeReco$4 Wstate) {

    n0aeN0ae6iel4 W'iel4 " inputStateget(#); n0aeN0ae6iel4 Wstate'iel4 " stateget(#); i' ('iel4isNull()) $etu$n;

    i' (state'iel4isNull()) { state'iel4setNull('alse); state'iel4 " 'iel4; ! else { switch ('iel4tpe()) { case N0aeDataTpesN-3DS3DG7E+RI+?,. case N0aeDataTpesN-3DS3DG76IG.D case N0aeDataTpesN-3DS3DG7N+TION+,7E+RI+?,. case N0aeDataTpesN-3DS3DG7N+TION+,76IG.D { st4st$ing Ws " ((n0aeN0aeSt$ing6iel4W)state'iel4); st4st$ing W' " ((n0aeN0aeSt$ing6iel4W)'iel4);

    i' (s ') state'iel4 " 'iel4; 1$ea2; ! case N0aeDataTpesN-3DS3DG7?OO, { i' ((int=

  • 8/11/2019 UDAP Developers Guide

    152/360

    =ser@Defined Analytic Process Developers 5uide

    { state'iel4 " 'iel4; ! 1$ea2; ! case N0aeDataTpesN-3DS3DG7INTA { i' ((intA7t)((n0aeN0aeIntA6iel4W)state'iel4) (intA7t)((n0aeN0aeIntA6iel4W)'iel4)) { state'iel4 " 'iel4; ! 1$ea2; ! case N0aeDataTpesN-3DS3DG7INT=

  • 8/11/2019 UDAP Developers Guide

    153/360

    CQQ anguage Aggregates

    1$ea2; ! case N0aeDataTpesN-3DS3DG7N3*.RIC=

  • 8/11/2019 UDAP Developers Guide

    154/360

    =ser@Defined Analytic Process Developers 5uide

    1$ea2; ! 4e'ault th$ow n0aeN0ae.xception(J3nexpecte4 tpeJ);

    ! ! !

    %oi4 'inalResult(n0aeN0ae+gg$egateW api& n0aeN0aeReco$4 WinputState& n0aeN0aeReco$4 W$esult) {

    n0aeN0ae6iel4 W'iel4 " inputStateget(#); n0aeN0ae6iel4 W$es " $esultget(#); i' ('iel4isNull()) $essetNull(t$ue); else { $es " 'iel4; !

    !

    !;static int $un(n0aeN0ae+gg$egate >ae+gg){ ae+gg8$un+gg$egation(new *@an4le$()); $etu$n #;!

    Co%pilation

    $se the standard compi*e:FN-7.GPORT7DIR/ae/utilities/1in/compile7ae 88language cpp X 88template compile 88exe maxae 88compa$gs J8g 89allJ X 88lin2a$gs J8gJ maxcpp 88%e$sion ;pu1lic class Testa%a*ax {

    p$i%ate static 'inal .xecuto$ exec " .xecuto$snewCache4Th$ea4Pool();

    pu1lic static 'inal %oi4 main(St$ing KL a$gs) { t$ { mainImpl(a$gs); ! catch (Th$owa1le t) { Ssteme$$p$intln(ttoSt$ing()); N0ae3tillog.xception(t& JmainJ); ! !

    00?200.@03 Rev9 2 1&&

  • 8/11/2019 UDAP Developers Guide

    156/360

    =ser@Defined Analytic Process Developers 5uide

    pu1lic static 'inal %oi4 mainImpl(St$ing KL a$gs) { N0ae+pi5ene$ato$ helpe$ " new N0ae+pi5ene$ato$();

    while (t$ue) { 'inal N0ae+pi api " helpe$get+pi(N0ae+pi+55R.5+TION); i' (apiapiTpe "" N0ae+pi+55R.5+TION) { i' (Zhelpe$isRemote()) { $un(apiae+gg$egate); 1$ea2; ! else { Runna1le tas2 " new Runna1le() { pu1lic %oi4 $un() { t$ { Testa%a*ax$un(apiae+gg$egate); ! 'inall { apiae+gg$egateclose(); ! ! !;

    execexecute(tas2);

    ! ! !

    helpe$close();

    !

    pu1lic static class *@an4le$ implements N0ae+gg*essage@an4le$

    { pu1lic %oi4 initiali0eState(N0ae+gg api& N0aeReco$4 state) { stateset6iel4(#& null);

    ! pu1lic %oi4 accumulate(N0ae+gg api& N0aeReco$4 input& N0aeReco$4 state) { O1Hect 'iel4 " inputget6iel4(#); i' ('iel4 "" null) $etu$n; O1Hect state'iel4 " stateget6iel4(#);

    i' (state'iel4 "" null) { state'iel4 " 'iel4; ! else { i' (((Compa$a1le)state'iel4)compa$eTo('iel4) #) state'iel4 " 'iel4; ! stateset6iel4(#& state'iel4);

    1&( 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    157/360

    ?ava anguage Aggregates

    !

    pu1lic %oi4 me$ge(N0ae+gg api& N0aeReco$4 inputState& N0aeReco$4 state) {

    O1Hect 'iel4 " inputStateget6iel4(#); i' ('iel4 "" null) $etu$n; O1Hect state'iel4 " stateget6iel4(#);

    i' (state'iel4 "" null) { state'iel4 " 'iel4; ! else { i' (((Compa$a1le)state'iel4)compa$eTo('iel4) #) state'iel4 " 'iel4; ! stateset6iel4(#& state'iel4); !

    pu1lic %oi4 'inalResult(N0ae+gg api& N0aeReco$4 inputState& N0aeReco$4 $esult) {

    O1Hect 'iel4 " inputStateget6iel4(#); $esultset6iel4(#& 'iel4); ! ! p$i%ate static 'inal %oi4 $un(N0ae+gg ae) { ae$un+gg$egation(new *@an4le$()); !!

    Co%pilation$se the standard compi*e:

    FN-7.GPORT7DIR/ae/utilities/1in/compile7ae 88language Ha%a X 88template compile Testa%a*axHa%a \8%e$sion )E4AE'E''I>4%ets the session I- as part of the connection point name. The defau*t 'a*ue is ", the I- is not part of

    the name. ?hen set to #, session I- +ecomes part of the connection point name. This 'a*ue can a*so

    +e set +y the @@rsessionoption of register@ae.

    N-+.7R.*OT.7N+*.7S.SSION"K# L

    1*2 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    173/360

    Co%%only =sed AE Environ%ent Mariable

    4AERE>)E4AE)RA4'AC)I>4%ets the transaction I- as part of the connection point name. The defau*t 'a*ue is ", the I- is not part

    of the name. ?hen set to #, transaction I- +ecomes part of the connection point name. This 'a*ue

    can a*so +e set +y the @@rtransoption of register@ae.

    N-+.7R.*OT.7N+*.7TR+NS+CTION"K# L

    4AERE>)E4AEDA)A'ICE%ets the datas*ice I- as part of the connection point name. The defau*t 'a*ue is ", the I- is not part of

    the name. ?hen set to #, datas*ice I- +ecomes part of the connection point name. This 'a*ue can

    a*so +e set +y the @@rdatasliceoption of register@ae.

    N-+.7R.*OT.7N+*.7D+T+7S,IC."K# L

    4AERE>)E)IE>=)'EC>4D'%ets the num+er of seconds the 9 runtime system /aits for a response +efore returning a timeout

    error. The defau*t 'a*ue is =" seconds.

    N-+.7R.*OT.7TI*.O3T7S.CONDS"secon4s

    4AERE>)EA=4C8%ets /hether the registration *aunches a ne/ 9 remote process. The defau*t 'a*ue is ", a ne/

    process is not *aunched. ?hen set to #, the registration *aunches a ne/ remote process.

    N-+.7R.*OT.7,+3NC@"K# L

    4AERE>)EA=4C8MER>'E%ets /hether the registration *aunches a ne/ 9 remote process and disp*ays information descri+ing

    the ne/ process. The defau*t 'a*ue is ", a ne/ process is not *aunched. ?hen set to #, the

    registration *aunches a ne/ remote process /ith detai*s. This 'a*ue can a*so +e set +y the @@launc$

    option of register@ae.

    N-+.7R.*OT.7,[email protected]?OS."K# L

    AE Environ%ent Mariable Prefixes

    9 en'ironment prefi0es are appended to the front of an en'ironment key name, pro'iding

    instructions to the 9 runtime system. The prefi0es are processed and remo'ed, *ea'ing on*y the

    actua* key name. u*tip*e prefi0es may +e used, +ut must appear in a definite order determined +y

    order of precedence.

    Precedence evel 0

    ?hen present, these prefi0es restrict the en'ironment 'aria+*e to either the host or the %B*ade%$;.

    N-+.7@OST7ON,B7N-+.7SP37ON,B7

    Precedence evel 1

    00?200.@03 Rev9 2 1*3

  • 8/11/2019 UDAP Developers Guide

    174/360

    =ser@Defined Analytic Process Developers 5uide

    ?hen present, this prefi0 instructs the 9 runtime not to perform 'aria+*e `Z ]; or shared *i+rary

    name su+stitution `[ \;.

    N-+.7NO7S3?STIT3T.7

    Precedence evel 2

    These prefi0es indicate that the 'a*ue is to +e appended or prepended to an e0isting en'ironment'a*ue. If the key is not defined, then it is created /ith this 'a*ue. 9 runtime first searches for an 9

    en'ironment key and then a inu0 % en'ironment key to find an e0isting 'a*ue.

    N-+.7+PP.ND7N-+.7PR.P.ND7

    The fo**o/ing prefi0es are used to inc*ude fi*es containing 9 en'ironment 'aria+*es, as descri+ed in

    the section9 9n'ironment Aaria+*e Inc*ude 6i*es NV9@INC$-9@B96(9 and

    NV9@INC$-9@6T9(@ are a**o/ed in inc*ude fi*es +ut do not affect processing order. The prefi0es

    must appear in precedence order, from *o/ to high. n*y one entry from each precedence *e'e* can

    +e used /ith a key. 6or more information on 'aria+*e processing, see rder of Aaria+*e arsing.

    N-+.7INC,3D.7;N-+.7INC,3D.7?.6OR.7;N-+.7INC,3D.7+6T.R7;

    The fo**o/ing e0amp*e demonstrates an e0isting key used on*y on the %B*ade ca**ed

    1@(N-@%CII@C2(%. Aaria+*e or shared *i+rary su+stitution is not desira+*e, +ut cou*d occur

    since X`[X and X`ZX seMuences are possi+*e. If an en'ironment 'a*ue a*ready e0ists for this key, the

    'a*ue shou*d +e appended after it:

    N-+.7SP37ON,B7N-+.7NO7S3?STIT3T.7N-+.7+PP.ND7*B7R+NDO*7+SCII7C@+RS"Ja]YK KKYYY{!!! #J

    This e0amp*e demonstrates prefi0 usage in its most genera* form, a*though it is *ike*y more comp*e0

    than most app*ications reMuire.

    'etting Dyna%ic AE Environ%ent Mariables

    9 en'ironment 'aria+*es can +e set at Muery run time, inc*uding 'aria+*es that affect the +eha'ior of

    the 9 runtime system. pp*icationspecific 'aria+*es can a*so +e set. They are set using an e0tra

    string in the *ast argument, usua**y a constant, to ta+*e or sca*ar %4 6unctions7.If the argument

    comes from a ta+*e co*umn, the co*umn must ha'e the same 'a*ue in e'ery ro/ or the resu*t is

    undefined. By defau*t, the target 9 recei'es the settings as the 9 en'ironment 'aria+*es and not as

    an argument.

    7 This feature is not supported for aggregates

    1* 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    175/360

    'etting Dyna%ic AE Environ%ent Mariables

    Exa%ple

    S.,.CT > 6RO* 4emo& T+?,. 9IT@ 6IN+,(ae4emo('&'

    registration, then the dynamic 'aria+*e o'errides the registered 'aria+*e.

    The en'ironment 'aria+*es are specified as name#Y'a*ue#, name!Y'a*ue!, and so on. pp*y the

    fo**o/ing synta0 for en'ironment 'aria+*es:

    $se a comma as 'aria+*e separator.

    $se dou+*e Muotes as the Muoting character.

    Aa*ues that inc*ude /hite space andor commas must +e enc*osed /ith dou+*e Muotes

    other/ise, /hite space is ignored.

    If a 'a*ue is Muoted, the dou+*e Muotes are remo'ed after parsing.

    If the dou+*e Muote character is reMuired as part of a 'a*ue, use the +acks*ash FLG; character to

    escape.

    Exa%pleN-+.7.G.C3T?,.7P+T@"/1in/ae& 4ata"J&

  • 8/11/2019 UDAP Developers Guide

    176/360

    =ser@Defined Analytic Process Developers 5uide

    fter parsing, this +ecomes:

    N-+.7.G.C3T?,.7P+T@"/1in/ae4ata"&rder of Mariable Parsing

    9 en'ironment 'aria+*es are processed in a defined order. This can resu*t in 'aria+*es o'erriding

    those of the same name that /ere processed ear*ier. The order of processing is:

    1*( 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    177/360

    >rder of Mariable Parsing

    )able 3B >rder of variable parsing

    )ype Description >verrides999

    . Include before Includes these files before the top-levelregistration environment variables. +ee %'

    'nvironment /ariable Include Files.0

    1. Registered &refaced 2ith 34%'_R'5I+6'R_, 2hich isstripped from the name, +ee %''nvironment /ariables and register_ae ,0the --environment description. 6hese are aspecial case of top-level registrationvariables.

    Include before

    7. 3ormal +pecified as a variable in a +89 :R'%6'command #as done by register_ae$ . 6heseare also "no2n as top-level registrationvariables.

    Registered orinclude before

    ;. Include after Includes these files after the top-levelregistration environment variables. +ee %''nvironment /ariable Include Files.

    3ormal,registered, orinclude before

  • 8/11/2019 UDAP Developers Guide

    178/360

    =ser@Defined Analytic Process Developers 5uide

    inc*uding e0ecuta+*es, Ea'a c*ass fi*es, and te0t fi*es. In practice, the shared *i+rary fi*e is made

    a'ai*a+*e to an 9 at runtime through an en'ironment 'aria+*e or the 9 I that pro'ides the

    *ocation of the fi*e on the fi*e system. The fi*e is readon*y and shou*d not +e modified or de*eted.

    *though the 9 shared *i+rary functiona*ity a**o/s fi*e resources reMuired +y an 9 to +e packaged

    into the N% system data+ase, the a*ternati'e is to p*ace these fi*e resources in the 9 e0port

    directory tree on a net/ork shared dri'e. %e*ection is +ased on the characteristics of the app*ication.6or e0amp*e, the 9 shared *i+rary functiona*ity may pro'ide +etter organiationa* efficiency. 6or

    some app*ications, it may +e used to a'oid net/ork contention on the 9 e0port directory tree.

    2o/e'er, the 9 shared *i+rary functiona*ity might +ecome cum+ersome if too many fi*es are

    in'o*'ed, making the 9 e0port directory tree a +etter choice.

    s an e0amp*e, the he**oae code sho/n in the %imp*e C anguage 9section can +e modified to use

    9 shared *i+raries instead of the 9 e0port directory tree. In the e0amp*e, the host e0ecuta+*e is

    *ocated at ne0portaeapp*icationsde'da'idhosthe**oae and the %$ e0ecuta+*e is *ocated

    at ne0portaeapp*icationsde'da'idspuhe**oae. 6irst, use an %4 command to store these as

    9 shared *i+raries in the data+ase:

    CR.+T. OR R.P,+C. ,I?R+RB @.,,O+.*+N3+, ,O+D.GT.RN+, @OST O?.CT :/n0/expo$t/ae/applications/4e%/4a%i4/host/helloae:.GT.RN+, SP3 O?.CT :/n0/expo$t/ae/applications/4e%/4a%i4/spu/helloae:;

    %ome notes a+out this %4:

    N$ - te**s the Netea system not to automatica**y *oad this 9 shared *i+rary. n*y

    true inu0 .so shared *i+rary fi*es shou*d +e automatica**y *oaded.

    Both the host and %$ copies of the he**oae e0ecuta+*e are stored in the data+ase. The data+ase

    does not maintain a pointer +ack to the fi*e 'ersions on disk.

    299 is the e0terna* name of this 9 shared *i+rary.

    6or ad'anced N% system users, an 9 shared *i+rary that is not a true .so fi*e shou*d not +e adependency for another *i+rary.

    This 9 is registered as fo**o/s:

    884eps in0a,I?N-+.+D+PT.R&in0a,I?N-+.C@I,D&@.,,O+.N-+.7.G.C3T+?,.7P+T@"YK@.,,O+.L

    %ome notes a+out this registration:

    The @@depsoption te**s register@ae or nud0compi*e /hich shared *i+raries are reMuired +y this

    9 app*ication. 299 has +een added to the defau*t *ist.

    `[299\ is an e0amp*e of 9 shared *i+rary path su+stitution. t runtime the 9 runtime

    system con'erts this to a fi*e path pointing to a copy of the shared *i+rary. Note the synta0 is the

    percent sign `; fo**o/ed +y sMuare +rackets [ \; that enc*ose the e0terna* 9 shared *i+raryname.

    The 9 shared *i+rary takes care of p*acing the host and %$ e0ecuta+*es on the appropriate

    machines. n*y one NV9@989C$TB9@T2 /ithout a *ocation prefi0 is reMuired.

    This 9 app*ication, as sho/n, does not access the 9 e0port directory tree.

    The 9 shared *i+rary functiona*ity can +e used for any fi*e type. 9 shared *i+rary path su+stitution

    1*- 00?200.@03 Rev9 2

  • 8/11/2019 UDAP Developers Guide

    179/360

    !or"ing #it$ AE '$ared ibraries

    can +e used in any en'ironment 'a*ue.

    To 'ie/ on*y the 8 fi*e associa