Snap Developer Guide - SnapLogic

Snap Developer Guide

Document Release: December 2012

SnapLogic, Inc.71 East Third AvenueSan Mateo, California 94401U.S.A.

www.snaplogic.com

http://www.snaplogic.com/

Copyright Information

© 2012 SnapLogic, Inc. All Rights Reserved. Terms and conditions, pricing, andother information subject to change without notice. ”SnapLogic” and “SnapStore”are among the trademarks of SnapLogic, Inc. All other product and company namesand marks mentioned are the property of their respective owners and are men-tioned for identification purposes only. “SnapLogic” is registered in the U.S. andTrademark Office.

Table of Contents

Snap Developer Guide 1

Table of Contents 3

Preface 5

Snap Development 7

Prerequisites for Snap Development 7

Development Methods 7

Snap Development Using Eclipse 7

Snap Development Using the Command Line 14

Creating a Component for a Snap 15

SnapLogic Maven Artifacts 34

Snap Maven Archetype 35

Debugging Snaps with JPDA 36

Submitting the Snap to the SnapLogic SnapStore 38

Snap Installer 41

- 3 -

SnapLogic® User Guide

- 4 -

PrefaceA Snap is the encapsulation of an integration task or subtask. It is a “pluggable” piece of codethat has been conveniently packaged to run seamlessly on the SnapLogic Server. You canbuild your own Snap or download Snaps built by the SnapStore community. A Snap in Snap-Logic is comparable to a smart phone app, a browser Add-On, or an application plug-in. Thisdocument contains information for SnapLogic users who want to develop Snaps.

- 5 -

4

- 6 -


Snap DevelopmentSnapLogic provides a very easy and intuitive way of developing Snaps through the ApacheMaven build infrastructure.

Prerequisites for Snap Developmentl SnapLogic Server should be installed locally on the Snap development machine.

Development Methodsl Snap Development using Eclipse

l Snap Development using Command Line

For more information on SnapLogic Maven Artifacts, please follow this link.

Snap Development Using EclipseThe following tutorial explains how to create a new Snap project in Eclipse then build anddeploy the Snap in a matter of few minutes.

Known Issue

The first time, Snap Creation wizard will fail with an error. This is because Eclipsehas not downloaded the Maven archetype to the local repository.

Work around

Try the Snap creation process again. Snap creation wizard actions following thefirst run will succeed. This is because Eclipse would have downloaded the SnapArchetype at the end of the first run. Therefore, all the required properties will beshown to the user during the Snap Creation phase.

Tutorial

1. Create a new Maven project using the New Maven Project wizard.

- 7 -

5


2. Provide the project location details for creating the project.

3. Configure the Remote SnapLogic Maven repository by providing the archetype-cat-alog.xml file URL: http://maven.-snaplogic.org/content/groups/public/archetype-catalog.xml

- 8 -

Snap Development

4. Choose the newly configured SnapLogic Maven archetype catalog from the drop-downlist and select the SnapArchetype as the archetype.

- 9 -


5. Enter the project artifact details like GroupId and ArtifactId and click Finish.

If this is the first run, the project creation will fail with the error: Unable to createproject from archetype. The detailed message about the error lists several items aremissing.

- 10 -

Snap Development

The workaround is to repeat the creation process.

6. Repeat this process starting from Step 1. This time, Step 5 for entering the projectdetails will show all of the required properties that are used by the SnapArchetype.Ensure that the server property is updated with the correct path of your installed server.

7. Click Finish to create a new Snap project. The project structure from the Eclipseproject explorer is shown below.

- 11 -


8. To build the project, navigate to the base directory of the project in the shell. Use the fol-lowing Maven command to build the project:

mvn clean install

This will create a Snap deployment unit as a .zip file.

- 12 -

Snap Development

9. The Snap .zip can be deployed to the server using the following Maven command:

mvn snap:deploy

This will deploy the Snap to the server. You will be prompted for the SnapLogic Serveradmin password while deploying the Snap.

- 13 -


Snap Development Using the Command LinePrerequisites

Snap development using the command line requires Apache Maven to be installed. You candownload Apache Maven from this page.

Tutorial

This tutorial helps you in creating a simple Snap project from the command line, making somechanges to the project and finally deploying the project to the SnapLogic Server.

Creating a New Snap Project from the Archetype

1. Open the shell and go to your workspace.

2. Use the following command to create a new Snap project.

mvn archetype:generate -

DarchetypeCatalog=http://maven.snaplogic.org/content/groups/public/ -

DarchetypeGroupId=com.snaplogic -DarchetypeArtifactId=SnapArchetype

You will be prompted for groupId, artifactId, version, package - much like in theEclipse example.

Building the Snap Project

1. Edit the Component files and provide a valid implementation for all the Component classimplementations.

2. Build the project to create the Snap .zip file.

mvn clean install

- 14 -

http://maven.apache.org/download.html

Snap Development

Deploying the Snap

l Deploy the Snap to the SnapLogic Server using the following goal. Note that you will beprompted for the SnapLogic Server admin password while deploying the Snap.

mvn snap:deploy

Creating a Component for a SnapThis page describes the various parts of a SnapLogic Component. You can study any of theexisting SnapLogic Components or the sample Component located in the demo/componentsdirectory in your installation or in the source tree.

Components are implemented as Python or Java classes. The SnapLogic Component API doc-umentation contains all the information about the interface calls available to a Componentauthor.

Example Components can be found in your <version>/demo/components directory and youcan also study the SnapLogic default Components in the <version>/snaplogic/componentsdirectory.

How Components Work

There are three basic types of Components corresponding to whether the Component hasInput, Output or both Input and Output views:

l Producers (Output views only)

l Consumers (Input views only)

l Transformers (Input and Output views)

A Producer Componentmay read data from a source (such as a flat file, a database or an appli-cation interface) or generate it spontaneously (eg. a random number or sequence generator)and writes it to an Output View.

A Consumer Component does exactly the opposite, reading data from an Input View and writ-ing to a destination (such as a flat file, a database or an application interface).

A Transformer Component takes input from one or more Input Views and performs someaction on the data before writing the results to one or more Output Views.

SnapLogic supports two modes of data representation, record mode and binary mode. Recordmode is the most intuitive and corresponds to a single row of data that is organized in a tab-ular format like a spreadsheet or database table. Binary mode is useful for datastreams thatare not "rectangular" such as audio, image or datastream formats. A Component can haveboth Record mode views and Binary mode views. For example, the SnapLogic RSS Writer Com-ponent has both a Record mode output view record and a Binary mode output view rss.

When a SnapLogic client, such as the SnapLogic API (SnAPI) or the SnapLogic Designercreates a Resource based on a Component, the client program connects to the server and isreturned a data structure which represents an instantiation of that Component. The rep-resentation was created from the template defined by the Component itself. The client then

- 15 -


further configures the client-side representation, specifying such things as property values,input and/or output view definitions.

If the Component supports it, it may be able to suggest configuration values itself. For exam-ple, the Component author could derive the most likely value a CSV file delimiter based oninspection of the file itself and perhaps even define the Output View based on that delimiter.When invoking the suggest method, the client sends the Resource representation to the serverand is returned a new data structure representation of the Resource containing the suggestedconfiguration. The original Resource data structure is not modified in any way and it is up tothe client to decide if all, any or none of the suggestions are to be incorporated.

Whenever the client desires, it can request that the representation be validated by the serverfor correctness and completeness. The representation is transmitted to the server, which theninvokes the Component's validation method. The server will return to the client a list of anyerrors it detected. It is up to the client to rectify any errors reported.

The client can at any time save the Resource to the server. It is not required that the Resourcebe correct or complete. The Resource can then be executed, added to a Pipeline, retrieved ata later time or shared with other users.

Creating a Custom Component in Java

This section explains how to write a new SnapLogic Component in the Java language and runit. We will write a Component that generates Fibonacci numbers as an example. You can studythe source code for this Component, it is installed by the SnapLogic installer in the <ver-sion>/demo/components/FibProducer.java file and is also available online in the sourcetree.

Example Components can be found in your <version>/demo/components directory and youcan also study the SnapLogic default Components in the <version>/snaplogic/componentsdirectory.

For information on how to create a Component in the Python language please see the Creatinga Custom Component in Python page.

Java Component Sections

The following discussion illustrates the anatomy of a Java Component.

API Version, Labels and Documentation Link

Each Component must indicate which version of the SnapLogic Component API it is written for.The current version is 1.0. The Component must also provide a label for itself, a descriptionand a link to web page where the Component is documented.

Note also that even though the FibProducer is not using a package declaration, it is a goodidea to do so for your custom Components. Follow the standard Java convention on packagenaming, such as org.company.

import java.math.*;

import java.util.*;

import org.snaplogic.cc.*;

- 16 -

Snap Development

import org.snaplogic.cc.prop.*;

import org.snaplogic.cc.prop.SimpleProp.*;

import org.snaplogic.common.*;

public class FibProducer extends ComponentAPI {

public String getAPIVersion() {

return "1.0";

}

public String getComponentVersion() {

return "1.0";

}

public String getLabel() {

return "Fibonacci Producer Component";

}

public String getDescription() {

return "This component produces Fibonacci numbers as output.";

}

public String getDocURI() {

return String.format("http://www.fibonacci.it/snaps/FibProducer/1.0");

}

Capabilities

The capabilities method determines whether the Component is a Producer, Consumer or Trans-former. It also indicates if the Component supports pass-through, for transformer Com-ponents. This method must be implemented.

Example of a Producer Component:

// This producer component has exactly one output view.

public Capabilities getCapabilities() {

Capabilities capabilities = new Capabilities();

capabilities.put(Capability.INPUT_VIEW_LOWER_LIMIT, 0);

capabilities.put(Capability.INPUT_VIEW_UPPER_LIMIT, 0);

capabilities.put(Capability.OUTPUT_VIEW_LOWER_LIMIT, 1);

capabilities.put(Capability.OUTPUT_VIEW_UPPER_LIMIT, 1);

return capabilities;

}

Example of a Transformer Component that supports pass-through:

// This transformer component supports pass-through

- 17 -


public Capabilities getCapabilities() {

Capabilities capabilities = new Capabilities();

capabilities.put(Capability.INPUT_VIEW_LOWER_LIMIT, 1);

capabilities.put(Capability.INPUT_VIEW_UPPER_LIMIT, 1);

capabilities.put(Capability.OUTPUT_VIEW_LOWER_LIMIT, 1);

capabilities.put(Capability.OUTPUT_VIEW_UPPER_LIMIT, 1);

capabilities.put(Capability.ALLOW_PASS_THROUGH, true);

return capabilities;

}

Resource Template

public void createResourceTemplate() {

The resource template is used to instantiate a SnapLogic Resource based on a particular Snap-Logic Component. The template defines the Properties of the Resource. Properties are var-iables that can be set by the user to influence the execution of the Component. Properties canbe strings, numbers, date/times, and Booleans (and lists and dictionaries of these types).

Properties can be required or optional. They can be populated with default values, or leftblank for the user to specify. The allowable values can be constrained to a set of values basedon the property type (eg. a boolean property can be only be true or false), based on a list pro-vided by the Component author (eg. an aggregate operator can only be 'min', 'max', 'sum',etc), or constrained to be of a valid input or output field name. These constraints will then bevalidated automatically by the SnapLogic Server without having to write your own validationcode.

The resource template can also specify default or mandatory input or output views. For exam-ple, if a Component can only have one particular output format, then the Component authorcan pre-configure the output view to save the user from having to create it themselves.

Validation

public void validate(ComponentResourceErr errObj) {

Validation is a very important part of any Component, and it can often be the largest portionof a Component. This section implements any checks that must pass before the Component isconsidered "correct" and ready for execution. If a check fails, an error message is set in thevalidation error object which is passed back to the client as a response to the validationrequest.

Component validation is performed in two passes. The first part doesn't really involve theComponent's validation section at all! It is performed by the Component container based onthe definition of the Component's capabilities and properties. The CC will check that anyrequired views have been defined, all required properties have been set, and that any prop-erty value constraints pass.

When the first pass checks pass cleanly, then the Component's validate method is invoked toperform the second pass. In this optional section, the Component author can perform complexor domain specific checks on the property values, or view definitions.

- 18 -

Snap Development

Suggest

public void suggestResourceValues(ComponentResourceErr errObj) {

The suggest section allows the Component author to suggest appropriate values for propertiesor views to the user based on the input received so far. For example, the CSV Reader Com-ponent will suggest an appropriate value for the "delimiter" property based on the value of the"filename" property. It will also suggest an appropriate output view by inspecting the data itfinds in the input file. If the input file contains a header row, it will name the output fieldsappropriately, otherwise it will use the generic field1, field2 names.

The suggest mechanism does not, however, modify the user's original Resource. It worksupon a copy of the Resource which is returned to the user. It is up to the user to decide if hewishes to use the result of the suggest operation.

The SnapLogic Designer invokes the suggest method whenever the user clicks on the Suggestbutton of a Resource.

Execution

public void execute(Map<String, InputView> inputViews, Map<String,

OutputView> outputViews) {

The execute section is where the actual execution logic of a Component resides. The executemethod is invoked by the Component Container and is passed the list of input and outputviews defined for the operation.

Records are read from input views by selecting the desired input view from the list and invok-ing the readRecord() method.


OutputView> outputViews)

InputView inputView = inputViews.values().iterator().next();

while (true) {

Record record;

record = inputView.readRecord();

if (record == null)

break;

// Process record ...

The readRecord() method will block until a record is available or "end of data" has beenreceived, in which case it will return None.

Similarly, records are written to the output view with the writeRecord() method.


OutputView> outputViews)

OutputView outputView = outputViews.values().iterator().next();

while (true) {

Record record = outputView.createRecord();

// set the individual fields of the record

...

// Write the record to the output view

outputView.writeRecord(record);

- 19 -


// if done, break

// No more records to write. Indicate completed.

outputView.completed();

Each output record must be created by calling the createRecord() method of the cor-responding output view. This will create an output record that is of the correct form and allfields initialized to null.

Components are obligated to indicate that all data has been written to a particular output viewby invoking the completed() method of the view.

It is good programming practice to check that views are connected before attempting to readfrom or write to them. For example, a Producer Component may have a single Output Viewwhich must be connected, or it may have a second Output View which may be optionally con-nected.

public class SomeWriter extends ComponentAPI {

...


OutputView> outputViews) {

// If your component has an output view which must be written to, then

// make sure that the view is connected to, upfront, as follows:

if (!outputViews.containsKey("some_required_view"))

throw new SnapComponentException("Output view 'some_required_view' must

be connected");

while (true) {

// If your component has an output view which does not need to

// be connected to, in order for the component to be useful,

// you can write conditionally to that output view as follows:

OutputView optionalView = outputViews.get("some_optional_view");

if (optionalView != null) {

Record rec = optionalView.createRecord();

rec.set("field001", data);

optionalView.writeRecord(rec);

// ...

If your Component will provide pass-through support then your Component must append anypass-through fields after the output record has been computed.

Record inputRecord = inputView.readRecord();

while (inputRecord != null) {

// Create an output record

Record outputRecord = outputView.createRecord();

// Set the fields my component computes

outputRecord.set("field001", something);

// Append any pass-through fields from the input record

outputRecord.transferPassThroughFields(inputRecord);

}

- 20 -

Snap Development

// We are done

outputView.completed();

Upgrade

The SnapLogic architecture allows for Components to evolve. When improvements or enhance-ments to a Component are made available, the Component should increment its Componentversion number and provide an upgrade method for evolving existing Resources based on theolder Component.

For example, if a Component introduces a new property in version 1.1, the Resource upgrademethod for evolving 1.0 resources will create the new property and perhaps set a reasonabledefault value for it.

The upgrade method is invoked during Snap installation to evolve any existing Resourcesbased on the Component.

Datatypes

It is the Component author's decision as to whether the Component should be strongly typedor weakly typed. A strongly typed Component is one where the output view field types mustmatch the input view field types. A weakly typed Component allows the user of the Com-ponent to transmute the type of a field as it is processed, for example, converting a numberinto a string. Some conversions may result in an exception and must the Component authormust decide if the error should be ignored and the record rejected or if it should cause theexecution to abort. For example, the string "123" can be converted to the number 123, but"abc" cannot.

Data read from input views or written to output views must be of the following the types:

l java.lang.String

l java.math.BigDecimal

l java.sql.Timestamp

In addition, Components must support reading/writing null (the null data indicator).

Writing any other datatype to an Output View will result in an exception from the SnapLogiccommunication layer (SnapStream) which handles the data after it is written to the view.

The following example shows how one might process fields in a weakly typed Component.

...

Record outRec = outputView.createRecord();

int i = 0;

for (Field field: outputView.getFields()) {

// Coerce the field data to output field type.

// Null must be treated specially since it cannot be

coerced.

// Note that these coercions could cause an exception that

author

// may or may not handle.

if (row[i] == null)

- 21 -


outRec.set(field.getName(), row[i]);

else if (field.getType() == SnapFieldType.SnapNumber)

outRec.set(field.getName(), new BigDecimal(row[i].toString());

else if (field.getType() == SnapFieldType.SnapDateTime) {

java.text.SimpleDateFormat formatter = new java.text.SimpleDateFormat

("yyyy-MM-dd HH:mm:ss");

formatter.parse(row[i].toString());

java.util.Calendar date = formatter.getCalendar();

outRec.set(field.getName(), new java.sql.Timestamp

(date.getTimeInMillis()));

} else {

// String

outRec.set(field, row[i]);

}

i += 1

}

outputView.writeRecord(outRec);

...

Installing a Component

Standard Java Components that come with a SnapLogic release are located in the <ver-sion>/snaplogic/lib directory in the javacomps.jar file. When the SnapLogic ComponentContainer starts, it will discover any and all Components present in this directory.

It is recommended, however, that custom Components be kept separate from the standardset. Instead, custom Components should be placed in a directory under exten-sions/components. For example, extensions/components/FibProducer.jar. By default,the server is configured to search extensions/components for any Components.

Here are sample commands to make and deploy the FibProducer Component. These com-mands are executed with the current directory being the directory where SnapLogic isinstalled.

Commands to Build and Install the FibProducer Component on Linux

Note: Building Java Components requires a version of Java SDK that is version1.6 or higher.

You can check the Java compiler version by running the command:

javac -version

The commands given here assume that SnapLogic is installed in the /opt/snaplogic direc-tory, and that Java SDK 1.6 or higher is installed and added to the PATH.

$ bash

$ cd /opt/snaplogic

$ source bin/snaplogic_env.sh

$ cd $SNAP_HOME/demo/components

$ javac -cp $SNAP_HOME/lib/javacc.jar FibProducer.java

- 22 -

Snap Development

$ ls *.class

FibProducer$1.class FibProducer.class

$ echo -e "SnapLogic-Components: FibProducer\n" > manifest.txt

$ jar cfm FibProducer.jar manifest.txt *.class

$ ls *.jar

FibProducer.jar

$ cp FibProducer.jar $EXTENSIONS_COMPONENTS

$ snaplogic.rc restart

Shutting down SnapLogic Data Server:

Shutting down SnapLogic Management Server:

Shutting down SnapLogic Component Container cc1:


Starting SnapLogic Data Server

Starting SnapLogic Management Server

Starting SnapLogic Component Container cc1_python

Starting SnapLogic Component Container cc2_java

$ grep FibProducer $SNAP_HOME/../logs/cc2.log

2010-01-26T10:00:44 INFO cc2.main CC - - - - Registered component

FibProducer FibProducer

(/opt/snaplogic/extensions/components/FibProducer.jar)

After restarting SnapLogic, the new FibProducer Component gets registered with the Snap-Logic Component Container.

These manual install steps above are helpful in understanding what needs to happen in orderfor the custom Component to get installed. However there is an automated Snap Installer thatcould be used to provide this functionality.

Note: In SnapLogic release 2.2.0 FibProducer doesn't use a Java package. Thiswas corrected in the 2.2.1 release. It is now required that SnapLogic Componentsuse package declaration, e.g. "package org.company;". When using a package,the build instructions above must be slightly modified accordingly, and the man-ifest declaring the Component must also declare the fully qualified Componentname including the package, e.g.: "SnapLogic-Components: org.co-mpany.FibProducer".

Commands to Build and Install the FibProducer Component on Windows

The commands given here assume that SnapLogic is installed in the c:\Program Files\sna-

plogic directory, and that Java SDK 1.6 or higher is installed and added to the PATH.

C:\> cd "c:\Program Files\snaplogic"

C:\Program Files\snaplogic> bin\snaplogic_env.bat

C:\Program Files\snaplogic> cd %SNAP_HOME%\demo\components

C:\Program Files\snaplogic\2.3.0PE\demo\components> javac -cp %SNAP_

HOME%\lib\javacc.jar FibProducer.java

C:\Program Files\snaplogic\2.3.0PE\demo\components> echo SnapLogic-

Components: FibProducer > manifest.txt

C:\Program Files\snaplogic\2.3.0PE\demo\components> echo. >>

- 23 -


manifest.txt

C:\Program Files\snaplogic\2.3.0PE\demo\components> jar cfm

FibProducer.jar manifest.txt *.class

C:\Program Files\snaplogic\2.3.0PE\demo\components> copy

FibProducer.jar %EXTENSIONS_COMPONENTS%

C:\Program Files\snaplogic\2.3.0PE\demo\components> cd "c:\Program

Files\snaplogic"

C:\Program Files\snaplogic> bin\snapctl.bat stop

C:\Program Files\snaplogic\2.3.0PE> bin\init.d\cc2_java.bat remove_

service

C:\Program Files\snaplogic\2.3.0PE> bin\init.d\cc2_java.bat install_

service

C:\Program Files\snaplogic\2.3.0PE> bin\snapctl.bat start

Shutting down SnapLogic Data Server:

Shutting down SnapLogic Management Server:



Starting SnapLogic Data Server

Starting SnapLogic Management Server

Starting SnapLogic Component Container cc1_python

Starting SnapLogic Component Container cc2_java

C:\Program Files\snaplogic\2.3.0PE> findstr FibProducer ..\logs\cc2.log

2010-01-26T10:00:44 INFO cc2.main CC - - - - Registered component

FibProducer FibProducer (C:\Program

Files\snaplogic\extensions\components\FibProducer.jar)

After restarting SnapLogic, the new FibProducer Component gets correctly registered by theComponent Container.

These manual install steps above are helpful in understanding what needs to happen in orderfor the custom Component to get installed. However there is an automated Snap Installer thatcould be used to provide this functionality.

Note that if you are running SnapLogic PE release 2.2.0 or earlier you must issue the followingcommands after having run the install (this is due to bug #2290 fixed in SnapLogic release2.2.1):

C:\Program Files\snaplogic\2.3.0PE\demo\components> cd "c:\Program

Files\snaplogic"

C:\Program Files\snaplogic> bin\snapctl.bat stop

C:\Program Files\snaplogic\2.3.0PE> bin\init.d\cc2_java.bat remove_

service

C:\Program Files\snaplogic\2.3.0PE> bin\init.d\cc2_java.bat install_

service

In addition to that, the FibProducer in SnapLogic release 2.2.0 doesn't use a Java package.This was corrected in the 2.2.1 release. It is now required that SnapLogic Components usepackage declaration, e.g. "package org.company;". When using a package, the build instruc-tions above must be slightly modified accordingly, and the manifest declaring the Component

- 24 -

Snap Development

must also declare the fully qualified Component name including the package, e.g.: "Snap-Logic-Components: org.company.FibProducer".

Troubleshooting Component Loading

If the Component Container log does not report registering your new Component, check thefollowing list of common issues:

• Is your Component JAR file in the correct directory location? As explained in the sectionInstalling a Component, the Component should be placed inside the extensions/componentsdirectory.

• Is your Component implemented correctly? Follow the design of FibProducer.java closely,and check the javadoc documentation installed in <version>/doc/java. Check that there areno compiler errors, and that the jar file contains the Component class, as well as any otherthird-party classes it depends on. If there are third-party libraries that this Componentdepends on, these must be copied into the extensions/components directory as well.

• Turn on verbose logging for the Java Component Container: editing the con-fig/snapserver.conf and locate the section [cc2]. Add a new propertylog_level =

DEBUG somewhere in this section. Your new section may look something like this:

[[cc2]]

cc_hostname = localhost

cc_port = 8090

component_conf_dir = /opt/snaplogic/component_config

log_dir = /opt/snaplogic/logs

component_dirs = $SNAP_HOME/lib,

/opt/snaplogic/extensions/components

# Set log level to include debug and diagnostic messages

log_level = DEBUG

After restarting SnapLogic, examine the cc2.log file to see if there were any helpful diag-nostic messages generated regarding the Component registration of the customer Com-ponent.

Debugging Your New Component

Within a Component, you can write messages to the Component Container log file using thelogging methods debug(), info(), warning(), error(). To log messages to the ComponentContainer exceptions log, use elog(). Note that in order for debug messages to show up inthe logs, verbose logging needs to be turned on as described above.

...

// Log a message to the cc log file

info("This is a log message from my component");

try {

...

- 25 -


} catch (NullPointerException e) {

// Log the exception

elog(e);

}

When installing a new iteration of your Component, you may want to delete any existingResources created using that Component to avoid having any Resources that were createdusing older versions of you Component. If you are evolving your Component and have alreadyreleased a version of your Component, you will want to consider using the upgrade methodsas described earlier.

Debugging with JPDA (Java Platform Debugging Architecture)

See "Debugging Snaps with JPDA"

Packaging a Component

SnapLogic has developed a simple distribution and installation format that works easily onboth Windows and Linux platforms. The Snap Installer supports fetching and installing depend-encies, installing Components, executing configuration scripts and importing samples into theSnapLogic Data Server.

For more details, see the "Snap Installer" page.

Creating a Custom Component in Python

This page describes the various parts of a SnapLogic Python Component. You can study any ofthe existing SnapLogic Components or the sample Component located in the demo/com-ponents directory in your installation or in the source tree.

For information on how to create a Component in the Java language, see the Creating a Cus-tom Component in Java page.

Python Component Sections

The following discussion illustrates the anatomy of a Python Component.

API Version, Labels and Documentation Link

Each Component must indicate which version of the SnapLogic Component API it is written for.The current support version is 1.0. The Component must also provide a label for itself, adescription and a link to web page where the Component is documented.

from snaplogic.snapi_base import keys

from snaplogic.cc.component_api import ComponentAPI

class MyComponent(ComponentAPI):

api_version = '1.0'

component_label = "My Custom Component"

component_description = "Computes the coefficient of drag

- 26 -

Snap Development

for proteins"

component_doc_uri =

"http://www.snapmaster.com/snaps/MyCustomComponent/1.1"

Capabilities

The capabilities section determines whether the Component is a Producer, Consumer or Trans-former. It also indicates if the Component supports pass-through, for transformer Com-ponents. This section must be present.

Example of a Producer Component

# This producer component has exactly one output view.

capabilities = {

ComponentAPI.CAPABILITY_INPUT_VIEW_LOWER_LIMIT : 0,

ComponentAPI.CAPABILITY_INPUT_VIEW_UPPER_LIMIT : 0,

ComponentAPI.CAPABILITY_OUTPUT_VIEW_LOWER_LIMIT : 1,

ComponentAPI.CAPABILITY_OUTPUT_VIEW_UPPER_LIMIT : 1,

}

Example of a Transformer Component that supports pass-through

# This transformer component supports pass-through

capabilities = {

ComponentAPI.CAPABILITY_INPUT_VIEW_LOWER_LIMIT : 1,

ComponentAPI.CAPABILITY_INPUT_VIEW_UPPER_LIMIT : 1,

ComponentAPI.CAPABILITY_OUTPUT_VIEW_LOWER_LIMIT : 1,

ComponentAPI.CAPABILITY_OUTPUT_VIEW_UPPER_LIMIT : 1,

ComponentAPI.CAPABILITY_ALLOW_PASS_THROUGH : True

}

Resource Template

def create_resource_template(self):

The resource template is used to instantiate a SnapLogic Resource based on a particular Snap-Logic Component. The template defines the Properties of the Resource. Properties are var-iables that can be set by the user to influence the execution of the Component. Properties canbe strings, numbers, date/times, and Booleans (and lists and dictionaries of these types).

Properties can be required or optional. They can be populated with default values, or leftblank for the user to specify. The allowable values can be constrained to a set of values basedon the property type (a Boolean property can be only be True or False), based on a list pro-vided by the Component author (an aggregate operator can only be 'min', 'max', 'sum', etc),or constrained to be of a valid input or output field name. These constraints will then be val-idated automatically by the SnapLogic server without having to write your own validationcode.

The resource template can also specify default or mandatory input or output views. For exam-ple, if a Component can only have one particular output format, then the Component authorcan pre-configure the output view to save the user from having to create it themselves. The

- 27 -


SnapLogic RSS Writer Component is a good example of this as it configures nine input views,with predefined view names and field names.

Validation

def validate(self, err_obj):

Validation is a very important part of any Component, and it can often be the largest portionof a Component. This section implements any checks that must pass before the Component isconsidered "correct" and ready for execution. If a check fails, an error message is set in thevalidation error object which is passed back to the client as a response to the validationrequest.

Component validation is performed in two passes. The first part doesn't really involve theComponent's validation section at all! It is performed by the Component Container based onthe definition of the Component's capabilities and properties. The Component Container (CC)will check that any required views have been defined, all required properties have been set,and that any property value constraints pass.

When the first pass checks pass cleanly, then the Component's validate method is invoked toperform the second pass. In this optional section, the Component author can perform complexor domain specific checks on the property values, or view definitions.

Suggest

def suggest_resource_values(self, err_obj):

The suggest section allows the Component author to suggest appropriate values for propertiesor views to the user based on the input received so far. For example, the CSV Reader Com-ponent will suggest an appropriate value for the "delimiter" property based on the value of the"filename" property. It will also suggest an appropriate output view by inspecting the data itfinds in the input file. If the input file contains a header row, it will name the output fieldsappropriately, otherwise it will use generic field1, field2 etc names.

The suggest mechanism does not, however, modify the user's original Resource. It worksupon a copy of the Resource which is returned to the user. It is up to the user to decide if hewishes to use the result of the suggest operation.

The SnapLogic Designer invokes the suggest method whenever the user clicks on the Suggestbutton of a Resource.

Execution

def execute(self, input_views, output_views):

The execute section is where the actual execution logic of a Component resides. The executemethod is invoked by the Component container and is passed the list of input and output viewsdefined for the operation.

Records are read from input views by selecting the desired input view from the list and invok-ing the read_record() method.


- 28 -

Snap Development

input_view = input_views.values()[keys.SINGLE_VIEW]

while True:

record = input_view.read_record()

if record is None:

break

# Process record ...

The read_record() method will block until a record is available or "end of data" has beenreceived, in which case it will return None.

Similarly, records are written to the output view with the write_record() method.


output_view = output_views.values()[keys.SINGLE_VIEW]

while True:

record = output_view.create_record()

# set the individual fields of the record

...

# Write the record to the output view

output_view.write_record(record)

# if done, break

# No more records to write. Indicate completed.

output_view.completed()

Each output record must be created by calling the create_record() method of the cor-responding output view. This will create an output record that is of the correct shape and allfields initialized to null.

Components are obligated to indicate that all data has been written to a particular output viewby invoking the completed() method of the view.

It is good programming practice to check that views are connected before attempting to readfrom or write to them. For example, a Producer Component may have a single Output Viewwhich must be connected, or it may have a second Output View which may be optionally con-nected.

class SomeWriter(ComponentAPI):

...


# If your Component has an output view which must be written to,

then

# make sure that the view is connected to, upfront, as follows:

if "some_required_view" not in output_views:

raise SnapComponentError("Output view 'some_required_view'

must be connected")

while True:

data = fp.read()

# If your Component has an output view which does not need

- 29 -


to

# be connected to, inorder for the Component to be useful,

then

# you can write conditionally to that output view as

follows:

optional_view = output_views.get("some_optional_view")

if optional_view is not None:

rec = optional_view.create_record()

rec['field001'] = data

optional_view.write_record(rec)

# ...

If your Component will provide pass-through support then your Component must append anypass-through fields after the output record has been computed.

input_record = input_view.read_record()

while input_record is not None:

# Create an output record

output_record = output_view.create_record()

# Set the fields my Component computes

output_record[output_field] = something

# Append any pass-through fields from the input record

output_record.transfer_pass_through_fields(input_record)

# We are done

output_view.completed()

Upgrade

The SnapLogic architecture allows for Components to evolve. When improvements or enhance-ments to a Component are made available, the Component should increment its Componentversion number and provide an upgrade method for evolving existing Resources based on theolder Component.

For example, if a Component introduces a new property in version 1.1, the Resource upgrademethod for evolving 1.0 resources will create the new property and perhaps set a reasonabledefault value for it.

The upgrade method is invoked during Snap installation to evolve any existing Resourcesbased on the Component.

The following example is taken from the SnapLogic Filter Component. In version 1.1 of theFilter Component, the Wildcard Bypass property was introduced and later, in version 1.2, aconstraint to the Field property was introduced to require that the value for this property bean existing input view field name. The Filter Component module contains the following meth-ods to evolve and existing Resources based on the Filter Component. The methods are appliedsuccessively as required to make the Resource current with the Component version.

- 30 -

Snap Development

def upgrade_1_0_to_1_1(self):

"""

Change 1.

Add the Wildcard Bypass value.

"""

# Create the new properties

self.set_property_def("Bypass", prop.SimpleProp("Wildcard for bypass",

SnapString,

"String that indicates that Filter should be bypassed", None, False))

def upgrade_1_1_to_1_2(self):

"""

Add LOV constraint to Field property

"""

# Save the property value.

# We need to recreate the property, which resets the value

property_value = self.get_property_value("Field")

# Redefine "Field" property to include the constraint

self.del_property_def("Field")

self.set_property_def("Field", prop.SimpleProp("Field", SnapString,

"Field to filter on",

{'lov': [ keys.CONSTRAINT_LOV_INPUT_FIELD] } ,

True))

# Restore the value

self.set_property_value("Field", property_value)

Datatypes

It is the Component author's decision as to whether the Component should be strongly typedor weakly typed. A strongly typed Component is one where the output view field types mustmatch the input view field types. A weakly typed Component allows the user of the Com-ponent to transmute the type of a field as it is processed, for example, converting a numberinto a string. Some conversions may result in an exception and must the Component authormust decide if the error should be ignored and the record rejected or if it should cause theexecution to abort. For example, the string "123" can be converted to the number 123, but"abc" cannot.

Data read from input views or written to output views must be of the following the types:

l Decimal

l Unicode String

l Datetime

In addition, Components must support reading/writing None (the null data indicator).

- 31 -


Writing any other datatype to an Output View will result in an exception from the SnapLogiccommunication layer (SnapStream) which handles the data after it is written to the view.

The following example shows how one might process fields in a weakly typed Component.

...

# Get the output view field's datatypes

types_list = [output_field[keys.FIELD_TYPE] for output_field in output_

view_fields]

out_rec = output_view.create_record()

i = 0

for field in out_rec.field_names:

# Coerce the field data to output field type.

# None must be treated specially since it cannot be coerced.

# Note that these coercions could cause an exception that author

# may or may not handle.

if row[i] is None:

out_rec[field] = row[i]

elif types_list[i] == SnapNumber:

out_rec[field] = Decimal(str(row[i]))

elif types_list[i] == SnapDateTime:

out_rec[field] = datetime(*(time.strptime(row[i],

DATE_FORMAT_STRING)[0:6]))

else:

out_rec[field] = row[i]

i += 1

output_view.write_record(out_rec)

...

Installing a Component

The standard set of Components that comes with a SnapLogic release are located in the <ver-sion>/snaplogic/components directory. When the SnapLogic Component Container starts, itwill discover any and all Components present in this directory. For Java Components, thoseare contained in the javacomps.jar file in the <version>/snaplogic/lib directory.

It is recommended, however, that custom Components be kept separate from the standardset. Instead, custom Components should be placed in a directory under exten-sions/components. For example, extensions/components/my_custom/. By default, theserver is configured to search extensions/components for any Components.

For Python Components, you will need to create the __init__.py file for that directory thatcontains the following:

import sys

from snaplogic.cc import registration

def get_component_list():

return registration.discover_package_components(sys.modules[__name__])

- 32 -

Snap Development

After restarting the Component container, check the cc log file (typically cc1.log) to verify thatyour Component was correctly registered with the Component Container. Your log outputshould look something like:

20090217134815 INFO CC cc1 - "Discovering components under directory

/opt/snaplogic/extensions/components"

20090217134815 INFO CC cc1 - "Registered component my_

custom.MyComponent"

Troubleshooting Component Loading

If the Component Container log does not report registering your new Component, check thefollowing list of common issues:

l Is your Component module in the correct directory location? As explained in the section"Installing a Component", the Component should be placed inside the exten-sions/components directory.

l Is the Component __init__.py discovery code file present? Have you added the __init__.py attached to this page to the Component?

l Does the Component follow the proper naming convention? The name of the Componentmodule (the .py file) must match the name of the Component class.

l Is your Component implemented correctly? Are there Python errors in your code? Checkthe log file logs/cc1_exceptions.log to see if any syntax errors prevented the Com-ponent from being loaded.

Debugging Your New Component

Within a Component, you can write debugging messages to the Component Container log fileusing the logging method log(). To log messages to the Component Container exceptions log,use elog().

You will need to enable debug logging for the Component Container by setting the log_levelparameter in the SnapLogic configuration file.

...

# Configuration of Component Containers (CC)

[component_container]

# Name of CC

[[cc1]]

log_level = DEBUG

...

...

from snaplogic.common import snap_log

...

...

- 33 -


# Log a message to the cc log file

self.log(snap_log.LEVEL_DEBUG, "This is my debug statement

from my Component")

# Log a message to the cc exceptions log file

self.elog(e) # Where e is the exception object

self.elog(e, "Some message") # Logs the exception and a

message associated with that exception.

...

When installing a new iteration of your Component, you may want to delete any existingResources created using that Component to avoid having any Resources that were createdusing older versions of your Component. If you are evolving your Component and havealready released a version of your Component, you will want to consider using the upgrademethods as described earlier.

Packaging a Component

SnapLogic has developed a simple distribution and installation format that works easily onboth Windows and Linux platforms. The Extension Installer supports fetching and installingdependencies, installing Components, executing configuration scripts and importing samplesinto the SnapLogic Data Server.

SnapLogic Maven ArtifactsSnapLogic Maven Repository

Repository Url http://maven.snaplogic.org/content/groups/public/

Index Url http://maven.snaplogic.org/content/groups/public/.index/

SnapLogic Maven Artifacts

SnapLogic provides two Maven artifacts for simplifying the Snap development process - Sna-pArchetype and SnapPlugin.

Group Id Artifact Id Latest Versioncom.snaplogic SnapArchetype 1.0

com.snaplogic SnapPlugin 1.0

What Does SnapArchetype Do?

SnapArchetype is a Maven archetype that contains the basic Snap project structure, buildscripts for generating Snap .zip as well as sets up the classpath with all the required depend-encies for Snap development. This simplifies the task of creating the required directory treestructure as well as bundling them into a single deployable unit. The SnapArchetype uses thefollowing properties to generate the Snap project.

Property Descriptionserver This is location of the SnapLogic Server on the filesystem.

Example: /opt/snaplogic/3.x.y.z

snap This is the name of the Snap.

- 34 -

Snap Development

Property Descriptioncompany This is the name of the company that is developing the Snap.

serverURL This is the URL of the SnapLogic Server where the snap should bedeployed.Example: http://localhost:8088

What Does SnapPlugin Do?

SnapPlugin is a Maven plug-in that is very handy when developing and testing Snaps. The firstuse of the Snap plug-in is to automatically update the MANIFEST.MF with the list of SnapLogicComponents that will be bundled in the Snap. The second use of SnapPlugin is to allow theusers to easily deploy the Snaps directly from their IDE/shell. The SnapPlugin exposes the fol-lowing goals.

Plug-inGoal

Description

deploy This goal is used to deploy the Snap .zip to the server URL."user" property, if defined, is used as the admin username. If not, the defaultuser name "admin" is used as the username."password" property, if defined, is used as the admin password. If not, defaultSnapLogic Server admin password is assumed.

update-com-ponent-list

This goal is used to automatically scan the project source tree and update theMANIFEST.MF file with the SnapLogic Component list.

Snap Maven ArchetypeSnapLogic has a Maven archetype for Snap development. Using the Maven archetype, theSnap developers can get started in minutes and can deploy their Snap to the SnapLogic Serverdirectly from their IDE.

Prerequisites

This tutorial assumes that the SnapLogic Server is installed locally on the machine where theSnap project will be created.

SnapLogic Maven Repository

Repository URL http://maven.snaplogic.org/content/groups/public/

Index URL http://maven.snaplogic.org/content/groups/public/.index/

SnapLogic Artifacts

There are two SnapLogic artifacts that are available exclusively in this Maven repository.

Group Id Artifact Id Latest Versioncom.snaplogic SnapArchetype 1.0

com.snaplogic SnapPlugin 1.0

- 35 -

http://maven.snaplogic.org/content/groups/public/

http://maven.snaplogic.org/content/groups/public/.index/


SnapArchetype

SnapArchetype is the Maven archetype that provides the basic skeleton project settings forthe Snap developers. SnapArchetype needs four properties from which a new project can begenerated.

Property Descriptionserver This is location of the SnapLogic Server on the filesystem.

Example: /opt/snaplogic/3.x.y.z

snap This is the name of the Snap .

company This is the name of the company that is developing the Snap.

serverURL This is the URLof the SnapLogic Server where the Snap should bedeployed.Example: http://localhost:8088

SnapPlugin

SnapPlugin is the Maven plug-in which helps the Snap developers to automatically generatethe Component list as well as deploy the Snaps to any SnapLogic Server.

Plug-in Goal Descriptiondeploy This goal is used to deploy the Snap zip to the server URL.

l "user" property, if defined, is used as the admin username. If not,default user name "admin" is used as the username.

l "password" property, if defined, is used as the admin password. If not,default SnapLogic Server admin password is assumed.

update-com-ponent-list

This goal is used to automatically scan the project source tree and update theMANIFEST.MF file with the SnapLogic Component list.

Debugging Snaps with JPDAJPDA is the Java Platform Debugging Architecture. It provides services for debugging byattaching to a debugged process or by launching a process under a debugger, allowing one toset breakpoints, step through code, examine variables, evaluate code snippets, etc. Addi-tionally, hot swap is supported, allowing you to modify code in the IDE and have it auto-matically deploy into the debugged process without relaunching the process or withoutexplicitly recompiling the code in many cases.

Common Java IDEs, such as Eclipse, integrate with JPDA for a visual debugging experience.This page provides examples for using Eclipse when debugging, but they should easily trans-late to other IDEs, such as IntelliJ or NetBeans.

Debugging Java Snaps

Modifying the Startup Script

- 36 -

http://java.sun.com/javase/technologies/core/toolsapis/jpda/




Snap Development

Modify the STARTCOMMAND= line in <SNAP_HOME>/bin/cc2_java.sh(similar changes apply tothe .bat file) by inserting the following line right after STARTCOMMAND="${JAVA_HOME}/bin/java \:

-agentlib:jdwp=transport=dt_socket,server=y,address=9999,suspend=n

In other words, modify

STARTCOMMAND="${JAVA_HOME}/bin/java $SNAP_JAVA_DEBUG $SNAP_JAVA_MEMORY \${LOGGING_OPTIONS} ${PROXY_OPTIONS} \

to be

STARTCOMMAND="${JAVA_HOME}/bin/java $SNAP_JAVA_DEBUG $SNAP_JAVA_MEMORY \-agentlib:jdwp=transport=dt_socket,server=y,address=9999,suspend=n \${LOGGING_OPTIONS} ${PROXY_OPTIONS} \

Note: The choice of port 9999 here is of course for example only. You may pickany port you wish.

Now, restart the CC by executing <SNAP_HOME>/bin/snapctl.sh restartcc. Java CC is nowrunning in a mode that will allow a debugger to attach to it at any time. See "Setting upEclipse" for how to configure the debugger to attach to it.

Note: On Windows, remote debugging will not work while Java CC is run as a Win-dows service. After changing the startup script as above, and prior to restartingthe CCs, remove the Java CC from services so that it starts as a normal program,as follows:<SNAP_HOME>\bin\init.d\cc2_java.bat remove_service

Setting up Eclipse

1. Go to Run-Debug Configurations.

2. Select Remote Java application and click New debug configuration. In the resultingpreference page:

l Choose the project.

l Keep the Connection Type as default (Standard (Socket Attach))

l Fill in the port number that you defined above (9999).

l Fill in the host if distinct from localhost.

l Apply.

3. Click Debug. You will now be connected to the running Java CC process.

See also:

• JPDA Connection and Invocation Details

- 37 -

http://java.sun.com/j2se/1.5.0/docs/guide/jpda/conninv.html






Submitting the Snap to the SnapLogic SnapStoreThis page describes how to package Snaps for inclusion in the SnapLogic SnapStore. It isintended to be used as a checklist for Snap developers who have already developed a Snapeither in Python or Java and describes technical requirements for Snaps submitted to the Snap-Logic SnapStore.

If you haven't developed a Snap yet you can learn how to write one by following the instruc-tions for Java or Python Snaps.

Submission Files

When submitting your Snap make sure to include two .zip files:

l ZIP1: The .zip file with the Snap that will be placed for sale in the SnapStore. This .zipfile will be put, without any modifications, into the SnapStore, so it must be installableand working. This is what the user receives when they buy your Snap.

l ZIP2: The .zip file with the source code and testing information for SnapLogic use only.If your Snap requires a connection to an external system, make sure to provide a user-name and password. This .zipfile will not be shared with anyone outside of SnapLogic.

Note: It is very important that before submitting you check that the Snap is instal-lable and working by doing the following: Install a fresh copy of SnapLogic, andinstall your Snap on it using ZIP1. Go to SnapLogic Designer, and verify that yourSnap is installed and working as intended.

ZIP1 Requirements

ZIP1 must conform to the following requirements:

A Snap must be packaged as a .zip File

Snap must be packaged as a .zip file named using the following convention: companyname-snapname-major.minor.zip where:

l companyname is the company or organization/party name, such as "acme"

l snapname is the name of the Snap, such as "inventory"

l major is the major version number

l minor is the minor version number

For example if your company name is Acme, and your Snap name is "Inventory Access", youcould name the zip file acme-inventory-1.0.zip.

The structure of the .zip file should be as follows (this example is for a Java Snap):

company-snap-major.minor/

company-snap-major.minor/install.bat

company-snap-major.minor/install.sh

company-snap-major.minor/common.py

company-snap-major.minor/company-snap/

company-snap-major.minor/company-snap/data/

- 38 -

Snap Development

company-snap-major.minor/company-snap/scripts/

company-snap-major.minor/company-snap/import/

company-snap-major.minor/company-snap/directives

company-snap-major.minor/company-snap/doc/

company-snap-major.minor/components/

company-snap-major.minor/components/company/

company-snap-major.minor/components/company/snap/

company-snap-major.minor/components/company/snap/snap.jar

company-snap-major.minor/components/company/snap/library.jar

For example if your company name is Acme, and your Snap name is "Inventory Access", andversion number is 1.2, the contents of the .zip file will look like this:

% unzip -l acme-inventory-1.2.zip

acme-inventory-1.2/

acme-inventory-1.2/install.bat

acme-inventory-1.2/install.sh

acme-inventory-1.2/common.py

acme-inventory-1.2/acme-inventory/

acme-inventory-1.2/acme-inventory/data/

acme-inventory-1.2/acme-inventory/scripts/

acme-inventory-1.2/acme-inventory/import/

acme-inventory-1.2/acme-inventory/directives

acme-inventory-1.2/acme-inventory/doc/

acme-inventory-1.2/components/

acme-inventory-1.2/components/acme/

acme-inventory-1.2/components/acme/inventory/

acme-inventory-1.2/components/acme/inventory/inventory.jar

acme-inventory-1.2/components/acme/inventory/library.jar

Java Snaps Must Declare a Package

If you've developed a Java Snap, the Component must declare a package. The package mustfollow the general Java guidelines for package naming and include the company name and theSnap name.

For example, the following package names are acceptable:

l com.acme.inventory

l com.acme.inventory.control

The following package names are not acceptable:

l com.acme

l acme

l my.component

l no package declaration (default package)

- 39 -


For example, if your company name is Acme, you could declare the package as follows in theJava code:

package com.acme.inventory;

Java Snaps Must Include the Manifest

Java Snaps are packaged as .jar files. The .jar file must include a manifest declaring the Snap-Logic Components contained in the .jar file.

The steps for building the .jar file including the manifest can be found here. Note that Snap-Logic release 2.2.0 didn't require manifests, however since the 2.2.1 release it is now nec-essary to include the manifest with your Java Snaps.

If you already have an existing .jar file with a Snap, it is easy to add a manifest to it by fol-lowing the instructions below. In this example we have a Snap in a .jar file acme.jar. This .jarfile contains two Components using the Java class names org.acme.Reader and org.acme.Wr-iter.

To add a manifest to the JAR file:

1. Create a text file that you may call header.txt.

2. Edit header.txt to declare your SnapLogic Components, such as:

SnapLogic-Components: org.acme.Reader, org.acme.Writer

In this example org.acme.Reader and org.acme.Writer are the Java class names of theSnapLogic Components (classes implementing the SnapLogic ComponentAPI interface).

3. Add a couple of empty lines to the end of header.txt, it is needed for the "jar" tool toprocess it correctly.

4. Run the following command to add the manifest to the JAR file:

jar umf header.txt acme.jar

In this example we are adding the header from file header.txt to the manifest of .jar fileacme.jar. Both files are located in the current directory.

Snap Must Include an Installer

The Snap must have an installer. You may use the SnapLogic Snap Installer to package theSnap.

Third-party Libraries Must be Included

If the Snap includes any third-party libraries, they must be included with the Snap andinstalled by the installer into the same directory where the Snap is installed or a subdirectoryof that directory.

Snap Must Install into the Directory Containing Company and Snap Name

After the user runs the installer, the target directory must include the company and Snapname.

- 40 -

Snap Development

For example, if company name is Acme and Snap names are "Point Of Sale" and "InventoryAccess" the extensions directory may look like this after the snaps have been installed:

l <snaplogic-directory>/extensions/components/acme/pointofsale

l <snaplogic-directory>/extensions/components/acme/inventory

The following directories are not acceptable:

l <snaplogic-directory>/extensions/components/pointofsale

l <snaplogic-directory>/extensions/components/acmepos

l <snaplogic-directory>/extensions/components/acme

Snap Must be Tested on Current SnapLogic Releases

The Snap must be tested with the current or upcoming SnapLogic release. Please contact Snap-Logic to obtain it.

Snap InstallerThis page describes how to package a custom Component for installation as an Snap into aSnapLogic installation. For details about how to construct a custom Component and how to con-figure SnapLogic to use it, see the Creating a Component page.

Snap Packaging

Snaps are delivered as a .zip archive file. The archive contains:

l the Component to be installed

l the directives file

l various documentation and post installation scripts (optional)

The Snap Installer

Directory Layout

By default, Snaps written in Python or Java install into the SnapLogic .../e-xtensions/components/ directory.

In preparation for creating the Snap archive, your temporary directory structure should looklike this:

Python Example:

./companyname-productname-1.00/components/companyname/__init__.py

(required)

./companyname-productname-1.00/components/companyname/productname/__

init__.py (required)

./companyname-productname-

1.00/components/companyname/productname/MyComponent.py (required)

./companyname-productname-1.00/companyname-

productname/scripts/GenerateMySnapResources.py (optional)


- 41 -


productname/patches/MyPatchScript.py (optional)


productname/patches/PatchFile1.py (optional)


productname/import/MyResources.dmp (optional)

./companyname-productname-1.00/companyname-productname/directives

(required)

./companyname-productname-1.00/companyname-productname/doc/README.txt

(optional)

Java Example:

./companyname-productname-

1.00/components/companyname/productname/MyComponent.jar (required)


productname/scripts/GenerateMySnapResources.py (optional)


productname/patches/MyPatchScript.py (optional)


productname/patches/PatchFile1.py (optional)


productname/import/MyResources.dmp (optional)

./companyname-productname-1.00/companyname-productname/directives

(required)

./companyname-productname-1.00/companyname-productname/doc/README.txt

(optional)

The leading Component of the path is the name of your Snap followed by a version number.There are two install scripts, one for execution on Windows and the other for Linux.

A skeleton Snap may be downloaded here .

The Directives File

The Snap installer is controlled by a directives file.

l The directives file allows the author to specify:

l what minimum SnapLogic version is Required for this Snap.

l any external package dependencies that should be automatically fetched from thePython cheese shop prior to installing the new Component (Python Snaps only).

l any patches that must be installed prior to installing the new Component.

l any scripts that should be run after installing the new Component.

l any resource exports to import to SnapLogic after installing the new Component.

An example directives file might be:

MIN_VERSION 2.3.0

NAME MY_FANCY_SNAP_NAME

DEPENDS zsi<=2.0

PATCH patches/MyPatchScript.py

- 42 -

Snap Development

RUN scripts/GenerateMySnapResources.py

IMPORT import/MyResources.dmp

The above example specifies that the Snap requires at least SnapLogic version 2.3.0. TheNAME directive is used to specify the name of the Snap which gets displayed in the Designerwhen the Snap is installed. The Snap has a dependency on the Python package ZSI version 2.0or lower and it will attempt to fetch it from the Python repository. It will then run the patchscript MyPatchScript.py which could, for example, apply a patch to the ZSI package, or per-form some package configuration if needed. It then specifies that the Gen-erateMySnapResources.py script should be run. This script could be a SnAPI program whichgenerates Resources using the newly installed Snap Components. Lastly the directives fileinstructs the Snap Installer to import resources contained in MyResources.dmp. These couldbe sample resources that utilize the new Snap Components.

Each directive may be specified more than once except for the MIN_VERSION, NAME and EDI-TIONS directives.

PATCH and RUN Scripts

The Snap Installer can invoke custom scripts that can be used to perform pre and post installactions, generating Resources using the newly installed Components. These scripts must bewritten in the Python language regardless of whether the custom Component is written inPython or Java.

For example, using the RUN directive to invoke a custom GenerateMySnapResources.pyscript, you could prompt the user to enter information, such as connection information, andthen, using the SnAPI interface, create Resources with the relevant properties configured withthat information.

Order of Execution

The Snap installer will execute the verbs in the following order.

1. MIN_VERSION

2. DEPENDS

3. PATCH

At this point the server will be restarted.

4. RUN

5. IMPORT

Environment Variables

A limited number of environment variables are exported by the Snap Installer and are thusavailable to Python scripts run during the installation (e.g., during processing of the PATCHand RUN sections of the directives file).

The current list of variables exported includes:

- 43 -


l PYTHONEXE : path to the Python interpreter executable for a particular SnapLogic instal-lation.

l PLATFORM : platform on which the installer is running. Currently either 'Windows' or'Linux'.

l INSTDIR : base installation directory for SnapLogic.

l SITEPACKAGES : path to the Python site-packages directory for a particular SnapLogicinstallation.

l PYTHONPATH : Python PYTHONPATH for a particular SnapLogic installation.

l DATAPORT : the TCP port used for the SnapLogic Data Server for a particular SnapLogicinstallation.

l SNAPLOGIC_USER : the user with appropriate permissions to create/modify resourcesvia the SnapLogic server

l SNAPLOGIC_PASSWORD : the password for SNAPLOGIC_USER

- 44 -

C

Components

creating 15

Java 16

Python 26

Java

API version 16

capabilities 17

datatypes 21

debugging 25

debugging with JPDA 26

documentation link 16

execution 19

installing on Linux 22

installing on Windows 23

labels 16

packaging 26

resource template 18

suggest 19

troubleshooting Component load-ing 25

upgrading 21

validation 18

Python

API version 26

capabilities 27

datatypes 31

debugging 33

documentation link 26

execution 28

installing 32

labels 26

packaging 34

resource template 27

suggest 28

troubleshooting Component load-ing 33

upgrading 30

validation 28

types 15

S

Snap

development

using command line 14

Snap Installer

diectory layout 41

directives file 42

environment variables 43

execution order 43

scripts 43

Snap Maven archetype

prerequisites 35

Snap Maven repository 35

SnapLogic artifacts 35

SnapArchetype 34, 36

SnapLogic Maven artifacts 34

SnapPlugin 35-36

- 45 -

Index


Snaps

debugging with JPDA 36

development

prerequisites 7

using command line 14

using Eclipse 7

packaging 41

submitting

file requirements 38

submission files 38

- 46 -

Snap Developer Guide - SnapLogic

Documents

Transcript of Snap Developer Guide - SnapLogic