...

In this

...

article

...

Overview

You can use the Script Snap to execute Javascript, Python, or Ruby

...

scripts using the JVM ScriptEngine mechanism.

...

The document data can convert to and from the JSON data interchange language. By convention, the root of every document is conceptually a JSON object -- a collection of name/value pairs, where each name is a string and each value is an object, an array, a string, a number, a boolean, or a null. Every modern programming language has a corresponding type for this concept:

When you are writing a script for the Script Snap, each input document is an object that implements the Java Map interface, and can be accessed as an instance of the scripting language’s native object class; for example, as a Python dictionary.

...

Accounts are not required to work with this Snap.

...

Snap Type

The Script Snap is a write-type Snap.

Support for Ultra Pipelines

Works in Ultra Pipelines if you pass the original document to the write() method for the output view. For example, in the JavaScript template, the body of the loop is per the following: 

// Read the next document, wrap it in a map and write out the wrapper
var doc = this.input.next();
var wrapper = {
    "original" : doc
}
this.output.write(doc, wrapper);

Limitations and Known Issues

For JavaScript, objects written to the output view should be composed of serializable Java types.

...

Some downstream Snaps

...

, such as the

...

Copy Snap, require this. To write out a

...

map to the output view in a

...

JavaScript, use a Java Map implementation, such as HashMap

...

General Instructions for All Scripting Languages

The script author should declare a global variable named 'hook' (note that this variable name is case sensitive). The Script engine makes the following four global variables available to the script as defined in the 1437904 section:

• Variable input is of type ScriptHook.Input
• Variable output is of type ScriptHook.Output
• Variable error is of type ScriptHook.Error
• Variable log is of type org.slf4j.Logger

...

Modes

• Ultra Pipelines: Works in Ultra Pipelines if you pass the original document to the write() method for the output view. For example, in the JavaScript template, the body of the loop is per the following: 

  Paste code macro
  language json
  // Read the next document, wrap it in a map and write out the wrapper var doc = this.input.next(); var wrapper = { "original" : doc } this.output.write(doc, wrapper);

  Note the two arguments to the output.write() method. The first argument is the input document -- doc. The second argument is the data for the output document -- wrapper. Both the arguments are needed so that the lineage of the output document can be tied to the input document. This is especially important for an Ultra pipeline responding to web requests. The initial request becomes an input document to the first Snap in the SnapLogic Pipeline, eventually resulting in an output document from the last Snap in the Pipeline. The Pipeline must maintain the lineage of each document so that each response can be correlated to the request that generated it.

...

importClass(java.util.LinkedHashMap);
...
var inDoc = this.input.next();
var outDoc = new LinkedHashMap();
outDoc.put("original", inDoc);
this.output.write(inDoc, outDoc);

...

Accounts are not used with this Snap.

...

or LinkedHashMap, per the following:

importClass(java.util.LinkedHashMap);
...
var inDoc = this.input.next();
var outDoc = new LinkedHashMap();
outDoc.put("original", inDoc);
this.output.write(inDoc, outDoc);

Breaking Changes

The following breaking changes apply to the pipelines using the Script Snap (or the deprecated Execute Script Snap) with the Python engine.

To implement Python in the Script Snap, we use Jython. We have upgraded the Jython engine from version 2.7-b3 (a beta version from 2014) to version 2.7.2(March 2020). The following are the resultant issues and workarounds:

• An open bug in 2.7 introduced a backward-incompatible change in the SnapLogic platform wherein the Jython engine automatically converts BigInteger values to primitive long values. This impacts all your scripts that perform numeric manipulation of integer values from documents (SnapLogic uses the BigInteger type to represent integers in documents). Your pipelines and Snaps with the Script Snap (or the deprecated Execute Script Snap) that use numeric manipulation scripts with integer or BigInteger data type may fail during execution. We recommend you prospectively replace integer or BigInteger values with long values.

  Example:
  sum = a.intValue() + b.intValue()
  Here, a and b are of BigInteger type that now fail as Jython 2.7.2 automatically and transparently calls longValue() on any BigInteger value it encounters. So a and b would need to use the long and not BigInteger type.

  The known fix is to rewrite the above calculation as sum = a + b by removing occurrences of .intValue() or .longValue() from your Python scripts. 

• Before the 4.22 release (August 2020), when using the Script Snap with the Scripting language option selected as Python, requesting a key that did not exist in a dictionary (for example, my_dict['missing_key']) would return None. Starting from the 4.22 release, the same request returns a KeyError exception. If you need to continue returning None, use the .get(key) method instead (for example, my_dict.get['missing_key']).

• - zlib.compress():
 The zlib library compresses the JSON files retrieved from the SnapLogic APIs and backs-up Pipelines and accounts to a database. The following Python code, when trying to compress displays an ascii … ordinal not in range(128) error.
  Original code: in_doc["json"] = zlib.compress(in_doc["json"])
  Fix: in_doc["json"] = zlib.compress(in_doc["json"].encode("utf-8"))

• {dictionary}.values().toArray()[i]:
Before the 4.22 release (August 2020), to subscript a {dictionary}.values() method, you had to append the toArray() method to values(); else, you would see the Failure: ‘java.util.LinkedHashMap$LinkedValues’ object is unsubscriptable error. After the 4.22 release, toArray() returns Failure: ‘list’ object has no attribute ‘toArray’. However, the requirement for toArray() is no longer necessary for the subscript.
  Original code: sLine = data.values().toArray()[0]
  Fix: sLine = data.values()[0]

Snap Views

...

Settings

...

Label:

...

...

• Javascript
• Python
• Ruby

Example: Javascript
Default value: Javascript

...

A script file that implements the ScriptHook interface. This field can be used if the script file is present in the SLDB. Click on the 'browse' icon to select the required script file from the SLDB. 

Example: transform.py

Default value: [None]

...

...

...

ScriptHook Interface

...

...

Snap Settings

ScriptHook Interface

package com.snaplogic.scripting.language;

import java.util.Iterator;

/**
 * ScriptHook is the interface that should be implemented as a callback mechanism for
 * ExecuteScript snap to call into the script.
*/

public interface ScriptHook {

    /**
     * Scripts should implement this method to provide application logic.
     */
    void execute();

    /**
     * Scripts should implement this 

...

method 

...

to 

...

cleanup 

...

any 

...

resources 

...

allocated 

...

in execute().
     */
    void cleanup();

    /**

...

...

    * Input is interface that is used by the script to read input from the 

...

snap's 

...

input 

...

view.
     */
   

...

 interface Input extends Iterator<Object> {
    

...

}

...

...

    /**
     * Output 

...

is 

...

interface that is used by the script to send output 

...

data to the snap 

...

Importing Third Party Libraries

While SnapLogic does not support importing third party libraries directly using the Script Snap, you can add their package/JAR files in a directory in your Groundplex nodes and then import them using this Snap.

For example, consider that you have added a JAR file, sample.jar, in the directory /opt/snaplogic/ext_jar/. 

We recommend you to configure an external library that works with both the old and new Jython engines by using a .jython file in the home directory of the user running the JCC node. The .jython file should specify a value for the python.path per the following:
python.path=/opt/snaplogic/ext_jar/sample.jar

...

The paths listed in python.path can be .jar files (Java libraries), directories containing Python libraries (compatible with Python 2.7), or .zip files packaging those Python libraries. For more on using the .jython file, see Jython Registry.

...

The python.path variable is Jython's version of CPython’s PYTHONPATH variable. To learn more about the PYTHONPATH, see the official Python documentation.

...

If you are using multiple Groundplex nodes then you must add the package/JAR files in each of those nodes.

...

output view.
     */
    interface Output {

        /**
         * Write the data to the snap output.
         *
         * @param data
         */
        void write(Object data);

        /**
         * Write the data that was generated for the given incoming data to the snap output.
         * This method carries the lineage data forward.
         *
         * @param incomingData
         * @param data
         */
        void write(Object incomingData, Object data);
    }

    /**
     * Error is interface that is used by the script to send error data to snap error view.
     */
    interface Error extends Output {
    }
}

Importing Third-Party Libraries

While SnapLogic does not support importing third-party libraries directly using the Script Snap, you can add their package/JAR files in a directory in your Groundplex nodes and then import them using this Snap. For example, consider that you have added the JAR file, mongo-java-driver-3.12.7.jar, in the directory /opt/snaplogic/ext_jar/. For your Python scripts to be able to use this library, create a file named .jython in the home directory of the user running the JCC process. The .jython file should specify a value for the python.path, as follows:

python.path=/opt/snaplogic/ext_jar/mongo-java-driver-3.12.7.jar

Here’s an example of Python script that imports and uses code from this library. Use the cleanup method to ensure that the mongoClient object is appropriately closed.

from com.snaplogic.scripting.language import ScriptHook
from com.mongodb.client import MongoClients

class TransformScript(ScriptHook):
    def __init__(self, input, output, error, log):
        self.input = input
        self.output = output
        self.error = error
        self.log = log

    def execute(self):
        try:
            self.mongoClient = MongoClients.create("mongodb://localhost:27017/?readPreference=primary&ssl=false")
            for d in self.mongoClient.listDatabases():
                self.output.write(d)
        except Exception as e:
            errDoc = {
                'error' : str(e.args)
            }
            self.error.write(errDoc)

    def cleanup(self):
        self.mongoClient.close()

hook = TransformScript(input, output, error, log)

Additional Information

The document data can be converted to and from the JSON data interchange language. By convention, the root of every document is conceptually a JSON object—a collection of name-value pairs, where each name is a string, and each value is an object, an array, a string, a number, a boolean, or a null. Every modern programming language has a corresponding type for this concept:

When writing a script for the Script Snap, each input document is an object that implements the Java Map interface and can be accessed as an instance of the scripting language’s native object class, such as a Python dictionary.

To write an output document, your script must create a new object. In Python or Ruby, you can create an instance of the required language’s native object type, a Python dictionary, or a Ruby hash. The values you add to these objects must be one of the JSON-compatible types, including objects, arrays, strings, numbers, and booleans. You can use the corresponding array or list type of the language for an array. Objects written to the output view should be of Java types. Some downstream Snaps require this, for example, the Join Snap. To write a Python map to the output view in a Python script, convert the map to a Java HashMap.

General Instructions for all Scripting Languages

The script author should declare a global variable named 'hook' (note that this variable name is case-sensitive). The Script engine makes the following four global variables available to the script as defined in the Script#ScriptHook Interface section:

• The variable input is of type ScriptHook.Input

• The variable output is of type ScriptHook.Output

• The variable error is of type ScriptHook.Error

• The variable log is of type org.slf4j.Logger

Type defined in the schema maps to the Java class per the following:

...

Example Scripts

JavaScript

script = {
   execute : function() {
       var fruits = ["apple", "banana", "cherry"]
       var map = {"fruits" : fruits}
       output.write(map)
   }
}
var hook = new com.snaplogic.scripting.language.ScriptHook(script)

Python

...

This example requires an input view be defined in order for it to work.

Paste code macro
language python
from com.snaplogic.scripting.language import ScriptHook from com.snaplogic.scripting.language.ScriptHook import * class TransformScript(ScriptHook): def __init__(self, input, output, error, log): self.input = input self.output = output self.error = error self.log = log def execute(self): self.log.info("Executing Transform script")

...

while self.input.hasNext(): data = self.input.next() data["firstLast"] = "%s-%s" %(data["first"],data["last"]) data["firstLast2"] = data["first"] + data["last"]

...

...

data["numberMath"] = data["counter"] + 22 data["numberMath2"] = data["counter"] + 23 data["dateMonthPlusOne"] = data["birthday"].plusMonths(1)

...

data["numberMathType"] = type(data["counter"]) data["dateType"] = type(data["birthday"])

...

...

try: data["mathTryCatch"] = data["counter2"] + 33 self.output.write(data) except Exception as e: data["errorMessage"] = e.message self.error.write(

...

data) self.log.info("Finished executing the Transform script") hook = TransformScript(input, output, error, log)

...

...

Script Snap Configuration Using Python, JS Script, and Ruby Script 

The following pipeline

...

demonstrates how the Script Snap

...

executes in all three supported languages.

...

This pipeline uses a simple JSON file with

...

First Name, Last Name, and Birthday

...

.

 Image Modified

The following is a snapshot of the input for the Script Snap:

Image Removed

The following scripts are executed:

...

Python Script

Script Snap uses the Jython engine to execute the scripts written in Python. 

from com.snaplogic.scripting.language import ScriptHook
from random import randint
from time import sleep
 
class TransformScript(ScriptHook):
   def __init__(self, input, output, error, log):
       self.input = input
       self.output = output
       self.error = error
       self.log = log
   def execute(self):
       self.log.info("Executing Transform script")
       i = 1
       while self.input.hasNext():
           data = self.input.next()
           sleep(randint(1,10))
           map = {"out": data}
           self.output.write(map)
       self.log.info("Finished executing the Transform script")
hook = TransformScript(input, output, error, log)

...

Java Script

The Script Snap uses

...

the Nashorn engine to execute the scripts written in JavaScript.

// Ensure compatibility with both JDK 7 and 8 JSR-223 Script Engines
try { load("nashorn:mozilla_compat.js"); } catch(e) { }
 
script = {
  execute : function() {
    while (input.hasNext()) {
      var in_data = input.next()
      var new_data = {}
      var keyArray = in_data.keySet().toArray()
      for (var index in keyArray) {
        var key = keyArray[index]
        new_data[key] = in_data.get(key)
      }
 
      new_data.firstLast = new_data.first + "-" + new_data.last
      new_data.firstLast2 = new_data.first + in_data.get("last")
      new_data.numberMath = (new_data.counter + 22) | 0
      new_data.numberMath2 = new_data.counter + 23
      new_data.dateMath = new_data.birthday.plusMonths(1).toString()
      
      new_data.mathType = typeof(new_data.counter)      
      new_data.dateType = typeof(new_data.birthday)              
 
      output.write(new_data)
    }
   }  
};
var hook = new com.snaplogic.scripting.language.ScriptHook(script)

Ruby Script

Script Snap uses the JRuby engine to execute the scripts written in Ruby.

class MyScript
    include com.snaplogic.scripting.language.ScriptHook
    attr_reader :log, :input, :output, :error
    def initialize(log, input, output, error)
        @log = log
        @input = input
        @output = output
        @error = error
        @array = [java.lang.Integer.valueOf(1), java.lang.Integer.valueOf(2)]
        
    end

    def execute()
  
        while input.hasNext()  do
          data = input.next()
        
            data["firstLast2"] = data["first"] + data["last"]
            
            data["numberMath"] = data["counter"] + 22
            data["numberMath2"] = data["counter"] + 23
            data["dateMonthPlusOne"] = data["birthday"].plusMonths(1)
   
            begin
                data["mathTryCatch"] = data["counter"] + 33
                output.write(data)
            rescue Exception=>e
                data["errorMessage"] = e.message
                error.write(data)
            end
        end

    end 
end
$hook = MyScript.new($log, $input, $output, $error)

A sample preview output of the successful execution for all the three Snaps is shown below:

...

Image Added

The exported

...

pipeline is available in the

...

Script#Downloads section below.

Script Snap Execution Using Pipeline

...

Parameters

The following example demonstrates the execution of the Script Snap

...

using the pipeline parameters

...

; the script file in the SLDB

...

is passed as a Pipeline parameter. The pipeline demonstrated above is modified to accept pipeline parameters

...

by configuring the Script file property. For the scripts to be passed as a pipeline parameter, the script file should be present in the project folder in SnapLogic. Confirm that the script files are present in the Files section inside the Manager

...

; if not, upload them by clicking on the '+' icon.

In this example, there are three files, one for each type of scripting language

...

:

...

...

Below is a snapshot of the

...

pipeline's properties

...

and configured pipeline parameters

...

.

 Image Modified

The individual Script Snaps are configured as shown below:

...

Image Added

...

Image Added

...

Image Added

As in the example above, this pipeline

...

produces the same output

...

on execution. The exported

...

pipeline and sample script files used are available as a zip file in the Downloads section below.

Downloads

...

Related Content

• Parameters and Fields

• SnapLogic Community Article: Configuring the Script Snap to use a configured HTTP proxy environment variable