Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

On this Page

Table of Contents
maxLevel2
excludeOlder Versions|Additional Resources|Related Links|Related Information

Snap type:Write
Description:

This Snap executes a Javascript, Python, or Ruby script using the JVM ScriptEngine mechanism. After selecting your language of choice, click the Edit Script button to open the script editor. By default, the editor will be populated with a skeleton script that you can modify. The skeleton simply reads an input document, wraps it in a map and writes the wrapper to the output view. If a script file is present in the SLDB, it can be uploaded to the Snap by using the Script file property; this property also accepts Pipeline parameters and upstream parameters. 

The document data can convert to and from the JSON data interchange language. By convention, the root of every document is conceptually a JSON object -- a collection of name/value pairs, where each name is a string and each value is an object, an array, a string, a number, a boolean, or a null. Every modern programming language has a corresponding type for this concept:

JavaMap
Python

Dictionary 

RubyHash
JavaScriptObject 

When you are writing a script for the Script Snap, each input document is an object that implements the Java Map interface, and can be accessed as an instance of the scripting language’s native object class; for example, as a Python dictionary.

To write an output document, your script must create a new object. In Python or Ruby, you can create an instance of the required language’s native object type, a Python dictionary or a Ruby hash. The values you add to these objects must be one of the JSON-compatible types including objects, arrays, strings, numbers, booleans. For an array, you can use the corresponding array or list type of the language. Objects written to the output view should be of Java types. This is required by some downstream Snaps. For example, the Join Snap. To write a Python map to the output view in a Python script, convert the map to a Java HashMap

General Instructions for All Scripting Languages

The script author should declare a global variable named 'hook' (note that this variable name is case sensitive). The Script engine makes the following four global variables available to the script as defined in the 1437904 section:

  • Variable input is of type ScriptHook.Input
  • Variable output is of type ScriptHook.Output
  • Variable error is of type ScriptHook.Error
  • Variable log is of type org.slf4j.Logger


Type defined in the schema maps to the Java class per the following:

 Data TypeJava class
 NUMBER   java.math.BigDecimal
 INTEGER java.math.BigInteger 
 STRING java.lang.String
 DATETIME org.joda.time. DateTime  
 LOCALDATETIME org.joda.time. LocalDateTime 
 BOOLEAN java.lang.Boolean 
 DATE org.joda.time.LocalDate 
 TIME org.joda.time.LocalTime
 BINARY java.lang.Byte 
 TABLE java.util.List 
 ANY java.lang.Object 
 COMPOSITE java.util.Map 

Modes

  • Ultra Pipelines: Works in Ultra Pipelines if you pass the original document to the write() method for the output view. For example, in the JavaScript template, the body of the loop is per the following: 

    Paste code macro
    languagejson
    // Read the next document, wrap it in a map and write out the wrapper
    var doc = this.input.next();
    var wrapper = {
        "original" : doc
    }
    this.output.write(doc, wrapper);

    Note the two arguments to the output.write() method. The first argument is the input document -- doc. The second argument is the data for the output document -- wrapper. Both the arguments are needed so that the lineage of the output document can be tied to the input document. This is especially important for an Ultra pipeline responding to web requests. The initial request becomes an input document to the first Snap in the SnapLogic Pipeline, eventually resulting in an output document from the last Snap in the Pipeline. The Pipeline must maintain the lineage of each document so that each response can be correlated to the request that generated it.

Prerequisites:None.
Limitations and Known Issues

For JavaScript, objects written to the output view should be composed of serializable Java types. This is required by some downstream Snaps such as the Copy Snap. To write out a map to the output view in a JavaScript script, use a Java Map implementation, such as HashMap or LinkedHashMap per the following:

Paste code macro
languagepython
importClass(java.util.LinkedHashMap);
...
var inDoc = this.input.next();
var outDoc = new LinkedHashMap();
outDoc.put("original", inDoc);
this.output.write(inDoc, outDoc);

Multiexcerpt include macro
namebreaking_change_script_snap
pageAugust 2020, 4.22 Release Notes

Account:

Accounts are not used with this Snap.

Views:


InputThis Snap has at most one document input view.
Output

This Snap has at most one document output view. 

ErrorThis Snap has at most one document error view and produces zero or more documents in the view.


Settings

Label:

Required. The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

Scripting Language:Required. Language in which the script is provided. The options available include:
  • Javascript
  • Python
  • Ruby

Example: Javascript
Default value: Javascript

Script file:

A script file that implements the ScriptHook interface. This field can be used if the script file is present in the SLDB. Click on the 'browse' icon to select the required script file from the SLDB. 

Example: transform.py

Default value: [None]

Note

This field accepts Pipeline parameters as well as upstream parameters provided that the script file is present in the SLDB. 


Edit Script:

Required. This property enables you to edit a script within the Snap instead of through an external file. From this page, you can export the script to a file in a project, import a script, or generate a template for the selected Scripting Language.

Default value: A skeleton for the chosen scripting language.  You can click the Generate Template button to regenerate the skeleton.

Multiexcerpt include macro
nameSnap Execution
pageSOAP Execute

Multiexcerpt include macro
nameExecution_Detail_Write
pageSOAP Execute


Note

You cannot create external process (like the popen function) on Cloudplex via the Script Snap or custom Snap. While external process creation on Groundplex is possible, this can be disabled upon your request to support@snaplogic.com.

ScriptHook Interface

Note

This example requires that an input view be defined for it to work.


Paste code macro
languagejson
package com.snaplogic.scripting.language;

import java.util.Iterator;

/**
 * ScriptHook is the interface that should be implemented as a callback mechanism for
 * ExecuteScript snap to call into the script.
*/

public interface ScriptHook {

    /**
     * Scripts should implement this method to provide application logic.
     */
    void execute();

    /**
     * Scripts should implement this method to cleanup any resources allocated in execute().
     */
    void cleanup();

    /**
     * Input is interface that is used by the script to read input from the snap's input view.
     */
    interface Input extends Iterator<Object> {
    }

    /**
     * Output is interface that is used by the script to send output data to the snap output view.
     */
    interface Output {

        /**
         * Write the data to the snap output.
         *
         * @param data
         */
        void write(Object data);

        /**
         * Write the data that was generated for the given incoming data to the snap output.
         * This method carries the lineage data forward.
         *
         * @param incomingData
         * @param data
         */
        void write(Object incomingData, Object data);
    }

    /**
     * Error is interface that is used by the script to send error data to snap error view.
     */
    interface Error extends Output {
    }
}

Importing Third Party Libraries

While SnapLogic does not support importing third party libraries directly using the Script Snap, you can add their package/JAR files in a directory in your Groundplex nodes and then import them using this Snap.

For example, consider that you have added a JAR file, sample.jar, in the directory /opt/snaplogic/ext_jar/. 

We recommend you to configure an external library that works with both the old and new Jython engines by using a .jython file in the home directory of the user running the JCC node. The .jython file should specify a value for the python.path per the following:
python.path=/opt/snaplogic/ext_jar/sample.jar

Note
  • The paths listed in python.path can be .jar files (Java libraries), directories containing Python libraries (compatible with Python 2.7), or .zip files packaging those Python libraries. For more on using the .jython file, see Jython Registry.

  • The python.path variable is Jython's version of CPython’s PYTHONPATH variable. To learn more about the PYTHONPATH, see the official Python documentation.

  • If you are using multiple Groundplex nodes then you must add the package/JAR files in each of those nodes.

  • You can import third party libraries only on Groundplex nodes.

Example Scripts


JavaScript

Paste code macro
script = {
   execute : function() {
       var fruits = ["apple", "banana", "cherry"]
       var map = {"fruits" : fruits}
       output.write(map)
   }
}
var hook = new com.snaplogic.scripting.language.ScriptHook(script)

Python

Note

This example requires an input view be defined in order for it to work.


Paste code macro
languagepython
from com.snaplogic.scripting.language import ScriptHook
 
class TransformScript(ScriptHook):
   def __init__(self, input, output, error, log):
       self.input = input
       self.output = output
       self.error = error
       self.log = log
 
   def execute(self):
       self.log.info("Executing Transform script")
      
       while self.input.hasNext():
           data = self.input.next()
           data["firstLast"] = "%s-%s" %(data["first"],data["last"])
           data["firstLast2"] = data["first"] + data["last"]
          
           data["numberMath"] = data["counter"] + 22
           data["numberMath2"] = data["counter"] + 23
 
           data["dateMonthPlusOne"] = data["birthday"].plusMonths(1)
          
           data["numberMathType"] = type(data["counter"])
           data["dateType"] = type(data["birthday"])
          
           try:
               data["mathTryCatch"] = data["counter2"] + 33
               self.output.write(data)
           except Exception as e:
               data["errorMessage"] = e.message
               self.error.write(data)
          
       self.log.info("Finished executing the Transform script")
 
hook = TransformScript(input, output, error, log)

Example Use Case 


The following pipeline is a demonstration of the Script Snap using all three supported languages. In this pipeline a simple JSON file with information like First Name, Last Name, and Birthday are passed to the Script Snaps.

 

The following is a snapshot of the input for the Script Snap:


The following scripts are executed:

Python Script

Script Snap uses Jython engine to execute the scripts written in Python. 

Paste code macro
languagepython
from com.snaplogic.scripting.language import ScriptHook
from random import randint
from time import sleep
 
class TransformScript(ScriptHook):
   def __init__(self, input, output, error, log):
       self.input = input
       self.output = output
       self.error = error
       self.log = log
   def execute(self):
       self.log.info("Executing Transform script")
       i = 1
       while self.input.hasNext():
           data = self.input.next()
           sleep(randint(1,10))
           map = {"out": data}
           self.output.write(map)
       self.log.info("Finished executing the Transform script")
hook = TransformScript(input, output, error, log)

JavaScript

The Script Snap uses the Nashorn engine to execute the scripts written in JavaScript.

Paste code macro
languagejavascript
// Ensure compatibility with both JDK 7 and 8 JSR-223 Script Engines
try { load("nashorn:mozilla_compat.js"); } catch(e) { }
 
script = {
  execute : function() {
    while (input.hasNext()) {
      var in_data = input.next()
      var new_data = {}
      var keyArray = in_data.keySet().toArray()
      for (var index in keyArray) {
        var key = keyArray[index]
        new_data[key] = in_data.get(key)
      }
 
      new_data.firstLast = new_data.first + "-" + new_data.last
      new_data.firstLast2 = new_data.first + in_data.get("last")
      new_data.numberMath = (new_data.counter + 22) | 0
      new_data.numberMath2 = new_data.counter + 23
      new_data.dateMath = new_data.birthday.plusMonths(1).toString()
      
      new_data.mathType = typeof(new_data.counter)      
      new_data.dateType = typeof(new_data.birthday)              
 
      output.write(new_data)
    }
   }  
};
var hook = new com.snaplogic.scripting.language.ScriptHook(script)

Ruby Script

Script Snap uses JRuby engine to execute the scripts written in Ruby.

Paste code macro
languageruby
class MyScript
    include com.snaplogic.scripting.language.ScriptHook
    attr_reader :log, :input, :output, :error
    def initialize(log, input, output, error)
        @log = log
        @input = input
        @output = output
        @error = error
        @array = [java.lang.Integer.valueOf(1), java.lang.Integer.valueOf(2)]
        
    end

    def execute()
  
        while input.hasNext()  do
          data = input.next()
        
            data["firstLast2"] = data["first"] + data["last"]
            
            data["numberMath"] = data["counter"] + 22
            data["numberMath2"] = data["counter"] + 23
            data["dateMonthPlusOne"] = data["birthday"].plusMonths(1)
   
            begin
                data["mathTryCatch"] = data["counter"] + 33
                output.write(data)
            rescue Exception=>e
                data["errorMessage"] = e.message
                error.write(data)
            end
        end

    end 
end
$hook = MyScript.new($log, $input, $output, $error)


A sample preview output of the successful execution for all the three Snaps is shown below:

Expand
titleOutput from the Python script

 


Expand
titleOutput from the JS script


Expand
titleOutput from the Ruby script

  

The exported Pipeline is available in the 1437904 section below.

Script Execution Using Pipeline Parameter

The Script Snap also accepts pipeline parameters, the script file in the SLDB can be passed as a Pipeline parameter. The pipeline demonstrated above is modified to accept pipeline parameters in this example by configuring the Script file property. For the scripts to be passed as a pipeline parameter, the script file should be present in the project folder in SnapLogic. Confirm in the Files section inside the Manager that the script files are present, if they are not then upload them by clicking on the '+' icon.

In this example there are three files, one for each type of scripting language supported: 

Below is a snapshot of the Pipeline's properties with the pipeline parameters also configured.

 

The individual Script Snaps are configured as shown below:

 

 

 

As in the example above, this pipeline also produces the same output upon execution. The exported Pipeline and sample script files used are available as a zip file in the Downloads section below.

Downloads

Multiexcerpt include macro
namedownload_instructions
pageOpenAPI

Attachments
patterns*.slp,*.zip
 

See Also

Insert excerpt
Script Snap Pack
Script Snap Pack
nopaneltrue