Configuring the Script Snap to Use an HTTP Proxy

HTTP-compatible Snap Packs can use an HTTP proxy configured in the Node Proxies configuration tab of a Snaplex in SnapLogic Manager. However, the Script Snap allows you to write scripts that call external processes (for example, curl), and these scripts are not aware of proxy configurations in the SnapLogic application.

You can use curl to configure a proxy directly via the --proxy argument. To enforce proxy usage across all usages of the Script Snap, set the http_proxy and/or https_proxy environment variables.

Environment variables declared in the /etc/sysconfig/jcc file are visible to the Snaplex application (OS-level environment variables are not visible). If the /etc/sysconfig directory and /etc/sysconfig/jcc file do not exist in your Snaplex, run the following command with your own username/password (if authentication is required), proxy-ip-address, and port (you could also add https_proxy) to create them:

sudo mkdir -p /etc/sysconfig; sudo sh -c"echo 'export http_proxy=username:password@proxy-ip-address:port' >> /etc/sysconfig/jcc"

After the file and its directory are created, run one of the following commands to restart the Snaplex application:

/opt/snaplogic/bin/jcc.sh restart
 
c:\opt\snaplogic\bin\jcc.bat restart

The http_proxy/https_proxy environment variable is now active in the SnapLogic product. You can now run your script to call the external process.

For example, the following Script Snap uses the subprocess library to execute curl and adds the response body to the output document:

# Import the interface required by the Script snap.
from com.snaplogic.scripting.language import ScriptHook
import subprocess

class TransformScript(ScriptHook):
    def __init__(self, input, output, error, log):
        self.input = input
        self.output = output
        self.error = error
        self.log = log

    # The "execute()" method is called once when the pipeline is started
    # and allowed to process its inputs or just send data to its outputs.
    def execute(self):
        self.log.info("Executing Transform script")
        while self.input.hasNext():
            try:
                # Read the next input document, store it in a new dictionary, and write this as an output document.
                inDoc = self.input.next()
                proc = subprocess.Popen(['curl','https://www.snaplogic.com'], stdout=subprocess.PIPE)
                (out, err) = proc.communicate()
                outDoc = {
                    'original' : out
                }
                self.output.write(inDoc, outDoc)
            except Exception as e:
                errDoc = {
                    'error' : str(e)
                }
                self.log.error("Error in python script")
                self.error.write(errDoc)

        self.log.info("Script executed")

    # The "cleanup()" method is called after the snap has exited the execute() method
    def cleanup(self):
        self.log.info("Cleaning up")

# The Script Snap will look for a ScriptHook object in the "hook" variable. The snap will then call the hook's "execute" method.
hook = TransformScript(input, output, error, log)

On execution, the proxy access log shows the request being routed through the proxy.

Snaplex Outbound Connections for Child Pipelines

The Snaplex process creates outbound WebSocket connections to the SnapLogic cloud to be able to process incoming requests from the cloud. If running several nested Pipelines, or Pipelines using ForEach, there could be timeout errors if no connection is available to process an inbound request. By default, 20 connections are created. This can be increased if a higher number of cloud-triggered Pipelines need to be run concurrently on the Snaplex node. The maximum allowed value is 60. Setting this too high causes increased network overhead. If this has to be set to a larger value, it is an indication that more nodes need to be added to the Snaplex. Pipeline execution requests are automatically load-balanced across the available nodes.

To increase this limit, change the default of 20 to a higher value. In the etc/global.properties by adding it to the Update Snaplex dialog, Node Properties tab, Global properties table:

SnapLogic Documentation

Snaplex Configuration for Snaps

Configuring the Script Snap to Use an HTTP Proxy

Snaplex Outbound Connections for Child Pipelines