Enterprise job schedulers that execute long-running Triggered Task Pipelines can report timeout or lost client connection errors. To help solve this issue, we provide the snaplogic_exec.sh script. It can execute on any server that has network access to call the Triggered Task URL.
Scheduler errors occur when network configuration limits the idle time for REST calls and a Triggered Task does not finish within that limit. For example, many network configurations limit the idle time for REST calls to one hour. With long-running Pipelines, such as for overnight batch processing, the synchronous call invoking the Triggered Task might not return within an hour. This can cause the scheduler to report errors even if the Pipeline is still running and eventually completes successfully.
Instead of invoking the Triggered Task directly, you can set up a scheduler to execute the Pipeline from the script. The script starts Pipeline execution and polls continuously for completion status. This keeps the connection alive and prevents the scheduler from reporting an error.
A user or service account with read and execute access for the project that contains the Triggered Task.
The URL for the Triggered Task, including any Pipeline parameters and their values.
Read and execute permissions on the node(s) that will run the Triggered Task.
For use on Windows servers, a Cygwin shell with curl.
Install the Script
Install the script on any server where the Scheduler has access to it and configure it with your SnapLogic credentials:
Extract the compressed file and save the snaplogic_exec.sh script to the appropriate location. For example, use /usr/local/bin on Linux machines and %systemroot%\System32\Repl\Imports\Scripts on Windows.
From the directory containing the script, invoke snaplogic_exec.sh --config in a terminal window. For example, on Windows:
Enter the SnapLogic user name and password.
Next, configure the Pipeline to work with the script.
Configure the Pipeline
To configure the Pipeline to work with the script, add a Mapper Snap that returns the Runtime ID (RUUID) value when the Triggered Task runs. The script will monitor execution using the RUUID.
In Snaplogic Designer, open the Pipeline that you want the script to execute.
Add a Mapper Snap to the canvas.
In the Settings tab, enter Map snaplogic_exec.sh return in the Label field.
In the Mapping table, add the Expression,pipe.ruuid with the Target pathruuid:
Select the Views tab.
Click - (minus) to remove the input view.
Leave the default output view, output0, as is, with Type document:
Save and close the Mapper Snap.
The Pipeline should look similar to the following (before validation):
Next, set up the scheduler to run the script.
Run the script from the scheduler
After installing the script and configuring the Pipeline, invoke the script from the scheduler. The invocation must include your Org name and the Triggered Task URL. For example: