In this Page

Create a URL

To create a URL that can be used to execute a Pipeline:

1. Select the Triggered (External URL) option when creating or updating a task.
   The option Do not start a new execution if one is already active lets you prevent a Pipeline from running again if a previous execution of that Pipeline is already running. If you attempt to call the triggered task when it is already running, you will receive a 409 error.

2. After saving the task, select Details from the context menu for that task to view the Details page.

3. Click Execute for the URL to run the Pipeline (you may be prompted to log in).

   Tasks, Snaplexes, and URLs

   A Snaplex consists of multiple nodes. If the triggered task is set to run in a Groundplex (also known as an On-premises Snaplex), options for Task URLs include one that is reachable from the Cloud (Cloud URL), and others that are limited to the on-premise network (On-premises URL, and an Override URL).

   Note that the Cloud URL has a 15-minute connection limit whether the connection is idle or not, so it should not be used for Tasks that stream data in or out beyond that time period.

   For information on API Throttles of triggered tasks, see Settings.

Using Bearer Tokens with Triggered Tasks

When you are creating a triggered task, the Bearer token is prepopulated with a token. You can either trigger this tokenized Task through a Cloudplex or Groundplex, typically for use with a service account user. 

Clearing this field disables token authentication; authentication must then be done with a user name and password. With Ultra tasks, an empty Bearer token means no authentication.

Passing Pipeline Arguments

To pass arguments to the Pipeline, you can append a query string to the URL with the key/value pairs of the parameters defined in the Pipeline properties. For example, if you created a Task named pipeline-with-args, which executed a Pipeline that had the parameters Age and Name, the end of the URL might look like the following example:

pipeline-with-args?Age=32&Name=Bob

Request Pipeline Arguments

Information from the HTTP request to the trigger URL is automatically passed as an argument when the Pipeline is executed. The following arguments are available with names similar to what you might find in a CGI script:

• PATH_INFO—The path elements after the Task part of the URL.
• REMOTE_USER—The name of the user that invoked this request.
• REMOTE_ADDR—The IP address of the host that invoked this request.
• REQUEST_METHOD—The method used to invoke this request.

The following HTTP headers from the request might also be available:

• CONTENT_TYPE
• HTTP_ACCEPT
• HTTP_ACCEPT_ENCODING
• HTTP_DATE
• HTTP_USER_AGENT
• HTTP_REFERER

Custom Headers

To pass a custom HTTP header, specify a header and its value through the parameter fields in Pipeline Settings. The request matches any header with Pipeline arguments and passes those to the Task, while the Authorization header is never passed to the Pipeline. 

Guidelines

• The header must be capitalized in its entirety. Headers are case-sensitive.
• Hyphens must be changed to underscores.
• The HTTP custom headers override both the Task and Pipeline parameters, but the query string parameter has highest precedence.

For example, if you want to pass a tenant ID (X-TENANT-ID) in a header, add the parameter X_TENANT_ID and provide a default or leave it blank. When you configure the expression, refer to the Pipeline argument following standard convention:  _X_TENANT_ID.  In the HTTP request, you add the header X-TENANT-ID: 123abc, which results in the value 123abc being substituted for the Pipeline argument X_TENANT_ID.

Semantic URL

Pipeline parameters can be sent to Triggered Tasks in the semantic URL form. These values are rendered into a variable name (like PATH_INFO), which must be defined as a parameter value in the Pipeline itself. 

In the following example, the parameter key/value pairs are displayed in the Edit Pipeline dialog, where you can configure the Pipeline Parameters.

• field1/G
• field2/F
• PATH_INFO/H

You can now append the trigger-URL with the new values:

triggered-url/newvariable1/newvariable2?field1=G&field2=F

Where the Pipeline parameters are:

• field1=G
• field2=F
• PATH_INFO=H 

Viewing the Output of a Pipeline

When you execute a Pipeline using a triggered task URL, its output can be included as the response to the HTTP request.  You can create a Pipeline to be used in this manner by leaving one output view unconnected.  For example, a Pipeline that consists solely of a File Reader Snap would return the contents of the file when the Task URL was requested.  

This feature works with both document and binary views.  With a document view, the response consists of a JSON-encoded array of each document read from the view. If the output document contains a Status field that is an integer, it is treated as the HTTP response status code. With a binary stream, the binary data's header is sent back to the client as the HTTP response header. The body of the binary data is directly streamed back to the client. A formatter, like the XML or CSV Formatter Snaps, can be used to transform the output of a document view into a binary view if the default JSON-encoding does not work.

POSTing Input to a Pipeline

If you would like to send a document to a Pipeline, you can leave one input view unlinked and then do an HTTP POST to the task URL.  The content of the POST request is then turned into a document and fed into the Pipeline.  If the input view accepts documents, then the content of the POST request should be a JSON-encoded object or an array-of-objects.  If it is a single object, that value is fed into the Pipeline.  If it is an array of objects, each object will be separately fed into the Pipeline as documents.  If the input view accepts binary streams, a single stream containing the raw contents of the POST request is fed into the view.

Authentication

The external URL created for a task requires you to be authenticated with the SnapLogic servers.  When using a web browser to access the URL, you might not have to provide credentials since you might already be logged into the Designer.  Using other methods to request the Task URL requires you to provide credentials using HTTP Basic Authentication.

Command-line Examples

This section covers some examples of using the CURL command-line tool to access Task URLs.  The following are only examples of the syntax and are not functioning URLs.

To execute a pipeline that takes no input and has an unlinked output view that writes documents:

$ curl -u 'user@example.com:mypassword' 'https://elastic.snaplogic.com/api/1/rest/slsched/feed/example/jobs/test-reader'
{ "msg" : "Hello, World!" }

To execute a Pipeline that takes a parameter and has an unlinked output view that writes a document:

$ curl -u 'user@example.com:mypassword' 'https://elastic.snaplogic.com/api/1/rest/slsched/feed/example/jobs/test-hello?Name=John'
{ "msg" : "Hello, John!" }

To execute a Pipeline using a Bearer token for authentication through a header:

$ curl -H 'Authorization: Bearer Xrcuwn2zVLqR7KXXXXXXXXX' 'https://elastic.snaplogic.com/api/1/rest/slsched/feed/example/jobs/test-hello?Name=John'
{ "msg" : "Hello, John!" }

To execute a Pipeline using a Bearer token for authentication through the bearer_token query parameter:

$ curl 'https://elastic.snaplogic.com/api/1/rest/slsched/feed/example/jobs/test-hello?Name=John&bearer_token=Xrcuwn2zVLqR7KXXXXXXXXX'
{ "msg" : "Hello, John!" }

To execute a Pipeline that accepts a JSON document and transforms it:

$ curl -u 'user@example.com:mypassword' --data-binary '{ "name" : "Bob" }' --header "Content-Type:application/json" https://elastic.snaplogic.com/api/1/rest/slsched/feed/example/jobs/test-transformer
{ "msg" : "Hello, Bob!" }

Debugging Mode

To make it easier to debug triggered executions of Pipelines, you can check the Debug option in the Create Task dialog to record the next five invocations of the task. When enabled, the headers and content from the HTTP request and response are saved in files on SLFS for each invocation. For example, when using a triggered task as a callback URL for another cloud service, enabling debugging lets you view the response that is being sent to the other cloud service. You can view the trace files in the Task Details page. The Open Trace Directory link opens the directory containing all of the trace files for this task.  You can also search for the files for a particular invocation by clicking the View Trace Files link in the details for that invocation in the execution history.

The Task Details page will display how many debug invocations are left next to the run count. After the five debug invocations have been done, the task will automatically revert to normal behavior. If you need to continue debugging, you can go back to the task configuration dialog and re-enable debug-mode.

Running an On-premise Pipeline from a URL

The On-premises URL refers to the Groundplex machine from which you send the request. Although the URL options are not tied to Snaplex type, these URLs depend on the user's ability to access those networks. For example, the Cloud URL is available to use from any computer with Internet access, whereas the On-premises and Override URLs require network access to the Groundplex or to a load balancer.

By default the Groundplex machine only accepts HTTP requests from the Loopback interface (127.0.0.1). To allow access from other machines, you must change the HTTP interface of the Groundplex configuration to listen on Any IP address (0.0.0.0). This may also require additional network changes, such as opening up a firewall, so consult your network administrator for guidance. 

When a triggered task is invoked through an On-premise or Override URL, the Task is cached for subsequent executions. The cached Tasks are then used on subsequent triggers of the URL, reducing task execution latency. Triggered tasks expire from the cache after an hour.

Additional Setup for Multiple Groundplex Nodes

A triggered task can run on any of the Groundplex nodes. Because of internal load-balancing between Snaplex nodes for triggered tasks, the node in the multi-node Snaplex designated in the On-premise URLs might not be the node that executes the Pipeline associated with that Task. Therefore, you need to configure the Groundplex nodes to accept requests from the other nodes so that data can be streamed back to the client. 

You can also determine if Snaplex nodes are not communicating by clicking Dashboard > Health, selecting the Snaplex, and viewing the Alerts for each node. If there are communication issues, an alert such as Neighbor connectivity alert might appear.

Override URLs

Two additional configurable URLs (Local override URL and Local override Secure URL) appear on the Task Details page. The Override URLs differ from the On-premises URLs in that you can use them to trigger a task to run on any Snaplex node, whereas the On-premises URL always runs on the Snaplex node that was specified when the Task was created.

If a triggered task is assigned to Snaplex A, then it always runs on that Snaplex if triggered with the On-premise URL. Therefore, even if triggered on a node on Snaplex B, the Task still runs on Snaplex A. However, if the Override URL is used, then the Task runs on Snaplex A if triggered on Snaplex A and Snaplex B if triggered on Snaplex B.

For example, if a Task is configured to run on Snaplex A, then it can be forced to run on Snaplex B by substituting the hostname and port of a JCC node or FeedMaster in Snaplex B, in place of the one configured in the Override URL. 

The Override URL was introduced for users interested in setting up their Tasks to run in High Availability configurations. The Override URL triggers a Pipeline to run on any Snaplex, regardless of what is defined in the Task. Unlike the On-premises URL, which always runs on the Snaplex defined in the Task, you are responsible for defining a HA architecture, and it is commonly done using a load balancer or the network's DNS.

On the same screen, you can review task status to determine if you need to run a triggered task on another Snaplex. Furthermore, clicking on the task detail displays a window with Pipeline statistics, parameters, and information such as path, node, and the user who ran the Pipeline.

Synchronous and Asynchronous Execution of Triggered Tasks

When a Task is triggered in SnapLogic using the URL, by default the request is synchronous. The Task responds with an HTTP response status code (for example, 200) when the it completes. However, if the pipeline has a single, unterminated Snap providing output, then the output from that Snap is provided as the response to the request. This behavior lets the developer make the pipeline asynchronous, in that the response is written back to the user.

For example, take the following Pipeline:

When you trigger the task, for example, from a browser, you receive an HTTP response status code on completion. For example:

{ http_status_code: 200 }

The request waits until the Pipeline has completed before displaying this message, which might take some time.

If you wanted to make the Pipeline asynchronous, you can add a Snap with an unterminated output, as shown in the following example:

Now when you trigger the Pipeline task, the output from the response generator Snap is provided almost as soon as the Pipeline executions starts. In this case, a JSON Generator responds with the message shown:

{ msg: "Started Ok", http_status_code: 200 }

The net effect of which is that the Pipeline execution becomes pseudo-asynchronous.

Example—Passing the execution status code through Binary Header Properties of the JSON Formatter Snap

In this Pipeline, the execution status is routed to the error view, which is redirected to reflect on the browser using the Binary Header Properties of the  JSON Formatter Snap. This allows us to view the   status on the browser same as on the output view of the Triggered Pipeline. 

The status code (405) as retrieved from the Pipeline is mapped to the response:

The JSON Formatter Snap passes the actual status code to the browser via the Binary Header Properties:

The status code (405) as reflected on the browser (Postman):

See Also

• Some Questions on Pipeline Execution Mechanism