In this Article

Overview

You can invoke a Pipeline execution through Tasks in the SnapLogic Platform. Tasks are how Pipelines become operational. As a general example, consider the Pipeline designer as a developer, typically with the background of an IT specialist, who builds Pipelines that fit the data processing needs of an organization. After you design a Pipeline, you then need to run it on a schedule or after an event occurs. Hence, Tasks provide an easy way to accomplish the productionization of your Pipelines. 

Tasks enable you to execute your Pipelines by either using a schedule or by triggering a URL. 

• Scheduled Tasks: Choose this option if you need to accomplish a job at a certain time, an interval, or a more complex schedule. 

• Triggered Tasks: Choose this option to enable triggering the Pipeline execution through an HTTP call. A Triggered Task can be used to build an endpoint of a web API. A Triggered Task also allows passing data into and retrieving data from a Pipeline.

• Ultra Pipeline Tasks: Choose this option for either specialized, low-latency jobs that need to process documents continuously or for Pipelines based on the always-on design.  For the former, a FeedMaster node is required in the Snaplex to queue the incoming messages. The URL method is similar to Triggered Tasks, but the Pipeline design limits the usage of certain Snaps. For the latter, the Pipeline can continuously poll the target messaging service, making the Ultra Task preferable to a Scheduled Task.

Interoperability Matrix of Tasks and Pipelines

Ultra and Triggered Tasks Compared

The difference between Ultra and Triggered Tasks is that the Ultra Task is a constantly running Pipeline, listening for the new documents streaming in.  By the time a document is sent to an Ultra Task, the underlying Pipeline is already prepared and can start processing the document instantly, whereas a Pipeline run through the URL that is created from the Triggered Task has to go through the Pipeline prepare stage first. Depending on a variety of criteria (Pipeline size, accounts, and types of Snaps used in the Pipeline), the prepare stage can take time, which makes the Ultra Task usage beneficial when the expected response time is a matter of sub-seconds. Since Ultra Pipelines are always running, they can be used to process documents continually from external sources like message queues. Also, data passed into an Ultra pipeline is more reliably processed.

In terms of Pipeline design, the Ultra Task is more restricting when compared to Triggered Tasks because of the number of unsupported Snaps and restrictions around the input and output Snaps and Pipeline parameters. In addition, the Snaplex on which the Ultra Task runs must have a FeedMaster. 

Task Authentication

Triggered and Ultra Tasks have the following four authentication mechanisms: 

• Basic Auth using Service Accounts: The easiest method to use, this type of authentication is shared across Tasks based on user ACLs. It supports password expiration.

• Bearer token: Also easy to use, the bearer token is by default unique per Task—one token per task. The token can be manually synced across tasks.

• JWT-based access control: This method suits Ultra Tasks. The authentication can be implemented in the Pipeline using JWT Snaps. This method allows for more fine-grained access control, at the project level or based on other criteria. JWT also allows for multiple tokens per Task.

• API Policy Manager: The various available authentication and authorization policies provide more features like the Callout Authenticator and Generic OAuth policies.

You can use the service account and bearer token for simple API integration use cases, and for complex use cases, you can use JWT and API Policy Manager.

Task Permissions

SnapLogic Tasks use the permissions as other Project Assets. As an Org admin or creator of the Task, you can set the following permissions for other users at the Project level:

• Read Only: View the Task configuration and details.

• Read & Write: Can view and modify the Task, but cannot run the Task.

• Read & Execute: View and run Tasks but cannot change Task configuration or details. 

• Full Access: View, run, and modify Tasks.

• Owner: Modify permissions on a Task, plus all of the above. 

When a user or group has different permissions at different levels, the lowest permission level is enforced. For example, if you have Read & Execute permissions to the Project Space containing the Task, but only read permissions to the Project, then the user cannot run the Task.

Anyone with the bearer token may execute a Triggered or Ultra Task. The owner of a Task is the user that executes the Pipeline when the Task is invoked.

Task Notifications

You can configure event-based notifications for your Tasks. In the dialog windows of all three Task types, you can add or remove recipients and set the event type for the notification. For example, a Task owner might enter the Org admin as a recipient and select Stopped as the event type, which indicates that the Task execution is not completed. 

For example, you can receive notifications for the Failed to Start task status for tasks that did not start. 

Both Email and Slack are supported for Task Notifications. For Slack recipients, enter the Slack Channel or direct username.