In this Article
Overview
. As a general example, consider the Pipeline designer as a developer, typically with the background of an IT specialist, who buildPipelines that fit the data processing needs of an organization. After you design a Pipeline, you then need schedule or after an event occurs. Hence, Tasks provide an easy way to accomplish the productionization of your Pipelines.
Tasks enable you to execute your Pipelines by either using a schedule or by triggering a URL.
Scheduled Tasks: Choose this option if you need to accomplish a job at a certain time, or an interval, or a more complex schedule.
Triggered Tasks: Choose this option to enable triggering the Pipeline execution through an HTTP call. A Triggered Task can be used to build an endpoint of a web API. A Triggered Task also passing data into and retrieving data from a Pipeline.
Ultra Pipeline Tasks: Choose this option for specialized, low-latency jobs that need to process documents continuously. The URL method is similar to Triggered Tasks, but the Pipeline design certain Snaps.
Interoperability Matrix of Tasks and Pipelines
Pipeline Mode | Scheduled Task | Triggered Task | Ultra Task |
---|
Standard | Supported | Supported | Supported |
Resumable | Supported | Supported | Not Supported |
eXtreme | Not supported | Not supported | Not supported |
Ultra and Triggered Tasks Compared
The difference between Ultra and Triggered Tasks is that the Ultra Task is a constantly running Pipeline, listening for the new documents streaming in. By the time a document is sent to an Ultra Task, the underlying Pipeline is already prepared and can start processing the document instantly, whereas a Pipeline run through the URL that is created from the Triggered Task has to go through the Pipeline stage first. Depending on a variety of criteria (Pipeline size, accounts and types of Snaps used in the Pipeline), the prepare stage can take time, which makes the Ultra Task usage beneficial when the expected response time is a matter of sub-seconds. Since Ultra Pipelines are always running, they can be used to process documents continually from external sources like message queues. Also, data passed into an Ultra pipeline is more reliably processed.
In terms of Pipeline design, the Ultra Task is more restricting when compared to Triggered Tasks because of the number of unsupported Snaps and restrictions around the input and output Snaps and Pipeline parameters. In addition, the Snaplex on which the Ultra Task runs must have a FeedMaster.
Task Authentication
Triggered and Ultra Tasks have the following four authentication mechanisms:
: The easiest method to use, this type of authentication is shared across Tasks based on user ACLs. It supports password expiration.
Bearer token: Also easy to use, the bearer token is by default unique per Task—one token per task. The token can be manually synced across tasks.
access control: This method suits Ultra Tasks. The authentication can be implemented in the Pipeline using Snaps. This method allows for more fine grained access control, at the project level or based on other criteria. JWT also allows for multiple tokens per Task.
API Policy Manager: The various available authentication and authorization policies providemore like the Callout Authenticator and Generic OAuth policies.
You can use the service account and bearer token for simple API integration use cases, and forcomplex use cases, you can use JWT and API Policy Manager.
Task Notifications
You can configure event-based notifications for your Tasks. In the dialog windows of all three Task types, you can add or remove recipients and set the event type for the notification. For example, a Task owner might enter the as a recipient and select Stopped as the event type, which that the Task execution did not complete.
. For Slack recipients, enter the Slack Channel or direct username.