Tasks are how Pipelines become operational. As a general example, consider the Pipeline designer as a developer, typically with the background of an IT specialist, who constructs Pipelines that fit the data processing needs of an organization. For example, after you build a Pipeline, you then need it to run on a schedule or after an event occurs. Hence, Tasks provide an easy way to accomplish the productionization of your Pipelines.
Tasks enable you to execute your Pipelines by either using a schedule or by triggering a URL.
Scheduled Tasks–Choose for the scenario where a job needs to be accomplished starting at a particular time, on an interval, or on a more complex schedule.
Triggered Tasks–Choose to enable triggering the Pipeline execution through a HTTP call. A Triggered Task can be used to build an endpoint of a Web API and allows passing data into and retrieving data from a Pipeline.
Ultra Pipeline Tasks–Choose for specialized, low-latency jobs that require continuous ability to process documents. The URL method is similar to Triggered Tasks, but the Pipeline design limits the usage of certain Snaps.
Ultra and Triggered Tasks Compared
The difference between Ultra and Triggered Tasks is that the Ultra Task is a constantly running Pipeline, listening for the new documents streaming in. By the time a document is sent to an Ultra Task, the underlying Pipeline is already prepared and can start processing the document instantly, whereas a Pipeline run through the URL that is created from the Triggered Task has to go through the Pipeline prepare stage first. Depending on a variety of criteria (Pipeline size, accounts and types of Snaps used in the Pipeline), the prepare stage can take time, which makes the Ultra Task usage beneficial when the expected response time is a matter of sub-seconds. Since Ultra Pipelines are always running, they can be used to process documents continually from external sources like message queues. Also, data passed into an Ultra pipeline is more reliably processed.
In terms of Pipeline design, the Ultra Task is more restricting when compared to Triggered Tasks because of the number of unsupported Snaps and restrictions around the input and output Snaps and Pipeline parameters. In addition, the Snaplex on which the Ultra Task runs must have a FeedMaster.
