Ultra Task Pipelines

On this Page

Overview

Ultra Pipelines provide the speed and scalability to run the most important integrations that require high availability, high throughput, and persistent execution. This feature ensures that data arrives at its final destination in an instant, regardless of the volume of data being processed, the variety of endpoints involved, or the complexity of the integration.

Ultra Pipelines are executed as Ultra Tasks, which enable the Pipelines to consume documents continuously from external sources (Pipeline executions that are always-on), including sources that are not compatible with Triggered Tasks or those that require low-latency processing.

An Ultra Task can manage and load-balance multiple Ultra Pipeline instances in multiple Snaplex nodes to process multiple documents simultaneously. The Ultra Task monitor ensures that the required number of instances are running and restarts the Task when an instance fails or a node restart is required.

The Ultra Task pulls requests from the FeedMaster's queue and assigns them to its associated Ultra Pipeline instances. Alternatively, an Ultra Task can be based on a Pipeline that uses a listening Snap, such as JMS Consumer, which runs indefinitely.

Key Features

  • Implement real-time web service integrations that require sub-second response times.
  • Support a combination of document or binary input and output views.
  • Error handling mechanism for preventing failures, disabling of the Ultra Pipeline, or service disruptions.
  • Capability of organizing large number of operations with child Ultra Pipeline, which use the Pipeline Execute Snap to call from Ultra Pipelines.
  • Supports High Availability configurations.


Ultra Pipeline Workflow


Prerequisites

  • Your Org must be subscribed to this feature before you can access it in Manager.
  • Your Pipelines must work in Ultra-mode.


Support and Limitations

  • Ultra Pipeline design limits the usage of certain Snaps: see Snap Support for Ultra Pipelines for details.
  • When you access the URL for a Triggered or Ultra Task that is in an Org different from the one you last selected, SnapLogic Manager displays the Task details, but the Org does not automatically change to the one that owns the Task. Use the selector at the top right in Manager to switch to a different Org.
  • Since the connection to the client has a 15-minute timeout, long running Pipelines invoked as Triggered or Ultra Tasks through a Cloudplex URL might not complete before the timeout occurs. For Groundplex users, this timeout does not apply to the On-premise URL.

  • The configuration for Auto-scaling of Ultra Tasks applies only to Response-and-request Ultra Pipeline Tasks. Ultra Auto-scaling setting is not available for Polling Ultra Pipeline Tasks.


Known Issues

  • In some cases, Ultra Pipeline Tasks duplicate messages that are being processed from the FeedMaster node, resulting in failed Retry errors. If you see this issue in your Org, see Enabling the Prevention of Duplicates Enhancement.
  • After changes are made to an Ultra pipeline, the instances will restart. Ultra pipelines have a rolling restart so multiple instances may run at the same time after making changes to the pipeline.