Ultra Task Pipelines

On this Page

Overview

Using traditional enterprise service bus (ESB) tools for low-latency data processing jobs is becoming unsustainable for many businesses. Ultra Pipelines provide the speed and scalability to run the most important integrations that require high availability, high throughput, and persistent execution. This subscription feature ensures that data arrives at its final destination in an instant, regardless of the volume of data being processed, the variety of endpoints involved, or the complexity of the integration.

Ultra Pipelines are executed as Tasks, which enable Pipelines to consume documents continuously from external sources. In this mode, Pipelines can consume documents from sources not compatible with Triggered Tasks or those that require low-latency processing of their inputs. A Task based on an Ultra Pipeline can manage multiple instances (runs) of a Pipeline and can consume documents simultaneously to attain load-balancing/reliability. 

Key Features

  • Implement real-time web service integrations that require subsecond response times.
  • Support a combination of document or binary input and output views.
  • Error handling mechanism for preventing failures, disabling of the Ultra Pipeline, or service disruptions.
  • Capability of organizing large number of operations with child Ultra Pipeline, which use the Pipeline Execute Snap to call from Ultra Pipelines.
  • Supports High Availability configurations.



Ultra Pipeline Workflow


Prerequisites

  • Your Org must be subscribed to this feature before you can access it in Manager.
  • Your Pipelines must work in Ultra-mode.


Support and Limitations

  • Ultra Pipeline design limits the usage of certain Snaps: see Snap Support for Ultra Pipelines for details.
  • When you access the URL for a Triggered or Ultra Task that is in an Org different from the one you last selected, SnapLogic Manager displays the Task details, but the Org does not automatically change to the one that owns the Task. Use the selector at the top right in Manager to switch to a different Org.
  • Since the connection to the client has a 15-minute timeout, long running Pipelines invoked as Triggered or Ultra Tasks through a Cloudplex URL might not complete before the timeout occurs. For Groundplex users, this timeout does not apply to the On-premise URL.


Known Issue

In some cases, Ultra Pipeline Tasks duplicate messages that are being processed from the FeedMaster node, resulting in failed Retry errors. If you see this issue in your Org, see Enabling the Prevention of Duplicates Enhancement.