Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Jira PLAT-8108

On this Page

Table of Contents
maxLevel2
excludeAdditional Resources

Overview

Using traditional enterprise service bus (ESB) tools for low-latency data processing jobs is becoming unsustainable for many businesses. Ultra Pipelines provide the speed and scalability to run the most important integrations that require high availability, high throughput, and persistent execution. This subscription feature ensures that data arrives at its final destination in an instant, regardless of the volume of data being processed, the variety of endpoints involved, or the complexity of the integration.

Ultra Pipelines are executed as Ultra Tasks, which enable the Pipelines to consume documents continuously from external sources . In this mode, Pipelines can consume documents from sources (always-on Pipeline executions), including sources that are not compatible with Triggered Tasks or those that require low-latency processing of their inputs. A Task based on an Ultra Pipeline can manage multiple instances (runs) of a Pipeline and can consume documents simultaneously to attain load-balancing/reliability. .

An Ultra Task can manage and load-balance multiple Ultra Pipeline instances in multiple Snaplex nodes to process multiple documents simultaneously. The Ultra Task monitor ensures that the required number of instances are running. As instances gracefully complete, new instances are created and are running within two minutes.

The Ultra Task pulls requests from the FeedMaster's queue and assigns them to its associated Ultra Pipeline instances. Alternatively, an Ultra Task can be based on a Pipeline that uses a listening Snap, such as JMS Consumer, which runs indefinitely.

Key Features

  • Implement real-time web service integrations that require subsecond response times.
  • Support a combination of document or binary input and output views.
  • Error handling mechanism for preventing failures, disabling of the Ultra Pipeline, or service disruptions.
  • Capability of organizing large number of operations with child Ultra Pipeline, which use the Pipeline Execute Snap to call from Ultra Pipelines.
  • Supports High Availability configurations.


Panel
bgColor#ebf7e1
borderStylesolid

In this Section

Child pages (Children Display)
depth2



Ultra Pipeline Workflow


Prerequisites

  • Your Org must be subscribed to this feature before you can access it in Manager.
  • Your Pipelines must work in Ultra-mode.


Support and Limitations

  • Ultra Pipeline design limits the usage of certain Snaps: see Snap Support for Ultra Pipelines for details.
  • When you access the URL for a Triggered or Ultra Task that is in an Org different from the one you last selected, SnapLogic Manager displays the Task details, but the Org does not automatically change to the one that owns the Task. Use the selector at the top right in Manager to switch to a different Org.
  • Since the connection to the client has a 15-minute timeout, long running Pipelines invoked as Triggered or Ultra Tasks through a Cloudplex URL might not complete before the timeout occurs. For Groundplex users, this timeout does not apply to the On-premise URL.


Known Issues

  • In some cases, Ultra Pipeline Tasks duplicate messages that are being processed from the FeedMaster node, resulting in failed Retry errors. If you see this issue in your Org, see Enabling the Prevention of Duplicates Enhancement.
  • After changes are made to an Ultra pipeline, the instances will restart. Ultra pipelines have a rolling restart so multiple instances may run at the same time after making changes to the pipeline.