Overview

Ultra Pipeline Tasks require consideration and preparation of your network depending on the purpose of the Ultra Pipeline.  The following deployment architectures cover the most common scenarios to setting up your environment to run Ultra Tasks.

No FeedMaster

Headless Ultra Pipelines do not require a FeedMaster, so setting up this Snaplex is simpler than other configurations.

• Use this configuration if you already have a data source that providers a listener-type of interface, such as JMS or file poller.
• The Ultra Pipeline functionality ensures that N number of listener Pipelines are always running. Since the source (or listener-type interface) is already a listener, you do not need to add a FeedMaster.
• This type of Ultra Pipeline does not provide a REST API interface for the Pipeline, this is just providing Always-On Pipeline functionality.
• One of the features of Ultra is the capability to have instances of Pipelines which are Always On, it does not use the FeedMaster infrastructure at all.
• When you use a JMS listener in a standard Pipeline execution, an individual instance of the Pipeline just runs, waiting for input documents to arrive from the JMS queue. Each instance is individual.
• When you define the same as an Ultra Task, you can ask for N instances to be instantiated across the Snaplex, which SnapLogic monitors and restarts automatically, ensuring that the full number of instances are continuously running.

FeedMaster Configured 

This configuration allows customers to build a low latency REST interface for talking to any kind of endpoint. It provides low latency and reliability as compared to a Triggered Task, which has dependency on the control plane). A FeedMaster can be installed in front of the execution node for any return-and-request Ultra Pipeline Tasks. Plain HTTP/S requests can be fed into a pipeline through a FeedMaster that is installed as part of a Snaplex. The HTTP request is turned into a document.  This document is sent to the Pipeline's unlinked input, which is turned into the HTTP response to the original request.

Standard Network Configuration - 2 FeedMasters and 1 Load Balancer

This configuration can be deployed for Groundplexes.  

The following diagram provides an overview of the standard network architecture required for using Ultra Pipeline Tasks, including the communication between the various components with a focus on the network ports.

The FeedMaster uses the following ports:

• 8084: The FeedMaster's HTTPS port. 
• 8089 - The FeedMaster's embedded ActiveMQ broker SSL port.

The machine hosting the FeedMaster must have those ports available on the local firewall. 

High Availability Configuration - 4 FeedMasters and 2 Load Balancers

The following diagram provides an overview of the High Availability architecture required for Ultra Pipeline Tasks. 

Configuration on DNS Server

• If you are deploying the Snaplex in an Active-Active configuration, set Route53  to support Active- Active across the F5 LoadBalancers (F5-1E and F5-1W)
• If you are deploying the Snaplex in an Active-Standby configuration, set Route53 s across the F5 LoadBalancers (F5-1E and F5-1W)

Deployment of Tasks

• All tasks deployed on Ultra East should have the same unique pre/postfix
• All tasks deployed on Ultra West should have the same unique pre/postfix
• For example we have a Task "<task name>_TaskOrgNameEast" that is deployed to "Ultra Plex East" -Where "TaskOrgNameEast" is a unique string that is present as pre/postfix for all Tasks deployed on Ultra Plex East.
• For example we have a Task "<task name>_TaskOrgNameWest" that is deployed to "Ultra Plex West" - Where "TaskOrgNameWest" is a unique string that is present as pre/postfix for all Tasks deployed on Ultra Plex West.

Configuration Guidelines for F5 Load Balancer

• Use an F5 LoadBalancer or API gateway that supports URI rewrites.
• Do not use the AWS elastic load balancer because it does not support URI rewrite. 
• "F5-1E" is configured to replace "_TaskOrgNameWest" in the URI to "_TaskOrgNameEast""F5-1W" is configured to replace "_TaskOrgNameEast" in the URI to "_TaskOrgNameWest"