Versions Compared

Old Version 5

changes.mady.by.user John Brinckwirth

Saved on

compared with

New Version 6

changes.mady.by.user John Brinckwirth

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Overview

Ultra Pipeline Tasks require consideration and preparation of your network depending on the purpose of the Ultra Pipeline.  Typically, 


No FeedMaster

Headless Ultra Pipelines do not require a FeedMaster, so setting up this Ultraplex is simpler than other configurations.

  • Use this configuration if you already have a data source that providers a listener-type of interface, such as JMS or file poller.
  • The Ultra Pipeline functionality ensures that N number of listener Pipelines are always running. Since the source (or listener-type interface) is already a listener, you do not need to add a FeedMaster.
  • This type of Ultra Pipeline does not provide a REST API interface for the Pipeline, this is just providing Always-On Pipeline functionality.
  • One of the features of Ultra is the capability to have instances of Pipelines which are Always On, it does not use the FeedMaster infrastructure at all.
  • When you use a JMS listener in a standard Pipeline execution, an individual instance of the Pipeline just runs, waiting for input documents to arrive from the JMS queue. Each instance is individual.
  • When you define the same as an Ultra Task, you can ask for N instances to be instantiated across the Snaplex, which SnapLogic monitors and restarts automatically, ensuring that the full number of instances are continuously running.


FeedMaster Configured 

This configuration allows customers to build a low latency REST interface for talking to any kind of endpoint. It provides low latency and reliability as compared to a Triggered Task, which has dependency on the control plane). A FeedMaster can be installed in front of the execution node for any return-and-request Ultra Pipeline Tasks. Plain HTTP/S requests can be fed into a pipeline through a FeedMaster that is installed as part of a Snaplex. The HTTP request is turned into a document.  This document is sent to the Pipeline's unlinked input, which is turned into the HTTP response to the original request.


Standard Network Configuration - 2 FeedMasters and 1 Load Balancer

This configuration can be deployed for Groundplexes.  

The following diagram provides an overview of the standard network architecture required for using Ultra Pipeline Tasks, including the communication between the various components with a focus on the network ports.

The FeedMaster uses the following ports:

  • 8084: The FeedMaster's HTTPS port. 
  • 8089 - The FeedMaster's embedded ActiveMQ broker SSL port.

The machine hosting the FeedMaster must have those ports available on the local firewall. 



High Availability Configuration - 4 FeedMasters and 2 Load Balancers


The following diagram provides an overview of the High Availability architecture required for Ultra Pipeline Tasks. 


Configuration on DNS
To support "Ultra Plex Server
If you are deploying the Snaplex in an Active-Active HA Configuration" configuration, Route53 set Route53  to support Active- Active across the F5 LoadBalancers (F5-1E and F5-1W)
To support "Ultra Plex If you are deploying the Snaplex in an Active-Standby HA Configuration" configuration, set Route53 set to Active Standby s across the F5 LoadBalancers (F5-1E and F5-1W)


Deployment of Tasks
All tasks deployed on "Ultra East" should have the same unique pre/postfix
All tasks deployed on "Ultra West" should have the same unique pre/postfix
For example we have a Task "<task name>_TaskOrgNameEast" that is deployed to "Ultra Plex East"
Where "TaskOrgNameEast" is a unique string that is present as pre/postfix for all Tasks deployed on Ultra Plex East.
For example we have a Task "<task name>_TaskOrgNameWest" that is deployed to "Ultra Plex West"
Where "TaskOrgNameWest" is a unique string that is present as pre/postfix for all Tasks deployed on Ultra Plex West.


Configuration on Guidelines for F5 LoadBalancer
Use an F5 LoadBalancer is used as we need a load balancer(or api gateway) that can support URI rewrite. AWS ELB cannot be used as it does not support URI rewrite. AWS ALB cannot be used as or API gateway that supports URI rewrites. Do not use the AWS elastic load balancer because it does not support URI rewrite.
 "F5-1E" is configured to replace "_TaskOrgNameWest" in the URI to "_TaskOrgNameEast"
"F5-1W" is configured to replace "_TaskOrgNameEast" in the URI to "_TaskOrgNameWest"