Overview
Although Snaplex nodes communicate within a Snaplex, Snaplexes do not directly communicate with each other, regardless of whether the Snaplex is managed by SnapLogic (Cloudplexes) or self-managed (Groundplexes). Because Snaplexes only communicate through the control plane, understanding the dependencies between the control plane and a Groundplex is helpful for implementation planning, pipeline design considerations, and troubleshooting. Pipelines can process documents for a while even if the node cannot communicate with the control plane.
Recommendations
To build pipelines that reduce dependency on the control plane:
Avoid using SLFS files (files uploaded to Manager and referenced in pipelines)
Avoid directly stored files on customer self-managed cloud storage like S3.
Avoid using dynamic references for assets like accounts; instead, use static references. Statically referenced assets (such as accounts and child pipelines) are cached on the Snaplex
The following types of pipelines run without access to the control plane using the aforementioned recommendations:
Processing documents through an Ultra pipeline task, which is already running, using the Low-latency Feed (Request and Response) design.
Pipelines, which are already started, either a standard mode pipeline or an Ultra pipeline task using the Always-On (Continuous Data Flow) design.
Dependencies between Groundplex and Control Plane
The following table lists the dependencies between a Grounplex and the control plane for pipelines invoked as Ultra pipeline tasks, triggered tasks, scheduled tasks, and child pipelines.
Tyoe of Integration | Asset | Dependency (Yes/No) | Notes |
---|---|---|---|
Ultra Task with No Child Pipelines - Running | Accounts | No |
|
Expression Library | No |
| |
Pipelines | No |
| |
SLDB File Read | Yes | Do not use SLDB as file source. | |
SLDB File Write | Yes | Do not use SLDB as file destination. | |
Metadata Snap Pack | Yes | None | |
Ultra Task with No Child Pipelines - Initializing | Accounts | Yes |
|
Expression Library | Yes |
| |
Pipelines | Yes |
| |
SLDB File Read | Yes | Do not use SLDB as file source. | |
SLDB File Write | Yes | Do not use SLDB as file destination. | |
Metadata Snap Pack | Yes | None | |
Ultra Task with child pipelines - initialization state or already running with child pipelines not running yet | Accounts | Yes | None |
Expression Library | Yes | ||
Pipelines | Yes | ||
SLDB File Read | Yes | ||
SLDB File Write | Yes | ||
Metadata Snap Pack | Yes | ||
Triggered Task - Cloud URL | Accounts | Yes | None |
Expression Library | Yes | ||
Pipelines | Yes | ||
SLDB File Read | Yes | ||
SLDB File Write | Yes | ||
Metadata Snap Pack | Yes | ||
Triggered Task - Alternative or Snaplex URL | Accounts | Yes | None |
Expression Library | Yes | ||
Pipelines | Yes | ||
SLDB File Read | Yes | ||
SLDB File Write | Yes | ||
Metadata Snap Pack | Yes | ||
Scheduled Task (Already Running) | Accounts | No | Assumes the task is running for a very long time (such as days or weeks). Sample use case: Salesforce Subscriber Snap always running, or a File Poller snap that polls indefinitely. However, you would not be able to view Pipeline Execution statistics in the dashboard if the control plane goes down. |
Expression Library | No | ||
Pipelines | No | ||
SLDB File Read | Yes | ||
SLDB File Write | Yes | ||
Metadata Snap Pack | Yes | ||
Scheduled Task (Initializing) | Accounts | Yes | None |
Expression Library | Yes | ||
Pipelines | Yes | ||
SLDB File Read | Yes | ||
SLDB File Write | Yes | ||
Metadata Snap Pack | Yes |
Troubleshooting
Scenario | Work-around |
---|---|
Most Scheduled Tasks and Ultra when it’s in the prepare state happen on the Snaplex. However, scheduled tasks still have a dependency on the control plane. | Change to an Ultra task pipeline. |
Statically defined OAuth accounts depend on Control Plane - specifically the refresh token. | ou can use client credentials in-memory to get around this which would depend on the Snaplex, instead of the Control Plane. |