Overview

The Pipeline Execute Snap executes a pipeline in a specific Snaplex with the specified parameters. You can use this Snap to execute multiple child pipelines from a single parent pipeline. Configure the Snap to execute the pipeline and data input.

To execute a SnapLogic pipeline that is exposed as a REST service, use the HTTP Client Snap.

Key Features

The Pipeline Execute Snap enables you to:

Structure complex pipelines into smaller segments through child pipelines
Initiate parallel data processing using the pooling option
Orchestrate data processing across nodes, in the Snaplex or across Snaplexes
Distribute global values through pipeline parameters across a set of child pipeline Snaps

Supported Modes for Pipelines

Modes	Description

Modes	Description
Standard mode (default)	A new child pipeline is launched for each input document in the standard mode. If you set the pool size to n (where n is any number with the default setting value of 1, then n concurrent child pipelines can run simultaneously. Each child pipeline processes one document from the parent and then completes.
Reuse mode	In reuse mode, child pipelines are started, and each child pipeline instance can process multiple input documents from the parent. If you set the pool size to n (default 1), then n number of child pipelines are started and they will process the input document in a streaming manner. The child pipeline needs to have an unlinked input view for use in reuse mode. Reuse mode is more performant, but it has the restriction that the child pipeline has to be a streaming pipeline. In reuse mode if a child pipeline fails, it causes the parent pipeline to fail.
Resumable Child pipeline	The resumable pipeline does not support the Pipeline Execute Snap. A regular mode pipeline can use the Pipeline Execute Snap to call a Resumable mode pipeline. If the child pipeline is a Resumable pipeline, then the batch size cannot be greater than one.
Ultra Task pipelines	Works in Ultra Task pipelines with the following exceptions: The reuse option works in the Ultra pipeline if the child pipelines run on the same node. Running on a different node in the same Snaplex fails with a lineage error. To ensure that the child pipelines run on the same node as the parent pipeline, set Execute on to LOCAL_NODE. For more information, refer to Snap Support for Ultra Pipelines. Without reuse enabled, the one-in-one-out requirement for Ultra means batching is not supported. A runtime check will fail the parent pipeline if the batch size is set to greater than 1. This would be similar to the current behavior in DB insert and other Snaps in Ultra mode.
ELT Mode	A pipeline can only use the Pipeline Execute Snap to call another pipeline with ELT Snaps in Standard Mode.
Pooling Enabled	The pool size and batch size can both be set to greater than one, in which case the input documents are spread across the child pipeline in a round-robin method to ensure that if the child pipeline does any slow processing external calls, then the processing is spread across the children in parallel. The limitation of this option is that the document order is not maintained.