| Table of Contents | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Overview
You can use this Snap to enable looping on pipeline executions based on a stop condition or iteration limit. The output document of each iteration is fed into the next iteration as the input document, and execution continues until the stop condition is met or the iteration limit is reached. Learn more.
...
Snap Type
The PipeLoop Snap is a Flow-type Snap.
Prerequisites
Child pipelines must have one input view and one output view.
Support for Ultra Tasks
Does not support Ultra Tasks.
Limitations
Reuse is currently not supported because of the requirement to close an input view to force flush all documents out.
Reads only the first output document for a specific input document (the remaining documents are ignored and dropped).
Supports expressions on the pipeline specification, but only evaluates the input document to PipeLoop, not for each iteration.
Known Issues
None.
Snap Views
...
Type
...
Format
...
Number of Views
...
Examples of Upstream and Downstream Snaps
...
Description
...
Input
...
Binary
Document
...
Min: 1
Max: 1
...
Mapper
Copy
...
The document or binary data to send to the child pipeline. For a single document input, only the first output document is captured in the output document and the remaining documents are ignored.
...
Output
...
Binary
Document
...
Min: 1
Max: 1
...
Mapper
...
If you select the Debug iterations output checkbox, an additional _iteration_documents is added to the output document alongside the output documents and passthrough documents.
_iteration_documents contains the following fields:
iteration: The index of the loop iteration.child_pipeline_output: The output document of the respective iteration.stop_condition_evaluation: The evaluated result of the stop condition.
...
Error
...
Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the pipeline by choosing one of the following options from the When errors occur list under the Views tab:
Stop Pipeline Execution: Stops the current pipeline execution if the Snap encounters an error.
Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.
Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.
Learn more about Error handling in pipelines.
Snap Settings
| Info |
|---|
Learn more about the icons in the Snap Settings dialog. |
...
Field Name
...
Field Type
...
Description
...
Label*
Default Value: PipeLoop
Example: SimpleLoop
...
String
...
Specify the name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snaps in your pipeline.
...
Pipeline*
Default Value: N/A
Example: demo-child-pipeline
...
String/Expression
...
Specify the path of the pipeline to execute.
...
Pipeline Parameters
...
Use this field set to define child pipeline parameters to use for executing the pipeline.
...
Parameter name
Default Value: N/A
Example: tools
...
String/Suggestion
...
Specify the parameter's name. You can also select the Pipeline Parameters defined for the pipeline selected in the Pipeline field.
...
Parameter value
Default Value: N/A
Example: $tools
...
String/Expression
...
If you configure the value as an expression based on the input, then each incoming document or binary data is evaluated against that expression when invoking the pipeline. The result of the expression is JSON-encoded if it is not a string. The child pipeline then needs to use the JSON.parse() expression to decode the parameter value.
...
Loop options
...
Define the iteration configuration.
Stop condition
Default Value: N/A
Example: $num >= 10
...
String/Expression
...
Specify the condition to be evaluated during execution that decides whether the execution should stop or continue. The iteration continues until the stop condition is met or the iteration limit is reached. The Snap errors out if the Stop condition cannot be evaluated at run time. The stop condition is evaluated on the output document from each iteration.
Iteration limit
Default Value: 10
Example: 15
...
Integer/Expression
...
Specify the maximum limit for the number of iterations to run to avoid infinite loops.
Debug iteration outputs
Default Value: Deselected
...
Checkbox
...
Select this checkbox to include the index, output document, and stop condition evaluation for each iteration as part of the output format.
Execution
...
Define the child pipeline execution configuration.
Execute on
Default Value: LOCAL_NODE
Example: LOCAL_SNAPLEX
...
Dropdown list
...
Select one of the following Snaplex options to specify the target Snaplex for the child pipeline:
LOCAL_NODE: Runs the child pipeline on the same node as the parent pipeline.
LOCAL_SNAPLEX: Runs the child pipeline on one of the available nodes in the same Snaplex as the parent pipeline.
Execution label
Default Value: N/A
Example: demo
...
String/Expression
...
Specify the label to display in the Dashboard for the current execution.
...
Snap Execution
Default Value: Validate & Execute
Example: Execute only
...
Dropdown list
...
Select one of the following three modes in which the Snap executes:
Validate & Execute: Performs limited execution of the Snap and generates a data preview during pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during pipeline runtime.
Execute only: Performs full execution of the Snap during pipeline execution without generating preview data.
Disabled: Disables the Snap and all Snaps that are downstream from it.
...
flow of Pipeloop Snap
The execution workflow of the PipeLoop Snap enables the parent pipeline to control and manage iterative processes using the child pipeline , for scenarios where repeated processing is needed until a specific condition is satisfied. Here are the key steps in the execution of the PipeLoop Snap:
The parent pipeline passes the input document to the child pipeline.
The child pipeline executes.
The PipeLoop Snap collects the output document from the child pipeline.
The PipeLoop Snap evaluates the stop condition based on the child pipeline’s output document.
If the stop condition is false, the Snap checks if the iteration limit is reached.
If the iteration limit is not reached, the Snap passes the output document to input (back to Step 1). The iteration stops if the stop condition is evaluated to true or the iteration limit is reached.”As shown in the example below, you can preview the child pipeline by hovering over the Pipeloop Snap while the parent pipeline is open in the Designer.
Parent pipeline: The parent pipeline generates the initial input using the JSON Generator Snap. The PipeLoop Snap iteratively calls the child pipeline until the condition specified is reached.
Child pipeline: The Mapper Snap in the child pipeline processes the mapping and transformation of data as needed during each iteration.
Example
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|
...