Execution flow of Pipeloop Snap

The execution workflow of the PipeLoop Snap enables the parent pipeline to control and manage iterative processes using the child pipeline, for scenarios where repeated processing is needed until a specific condition is satisfied. Here are the key steps in the execution of the PipeLoop Snap:

1. The parent pipeline passes the input document to the child pipeline.

2. The child pipeline executes.

3. The PipeLoop Snap collects the output document from the child pipeline.

4. The PipeLoop Snap evaluates the stop condition based on the child pipeline’s output document.

5. If the stop condition is false, the Snap checks if the iteration limit is reached.

6. If the iteration limit is not reached, the Snap passes the output document to input (back to Step 1). The iteration stops if the stop condition is evaluated to true or the iteration limit is reached.”

Example

The following example pipeline demonstrates the flow of PipeLoop Snap.

• Parent pipeline: The parent pipeline generates the initial input using the JSON Generator Snap. The PipeLoop Snap iteratively calls the child pipeline until the condition specified in the Stop condition is met or the iteration limit is reached.

• Child pipeline: The Mapper Snap in the child pipeline processes the mapping and transformation of data as needed during each iteration.

Pipeline Flow

Open

Parent Pipeline

1. Configure the JSON Generator Snap with the input parameters to pass to the PipeLoop Snap. This Snap generates the initial input data for the PipeLoop Snap and the JSON data produced here will be passed as input to the child pipeline.
   
   

   Open

   

2. Configure the PipeLoop Snap with the following settings:

   • Select the child pipeline to call within the PipeLoop Snap.

   • If your pipeline requires specific parameters for the run, set up the Pipeline parameters.

   • Specify the Stop condition. This condition determines when the iteration should terminate. The stop condition is evaluated after each iteration from the first (or only) document coming out from the child pipeline. If the stop condition is not met after the specified number of iterations based on the iteration limit, the loop will terminate automatically. Configure the Iteration limit:

   • Configure the error handling settings in the Views tab as needed.
     
     

Child Pipeline

This pipeline increments the $num variable by 1 and adds the passed-in _foo parameter to the output document.

1. Configure the Mapper Snap with the following settings:

   • Define the mappings between the input fields and the desired output fields.

   • Ensure that the output document produced by the Mapper Snap contains the necessary information for the stop condition in the PipeLoop Snap to evaluate.

Final Output: Once the stop condition is met or the iteration limit is reached, the pipeline's final output is collected and passed to the next step or processed further based on the overall workflow.

Download the pipelines:

• Parent pipeline

• Child pipeline

Next steps (if applicable):

1. If additional Snaps are required for processing, transformation, or routing in the child pipeline, configure those accordingly.

2. Examples might include:

   • Filter Snap to filter out unnecessary data.

   • Data Transformation Snaps for applying complex transformations.

   • HTTP Client or Database Snaps if external calls or data lookups are needed.

3. Return Output to Parent Pipeline:

   • Once the child pipeline completes its steps, it should return its output document to the parent pipeline.

   • This output will be used by the PipeLoop Snap in the parent pipeline to evaluate the stop condition or proceed with further iterations.

4. Execute the PipeLoop Snap. The parent pipeline will pass the generated JSON input to the PipeLoop Snap, which then initiates the iterative process by calling the child pipeline.

5. Evaluate the Output:

   • After each iteration, the PipeLoop Snap evaluates the output document from the child pipeline.

   • The stop condition will be assessed to determine if further iterations are necessary or if the loop should end.

Related links:

• Machine Learning Snaps

• AgentCreator