Iterative processing using the Pipeloop Snap

This example pipeline demonstrates how to use the PipeLoop Snap to iteratively execute a child pipeline that increments a numerical value until it reaches the specified threshold.

Broadly, the pipelines perform the following tasks:

• Parent pipeline: The parent pipeline generates the initial input using the JSON Generator Snap. The PipeLoop Snap iteratively calls the child pipeline until the condition specified in the Stop condition is met or the iteration limit is reached.

• Child pipeline: The Mapper Snap in the child pipeline processes the mapping and transformation of data as needed during each iteration.

Open

Parent Pipeline

1. Configure the JSON Generator Snap with the input parameters to pass to the PipeLoop Snap. This Snap generates the initial input data for the PipeLoop Snap and the JSON data produced here will be passed as input to the child pipeline.
   
   

   Open

   

2. Configure the PipeLoop Snap with the following settings:

   • Select the child pipeline to call within the PipeLoop Snap.

   • If your pipeline requires specific parameters for the run, set up the Pipeline parameters.

   • Specify the Stop condition. This condition determines when the iteration should terminate. The stop condition is evaluated after each iteration from the first (or only) document coming out from the child pipeline. If the stop condition is not met after the specified number of iterations based on the iteration limit, the loop will terminate automatically. Configure the Iteration limit:

   • Configure the error handling settings in the Views tab as needed.
     
     

Child Pipeline

This pipeline increments the $num variable by 1 and adds the passed-in _foo parameter to the output document.

1. Configure the Mapper Snap with the following settings:

   • Define the mappings between the input fields and the desired output fields.

   • Ensure that the output document produced by the Mapper Snap contains the necessary information for the stop condition in the PipeLoop Snap to evaluate.

Final Output: Once the stop condition is met or the iteration limit is reached, the pipeline's final output is collected and passed to the next step or processed further based on the overall workflow.

Download the pipelines:

• Parent pipeline

• Child pipeline

Next steps (if applicable):

1. If additional Snaps are required for processing, transformation, or routing in the child pipeline, configure those accordingly.

2. Examples might include:

   • Filter Snap to filter out unnecessary data.

   • Data Transformation Snaps for applying complex transformations.

   • HTTP Client or Database Snaps if external calls or data lookups are needed.

3. Return Output to Parent Pipeline:

   • Once the child pipeline completes its steps, it should return its output document to the parent pipeline.

   • This output will be used by the PipeLoop Snap in the parent pipeline to evaluate the stop condition or proceed with further iterations.

4. Execute the PipeLoop Snap. The parent pipeline will pass the generated JSON input to the PipeLoop Snap, which then initiates the iterative process by calling the child pipeline.

5. Evaluate the Output:

   • After each iteration, the PipeLoop Snap evaluates the output document from the child pipeline.

   • The stop condition will be assessed to determine if further iterations are necessary or if the loop should end.

Related links:

• https://docs.snaplogic.com/snaps/snaps-machine-learning/snaps-machine-learning-about.html

• https://docs.snaplogic.com/genai/genai-builder-about.html