Best Practices and Guidelines for the Pipeline Execute Snap

In this article

1 Best Practices
2 Ultra Pipelines
3 Guidelines for child pipelines
4 Guidelines for schema propagation

Best Practices

Scenario	Available functionality	Benefit

Scenario	Available functionality	Benefit
To run a pipeline, which ensures that the child pipeline runtime occurs on the same Snaplex node as the parent pipeline.	LOCAL_NODE	Faster and more reliable execution because of the absence of network traffic required to start the child pipeline on a different node. Facilitates usage for the user to import the pipeline.
To run the child pipeline on one of the available nodes in the same Snaplex as the parent pipeline.	LOCAL_SNAPLEX	No need to set an Org-specific property. Facilitates usage for the user to import the pipeline.
To run the child pipeline on a different Snaplex other than the parent pipeline.	SNAPLEX_WITH_PATH	Use a relative path for the Snaplex name and keep the same name for the Snaplex across Orgs (Development, QA, Stage, or Production) for easier management and organization of the pipelines. If the Snaplex property is set to cloud, the pipeline runs on the Snaplex-named cloud in the shared directory of the current Org. Use a pipeline parameter or expression library 1 to specify the path to the Snaplex. Create an expression library file with the environment's details for complicated setups and import that into the pipeline. In this model, each Org has its own library version with the appropriate settings for the environment. The pipeline can then be moved around to each Org without an udpate.

Ultra Pipelines

We do not recommend usage of the pipe.plexPath setting for Ultra pipelines, because requests should process quickly and when you send an execution to another node it can be resource-intensive and costly.

Ultra mode compatibility

If you select Reuse executions to process documents for the Pipeline Execute Snap in an Ultra pipeline, then the Snaps in the child pipeline must also be Ultra-compatible.
If you need to use Snaps that are not Ultra-compatible in an Ultra pipeline, you can create a child pipeline with those Snaps and use a Pipeline Execute Snap with Reuse executions to process documents disabled to invoke the pipeline. Because the child Pipeline is executed for every input document, the Ultra Pipeline restrictions do not apply.
For example, if you want to run an SQL Select operation on a table that returns more than one document, you can put a Select Snap followed by a Group By N Snap with the group size set to zero in a child Pipeline. In this configuration, the child Pipeline performs the select operation during execution, and then the Group By Snap gathers all of the outputs into either a single document or as binary data. Use that single output document or binary data can then be used as the output of the Ultra Pipeline.

Guidelines for child pipelines

No unlinked binary input or output views. When you enter an expression in the pipeline field, the Snap needs to contact the SnapLogic® cloud servers to load the pipeline runtime information. Also, if Reuse executions to process documents are enabled, the result of the expression cannot change between documents.
When reusing pipeline executions, there must be one unlinked input view and zero or one unlinked output view.
If the Reuse executions to process documents option is not enabled, use only one unlinked input view and one unlinked output view.
If the child Pipeline has an unlinked input view, ensure the Pipeline Execute Snap has an input view because the input document or binary data is fed into the child pipeline.
If you rename the child pipeline, then you must manually update the reference to it in the Pipeline field. If you do not do this, the connection between the child and parent pipeline breaks.
The child pipeline is executed in preview mode when you save the pipeline with the Pipeline Execute Snap. Consequently, any Snaps marked not to execute in preview mode do not execute, and the child pipeline only processes 50 documents.
You cannot have a Pipeline call itself: recursion is not supported.

Guidelines for schema propagation

If the child pipeline has a Snap that supports schema suggest, such as the JSON Formatter Snap or MySQL Insert Snap, then the schema is back-propagated to the parent pipeline. This configuration is useful in mapping input values in the parent pipeline to the corresponding fields in the child pipeline with a Mapper Snap in the parent pipeline.
For the schema suggest to work, the parent pipeline must be validated first; only then is the schema from the child pipeline visible in the parent pipeline.
The Pipeline Execute Snap can propagate schema in both directions: upstream and downstream.
- For a child Pipeline to run on a Snaplex node, memory is prioritized over slots. That means, regardless of the node a parent pipeline runs, the child pipeline runs on a node in 10% memory utilization.
- Difference between how a local node and a Snaplex run the child pipeline:
  - SNAPLEX_WITH_PATH: Runs the child pipeline on a user-specified Snaplex. Select this option to get the Snaplex Path field.
  - LOCAL_NODE: Runs the child pipeline on the same node as the parent pipeline.
  - LOCAL_SNAPLEX: Runs the child pipeline on one of the available nodes in the same Snaplex as the parent pipeline.