In this article
Table of Contents | ||||
---|---|---|---|---|
|
...
Supported Modes for Pipelines
Modes | Description |
---|---|
Standard mode (default) |
|
Reuse mode |
|
Resumable Child pipeline |
|
Ultra Task pipelines |
|
ELT Mode |
|
Pooling Enabled |
|
Info |
---|
Replaces Deprecated Snaps: This Snap replaces the ForEach and Task Execute Snaps and the Nested pipeline mechanism. |
...
If there are insufficient Snaplex nodes to execute the pipeline, the Snap waits until the resources become available. In this scenario, a message appears in the execution statistics dialog.
Only the last 100 completed child pipeline runs are saved for inspection in the Dashboard because this Snap generates many pipeline runtimes.
The Pipeline Execute Snap cannot exceed a depth of 64 child pipelines before they begin to fail.
The child pipelines do not display data preview details. However, you can view the data preview for any child pipeline after the Pipeline Execute Snap completes execution in the parent pipeline.Ultra Pipelines do not support batching.
Unlike the Group By N Snap, when you configure the Batch field, the documents are processed one by one by the Pipeline Execute Snap and then transferred to the child pipeline when the parent pipeline receives it. The child pipeline closes when the batch or input stream ends.
Error handling for child pipelines inside a parent Ultra Pipeline are not supported. Instead, configure error handling in the Ultra Pipeline with the Pipeline Execute Snap.
Known Issues
None.
Snap Views
Type | Format | Number of Views | Examples of Upstream and Downstream Snaps | Description |
---|---|---|---|---|
Input | Binary or Document |
|
| The document or binary data to send to the child pipeline. Retry is not supported if the input view is a Binary data type |
Output | Binary or Document |
|
|
|
Error | Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:
Learn more about Error handling in Pipelines. |
Snap Settings
Info |
---|
Learn more about the icons in the Snap Settings dialog. |
Field Name | Field Type | Field Dependency | Description | |||
---|---|---|---|---|---|---|
Label* Default Value: Pipeline Execute | String | None. | Specify the name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline. | |||
Pipeline* Default Value: N/A | String/Expression | None | Specify the child pipeline's absolute or relative (expression-based) path to run. If you specify only the pipeline name, Snap searches for the pipeline in the following folders in this order:
You can specify the absolute path to the target project using the Org/project_space/project notation. You can also dynamically choose the pipeline to run by entering an expression in this field when Expressions are enabled. For example, to run all of the pipelines in a project, you can connect the SnapLogic List Snap to this Snap to retrieve the list of pipelines in the project and run each one. | |||
Execute On Default Value: SNAPLEX_WITH_PATH | Dropdown list | N/A | Select one of the following Snaplex options to specify the target Snaplex for the child pipeline:
For more information, refer to the Best Practices. | |||
Snaplex Path Default Value:N/A | String/Expression | Appears when you select SNAPLEX_WITH_PATH for Execute On. | Enter the name of the Snaplex on which you want the child pipeline to run. Click | |||
Execution Label Default Value: N/A | String/Expression | N/A | Specify the label to display in the Pipeline view of the Dashboard. You can use this field to differentiate each pipeline execution. | |||
Pipeline Parameters | Use this fieldset to define the Pipeline Parameters for the pipeline selected in the Pipeline field. When you select Reuse executions to process documents, you cannot change parameter values from one Pipeline invocation to the next. | |||||
Parameter Name Default Value: N/A | String | Debug mode checkbox is not selected. | Enter the name of the parameter. Select the defined Pipeline Parameters in the Pipeline field. | |||
Parameter Value Default Value: N/A | String | None | Enter the value for the Pipeline Parameter, which can be an expression based on incoming documents or a constant. If you configure the value as an expression based on the input, then each incoming document or binary data evaluates against that expression when to you invoke the pipeline. The result of the expression is JSON-encoded if it is not a string. The child Pipeline then needs to use the When Reuse executions to process documents is enabled, the parameter values cannot change from one invocation to the next. | |||
Reuse executions to process documents Default Value: Deselected | Checkbox | None | Select this checkbox to start a child pipeline and pass multiple inputs to the pipeline. Reusable executions continue to live until all of the input documents to this Snap are fully processed.
| |||
Batch size* Default Value: 1 | Integer/Expression | None | Specify the number of documents in the batch size. If Batch Size is set to N, then N input documents are sent to each child pipeline that is started. After N documents, the child pipeline input view is closed until the child pipeline completes its execution. The output of the child pipeline (one or more documents) passes to the Pipeline Execute output view. New child pipelines are started after the original pipeline is complete.
| |||
Pool Size* Default Value: 1 | Integer/Expression | None | Specify an execution pool size to process multiple input documents or binary data concurrently. When the pool size is greater than one, the Snap starts Pipeline executions as needed up to the specified pool size. When you select Reuse executions to process documents, the Snap starts a new execution only if either all executions are busy working on documents or binary data and the total number of executions is below the pool size. | |||
Timeout (in seconds) Default Value: N/A | Integer/Expression | N/A | Specify the number of seconds for which the Snap must wait for the child pipeline to complete the runtime. If the child pipeline does not complete the runtime before the timeout, the execution process stops and is marked as failed. | |||
Retry limit Default Value: N/A | Integer/Expression | N/A | Specify the maximum number of retry attempts that the Snap must make in the case of a failure. If the child pipeline does not execute successfully, an error document is written to the error view. If the child pipeline is not in a completed state, then it will retry. The pipeline failure at the application level could have various causes, including network failures.
| |||
Retry interval Default Value: N/A | String/Expression | N/A | Specify the minimum number of seconds the Snap must wait between two retry requests. A retry happens only when the previous attempt results in an error.
| |||
Snap Execution Default Value: | Dropdown list | N/A | Select one of the following three modes in which the Snap executes. The available options are:
|
|
|
|
Troubleshooting
Error | Reason | Resolution |
---|---|---|
Account validation failed. | The Pipeline ended before the batch could complete execution because of a connection error. | Verify that the Refresh token field is configured to handle the inputs properly. If you are not sure when the input data is available, configure this field as zero so the connection is always open. |
Access pipelines in the Pipeline Catalog in the Designer to create a Child pipeline
...
The Pipeline Execute Snap can replace some uses of the Nested Pipeline mechanism and the ForEach and Task Execute Snaps. For now, this Snap only supports child Pipelines with unlinked document views (binary views are not supported). If these limitations are not a problem for your use case, read on to find out how to transition to this Snap and the advantages of doing so.
...