In this Page
| Table of Contents | ||||
|---|---|---|---|---|
|
Overview
Pipeline design requires adding and removing Snaps, as well as configuring the various settings in each Snap, before testing to see if the Pipeline validates and executes reliably. This process of trial and error is particularly involving if the Pipelines are long (contain many Snaps) and complex (contain many different types of Snaps with accounts and parameters). SnapLogic Designer provides functionality, easing the process of testing Pipelines-in-development.
Prior to the May 2019 release (SnapLogic 4.17), Pipeline development often necessitated opening multiple Designers to do this work. If you worked on developing a complex Pipeline from scratch, you would have to copy different parts to another Designer to test the new branch of the Pipeline in isolation. Likewise, when building upon legacy Pipelines, you might have copied that Pipeline, imported it onto a separate canvas, then added the new section and configured the new Snaps. Once the Pipeline-in-development executed successfully, you could then make the changes to the legacy Pipeline or replace it.SnapLogic introduces this functionality to make Pipeline development easier by allowing you to disable a Snap. Disabling a Snap prevents that Snap and subsequently all downstream Snaps from executing. In SnapLogic Designer, the Execute during Preview checkbox option is replaced with a drop-down menu with the following options:
- Validate and Execute
- Execute only
- Disabled
Although only a limited number of Snaps contained the Execute during Preview checkbox option, the Disabled Snap functionality is available in all Snaps. This functionality enables you to test isolated sections of your Pipeline-in-development by disabling a Snap at the point in the Pipeline where validation/execution testing is not required. When you disable a Snap, only Snaps upstream of the disabled Snap validate and execute. Once executed, these upstream Snaps turn green, while the downstream Snaps turn a light shade of gray. The disabled Snap itself is displayed with a circle and a slash over it.
This functionality simplifies Pipeline development since you can disable certain Snaps temporarily, like Snaps that require access to accounts, whose configuration details you may not have at the moment. You can also test multiple sections of a Pipeline conveniently in the same canvas. This type of Pipeline execution is available now in all Standard-mode Pipelines.
Execution/Validation Options Before and After May 2019
Execute during Preview checkbox selected
or
No checkbox
as before
Validation: The Snap validates and executes a limited set of documents.
Execution: The Snap executes in its entirety.
Execute Only
Validation: The Snap does not validate separately.
Execution: The Snap executes in its entirety.
Validation: None
Execution: None
In the Pipeline, all Snaps downstream from the disabled Snaps do not validate/execute. However, Snaps upstream from the disabled Snap validate and execute.
Support and Limitations
All Snap Packs are supported except the Spark SQL 2.x Snap Pack, all eXtreme Snap Packs, and deprecated Snap Packs, as well as obsolete Snaps, such as Nested
Snap).Snap
Packs prior to the May 2019 release (SnapLogic 4.17)
. If you decide to roll-back a Snap Pack to a previous version, you must change any Snap Execution Type from Disabled to Execute Only or Validate and Execute before the Snap Pack is rolled back.
Best Practices
For developing Pipelines, disable any Snap to stop execution of the pipeline at that point. All downstream Snaps are disabled as well.
- If a Pipeline consists of numerous unconfigured account Snaps, you can disable them or the Snaps that are upstream from them.
- For Triggered Tasks, Ultra Tasks, and Child Pipelines, the last non-disabled Snap becomes the open output view.
- For Join Snaps, if one input view of any Snap is downstream from a Disabled Snap, then that Snap is disabled. If you want those downstream Snaps to run, then disconnect the Snap with the disabled input view.
- You cannot discern whether a non-Disabled Snap without an output view is executing unless you open the Snap settings pop up associated with the Snap. Add an output view to see if such Snaps are being executed.
Examples
The following examples display how validation works in a Pipeline where certain Snaps are disabled.
| Expand | ||
|---|---|---|
| ||
In this Pipeline with multiple Join Snaps, the Join Snap in the primary branch of the Pipeline is Disabled from executing.
In this Pipeline, one of the Join Snaps is disabled, stopping validation from all downstream Snaps. However, notice that the MySQL Select Snap is validated because it precedes the branch that has been disabled.
In this Pipeline, the MySQL-Select, which precedes the Join Snap in the Main branch, is disabled, stopping the downstream Snaps from validating.
|
