Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

On this pageIn this article

Table of Contents
maxLevel2
excludeRelated Links

...

Overview

Pipeline properties refer to the configurable settings, variables, or inputs you use to customize the behavior of a pipeline. These parameters provide a means of making pipelines more adaptable, maintainable, and reusable by allowing you to configure their behavior without altering the core logic of the pipeline. When you add a new pipeline, you initialize its properties.

Pipeline Parameters

To update the properties of an existing pipeline, view the pipeline and click the Edit Pipeline Properties button icon ( (blue star) ) on the toolbar. Note that you cannot update all fields. The pipeline properties dialog includes the following tabs:

Settings

...

Label

Required. The name for the Pipeline.

Each

The Pipeline name must be unique within

a Project must have a unique name. The label

the Project and must not exceed 100 characters.

Project

Required. The Project in which you save the Pipeline, displayed as <project_space>/<project_name>. All necessary resources (such as files and Accounts)

should

must be within the same Project unless they are part of the Shared project.

Note
This is only configurable before

You cannot update this property after you save your

Pipeline

pipeline the first time. To change the Project later, you must move the

Pipeline

pipeline to a new Project or make a copy.

Type

  • Standard. The default pipeline type.

  • eXtreme. No longer supported.

Resumable Mode

Select this checkbox to convert your pipeline into a resumable pipeline, which pauses when an error occurs and can be resumed at the point of error later.

Learn more: Resumable Pipelines

Error Pipeline

The pipeline that handles any error documents produced by this Pipeline.

Parameters

Selecting this checkbox displays the Pipeline parameter values in

Error Pipeline Parameters

Key-value pairs containing parameters to pass to the Error Pipeline if an error occurs.

Error Pipeline Parameters - Key and Value fieldsImage Added

Parameters

Key-value pairs containing parameters to pass to this Pipeline during execution.

Pipeline Parameters - Capture, Key, and Value fieldsImage Added

If the Capture checkbox is selected, the Pipeline parameter value is displayed in the Pipeline Execution Statistics window on SnapLogic Dashboard, when the Pipeline run is

run through

started by a Task or by another Pipeline

with You can define Parameters to be used during Pipeline execution. Pipeline parameters can only contain alpha-numeric characters, such as: myParam1.

using the PipeExecute Snap.

Image Removed

The keys and values of a Pipeline parameter can contain alphanumeric characters only.

You can also specify the values for the parameters within a Task that runs this Pipeline.

To use the value of the parameter as part of an expression within a Snap, you must

use

prefix the parameter name with the underscore character ( _ )

 as a prefix to

. For example, if the parameter name is myParam1,

as in the following: 

you can refer to its value by using _myParam1 as follows: '/out' + _myParam1 + '.json'

Parameters can also have their values defined on a Task.

You can also reference an account with an expression through Pipeline parameters, and therefore, configure multiple Pipelines to use a single account reference (credentials) through that Pipeline parameter.
Note
titleUse the Pipeline Execute Snap for large Pipeline parameters
We recommend that you do not pass excessive data through Pipeline Parameters. You should use Pipeline parameters to pass metadata—not data. You can save data to a file, and then pass the file name as a Pipeline parameter. Similarly, when using the Pipeline Execute Snap, the data can be passed

To use the same account in multiple Pipelines, you can reference the account credentials using an expression in a Pipeline parameter value.

Info

IMPORTANT: Use pipeline parameters to pass metadata, not data.

To pass data, use any of the following methods:

  • Save the data to a file and then pass the filename as a pipeline parameter value.

  • Use the Pipeline Execute Snap and pass the data through the input view of the child Pipeline.

When the Capture options

If you pass large data sets as parameter values, the following built-in limitations in the SnapLogic Platform can cause parameter values to be truncated or can prevent the Pipeline from running:

  • If Capture is selected, the first 256 bytes of the captured Pipeline parameter

values
  • value are stored in the runtime for debug purposes. Each Snaplex node pushes information to the control plane about all the Pipelines running on the

node. This data
  • JCC node, and this information is limited to 250MB

of data
  • for all Pipelines on the

JCC
  • node.

In the 4.20 release, an additional limit is added which caps the
  • If the total size of the Pipeline parameter values for one Pipeline execution (including parameters that are not captured)

to 8MB. If the Pipeline parameter values exceed 8MB in size, then
  • exceeds 8 MB, the Pipeline does not execute.

Expression Libraries

List

The list of expression libraries to import

. See Expression Libraries for more information.

and use with this Pipeline.

Expression Libraries - Path and As fieldsImage Added
  • Path: Path to the file that contains the expression to import. The path can be absolute or relative to the current project.

  • As: The name to use for the library under the

'
  • lib

'
  • variable. Defaults to the base name of the file

.

...

  • .

Learn more: Expression Libraries

Cache Pipelines

The data to pre-cache in a Cache Pipeline. The alias and corresponding pipeline to run and store the data under the provided alias.. You can access this cache with Pipeline Cache properties, where you can call it from the cache and filter the conditions to return back the data specified by the function. This reference provides the full document from the match to then use the output from the target match.

Info

TIP: You can use captured Pipeline parameter values and the error Error Pipeline parameter values to search for a Pipeline in the Pipeline wall on the Dashboard. For details, see Pipeline Search/Pipeline Filter.

Open API

The You can use the Open API tab can be used to add descriptive information information, such as data type and description, about the Pipeline parameters like data type and description. This information is helpful to helps other Pipeline developers in understanding understand what each of the Pipeline parameter represents.

...

Pipeline parameters are not populated in this tab unless are populated after the Pipeline has been is saved. If you have added new parameters since the last time you saved the Pipeline, save the Pipeline again to update this tab.

Key

The name of the Pipeline parameter as

given

specified in

the 

the Parameters

 section

 section (Key). 

Type

The data type of the parameter. Choose

appropriately from the drop-down. The

one of the following data types

are listed

from the dropdown list:

  • string

  • number

  • byte

  • binary

  • boolean

  • date

  • date-time

  • password

Default value: string

For more information regards the data types refer to 

Learn more: Data Types

.

Description

A description of the parameter

, this description will be shown

. This description is included in the auto-generated API document

that is created of the Pipeline properties. Makes it easier for the user to understand the parameter.

about the Pipeline’s properties.

Default value:

[

None

]

Required

Specifies whether the parameter is required.

Default value: Not selected

Info

...

Notes

Additional information about this pipeline.

Purpose

The purpose of this pipeline.

Author

The creator of the pipeline. If you are adding a new pipeline, this field is prepopulated with your username or email address.

Doc Link

The link to the documentation about this pipeline (if any).

...

Related Content: