Input 

This Snap has at most one document input view. Each document contains data in the form of json.

Output

This Snap has at most one document output view. Each document contains a corresponding response in the form of JSON.

Error

This Snap has at most one document error view and produces zero or more documents in the view. Each document contains the corresponding error record if snap is not able to process data to Custom SFTP.

Label

required

The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

Default value: [None]

Directory 

required

The URL for the data source (directory). It should start with a protocol.

File filter

Conditionally

required

The GLOB pattern can be applied to select the contents (files/sub-folders) of the directory. You cannot recursively navigate through the directory structures.

The File filter property can be a JavaScript expression, which will be evaluated with the values from the input view document.

Example:

• *.txt

• ab????xx.*x

• *.[jJ][sS][oO][nN](as of the May 29th, 2015 release)

Default value: [None]

Ignore empty result

Conditionally

required

If selected, no document will be written to the output view when the result is empty. If this property is not selected and the Snap has received an input document, the input document will be passed through to the output view. If this property is not selected and there is no input document, an empty document will be written to the output view.

Default value: Selected

Display update time and owner for S3 subdirectories

Conditionally required

Applicable only for S3 directory.

If selected, the Snap produces Update time and Owner attributes in the output documents for S3 subdirectories. However, it may take a long time or even freeze if there are very large number of files and subdirectories in the file structure under the specified S3 directory. This is because the Snap has to completely examine the S3 objects under the directory recursively in order to get Update time and Owner attributes for subdirectories.

If not selected, the Snap does not produce Update time and Owner attributes in the output documents for S3 subdirectories. Snap output is faster since it examines only S3 files and subdirectories immediately under the specified S3 directory.

Default value: Selected

Number of retries

optional

Specifies the maximum number of retry attempts in case of a network failure.

Example:  3

Minimum value: 0

Default value: 0

Retry interval (seconds)

Specifies the minimum number of seconds for which the Snap must wait before attempting recovery from a network failure.

Example:  3

Minimum value: 1

Default value: 1

Advanced properties

Use this fieldset to define specific settings for directory browsing. Click + to add a new row for defining an advanced property. This fieldset comprises the following fields:

• SAS URI

• Use fast file-type check for wasb/wasbs protocols

Snap Execution

Indicates how the Snap must be executed. Available options are:

• Validate & Execute: Performs limited execution of the Snap (up to 50 records) during Pipeline validation; performs full execution of the Snap (unlimited records) during Pipeline execution.

• Execute only: Performs full execution of the Snap during Pipeline execution; does not execute the Snap during Pipeline validation.

• Disabled: Disables the Snap and, by extension, its downstream Snaps.

Default value: Validate & Execute