Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Table of Contents
minLevel1
maxLevel2
absoluteUrltrue

Overview

You can use this Snap to restructure and write input JSON data as JSON or Parquet files into Adobe Enterprise Platform (AEP)retrieve attributes of contents of a given directory.

Snap Type

Adobe Experience Platform Write Directory Browser Snap is a Write Read -type Snap that writes

Prerequisites

  • The dataset into which you want to write the output of the Adobe Experience Platform Write Snap. If you do not have the dataset ready, you must create it first using the Adobe Experience Platform UI or the Adobe Experience Platform S3 Connector Snap.

Limitations

While the Dataset Name field is expression enabled, its purpose is to read Pipeline parameter values. This Snap does not read values from upstream Snaps. You can, however, add this Snap in Pipelines with other Snaps to execute them in a sequence.reads attributes of contents of a given directory.

Prerequisites

  • None

Limitations

  • None

Known Issues

  • None.

Snap

...

Input/Output

...

Type of View

...

Number of Views

...

Views

Views

Description

Input 

Document

  • Min: 1

  • Max: 1

  • Mapper Snap

  • Binary to Document Snap

  • Sort Snap

Each input document contains data that you want to write to AEP.

Output

Document

  • Min: 0

  • Max: 1

  • Mapper Snap

  • Join Snap

  • Union Snap

Each output document lists out the status and other details associated with the write processThis Snap has at most one document input view. Each document contains data in the form of json.

Output

This Snap has at most one document output view. Each document contains a corresponding response in the form of JSON.

Error

This Snap has at most one document error view and produces zero or more documents in the view. Each document contains the corresponding error record if snap is not able to process data to Custom SFTP.

Snap Settings

Error

Reason

Resolution

Missing property value

The snap settings are not correct.

You may see this error because you did not provide a mandatory value.

Check to ensure that all mandatory values are provided and valid.

Invalid Snap configuration

Target Dataset Name is null or missing.

This typically means that you did not specify the dataset into which the parquet file must be written or that the dataset name specified doesn't exist in AEP.

Ensure that the property Dataset Name has a valid dataset name. Use the drop-down button adjacent to the Dataset Name field to select a valid dataset.

Unable to load the private key for the given alias

File not found on <your SnapLogic instance> at <account location>

This error appears when the account doesn't have valid values. Ensure that the keystore path, keystore passphrase, private key alias, and private key passphrase are correct. Also, you must upload the keystore in the Keystore Path that you specify.

Unable to obtain access token

This message appears when the endpoint account details are incorrect.

Check your Organization IDTechnical Account IDClient IDClient Secret Key details and try again.String

Parameter Name

Data Type

Description

Default Value

Example 

Label

Required

Required. The name for the Snap. Modify You can modify this to be more specific, especially if there is you have more than one of the same Snap in the Pipeline.

Adobe Experience Platform Write

Write Documents into AEP

Dataset Name

String

Required. The name of the dataset into which you want to write the input data. This is a suggestible field. Click  to view dataset suggestions that you can use.

 

This property is expression-enabled. For more information on the expression language, see Understanding Expressions in SnapLogic and Using Expressions. For information on Pipeline Parameters, see Pipeline Properties.

This Snap reads Pipeline parameters, but not values from upstream Snaps. You can, however, add this Snap in Pipelines with other Snaps to execute them in a sequence.

 

N/A

Adobe Generic Dataset

Schema Type

String

Required. The schema type that you want to use to create the file that must be written. Click to select from the following options:

  • Parquet

  • JSON

Parquet

Parquet

Snap Execution

String

Select one of the three modes in which the Snap executesyour pipeline.

Default value: [None]

Directory

Required

The URL for the data source (directory). It should start with a protocol.

File Filter

Conditionally

required

The GLOB pattern can be applied to select the contents (files/sub-folders) of the directory. You cannot recursively navigate through the directory structures.

The File filter property can be a JavaScript expression, which will be evaluated with the values from the input view document.

Example:

  • *.txt

  • ab????xx.*x

  • *.[jJ][sS][oO][nN](as of the May 29th, 2015 release)

Default value: [None]

Ignore empty result

Conditionally

required

If selected, no document will be written to the output view when the result is empty. If this property is not selected and the Snap has received an input document, the input document will be passed through to the output view. If this property is not selected and there is no input document, an empty document will be written to the output view.

Default value: Selected

Display update time and owner for S3 subdirectories

Conditionally required

 

Applicable only for S3 directory.

If selected, the Snap produces Update time and Owner attributes in the output documents for S3 subdirectories. However, it may take a long time or even freeze if there are very large number of files and subdirectories in the file structure under the specified S3 directory. This is because the Snap has to completely examine the S3 objects under the directory recursively in order to get Update time and Owner attributes for subdirectories.

If not selected, the Snap does not produce Update time and Owner attributes in the output documents for S3 subdirectories. Snap output is faster since it examines only S3 files and subdirectories immediately under the specified S3 directory.

Default value: Selected

Number of retries

optional

Specifies the maximum number of retry attempts in case of a network failure.

Example:  3

Minimum value: 0

Default value: 0

Retry interval (seconds)

Specifies the minimum number of seconds for which the Snap must wait before attempting recovery from a network failure.

Example:  3

Minimum value: 1

Default value: 1


Advanced properties

Use this fieldset to define specific settings for directory browsing. Click + to add a new row for defining an advanced property. This fieldset comprises the following fields:

  • SAS URI

  • Use fast file-type check for wasb/wasbs protocols

Snap Execution

Indicates how the Snap must be executed. Available options are:

  • Validate & Execute: Performs limited execution of the Snap , and generates a data preview (up to 50 records) during Pipeline validation. Subsequently, ; performs full execution of the Snap (unlimited records) during Pipeline runtimeexecution.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview dataSnap during Pipeline execution; does not execute the Snap during Pipeline validation.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Execute only

Validate and Execute

Limitations

While the Dataset Name field is expression enabled, its purpose is to read Pipeline parameter values. This Snap does not read values from upstream Snaps. You can, however, add this Snap in Pipelines with other Snaps to execute them in a sequence.

Troubleshooting

  • , by extension, its downstream Snaps.

Default value: Validate & Execute