Input 

Document

• Min: 0

• Max: 1

• Mapper

• S3 Copy

Directory Path to be browsed and File Filter Pattern to be applied. It contains values for the directory path to be browsed and the GLOB filter to be applied to select the contents.

Output

Document

• Min: 1

• Max: 1

• CSV Formatter

• File Writer

The attributes (name, size, and so on) of the contents of the directory path matching the filter pattern. This Snap provides the various attributes (such as Name, Type, Size, Owner, Last Modification Time) of the contents of the given directory path.  Only those contents are selected that match the given GLOB filter.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

• Stop Pipeline Execution: Stops the current pipeline execution when the Snap encounters an error.

• Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

• Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Label*

Default Value: Directory Browser
Example: Directory Browser

String

Specify a unique name for the Snap.

Directory 

Default Value: N/A

Examples: 

• Read the file from a location of the storage

• : adl://storename/

• folder/

• filename 

• If the Snap is executed in the Windows Groundplex and needs to access D: drive): file:///D:/testFolder/ 

• ftp://ftp.mock.com/vehicles

• If the bucket name is 'testBucket': gs:///testBucket/testDir/ 

• project-name

• s3:///<S3_bucket_name>@s3.<region_name>.amazonaws.com/

• <path>

• File in VPC: s3:///snaplogic-bucket@bucket.vpce-0e340d8d5b0eae90a-k9wbom46-us-west-2a.s3.us-west2.vpce.amazonaws.com

• sftp://sftp.mock.com/students/grades/

• share

• sldb:///shared

• sldb:///project-name

• If the target folder is /share/folder in server: smb://server/share/folder/

•  

• If the name of the container is 'Snaplogic': wasb:///Snaplogic/testDir/

• 
  or wasbs:///Snaplogic/testDir

• / 

String/Expression

Specify the URL for the data source (directory) that should begin with a protocol. Following are the protocols that are currently supported:

• sldb:

• s3:

• ftp:

• ftps:

• sftp:

• hdfs:

• wasb:

• wasbs:

• smb:

• gs:

• adl:

• file:

File filter

Default Value: N/A
Example:

• *.txt

• ab????xx.*x

• *.[jJ][sS][oO][nN]

String/Expression

Specify the Glob pattern for filtering the files. The Glob pattern can be applied to select the contents (files/sub-folders) of the directory. You cannot recursively navigate through the directory structures.

Ignore empty result

Default Value: Selected

Checkbox

Select this checkbox to enable the Snap to not write a document to the output view when the result is empty.

Deselect this checkbox in the following cases:

• If the Snap receives an input document, the input document

• is passed through to the output view.

• If the Snap does not receive an input document, an empty document

• is written to the output view.

Display update time and owner for S3 subdirectories

Default Value: Selected

Checkbox

Applicable only for S3 directory.

If selected, the Snap

produces Update time

 and Owner attributes in the output documents for S3 subdirectories. However, it may take a long time or even freeze if there are very large number of files and subdirectories in the file structure under the specified S3 directory. This is because the Snap has to completely examine the S3 objects under the directory recursively in order to

get Update time 

and Owner

 attributes for subdirectories.

If not selected, the Snap does not

produce Update time

 and Owner attributes in the output documents for S3 subdirectories. Snap output is faster since it examines only S3 files and subdirectories immediately under the specified S3 directory.

Number of retries

Default Value: 0
Example:  3

String/Expression

Specify the maximum number of retry attempts in case of a network failure.

Minimum

value: 0

Retry interval (seconds)

Default Value:1
Example: 3

String/Expression

Specify the minimum number of seconds

the Snap must wait before attempting recovery from a network failure.

Minimum value: 1

Advanced properties

Use this field set to define specific settings for directory browsing.

Properties

SAS URI

Dropdown list

Use fast file-type check for wasb/wasbs protocols

String/Expression

Snap Execution

Default Value: Validate & Execute
Example: Execute only

Dropdown list

Select one of the three modes in which the Snap executes. Available options are:

• Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

• Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

• Disabled: Disables the Snap and all Snaps that are downstream from it.