Input

Document

Min: 0

Max: 1

• Mapper

• File Writer

Any document with key-value pairs to evaluate expression properties in the File Operator Snap can be used, where each input document will cause one complete execution of the Snap.

Output

Document

Min: 0

Max: 1

• CSV Formatter

• JSON Formatter

• Mapper

A typical output from this Snap is a document with column headers, such as Source, Target, and Status (Moved or Copied).

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab. The available options are:

• Stop Pipeline Execution: Stops the current pipeline execution when the Snap encounters an error.

• Discard Error Data and Continue: Ignores the error, discards that record, and continues with the rest of the records.

• Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Label*

Default Value: File Operation
Example: File Operation

String

Specify the name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your Pipeline.

Source*

Default Value: [None]
Example:

• file:///tmp/test.csv (if you're using Linux)

• ftp://ftp.snaplogic.com/home/mrtest/source.csv (if your file is on an FTP server)

• sftp://ftp.snaplogic.com/home/mrtest/source.csv (if you are using Secure FTP)

• s3:///test_bucket/folder1/test.json (if you are using AWS without specifying any region)

• s3:///mybucket@eu-west-1/folder1/test.json (if you're using an AWS account that is region-specific)

• wasb:///test_container/testFolder/sample.json (if you are using Windows Azure Storage Blob)

• wasbs:///test_container/testFolder/sample.json (if you are using Secure Windows Azure Storage Blob)

• _source (A key/value pair with "source" key should be defined as a Pipeline parameter.)

• $source(A key/value pair with "source" key should be defined the input document.)

• file:///Z:/somedirectory/somefile.csv (if you have a Groundplex on Windows)

• file:////somedirectory/somfile.csv (if you're using the Universal Naming Convention

• abfs(s):///filesystem/<path>/

• abfs(s)://filesystem@accountname.endpoint/<path>

String/Expression

Specify the URL for the source, where the binary data is read from. This Snap also supports S3 Virtual Private Cloud (VPC) endpoint. For example, s3://my-bucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

Example of Source as an expression: "s3:///mybucket/out_" + Date.now() + ".csv"
the evaluated filename will be: s3:///mybucket/out_2013-11-13T00:22:31.880Z.csv

This property should have the syntax: scheme://[hostname:port]/[path to source]

You can also copy or move the file from your local system to Azure blob container for better performance.

Target*

Default Value: [None]
Example:

• file:///tmp/test.csv

• ftp://ftp.snaplogic.com/home/mrtest/source.csv

• ftp://ftp.snaplogic.com/home/mrtest/

• sftp://ftp.snaplogic.com/home/mrtest/source.csv

• sftp://ftp.snaplogic.com/home/mrtest/

• s3:///test_bucket/folder1/test.json

• s3://test_bucket/folder1/

• wasb:///test_container/testFolder/sample.json

• wasbs:///test_container/testFolder/sample.json

• _target (A key/value pair with "target" key should be defined as a Pipeline parameter.)

• $target (A key/value pair with "target" key should be defined the input document.)

• abfs(s):///filesystem/<path>/

• abfs(s)://filesystem@accountname.endpoint/<path>

String/Expression

This property specifies the URL of the destination where the selected file operation must be performed. This Snap also supports S3 Virtual Private Cloud (VPC) endpoint. For example, s3://my-bucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

File Operation

Default Value: Move
Example:

• Copy

• _operation (A key/value pair with "operation" key should be defined as a Pipeline parameter.)

• $operation(A key/value pair with "operation" key should be defined the input document.)

String/Expression

Enter or select the operation you want the Snap to perform on the file. Available options are: 

• Move - Moves the source file to the target location.

• Copy - Creates a copy of the source file in the target location.

• Rename - Renames a file in the source location. Rename is handled the same as Move. The source URL and target URL are the same, with the exception of the file name. 

Error if exists

Default Value: Selected

Checkbox

If enabled, the Snap displays an error when the target exists. If disabled, the Snap replaces or overwrites the target with the source.

Advanced Properties

Use this field set to customize or control the Snap's validation and execution mechanism. 

Properties

Default Value: SAS URI
Example: Simple file operation

Dropdown list

The available options are:

• SAS URI:

• Simple file operation: Select and set this property to true to allow the Snap carry out the move or copy operation without checking if the source and target are directory or regular file or does not exist. This feature is useful when the account does not have the permissions to list the source and/or target directories.
  Note: This feature is currently supported only for FTP file operations.

• Az copy absolute path: Specify the directory and wildcard for Azure blob storage for Copy, Move, and Rename operations. For example,  "\t" + blob_demo

You can also copy or move the file from your local system to Azure blob container for better performance.

Values

Default Value: None

String/Expression

Specify a value for the above property.

Snap Execution

Dropdown list

--

--

x

State does not affect outcome

x

State does not affect outcome

00

Does not exist

00

Does not exist

0

Not selected

01

Regular file

01

Regular file

1

Selected

10

Directory

10

Empty directory

--

--

--

--

11

Non-empty directory

--

--