In this article

Overview

You can use this Snap to perform a file operation (move/copy/rename) for a file from a source to a target in the same server. The supported file protocols are: local file (file:///), FTP, SFTP, S3, WASB, and WASBS.

Snap Type

File Operation Snap is a WRITE-type Snap that writes files to the target.

Prerequisites

• The provided account must have 'write' access to the specified directory and file in order to perform the file operation successfully.

• IAM Roles for Amazon EC2 

Support for Ultra Pipelines

Works in Ultra Task Pipelines.

Limitations

• The Snap can move, rename, or copy a file within the same file server, but not across file servers.
• The Snap can move or copy S3 files across buckets within the same region, but not across regions.

Known Issues

• This Snap does not support using the ABFS protocol with a Windows-based Snaplex.
• When you use special characters, such asðø©¢¾A²½µ®÷¶þ~ for Source and Target directory and filenames, this Snap fails and results in the following error as the special characters are not supported.
  Error: Illegal character in fragment at index 71:abfs://bigdataqa@bigdataqassl.dfs.core.windows.net/simplechar/owner!@#$^&()_¢äâêîôûñç¡¿ÉÙËǨ°¸ðø©¢¾A²½µ®§÷¶þ~.json
• The format: abfs(s)://filesystem@accountname.endpoint/<path> for file path for Source and Target fields does not work as this URL syntax is not supported. The file protocol must begin with abfs(s):///, else the container, account name, and endpoint are interpolated into the URL at runtime, which results in the following error. 
  Error: Unsupported protocol or URL syntax error in abfs(s)://filesystem@accountname.endpoint/<path>,
  Workaround: Use the supported file protocol and correct URL syntax.

Snap Views

{
	"Source": "ftp://ftp.snaplogic.com/home/mrtest/new1/sample.csv",
	"Target":  "ftp://ftp.snaplogic.com/home/mrtest/new2/new.csv",
	"Status": "Moved"
}

Snap Settings

Snap Behavior for Key Operations

FTP, SFTP, and File Protocols

S3, WASB, and WASBS Protocols

Examples

Copying and Moving of File from One Folder to Another Folder

This example Pipeline demonstrates copying and moving of files with ABFS protocol from one folder to another folder.

First, we configure the JSON Generator Snap to pass the following sample data. After validation we see the following output.

Next, we format the data using the JSON Formatter Snap and then, use the HDFS Writer Snap to write the sample file (that uses ABFS protocol and ADLS Gen 2) to the target location.

Next, we configure two File Operation Snaps to copy and move files from one folder to another folder. We use the first File Operation Snap to copy the sample file from abfss:///bigdataqa/Samplefile/ to abfss:///bigdataqa/samplefile_diff/).

We add the second File Operation Snap to move the same sample file (uni_code.json) to another file (uni_code_move.json).

The target location of the copied and moved file can be viewed in the output preview of the Snap.

Finally, we connect the HDFS Reader Snap to read the target file.

Basic Use Case

The following Pipeline describes how the Snap functions as a standalone Snap in a Pipeline:

Download the Pipeline here or from the Downloads section below.

In the Pipeline shown above, a copy of the file Emp_Data_03032020.json is created in the specified target location. The Snap's configuration is as below:

A preview of the output from executing this Pipeline:

The exported Pipeline is available in the Downloads section below. More examples of operations are also available in the Downloads section - variousCasesOfFileOperations.pdf.

Typical Snap Configurations

• Without Expressions: Values are passed in the Snap directly:

• With Expressions:
  • Using Pipeline parameters: Values are passed as Pipeline parameters:

Downloads