{
        "filename": "s3:///mybucket/qatest/user_manual.json",
        "result": "overwritten",
        "original": {
            "content-type" : "application/json"
        }
    }

The file name must start with s3:///. You can use the suggest feature to view the list of buckets, subdirectories and files. Bucket names are suggested if the property is empty or s3:///. Once you select a bucket, it can list subdirectories and files immediately below the bucket. Names of subdirectories end with a forward slash ("/"). The suggest feature is not supported if the properties in the S3 Dynamic account are parameters.

s3:///<S3_bucket_name>@s3.<region_name>.amazonaws.com/<path>

Region name is optional only if the region is us-east-1. In all other cases the region name must be specified based on the syntax above. For example, mybucket@eu-west-1. 

For more information about regions, see AWS Regions and Endpoints.

We recommend you to use fully-qualified suggestions from the Suggestions list if you are using an instance in the gov cloud.

Even though it is listed, Append is not supported for the S3 file protocol.

If the header has no keys other than Content-Type or Content-Encoding, the .header file is not written.

Ensure that the local drive has sufficient free disk space as large as the expected target file size.

If the value is larger than 0, the Snap first downloads the target file into a temporary local file. If any error occurs during the download, the Snap waits for the time specified in the Retry interval and attempts to download the file again from the beginning. When the download is successful, the Snap streams the data from the temporary file to the downstream Pipeline. All temporary local files are deleted when they are no longer needed.

• The minimum data size you can upload is 6 MB.

• The maximum data size you can upload is limited to 10000 times the buffer size.

• To upload S3 files that are more than 100 GB in size, we recommend you to set the Buffer size to 100 MB or more. Also, set the Maximum upload threads to 10; otherwise, many parallel HTTP connections may be opened, leading to the following error: "AmazonClientException: Unable to execute HTTP request: Timeout waiting for connection from pool".

To upload S3 files that are more than 100 GB in size, we recommend you to set the Maximum upload threads to 10. Also, set the Buffer size to 100 MB or more; otherwise, many parallel HTTP connections may be opened, leading to the following error: "AmazonClientException: Unable to execute HTTP request: Timeout waiting for connection from pool".

The account that you use for the Snap must have the required permissions to set the ACLs for the S3 object it is writing to.

A grantee can be an AWS account or one of the predefined Amazon S3 groups. The following note is an excerpt from AWS document:

"An email grantee is a grantee identified by their email address and authenticated by an Amazon system. email grants are internally converted to the canonical user representation when creating the ACL. If the grantee changes their email address, it will not affect existing Amazon S3 permissions. Adding a grantee by email address only works if exactly one Amazon account corresponds to the specified email address. If multiple Amazon accounts are associated with the email address, an AmbiguousGrantByEmail error message is returned. This happens rarely, but usually occurs if a user created an Amazon account in the past, forgotten the password, and created another Amazon account using the same email address. If this occurs, the user should contact Amazon customer service to have the accounts merged. Alternatively, grant user access specifying the canonical user representation."

"Everyone” option allows anyone in the world to access the file (authenticated or anonymous). Even when the bucket is protected with permission and if the file operation in the Snap under ACL->Grantee is set to 'Everyone', any user (authenticated or anonymous) can access the file. So, we highly recommend not to use this option as its unsafe. 

You can use Email addresses to specify a grantee only in the following regions:

• US East (N. Virginia)
• US West (N. California)
• US West (Oregon)
• Asia Pacific (Singapore)
• Asia Pacific (Sydney)
• Asia Pacific (Tokyo)
• EU (Ireland)
• South America (São Paulo)

For more information, see Specifying Grantee.