Input 

Binary

• Min: 1

• Max: 1

• CSV Formatter

• JSON Formatter

• XML Formatter

• File Reader

Output

Document

• Min: 0

• Max: 1

• Mapper

• File Reader

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

• Stop Pipeline Execution: Stops the current Pipeline execution when the Snap encounters an error.

• Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

• Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Label*

Default Value: S3 Upload
Example: S3 Upload

String

N/A

Specify a unique name for the Snap.

Bucket*

Default Value: N/A
Examples: 

• mybucket

• mybucket@eu-west-1

• Example of S3 Bucket in S3 region:

  • mybucket@s3.eu-west-1.amazonaws.com

• Example of S3 VPC endpoint:

  • my-bucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

  • testBucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

• Example of S3 Express One Zone bucket:

  • snapexpress--usw2-az2--x-s3

String/Expression/Suggestion

N/A

Specify the S3 bucket name, from where an S3 object is to be downloaded.

Bucket names are unique globally and can be accessed without the region name in most cases. If you cannot access a bucket name without its region name, you can specify the region information with the following syntax:

<S3_bucket_name>@<region_name>

Note: If you enter an incorrect region name, but the bucket name is valid, the AWS S3 service might successfully access the bucket without errors.

• You can access an S3 bucket in an S3 Virtual Private Cloud (VPC) endpoint by specifying the bucket name with the following syntax:

  • <S3_bucket_name>@<VPC_S3_endpoint>

• You can access an S3 Express One Zone bucket with the following syntax:

  • <bucket-name>--<region>-<available-zone>--x-s3

• S3 Express One Zone does not support the following bucket name pattern:

  • <bucket>@<region_info>.

• S3 Express One Zone does not support VPC.
  

Object Key

Default Value: N/A
Examples: 

• test.csv

• abc/test.json

• abc/xyz/test.xml

String/Expression/Suggestion

N/A

Specify or select the S3 object key name, which may include one or more forward-slash '/' characters.

Show Advanced Properties

Default Value: Deselected 

Checkbox

Select this checkbox to display the advanced properties.
Deselect this checkbox to hide the advanced properties.

Thread Pool Size

Default Value: 10
Example: 5

Integer/Expression

Appears when you select Show Advanced Properties checkbox.

Specify the maximum number of threads to use to download multiple S3 objects simultaneously.

Maximum Retries*

Default Value: 3
Example: 5

Integer/Expression

Appears when you select Show Advanced Properties checkbox.

Specify the maximum number of retry attempts to make in case of a temporary network failure.

Write Empty File

Default Value: Deselected
Example: Selected

Checkbox

Appears when you select Show Advanced Properties checkbox.

Select this checkbox to write an empty S3 object when the incoming binary document has empty data. If there is no incoming document at the input view of the Snap, no S3 object is written regardless of the value in this field.

Multipart Size (MB)

Default Value: 10
Example: 10

Integer/Expression

Appears when you select Show Advanced Properties checkbox.

Specify the multipart size in MB:

• The minimum multipart size is 5 MB.

• The maximum multipart size is 5 GB.

• The maximum number of parts in an S3 object is 10,000, and the maximum size of an S3 object is 5 TB.

• To upload S3 objects that are more than 100 GB, you must increase the Multipart Size to at least the possible S3 object size divided by 10,000.

• The larger the Multipart Size is, the larger the memory size that the Snap uses. For example, if the Multipart Size is 10 MB and the Thread Pool Size is 10, the memory size the Snap uses is 100 MB.

Learn more about uploading multipart files to S3.

Access Control List

Use this fieldset to define the Access Control List (ACL) to the specified S3 object.

Grantee

Default Value: N/A
Examples:      

• "Everyone"

• "Authenticated user"

•  a canonical AWS ID: 1700891f3927e316dc4c9e18c789b32131880f48d3e03ac110aaf695b212573e

String/Expression/Suggestion

Appears when you select Show Advanced Properties checkbox.

Select a grantee from the suggested list or enter a valid email address or a canonical ID associated to an AWS account. A canonical AWS ID can be obtained in the Security Credentials page of the AWS console.

Read

Checkbox

Appears when you select Show Advanced Properties checkbox.

Select this checkbox to grant permission to read the file.

View Permissions

Checkbox

Appears when you select Show Advanced Properties checkbox.

Select this checkbox to grant permission to read the ACL.

Full Control

Checkbox

Appears when you select Show Advanced Properties checkbox.

Select this checkbox to grant full control to the file.

Canned ACL

Default Value: None
Example: PublicRead

String/Expression

Appears when you select Show Advanced Properties checkbox.

The predefined ACL grants (from AWS) to use when writing a file to S3.

Choose a Canned ACL from the available options:

• None

• Private

• PublicRead

• PublicReadWrite

• AuthenticatedRead

• LogDeliveryWrite

• BucketOwnerRead

• BucketOwnerFullControl

• AwsExecRead

Watch the video for more information about AWS Canned ACL.
Learn more about AWS Canned ACLs.

User-defined Object Metadata

Use this field set to define key-value pairs for user-defined object metadata of an S3 object. Learn more about Using Metadata.

Key

Default value: N/A

String/Expression

Appears when you select Show Advanced Properties checkbox.

Specify the key name of the object metadata.

Value

Default value: N/A

String/Expression

Appears when you select Show Advanced Properties checkbox.

Specify the value for the key entered above.

Object Tags

Use this field set to define key-value pairs for object tags of an S3 object. Object tags enable you to categorize existing and new objects using key-value combinations. Learn more about Object Tagging. 

Key

Default value: N/A

String/Expression

Appears on selecting Show Advanced Properties checkbox.

Specify the key name of the object tag.

Value

Default value: N/A

String/Expression

Appears on selecting Show Advanced Properties checkbox.

Specify the value for the key entered above.

Pass Through

Default Value: Not selected

Checkbox

Appears on selecting Show Advanced Properties checkbox.

Select this checkbox to pass the input header as a value of the ‘original’ key in the output document.

Snap Execution

Default Value: Execute only
Example: Validate & Execute

Dropdown list

N/A

Select one of the three modes in which the Snap executes. Available options are:

• Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

• Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

• Disabled: Disables the Snap and all Snaps that are downstream from it.