Input 

Document

• Min: 0

• Max: 1

• Mapper

An upstream Snap is optional. Any document with key-value pairs to evaluate expression properties. Each input document, if any, results in one download operation of the Snap.

Output

Binary

• Min: 1

• Max: 1

• Mapper

• CSV Parser

• JSON Parser

• XML Parser

Binary data downloaded from AWS S3 is specified in the Bucket and Object Key fields with header information about the binary stream.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

• Stop Pipeline Execution: Stops the current pipeline execution when the Snap encounters an error.

• Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

• Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Label*

Default Value: S3 Download
Example: S3 Download

String

N/A

Specify a unique name for the Snap.

Bucket*

Default Value: None
Examples: 

• mybucket

• mybucket@eu-west-1

• Example of S3 Bucket in S3 region:

  • mybucket@s3.eu-west-1.amazonaws.com

• Example of S3 VPC endpoint:

  • my-bucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

  • testBucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

• Example of S3 Express One Zone bucket:

  • snapexpress--usw2-az2--x-s3

String/Expression/Suggestion

N/A

Specify the S3 bucket name, from where an S3 object is to be downloaded.

Bucket names are unique globally and can be accessed without the region name in most cases. If you cannot access a bucket name without its region name, you can specify the region information with the following syntax:

<S3_bucket_name>@<region_name>

Note: If you enter an incorrect region name, but the bucket name is valid, the AWS S3 service might successfully access the bucket without errors.

• You can access an S3 bucket in an S3 Virtual Private Cloud (VPC) endpoint by specifying the bucket name with the following syntax:

  • <S3_bucket_name>@<VPC_S3_endpoint>

• You can access an S3 Express One Zone bucket with the following syntax:

  • <bucket-name>--<region>-<available-zone>--x-s3

• S3 Express One Zone does not support the following bucket name pattern:

  • <bucket>@<region_info>.
    
    

Object Key

Default Value: None
Examples: 

• test.csv

• abc/test.json

• abc/xyz/test.xml

String/Expression/Suggestion

N/A

Specify or select the S3 object key name, which may include one or more forward-slash '/' characters.

Show Advanced Properties

Default Value: Deselected

Checkbox

N/A

Select this checkbox to display the advanced properties.
Deselect this checkbox to hide the advanced properties.

Thread Pool Size

Default Value: 10

Integer/Expression

Appears when you select Show Advanced Properties checkbox.

Specify the maximum number of threads to use to download multiple S3 objects in parallel.

Maximum Retries*

Default Value: 3
Example: 5

Integer/Expression

Appears when you select Show Advanced Properties checkbox.

Specify the maximum number of retry attempts to perform in case of a temporary network loss.

Version ID

Default Value:  N/A
Example:   xvcnB8gPi37l3hbOzlsRFxjVwQ.numQz

String/Expression/Suggestion

Appears when you select Show Advanced Properties checkbox.

Specify or select the version ID of the S3 file object. If you leave this field empty, the Snap downloads the latest version. S3 versioning is not supported in S3 Express One Zone.

Version ID Suggestion Interval

Use this field set to configure the time interval for the version ID suggestion.

Year

Default value: N/A
Example:  2017

Integer

Appears when you select Show Advanced Properties checkbox.

Enter the year as a 4-digit integer.

Month

Default value: N/A
Examples: 9, 09, 12

Integer

Appears when you select Show Advanced Properties checkbox.

Enter the month as an integer.

Date

Default value:  N/A
Examples: 28, 09, 12

Integer

Appears when you select Show Advanced Properties checkbox.

Enter the day of the month.

Zone

Default value:  N/A
Example: US/Pacific

String/Suggestion

Appears on selecting Show Advanced Properties checkbox.

Enter or select a time zone ID from the suggested list. For the UTC time zone, this field may be empty.

Get Object Tags

Default Value: Not selected

Checkbox

Appears when you select Show Advanced Properties checkbox.

Select this property to include object tags in the header of the output binary data. Learn more about object tags.

Enable Staging

Default Value: Deselected

Checkbox

Appears when you select Show Advanced Properties checkbox.

Select this checkbox to download the entire file content into a local temporary file. When the download is completed, the Snap streams the data from the temporary file to the output view.

Snap Execution

Default Value: Validate & Execute
Example: Execute only

Dropdown list

N/A

Select one of the three modes in which the Snap executes. Available options are:

• Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

• Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

• Disabled: Disables the Snap and all Snaps that are downstream from it.