In this article

Overview

You can use this Snap to download S3 objects from the S3 bucket.

Snap Type

S3 Download Snap is a READ-type Snap.

Prerequisites

None.

Support for Ultra Pipelines

Works in Ultra Pipelines.

Limitations

The current Snap functionality supports AWS S3 Cloud Service and will be applicable for AWSGovCloud setup.

Known Issues

None.

Snap Views

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input

Document

Min: 0
Max: 1

Mapper

An upstream Snap is optional. Any document with key-value pairs to evaluate expression properties in the S3 Download Snap. Each input document, if any, results in one download operation of the Snap.

Output

Binary

Min: 1
Max: 1

Mapper
CSV Parser
JSON Parser
XML Parser

Binary data downloaded from AWS S3 specified in the Bucket and Object Key fields with header information about the binary stream.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

Stop Pipeline Execution: Stops the current pipeline execution when the Snap encounters an error.
Discard Error Data and Continue: Ignores the error, discards that record, and continues with the rest of the records.
Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

Asterisk (*): Indicates a mandatory field.
Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.
Expression icon (): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.
Add icon (): Indicates that you can add fields in the fieldset.
Remove icon (): Indicates that you can remove fields from the fieldset.

Field Name		Type	Field Dependency	Description
Label* Default Value: S3 Download Example: S3 Download		String	N/A	Specify a unique name for the Snap.
Bucket* Default Value: [None] Examples: mybucket mybucket@eu-west-1 Example of S3 Bucket in S3 region: mybucket@s3.eu-west-1.amazonaws.com Example of S3 VPC endpoint: my-bucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com testBucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com		String/Expression/Suggestion	N/A	Specify the S3 bucket name, from where an S3 object is to be downloaded. Do not add S3:/// before bucket name, else the Snap can fail. Bucket names are unique globally and can be accessed without its region name in most cases. If you cannot access a bucket name without its region name, you can specify the region information in the following syntax `<S3_bucket_name>@<region_name>` You can also access S3 bucket in a S3 Virtual Private Cloud (VPC) endpoint by specifying the bucket name in the following syntax: `<S3_bucket_name>@<VPC_S3_endpoint>` Note: When you enter an incorrect region name, but if the bucket name is valid, the AWS S3 service may successfully access the bucket without any error.
Object Key Default Value: [None] Examples: test.csv abc/test.json abc/xyz/test.xml		String/Expression/Suggestion	N/A	Specify or select the S3 object key name, which may include one or more forward-slash '/' characters. The forward-slash character is part of the S3 object key name, and there is no folder object defined in AWS S3. The Snap uses the existing Object Key value as a prefix to produce the suggested list. The maximum length of the suggested list is 1,000.
Show Advanced Properties Default Value: Deselected		Checkbox	Displays Advanced Properties* when you select this checkbox.*	Select this checkbox to display the advanced properties. Deselect the checkbox to hide the properties.
Thread Pool Size Default Value: 10		Integer/Expression	Appears on selecting Show Advanced Properties* checkbox.*	Specify the maximum number of threads to use to download multiple S3 objects in parallel.
Maximum Retries* Default Value: 3 Example: 5		Integer/Expression	Appears on selecting Show Advanced Properties* checkbox.*	Specify the maximum number of retry attempts to to perform in case of a temporary network loss.
Version ID Default Value: N/A Example: xvcnB8gPi37l3hbOzlsRFxjVwQ.numQz		String/Expression/Suggestion	Appears on selecting Show Advanced Properties* checkbox.*	Specify or select the version ID of the S3 file object. If you leave this field empty, the Snap downloads the latest version. You can use the Suggestion icon to view the list of version IDs for the S3 object in the Object Key field. Each line in the suggested list also includes the last modified date and the file size to help select a version. If the versioning is not enabled in S3 bucket, no version ID is suggested. When you specify a static value in this field, you must enter only the version ID. The Snap ignores the last modified date and size information of a version when it downloads the version. The versions for the following cases are ignored in the suggested list because you cannot download them: If an S3 object had existed before enabling the versioning; hence, its version does not have any version ID assigned to it. Version ID's with 'Deleted Marker' resource type.
Version ID Suggestion Interval	Use this field set to configure the time interval for version ID suggestion. Enter two separate rows to enter a start date and an end date. If you provide only one row, the suggestion interval will be considered from the specified data until the current date. If you leave this field empty, the Snap suggests all version IDs. This may be helpful when a given S3 file has many versions. This property is used only for the Version ID suggestion, not during the Snap preview or execution.
	Year Default value: N/A Example: 2017	Integer	Appears on selecting Show Advanced Properties* checkbox.*	Enter the year as a 4-digit integer.
	Month Default value: N/A Examples: 9, 09, 12	Integer	Appears on selecting Show Advanced Properties* checkbox.*	Enter the month as an integer.
	Date Default value: N/A Examples: 28, 09, 12	Integer	Appears on selecting Show Advanced Properties* checkbox.*	Enter the day of the month.
	Zone Default value: N/A Example: US/Pacific	String/Suggestion	Appears on selecting Show Advanced Properties* checkbox.*	Enter or select a time zone ID from the suggested list. For UTC time zone, this field may be empty. Only zone IDs are supported in the suggested list.
Get Object Tags Default Value: Not selected		Checkbox	Appears on selecting Show Advanced Properties* checkbox.*	Select this property to include object tags in the header of the output binary data. Learn more about object tags. You must have the `S3:GetObjectTagging` permission to be able to use this feature.
Snap Execution Default Value: Validate & Execute Example: Execute only		Dropdown list	N/A	Select one of the three modes in which the Snap executes. Available options are: Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime. Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data. Disabled: Disables the Snap and all Snaps that are downstream from it.

Example

Refer Managing-Data-in-S3.

Downloads

Download and import the Pipeline into SnapLogic.
Configure Snap accounts as applicable.
Provide Pipeline parameters as applicable.

Error rendering macro 'excerpt-include' : No link could be created for 'PREVIEW | Amazon S3 Snap Pack'.

PREVIEW | S3 Download