In this article
Overview
You can use this Snap to download S3 objects from the S3 bucket.
Snap Type
The S3 Download Snap is a Read-type Snap.
Prerequisites
None.
Support for Ultra Pipelines
Works in Ultra Pipelines.
Limitations
The current Snap functionality supports the AWS S3 Cloud Service and is applicable for the AWSGovCloud setup.
Known Issues
None.
Snap Views
Type | Format | Number of Views | Examples of Upstream and Downstream Snaps | Description |
---|---|---|---|---|
Input | Document |
|
| An upstream Snap is optional. Any document with key-value pairs to evaluate expression properties. Each input document, if any, results in one download operation of the Snap. |
Output | Binary |
|
| Binary data downloaded from AWS S3 is specified in the Bucket and Object Key fields with header information about the binary stream. |
Error | Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:
Learn more about Error handling in Pipelines. |
Snap Settings
Asterisk (*): Indicates a mandatory field.
Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.
Expression icon (): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.
Add icon (): Indicates that you can add fields in the fieldset.
Remove icon (): Indicates that you can remove fields from the fieldset.
Field Name | Type | Field Dependency | Description | |
---|---|---|---|---|
Label* Default Value: S3 Download | String | N/A | Specify a unique name for the Snap. | |
Bucket* Default Value: None
| String/Expression/Suggestion | N/A | Specify the S3 bucket name, from where an S3 object is to be downloaded. Do not add S3:/// before bucket name, because the Snap can fail.
Note: When you enter an incorrect region name, but if the bucket name is valid, the AWS S3 service may successfully access the bucket without any error. | |
Object Key Default Value: None
| String/Expression/Suggestion | N/A | Specify or select the S3 object key name, which may include one or more forward-slash '/' characters. The forward-slash character is part of the S3 object key name, and there is no folder object defined in AWS S3. The Snap uses the existing Object Key value as a prefix to produce the suggested list. The maximum length of the suggested list is 1,000. | |
Show Advanced Properties Default Value: Deselected | Checkbox | N/A | Select this checkbox to display the advanced properties. | |
Thread Pool Size Default Value: 10 | Integer/Expression | Appears when you select Show Advanced Properties checkbox. | Specify the maximum number of threads to use to download multiple S3 objects in parallel. | |
Maximum Retries* Default Value: 3 | Integer/Expression | Appears when you select Show Advanced Properties checkbox. | Specify the maximum number of retry attempts to perform in case of a temporary network loss. | |
Version ID Default Value: N/A | String/Expression/Suggestion | Appears when you select Show Advanced Properties checkbox. | Specify or select the version ID of the S3 file object. If you leave this field empty, the Snap downloads the latest version. You can use the Suggestion icon to view the list of version IDs for the S3 object in the Object Key field. Each line in the suggested list also includes the last modified date and the file size to help you select a version. If the versioning is not enabled in the S3 bucket, no version ID is suggested. When you specify a static value in this field, you must enter only the version ID. The Snap ignores the last modified date and size information of a version when it downloads the version. The versions for the following cases are ignored in the suggested list because you cannot download them:
| |
Version ID Suggestion Interval | Use this fieldset to configure the time interval for the version ID suggestion.
| |||
Year Default value: N/A | Integer | Appears when you select Show Advanced Properties checkbox. | Enter the year as a 4-digit integer. | |
Month Default value: N/A | Integer | Appears when you select Show Advanced Properties checkbox. | Enter the month as an integer. | |
Date Default value: N/A | Integer | Appears when you select Show Advanced Properties checkbox. | Enter the day of the month. | |
Zone Default value: N/A | String/Suggestion | Appears on selecting Show Advanced Properties checkbox. | Enter or select a time zone ID from the suggested list. For the UTC time zone, this field may be empty. Only zone IDs are supported in the suggested list. | |
Get Object Tags Default Value: Not selected | Checkbox | Appears when you select Show Advanced Properties checkbox. | Select this property to include object tags in the header of the output binary data. Learn more about object tags. You must have the | |
Snap Execution Default Value: Validate & Execute | Dropdown list | N/A | Select one of the three modes in which the Snap executes. Available options are:
|
Example
Refer Managing-Data-in-S3.
Downloads
Download and import the Pipeline into SnapLogic.
Configure Snap accounts as applicable.
Provide Pipeline parameters as applicable.
Release | Snap Pack Version | Date | Type | Updates |
---|---|---|---|---|
August 2024 | main27765 |
| Stable | Updated and certified against the current Snaplogic Platform release. |
May 2024 | 437patches26643 |
| Latest | Fixed an issue with the S3 Browser Snap that could not initialize the output document properly, causing an error in the downstream Snaps. |
May 2024 | main26341 |
| Stable | Enhanced the S3 Select Snap to capture metadata and lineage information from the input document. |
February 2024 | 436patches25360 |
| Latest | Fixed an issue with the Amazon S3 Snaps that displayed a |
February 2024 | main25112 |
| Stable | Updated and certified against the current Snaplogic Platform release. |
November 2023 | 435patches24238 |
| Latest | Added support for Amazon S3 Express One Zone in the Amazon S3 Snap Pack. |
November 2023 | main23721 |
| Stable | Updated and certified against the current Snaplogic Platform release. |
August 2023 | main22460 |
| Stable | Updated and certified against the current SnapLogic Platform release. |
May 2023 | 433patches21816 |
| Latest | The Amazon S3 Snaps automatically detect the Maximum session duration value for the Cross-Account IAM role (1 through 12 hours). The Snaps round down the value to the nearest hour. So, if the Snap administrator sets the Maximum session duration at 3 hours and 45 minutes, the Snaps read it as 3 hours. The Snaps also refresh the session before it expires. However, the automatic session refresh does not support the case of very large file upload or download that takes longer than the maximum session duration. |
May 2023 | main21015 |
| Stable | Upgraded with the latest SnapLogic Platform release. |
February 2023 | 432patches20385 |
| Latest | Added support for Ultra Task Pipelines. |
February 2023 | main19844 |
| Stable | Upgraded with the latest SnapLogic Platform release. |
November 2022 | main18944 |
| Stable |
|
October 2022 | 430patches18674 |
| Latest |
|
August 2022 | 430patches17354 |
| Latest | The KMS Region field in the S3 Account now suggests the regions when you click the suggestion icon. |
August 2022 | main17386 |
| Stable | Introduced the Amazon S3 Snap Pack, which enables you to browse, copy, delete, download, or upload objects in S3. This Snap Pack contains the following Snaps:
|
Related Links
https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1439233/Glossary
https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1438341/Getting+Started