In this article
Overview
You can use this Snap to archive an S3 object in the S3 bucket. Aside from archiving S3 objects, you can also change the storage class from the current class to another if the transition meets the requirements of AWS S3.
AWS S3 does not define any folder object and the '/' characters are part of the S3 object key names.
Snap Type
The S3 Archive Snap is a Write-type Snap.
Prerequisites
A valid account with the required permissions.
Support for Ultra Pipelines
Works in Ultra Pipelines.
Limitations
None.
Known Issues
None.
Snap Views
Type | Format | Number of Views | Examples of Upstream and Downstream Snaps | Description |
|---|---|---|---|---|
Input | Document |
|
| An upstream Snap is optional. Any document with key-value pairs to evaluate expression properties. Each input document, if any, results in changing the storage class of an S3 object. |
Output | Document |
|
|  |
Error | Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:
Learn more about Error handling in Pipelines. | |||
Snap Settings
Asterisk ( * ): Indicates a mandatory field.
Suggestion icon (
): Indicates a list that is dynamically populated based on the configuration.Expression icon (
): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.Add icon (
): Indicates that you can add fields in the fieldset.Remove icon (
): Indicates that you can remove fields from the fieldset.Upload icon (
): Indicates that you can upload files.
Field Name | Field Type | Field Dependency | Description | |
|---|---|---|---|---|
Label*
Default Value: S3 Archive | String | None
| Specify a unique name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your Pipeline. | |
Bucket*
Default Value: None
| String/Expression/Suggestion | None
| Specify the S3 bucket name where the S3 object to be archived is located. Do not add S3:/// before the bucket name, because the Snap can fail.
Note: If you enter an incorrect region name, but the bucket name is valid, the AWS S3 service may successfully access the bucket without any errors. | |
Object Key*
Default Value: None
| String/Expression/Suggestion | None | Specify the S3 object key name, which may include one or more forward-slash ('/') characters. The forward-slash character is part of the S3 object key name and there is no folder object defined in AWS S3. The maximum length of the suggested list is 1,000.
| |
Change Storage Class
| Use this field set to change the storage class of S3 objects. | |||
From Default Value: None | String/Suggestion | None | This field is for information only and not used when the Snap is previewed or executed. The current storage class of the selected S3 object can selected from the suggestion. | |
To Default Value: None
| String/Expression/Suggestion | None | Enter or select the target storage class name from the suggested list. Supported storage classes are:
| |
Show Advanced Properties
Default Value: Deselected | Checkbox | None | Select this checkbox to display the advanced properties. | |
Thread Pool Size
Default Value: 10 | Integer/Expression | Appears when you select the Show Advanced Properties checkbox. | Specify the maximum number of threads to be used to archive multiple S3 objects in parallel with Bucket and Object Key properties expression-enabled. If the Thread Pool Size property is expression-enabled, it can be evaluated with a Pipeline parameter, but not with an input document. Therefore, the value of this property remains constant throughout the Pipeline execution. | |
Maximum Retries*
Default Value: 3 | Integer/Expression | Appears when you select the Show Advanced Properties checkbox. | Specify the maximum number of retry attempts to perform in case of a temporary network loss.
| |
Pass Through
Default Value: Deselected | Checkbox | Appears when you select the Show Advanced Properties checkbox. | Select this checkbox to pass the input document to the output view as a value to the ' | |
Snap Execution Default Value: Execute only | Dropdown list | None | Select one of the following three modes in which the Snap executes:
| |
Examples
Browsing and Archiving Amazon S3 Objects
This example Pipeline demonstrates how to browse an S3 object and then archive it.
Prerequisites: A valid AWS S3 Account
Configure the S3 Browser Snap to browse the test/test100.json object from the mrtest bucket.
Validate the Pipeline. The output displays the details of the browsed S3 object. Note that the storage class (Standard) is eligible for archiving.
Next, configure the S3 Archive Snap to archive the S3 object. In this example we are changing the storage class from Standard to GlacierFlexibleRetrieval.
Validate the Pipeline. The output displays the details of the archived S3 object. You can verify that the storage class is changed.
Downloads
Download and import the Pipeline into SnapLogic.
Configure Snap accounts, as applicable.
Provide Pipeline parameters, as applicable.
Snap Pack History
Release | Snap Pack Version | Date | Type | Updates |
|---|---|---|---|---|
November 2024 | 439patches29535 |
| Latest | Fixed the account validation issue in the Amazon S3 Account. Previously, the account validation failed even when Snap execution was successful within the same account. The fix includes an enhancement, introducing a new Bucket for account validation field that allows you to specify a bucket name for validating the account. |
January 2025 | 439patches29484 |
| Latest | Enhanced the S3 Account to access AWS S3 successfully in the ECS environment with IAM roles. Learn more. |
November 2024 | main29029 |
| Stable | Updated and certified against the current Snaplogic Platform release. |
August 2024 | 438patches28445 |
| Latest | Enhanced the S3 Download Snap with an Enable Staging checkbox that enables you to download an S3 object into a local temporary file. This enhancement addresses the cases where certain downstream Snaps take longer to process large volumes of data, potentially causing a connection reset error and pipeline failure. |
August 2024 | main27765 |
| Stable | Updated and certified against the current Snaplogic Platform release. |
May 2024 | 437patches26643 |
| Latest | Fixed an issue with the S3 Browser Snap that could not initialize the output document properly, causing an error in the downstream Snaps. |
May 2024 | main26341 |
| Stable | Enhanced the S3 Select Snap to capture metadata and lineage information from the input document. |
February 2024 | 436patches25360 |
| Latest | Fixed an issue with the Amazon S3 Snaps that displayed a |
February 2024 | main25112 |
| Stable | Updated and certified against the current Snaplogic Platform release. |
November 2023 | 435patches24238 |
| Latest | Added support for Amazon S3 Express One Zone in the Amazon S3 Snap Pack. |
November 2023 | main23721 |
| Stable | Updated and certified against the current Snaplogic Platform release. |
August 2023 | main22460 |
| Stable | Updated and certified against the current SnapLogic Platform release. |
May 2023 | 433patches21816 |
| Latest | The Amazon S3 Snaps automatically detect the Maximum session duration value for the Cross-Account IAM role (1 through 12 hours). The Snaps round down the value to the nearest hour. So, if the Snap administrator sets the Maximum session duration at 3 hours and 45 minutes, the Snaps read it as 3 hours. The Snaps also refresh the session before it expires. However, the automatic session refresh does not support the case of very large file upload or download that takes longer than the maximum session duration. |
May 2023 | main21015 |
| Stable | Upgraded with the latest SnapLogic Platform release. |
February 2023 | 432patches20385 |
| Latest | Added support for Ultra Task Pipelines. |
February 2023 | main19844 |
| Stable | Upgraded with the latest SnapLogic Platform release. |
November 2022 | main18944 |
| Stable |
|
October 2022 | 430patches18674 |
| Latest |
|
August 2022 | 430patches17354 |
| Latest | The KMS Region field in the S3 Account now suggests the regions when you click the suggestion |
August 2022 | main17386 |
| Stable | Introduced the Amazon S3 Snap Pack, which enables you to browse, copy, delete, download, or upload objects in S3. This Snap Pack contains the following Snaps:
|