S3 Restore

In this article

Overview

You can use this Snap to restore an archived S3 object in an S3 bucket.

S3 Restore Snap Settings

Snap Type

The S3 Restore Snap is a Write-type Snap.

Prerequisites

  • A valid account with the required permissions.

  • The S3 object should have been archived by changing its storage class to GlacierFlexibleRetrieval or DeepArchive before executing the S3 Restore Snap.

Support for Ultra Pipelines

Works in Ultra Pipelines. 

Limitations and Known Issues

None.

Snap Views

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input 

Document

 

  • Min: 0

  • Max: 1

  • Mapper

  • JSON Parser

An upstream Snap is optional. Any document with key-value pairs to evaluate expression properties.

Output

Document

 

  • Min: 0

  • Max: 1

  • Mapper

  • JSON Parser

When Check Status Until Completed is selected in Settings:

Preview data in a table

When Check Status Until Completed is not selected in Settings:

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

  • Stop Pipeline Execution: Stops the current pipeline execution if the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

  • Asterisk ( * ): Indicates a mandatory field.

  • Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon ( ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon ( ): Indicates that you can add fields in the fieldset.

  • Remove icon ( ): Indicates that you can remove fields from the fieldset.

  • Upload icon ( ): Indicates that you can upload files.

Field Name

Field Type

Field Dependency

Description

Field Name

Field Type

Field Dependency

Description

Label*

 

Default Value: S3 Restore
Example: Restore Invoice - March 2018

String

None

The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your Pipeline.

 

Bucket*

 

Default Value: None
Example:

  • mybucket

  • mybucket@eu-west-1

  • S3 Bucket in S3 region:

    • mybucket@s3.eu-west-1.amazonaws.com

  • S3 VPC endpoint:

    • my-bucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

    • testBucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

String/Expression/Suggestion

None

 

 

Specify the S3 bucket name where the S3 object to be restored is located.

Do not add S3:/// before the bucket name, because the Snap can fail.

  • Bucket names are unique globally and can be accessed without the region name in most cases. If you cannot access a bucket name without its region name, you can specify the region information using the following syntax:

    • <S3_bucket_name>@<region_name>

  • You can also access the S3 bucket in an S3 Virtual Private Cloud (VPC) endpoint by specifying the bucket name using the following syntax:

    • <S3_bucket_name>@<VPC_S3_endpoint>

Note: If you enter an incorrect region name, but the bucket name is valid, the AWS S3 service may successfully access the bucket without any errors.

Object Key*

Default Value: None
Examples: 

  • test.csv

  • abc/test.json

  • abc/xyz/test.xml

String/Expression/Suggestion

None

Specify the S3 object key name, which may include one or more forward-slash ('/') characters.

The S3 object should have been archived by changing its storage class to GlacierFlexibleRetrieval or DeepArchive before executing the S3 Restore Snap.

Expiration (Days)

Default Value: 1

Minimum Value: 1
Example: 2

Integer

Expression

None

Enter the lifetime of the active copy of the S3 object in days.

If an S3 object's restoring process is completed, the content of the S3 object can be accessed until its active copy expires.

Retrieval Tier

Default Value: None
Examples:

  • Bulk

  • Standard

  • Expedited

Dropdown list

None

Choose one of the available S3 Glacier retrieval tiers: Bulk, Standard, or Expedited.

Check Status Until Completed

Default Value: Deselected

Examples: Selected

Checkbox

None

Select this checkbox to poll the job status of restoring an archived S3 object until it is completed.

Check Status Interval (Minutes)

Default Value: 10

Minimum Value: 1
Example: 30

Integer/Expression

Appears when you select the Check Status Until Completed checkbox.

Enter the minimum amount of time in minutes to pause between the polling of the job status of restoring an archived S3 object.

 

Show Advanced Properties

Default Value: Deselected
Example: Selected

Checkbox

None

Select this checkbox to display the advanced properties.
Deselect this checkbox to hide the properties.

Thread Pool Size

Default Value: 10
Example: 20

Integer/Expression

Appears when you select the Show Advanced Properties checkbox.

Specify the maximum number of threads to be used to restore multiple S3 objects in parallel with Bucket and Object Key properties expression-enabled.

Maximum Retries*

Default Value: 3
Example: 2

Integer/Expression

Appears when you select the Show Advanced Properties checkbox.

Specify the maximum number of retry attempts to perform in case of a temporary network loss.

Pass Through

Default Value: Deselected
Example: Selected

Checkbox

Appears when you select the Show Advanced Properties checkbox.

Select this checkbox to pass the input document to the output view as a value to the 'original' key .

Snap Execution

Default Value: Execute only
Example: Validate & Execute

Dropdown list

None

Select one of the following three modes in which the Snap executes:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Examples

Browsing and Restoring Amazon S3 Objects

This example Pipeline demonstrates how to browse an S3 object and then restore it.

Prerequisites: A valid AWS S3 Account

  1. Configure the S3 Browser Snap to browse the mrtest/test100.json object from the mrtest bucket.

     

  2. Validate the Pipeline. The output displays the details of the browsed S3 object. Note that the storage class (GlacierFlexibleRetrieval) is eligible for restoration.

     

  3. Next, configure the S3 Restore Snap to restore the S3 object, which is currently archived.

     

  4. Validate the Pipeline. The output displays the details of the restored S3 object, including the restoration status of the object (Restore requested or Restore completed).

    When Check Status Until Completed is not selected:



    When Check Status Until Completed is selected:



Download this Pipeline. 

Downloads

  File Modified

File S3 Restore Example.slp

Nov 21, 2022 by Cindy Hall

Snap Pack History


Related Links

Â