S3 Presigned

Owned by Kalpana Malladi, created
Last updated: Mar 22, 2024 by Gouri Bhagchandani
6 min read

In this article

Overview

You can use this Snap to generate a presigned URL in the output document to access an S3 object. You can use the presigned URL in external applications to read or write S3 objects, for example, Postman, REST Get Snap, and REST Put Snap.

The Snap may produce a presigned URL without any error if Snap properties contain invalid values, and this may cause the external application to fail.

s3-presigned-overview.png

Snap Type

The S3 Presigned Snap is a Read-type Snap.

Prerequisites

None.

Support for Ultra Pipelines

 Works in Ultra Pipelines. 

Limitations and Known Issues

None.

Snap Views

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Input 

Document

  • Min: 0

  • Max: 1

  • Mapper

  • Copy

Output

Document

  • Min: 1

  • Max: 1

  • REST Get

  • REST Put

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

  • Stop Pipeline Execution: Stops the current pipeline execution when the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

  • Asterisk ( * ): Indicates a mandatory field.

  • Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon ( ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon ( ): Indicates that you can add fields in the fieldset.

  • Remove icon ( ): Indicates that you can remove fields from the fieldset.

Field Name

Field Type

Description

Field Name

Field Type

Description

Label*

Default Value: S3 Presigned
Example: GET presigned URL

String

Specify a unique name for the Snap.

 

Bucket*

 

Default Value: None
Examples: 

  • mybucket

  • mybucket@eu-west-1

  • Example of S3 Bucket in S3 region:

    • mybucket@s3.eu-west-1.amazonaws.com

  • Example of S3 VPC endpoint:

    • my-bucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

    • testBucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

String/Expression/Suggestion

Specify the S3 bucket name, from where you need to fetch or post the presigned URL.

Do not add S3:/// before bucket name, because the Snap can fail.

Object Key

Default Value: [None]
Examples: 

  • test.csv

  • abc/test.json

  • abc/xyz/test.xml

String/Expression/Suggestion

Specify or select the S3 object key name, which may include one or more forward-slash '/' characters.

Expire in (seconds)

Default Value: 3600
Example: 2000

Integer/Expression

Specify the time in seconds at which the generated presigned URL will expire.

Min: 1
Max: 604800 (7 days)

HTTP Method

Default Value: GET
Example: PUT

String/Expression

Specify the HTTP method verb to use for the presigned URL. The supported methods are GET and PUT.

Snap Execution

Default Value: 
Example: Validate & Execute

Dropdown list

Select one of the three modes in which the Snap executes. Available options are:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Examples

Generating a Presigned URL using GET Method

This example Pipeline demonstrates how to generate a presigned URL to get an S3 object.

Prerequisites: A valid AWS S3 Account

  1. Configure the S3 Download Snap to download the tmp/hello.json object from the mrtest Bucket.

  2. Validate the Pipeline. The output displays the details of the downloaded object from S3 in binary format.

  3. Next, configure the Mapper Snap to transform the binary content message to a string value.

  4. Configure an S3 Presigned Snap to generate a presigned URL to access the tmp/hello.json object using the GET HTTP Method. The validated output displays the generated pre-signed URL.

  5. Configure the REST Get Snap to send the HTTP Get request (the presigned URL).

  6. Validate the REST GET Snap. The output verifies the creation of the S3 object.

     

Download this pipeline.

Generating a Presigned URL Using PUT Method

This example demonstrates how to generate a presigned URL using HTTP PUT and GET methods.

  1. Configure an S3 Presigned Snap as follows with the PUT HTTP Method.

  2. Validate the Snap—it generates a presignedUrl as highlighted in the following image:

  3. Next, configure the Mapper Snap to pass the message and presignedUrl as inputs to the file that is to be written.

  4. Validate the Maper Snap—it generates the message and the presignedUrl.

  5. Configure the REST Put Snap with $presignedUrl as the Service URL, $content as HTTP Entity, and TEXT as Response entity type. The REST Put Snap creates an S3 object if it does not exist.

  6. Validate the REST Put Snap—it generates the response with the following status and the headers as shown below:

  7. Next, configure the second S3 Presigned Snap with the GET HTTP Method to generate the presigned URL.

  8. Validate the Presigned Snap. The Snap generates the presigned URL with the original input.

  9. Configure the REST Get Snap to verify the creation of the S3 object.

  10. Validate the Snap—it gets the response from the REST API service endpoint and generates the output as shown below:

Download this pipeline.

Downloads

  File Modified

File Example_S3 Presigned_PUT&GET.slp

Jul 22, 2022 by Kalpana Malladi

File Example_S3 Presigned_Get.slp

Jul 22, 2022 by Kalpana Malladi

Snap Pack History

Related Links

 

Have feedback? Email documentation@snaplogic.com | Ask a question in the SnapLogic Community
© 2017-2024 SnapLogic, Inc.