In this article

Overview

You can use this Snap to archive an S3 object in the S3 bucket. Apart Aside from archiving S3 objects, you can also change the storage class from the current class to another if the transition meets the requirements of AWS S3.

...

Snap Type

The S3 Archive Snap is a Write-type Snap.

Prerequisites

• A valid account with the required permissions.
  [I think this is required. However, Paul listed None.]

Support for Ultra Pipelines

• Works in Ultra Pipelines. 

Limitations

None.

Known Issues

None.

Snap Views

Type	Format	Number of Views	Examples of Upstream and Downstream Snaps	Description
Input	Document

Binary

Binary or Document

Min: 0 Max: 1

Mapper S3 Upload

Copy

..

Requires the EDI data and internal ID as a JSON document

JSON Formatter	An upstream Snap is optional. Any document with key-value pairs to evaluate expression properties. Each input document, if any, results in changing the storage class of an S3 object.
Output	Document

Binary

Binary or Document

Min: 1 Max:

• ..

• ..

The EDI transaction ID and order confirmation.

1	Mapper S3 Restore S3 Download	Image Added
Error	Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab: Stop Pipeline Execution: Stops the current pipeline execution if the Snap encounters an error. Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records. Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution. Learn more about Error handling in Pipelines.

Snap Settings

...

{"features":["title","icon","rounded","collapsible","border"],"title":"Documenting Fields Based On Data Type/UI Element","titleColor":"#000000","titleColorBG":"#ff5c30","titleSize":"14","titleBold":true,"titleItalic":false,"titleUnderline":false,"lozenge":"Hot stuff","lozengeColor":"#172B4D","lozengeColorBG":"#fff0b3","height":200,"panelPadding":12,"panelColor":"#172B4D","panelColorBG":"#FFFFFF","borderColor":"#ff5c30","borderRadius":3,"borderStyle":"solid","borderWidth":1,"icon":"editor/","iconPrimary":"#FFFFFF","iconSecondary":"#0052CC","newMacro":false}

**Delete Before Publishing**

Choose from the following sentences to document specific field types.

Drop-down lists/Option Buttons (radio buttons):

You must list the LoV and describe them if their meaning isn't apparent. In this case, format the LoV in italics, regular font for the LoV's description. In either case, list the LoVs as a bullet list.

• <State what the option should do in this field>. The available options are: <bullet list of LoVs>
  Specify the Salesforce API to be used. The available options are:...

• Select the <category> that you want to use. Available options are...
  * Option 1<italicized>. <third person singular form of the verb>
  * Option 2<italicized>. <third person singular form of the verb>
  Select the API that you want to use. Available options are:
  Bulk API. Sends the Snap execution request details as a bulk API call.
  REST API. ...
  OR
  Select one of the three following modes in which the Snap executes:
  * Validate & Execute. Performs limited execution of the Snap and generates a data preview during Pipeline validation, then performs full execution of the Snap (unlimited records) during Pipeline runtime.
  * Execute only. Performs full execution of the Snap during Pipeline execution without generating preview data.
  * Disabled. Disables the Snap and all Snaps downstream from it.
  

Check boxes:

• If selected, <Snap behavior>.
  If selected, an empty file is written when the incoming document has no data.

• If selected, <behavior>. If not selected/Otherwise, <behavior>
  Use "If not selected" if the first sentence is long.
  If selected, the Snap uses the file path value as is. Otherwise, the Snap uses the file path value in the URL.
  If selected, an empty file is written when the incoming document has empty data. If there is no incoming document at the input view of the Snap, no file is written regardless of the value of the property.

• Select to <action>
  Use this if the behavior is binary. Either this or that, where the converse behavior is apparent/obvious.
  Select to execute the Pipeline during validation.

Text Fields

• Describe what the user shall specify in this field. Additional details, as applicable, in a separate sentence. Include caveats such as the field being conditionally mandatory, limitations, etc.
  Enter the name for new account.
  Specify the account ID to use to log in to the endpoint.
  Required if IAM Role is selected.
  Do not use this field if you are using batch processing.
  

Numeric Text Fields

• Describe what the field represents/contains. Additional details, as applicable, in a separate sentence. Include caveats such as the field being conditionally mandatory, limitations, etc. Include special values that impact the field's behavior as a bullet list.
  The number of records in a batch.
  The number of seconds for which you want the Snap to wait between retries.
  The number of seconds for which the Snap waits between retries.
  Use the following special values:
  * 0: Disables batching.
  * 1: Includes all documents in a single request.

Notes in field descriptions

• Confluence’s new editor does not allow nesting of most macros inside another macro, especially the Note/Alert/Warning/Info (Panel) macros inside a table macro and Excerpt macros inside Expand or Panel Macro+ macros. So, as a workaround use the Footnotes approach as mentioned below:

  • Assign numbers at the Note locations in the form of follow through phrases like See Note 2 below this table. or such.

  • Add your Notes---an appropriate Note/Alert/Warning/Info (Panel) macro---immediately below the macro (for example, Table macro) beginning the content with the corresponding number assigned.

Info
Info
Asterisk ( * ): Indicates a mandatory field. Suggestion icon (): Indicates a list that is dynamically populated based on the configuration. Expression icon ( ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic. Add icon ( ): Indicates that you can add fields in the fieldset. Remove icon ( ): Indicates that you can remove fields from the fieldset. Upload icon ( ): Indicates that you can upload files. //is this missing by design?

Field Name		Field Type	Field Dependency	Description
Label*

 

Default Value: S3 Archive Example:

 Project

Project Archive

String

N/A

None	Specify a unique name for the Snap.

 

Label*

 

Default Value: S3 Archive
Example: Project Archive

You can modify this to be more specific, especially if you have more than one of the same Snap in your Pipeline.
Bucket*   Default Value: None Example: mybucket mybucket@eu-west-1 Example of S3 Bucket in S3 region: mybucket@s3.eu-west-1.amazonaws.com Example of S3 VPC endpoint: my-bucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com testBucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com		String/Expression/Suggestion	None

N/A

Specify the S3 bucket name where the S3 object to be archived is located. Note Do not add S3:/// before the bucket name, because the Snap can fail. Bucket names are unique globally and can be accessed without

its

the region name in most cases. If you cannot access a bucket name without its region name, you can specify the region information

in

using the following syntax: <S3_bucket_name>@<region_name> You can also access the S3 bucket in an S3 Virtual Private Cloud (VPC) endpoint by specifying the bucket name

in

using the following syntax: <S3_bucket_name>@<VPC_S3_endpoint> Note:

When

If you enter an incorrect region name, but

if

the bucket name is valid, the AWS S3 service may successfully access the bucket without any

error.

 

Bucket*

 

Default Value: None
Example:

• mybucket

• mybucket@eu-west-1

• Example of S3 Bucket in S3 region:

  • mybucket@s3.eu-west-1.amazonaws.com

• Example of S3 VPC endpoint:

  • my-bucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

  • testBucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

errors.
Object Key*   Default Value: None Examples:  test.csv abc/test.json abc/xyz/test.xml		String/Expression/Suggestion

N/A

None	Specify the S3 object key name, which may include one or more forward-slash ('/') characters. The forward-slash character is part of the S3 object key name and there is no folder object defined in AWS S3. The maximum length of the suggested list is 1,000.

Object Key*

 

Default Value: None
Examples: 

• test.csv

• abc/test.json

• abc/xyz/test.xml

Change Storage Class

Change Storage Class

Composite

N/A

Use this field set to change the storage class of S3 objects.

From Default Value: None Examples:

Standard

String/

Expression/

Suggestion

N/A

None	This field is for information only and not used when the Snap is previewed or executed. The current storage class of the selected S3 object can selected from the suggestion.

From

Default Value: None
Examples: 

• Standard

To Default Value: None Examples:  GlacierInstantRetrieval GlacierFlexibleRetrieval DeepArchive

String/Expression/Suggestion

N/A

None	Enter or select the target storage class name from the suggested list. Supported storage classes are: Standard GlacierInstantRetrieval GlacierFlexibleRetrieval DeepArchive IntelligentTiering StandardInfrequentAccess OneZoneInfrequentAccess ReducedRedundancy

To

Default Value: None
Examples: 

• GlacierInstantRetrieval

• GlacierFlexibleRetrieval

• DeepArchive

Show Advanced Properties   Default Value: Deselected Example: Selected

Checkbox

N/A

None	Select this checkbox to display the advanced properties. Deselect this checkbox to hide the properties.

Show Advanced Properties

 

Default Value: Deselected
Example: Selected

Thread Pool Size   Default Value: 10 Example: 20

Integer/Expression

Appears when you select the Show Advanced Properties checkbox.

Specify the maximum number of threads to be used to archive multiple S3 objects in parallel with Bucket and Object Key properties expression-enabled. If the Thread Pool Size property is expression-enabled, it can be evaluated with a Pipeline parameter, but not with an input document. Therefore, the value of this property remains constant throughout the Pipeline execution.

Thread Pool Size

 

Default Value: 10Example: 20

Maximum Retries*   Default Value: 3 Example: 2

Integer/Expression

Appears when you select the Show Advanced Properties checkbox.

Specify the maximum number of retry attempts to perform in case of a temporary network loss.

 

Maximum Retries*

Default Value: 3
Example: 2

Pass Through   Default Value: Deselected Example: Selected

Checkbox

Appears when you select the Show Advanced Properties checkbox.

Select this checkbox to pass the input document to the output view as a value to the 'original' key .

Pass Through

 

Default Value: Deselected
Example: Selected

Snap Execution Default Value: Execute only Example: Validate & Execute		Dropdown list

N/A

None	Select one of the following three modes in which the Snap executes

. Available options are

: Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime. Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data. Disabled: Disables the Snap and all Snaps that are downstream from it.

Snap Execution

Default Value: 
Example: Validate & Execute

Number of records

Default Value: 
Example:

String/Expression

Sampling Type is Number of records.

Enter the number of records to output.

Field set Name

Specify advanced parameters that you want to include in the request.

Field 1*

Default Value: <value> or None.
Example: <value>

String

Debug mode check box is not selected.

Field 2

Default Value: <value> or None.
Example: <value>

String

None.

Snap Execution

Default Value: 
Example: Validate & Execute

Dropdown list

N/A

Select one of the following three modes in which the Snap executes:

• Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

• Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

• Disabled: Disables the Snap and all Snaps that are downstream from it.

Troubleshooting

...

Error

...

Reason

...

Resolution

...

Account validation failed.

...

The Pipeline ended before the batch could complete execution due to a connection error.

...

Verify that the Refresh token field is configured to handle the inputs properly. If you are not sure when the input data is available, configure this field as zero to keep the connection always open.

Examples

Excluding Fields from the Input Data Stream

We can exclude the unrequired fields from the input data stream by omitting them in the Input schema field set. This example demonstrates how we can use the <Snap Name> to achieve this result:

...

Examples

Browsing and Archiving Amazon S3 Objects

This example Pipeline demonstrates how to browse an S3 object and then archive it.

...

Prerequisites: A valid AWS S3 Account

1. Configure the S3 Browser Snap to browse the test/test100.json object from the mrtest bucket.
   

   Image Added

2. Validate the Pipeline. The output displays the details of the browsed S3 object. Note that the storage class (Standard) is eligible for archiving.
   

   Image Added

3. Next, configure the S3 Archive Snap to archive the S3 object. In this example we are changing the storage class from Standard to GlacierFlexibleRetrieval.
   

   Image Added

4. Validate the Pipeline. The output displays the details of the archived S3 object. You can verify that the storage class is changed.
   

   Image Added

Download this Pipeline. 

Downloads

Snap Pack History

Related Links

...

Endpoint Doc Link 1

...

Endpoint Doc Link 2

...

Endpoint Doc Link 3

...

SnapLogic Glossary

...

Getting Started with SnapLogic

...

...