Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Support for Ultra Pipelines

Works in Ultra Pipelines

Limitations

...

None.

and Known Issues

None.

Snap Views

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input 

Document

  • Min: 0

  • Max: 1

  • Mapper

  • JSON Parser

An upstream Snap is optional. Any document with key-value pairs to evaluate expression properties.

Output

Document

  • Min: 0

  • Max: 1

  • Mapper

  • JSON Parser

When Check Status Until Completed is selected in Settings:

Preview data in a tableImage Modified

When Check Status Until Completed is not selected in Settings:

Preview data in a tableImage Modified

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

  • Stop Pipeline Execution: Stops the current pipeline execution if the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

...

Field Name

Field Type

Field Dependency

Description

Label*

Default ValueS3 Restore
ExampleRestore Invoice - March 2018

String

None

The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your Pipeline.

Bucket*

 

Default Value: None
Example:

  • mybucket

  • mybucket@eu-west-1

  • Example of S3 Bucket in S3 region:

    • mybucket@s3.eu-west-1.amazonaws.com

  • Example of S3 VPC endpoint:

    • my-bucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

    • testBucket@bucket.vpce-028b7814794578709-vu0vvauy.s3.us-west-2.vpce.amazonaws.com

String/Expression/Suggestion

None

 

 

Specify the S3 bucket name where the S3 object to be restored is located.

Note

Do not add S3:/// before the bucket name, because the Snap can fail.

  • Bucket names are unique globally and can be accessed without the region name in most cases. If you cannot access a bucket name without its region name, you can specify the region information using the following syntax:

    • <S3_bucket_name>@<region_name>

  • You can also access the S3 bucket in an S3 Virtual Private Cloud (VPC) endpoint by specifying the bucket name using the following syntax:

    • <S3_bucket_name>@<VPC_S3_endpoint>

Note: If you enter an incorrect region name, but the bucket name is valid, the AWS S3 service may successfully access the bucket without any errors.

Object Key*

Default Value: None
Examples

  • test.csv

  • abc/test.json

  • abc/xyz/test.xml

String/Expression/Suggestion

None

Specify the S3 object key name, which may include one or more forward-slash ('/') characters.

The S3 object should have been archived by changing its storage class to GlacierFlexibleRetrieval or DeepArchive before executing the S3 Restore Snap.

The forward-slash character is part of the S3 object key name and there is no folder object defined in AWS S3. The maximum length of the suggested list is 1,000.

Expiration (Days)

Default Value: 1

Minimum Value: 1
Example: 2

Integer

Expression

None

Enter the lifetime of the active copy of the S3 object in days.

If an S3 object's restoring process is completed, the content of the S3 object can be accessed until its active copy expires.

Retrieval Tier

Default Value: None
Examples:

  • Bulk

  • Standard

  • Expedited

Dropdown list

None

Choose one of the available S3 Glacier retrieval tiers: Bulk, Standard, or Expedited.

Check Status Until Completed

Default Value: Deselected

ExamplesSelected

Checkbox

None

Select this checkbox to poll the job status of restoring an archived S3 object until it is completed.

The restore process may take 5 to 12 hours. When this checkbox is selected, the Snap checks the S3 object restore status continuously in a given interval until the restore is completed.

Check Status Interval (Minutes)

Default Value: 10

Minimum Value: 1
Example30

Integer/Expression

Appears when you select the Check Status Until Completed checkbox.

Enter the minimum amount of time in minutes to pause between the polling of the job status of restoring an archived S3 object.

Show Advanced Properties

Default Value: Deselected
Example: Selected

Checkbox

None

Select this checkbox to display the advanced properties.
Deselect this checkbox to hide the properties.

Thread Pool Size

Default Value: 10
Example: 20

Integer/Expression

Appears when you select the Show Advanced Properties checkbox.

Specify the maximum number of threads to be used to restore multiple S3 objects in parallel with Bucket and Object Key properties expression-enabled.

If the Thread Pool Size property is expression-enabled, it can be evaluated with a Pipeline parameter, but not with an input document. Therefore, the value of this property remains constant throughout the Pipeline execution.

Maximum Retries*

Default Value: 3
Example: 2

Integer/Expression

Appears when you select the Show Advanced Properties checkbox.

Specify the maximum number of retry attempts to perform in case of a temporary network loss.

Pass Through

Default Value: Deselected
Example: Selected

Checkbox

Appears when you select the Show Advanced Properties checkbox.

Select this checkbox to pass the input document to the output view as a value to the 'original' key .

Snap Execution

Default ValueExecute only
Example: Validate & Execute

Dropdown list

None

Select one of the following three modes in which the Snap executes:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

...