Skip to end of banner
Go to start of banner

SQS Acknowledge

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 14 Current »

On this Page

Snap Type:Write
Description:

This Snap deletes the message by receipt handle coming from SQS Consumer when Auto Acknowledge is not enabled in the SQS Consumer Snap. 

ETL Transformations & Data Flow

The SQS Acknowledge Snap extracts receipt handle and queue URL from the input Document, and uses them to delete the corresponding message from the SQS instance.

Input & Output

  • InputThis Snap can have an upstream Snap that can pass a document output view. The input document must contain the keys ReceiptHandle and QueueURL.
  • Output: The Snap would generate a document that contains the result and the original document
Prerequisites:

Users of the Snap must have the following accesses to be able to use this Snap:

  • Valid AWS account. 
  • Access Key and Secret Key.
  • IAM Role access
Limitations and Known Issues
  • Works in Ultra Task Pipelines.
  • The maximum number of inflight messages per queue are:
    • Standard queue - 120,000 inflight messages per queue. 
    • FIFO queue - 20,000 inflight messages per queue.

For more information on queue-related limitations, see Queue-limits.

Configurations:

Account & Access

This Snap uses account references created on the Accounts page of SnapLogic Manager to handle access to this endpoint. See Configuring SQS Accounts for information on setting up this type of account.

Views

InputThis Snap has exactly one document input view.
OutputThis Snap has at most one document output view.
ErrorThis Snap has at most one document error view and produces zero or more documents in the view.
Troubleshooting:None at the moment.

Settings

Label

Required. The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

Maximum retries

The maximum number of retry attempts for failed retryable requests.

Example: 2

Default value: 3

  • If the value is larger than 0, the Snap first downloads the target file into a temporary local file. If any error occurs during the download, the Snap waits for the time specified in the Retry interval and attempts to download the file again from the beginning. When the download is successful, the Snap streams the data from the temporary file to the downstream Pipeline. All temporary local files are deleted when they are no longer needed.
  • Ensure that the local drive has sufficient free disk space to store the temporary local file.

Batch size

The number of documents the Snap would batch before processing. The maximum number is 10. Would be ignored if client-side buffering is enabled in the SQS Producer Snap.

Example: 5

Default value: 10

Batch timeout

The maximum time in milliseconds the snap waits before a batch is sent out.

Example: 100

Default value: 200

Max outbound batches in parallel

The maximum number of concurrent batches for each type of outbound request. The greater the number, the greater the throughput that can be achieved (at the expense of consuming more threads).

Example: 4

Default value: 5

Snap execution

Select one of the three modes in which the Snap executes. Available options are:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.
  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.
  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Examples


Basic Use Case

The following pipeline describes how the Snap functions as a standalone Snap in a pipeline:

The Snap is configured with the following parameters:

A preview of the output from executing this pipeline is as follows:

Exported pipelines are available in the Downloads section below.

Typical Snap Configurations

Key configuration of the Snap lies in how the values are passed. The values can be passed:

  • Without Expressions: Values are passed directly in the Snap:

  • With Expressions
    • Using pipeline parameters: Values are passed as pipeline parameters:

                     

Downloads

  File Modified
You are not logged in. Any changes you make will be marked as anonymous. You may want to Log In if you already have an account.
No files shared here yet.
  • Drag and drop to upload or browse for files
  •  

    Snap Pack History

     Click to view/expand
    Release
    Snap Pack Version 
    Date
    Type
    Updates
    May 2024main26341 StableUpdated and certified against the current SnapLogic Platform release.
    February 2024main25112 StableUpdated and certified against the current SnapLogic Platform release.
    November 2023main23721 Stable

    Updated and certified against the current SnapLogic Platform release.

    August 2023434patches23363 Latest

    Fixed the behavior change introduced in 4.32 GA related to Amazon SQS Accounts using the STS token. After the 4.32 GA, the account would automatically retry authentication on encountering 401 or 403 errors, and pipelines would not fail even after token expiration. The Amazon SQS Consumer Snap now behaves as expected and fails after the STS token expires.

    August 2023main22460 StableUpdated and certified against the current SnapLogic Platform release.
    May 2023main21015 StableUpgraded with the latest SnapLogic Platform release.
    February 2023432patches19990 Latest

    Added support for the SQS Account to reauthenticate when the authentication expires.

    Behavior Change

    With the 432patches19990, the Amazon SQS account reauthenticates when the authentication expires; the account automatically retires authentication on encountering 401 or 403 errors.

    February 2023main19844 StableUpgraded with the latest SnapLogic Platform release.
    November 2022main18944 Stable

    Upgraded with the latest SnapLogic Platform release.

    August 2022main17386 StableUpgraded with the latest SnapLogic Platform release.
    4.29main15993 StableUpgraded with the latest SnapLogic Platform release.
    4.28 Patch428patches14774 LatestFixed an issue in the SQS Consumer Snap where the Snap took longer time to process the messages when the Client-side buffering checkbox was selected.
    4.28main14627 StableUpgraded with the latest SnapLogic Platform release.
    4.27 Patch427patches14012 LatestFixed an issue in the SQS Consumer Snap where the Snap did not read message attributes when the Client-side buffering checkbox is selected.

    4.27

    main12833

     

    Stable

    Upgraded with the latest SnapLogic Platform release.
    4.26main11181 StableUpgraded with the latest SnapLogic Platform release.
    4.25 Patch425patches10994 Latest
    • Removed the duplicate strings in the request’s User Agent header sent by the Snaps in Amazon SQS Snap Pack.
    • Enhanced the Amazon SQS account to add Snap’s information to the User Agent header for Cross IAM account requests.
    • Enhanced the Amazon SQS Account with a new checkbox Enable Large Message Support that enables Amazon SQS client to send and receive messages up to 2GB through Amazon S3. 
    4.25 Patch425patches9596 Latest

    Added the Security token field to the Amazon SQS account.

    4.25main9554
     
    StableUpgraded with the latest SnapLogic Platform release.
    4.24main8556
    Stable

    Enhances the Snap Pack to support Cross-Account IAM role that allows to access the queues residing in another AWS account.

    4.23main7430
     
    StableUpgraded with the latest SnapLogic Platform release.
    4.22main6403
     
    StableUpgraded with the latest SnapLogic Platform release.
    4.21snapsmrc542

     

    StableUpgraded with the latest SnapLogic Platform release.
    4.20snapsmrc535
     
    StableUpgraded with the latest SnapLogic Platform release.
    4.19snaprsmrc528
     
    StableUpgraded with the latest SnapLogic Platform release.
    4.18 PatchMULTIPLE7778 Latest

    Updated the AWS SDK library version to default to Signature Version 4 Signing process for API requests across all regions.

    4.18snapsmrc523
     
    StableUpgraded with the latest SnapLogic Platform release.
    4.17ALL7402
     
    Latest

    Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers.

    4.17snapsmrc515
     
    Latest

    Added the Snap Execution field to all Standard-mode Snaps. In some Snaps, this field replaces the existing Execute during preview checkbox.

    4.16snapsmrc508
     
    StableUpgraded with the latest SnapLogic Platform release.
    4.15snapsmrc500
     
    StableUpgraded with the latest SnapLogic Platform release.
    4.14snapsmrc490
     
    StableUpgraded with the latest SnapLogic Platform release.
    4.13

    snapsmrc486

     
    StableUpgraded with the latest SnapLogic Platform release.
    4.12

    snapsmrc480

     
    Stable

    Introduced the Snap Pack in this release that includes SQS Producer, SQS Consumer, and the SQS Acknowledge Snaps.

    • No labels