Skip to end of banner
Go to start of banner

RabbitMQ Consumer

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 49 Next »

In this article

Overview

You can use this Snap to fetch messages from a RabbitMQ destination.

Snap Type

RabbitMQ Consumer Snap is a READ-type Snap.

Prerequisites

  • Access to the RabbitMQ Server.

  • Privileges to perform various actions on the Queues such as receive and send messages.

Support for Ultra Pipelines

Works in Ultra Pipelines

Limitations and Known Issues

None.

Snap Views

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input 

Document

  • Min:0

  • Max:1

Upstream Snap is optional. You can connect any Snap with a document output view, such as Mapper or Copy Snaps upstream of this Snap.

The Snap does not require input data. Input documents may be used to evaluate any JavaScript expression in the File property.

Output

Binary

  • Min:1

  • Max:1

  • File Writer

  • CSV Parser

  • JSON Parser

  • XML Parser

Message consumed from RabbitMQ server as binary document.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

  • Stop Pipeline Execution: Stops the current pipeline execution when the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the rest of the records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

  • Asterisk ( * ): Indicates a mandatory field.

  • Suggestion icon ((blue star)): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon ((blue star) ): Indicates whether the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon ( (blue star) ): Indicates that you can add fields in the fieldset.

  • Remove icon ( (blue star)): Indicates that you can remove fields from the fieldset.

Field Name

Field Type

Field Dependency

Description

Label*

Default Value: RabbitMQ Consumer
Example: RabbitMQ Consumer

String

None

Specify a unique name for the Snap.

Exchange

Default Value: N/A
Example: DemoExchange

String/Expression

None

Specify a name of the RabbitMQ exchange bound to the queue. If you do not specify a value for exchange, the default exchange provided by RabbitMQ is considered.

Condition: Either Exchange or Queue has to be specified.

Exchange type

Default Value: direct
Example: fanout

Dropdown list

None

The type of RabbitMQ exchange used to push messages. Available options are:

  • direct: Routes the messages to the queues only on matching with the routing key.

  • fanout: Routes the messages to the queues that are bound to it regardless of the routing keys and patterns.

  • headersRoutes the messages to the queues based on header values as specified in the binding. Provide the x-match and Header Properties if selected.

  • topic: Routes the messages to the queues based on the routing key and the binding pattern.

Durable Exchange

Default Value: Selected

Checkbox

None

Select this checkbox to enable the exchange to be durable.

Queue

Default Value: N/A
Example: DemoQueue

String/Expression

None

Conditional. Specify a name of the RabbitMQ queue from which messages have to be consumed. If there is no queue with given queue name on RabbitMQ, then a new queue is created and consumer starts listening. If the exchange name and routing key are also specified, then the queue is bound to the exchange with the given routing key.

Either Exchange or Queue has to be specified.

Auto delete

Default Value: Deselected
Example: Selected

Checkbox

None

Select this check box to indicate that the RabbitMQ autoDelete property associated with the existing queue is set to true.

Routing Key

Default Value: N/A
Example: ZYA123&$

String/Expression

None

Specify the routing key to be used for binding the queue with the exchange.

Processing Mode

Default Value: Synchronous
Example: Asynchronous


Dropdown list

None

Select one of the following modes for processing messages:

Synchronous: In synchronous mode, the consumer processes messages from the destination one at a time until a STOP is read or the Message Count is reached.

If a message is not available, the consumer will retry for an available message at an interval of every 3 secs until the Message count is reached.

For example, if the Queue sends 8 messages, and the Message count is set to 10, the consumer processes the 8 messages one after the other, and sleeps for every 3 seconds for the remaining two unavailable messages.

Asynchronous: In asynchronous mode, the consumer reads the messages from the destination whenever the message arrives until a 'STOP' is read or the Message Count is reached. The consumer registers a messagelistener asynchronously. It does not block, but calls (the processmessage) immediately once a message is available.

Message Acknowledge Mode*

Default Value: AUTO_ACKNOWLEDGE
Example: PIPELINE_CONTROL


Dropdown list

None

Select one of the following modes of message acknowledgement for non-transacted sessions:

  • AUTO_ACKNOWLEDGE the session acknowledges the receipt of a message when a call to receive method or when the message listener returns successfully.

  • PIPELINE_CONTROLthe client has the power when to acknowledge the message.  

Select the PIPELINE_CONTROL option when using the RabbitMQ Acknowledge Snap in a Pipeline.

  • The RabbitMQ Consumer Snap writes the messages to the output view along with headers containing a DeliveryTag which is then forwarded to the RabbitMQ Acknowledge Snap. 

  • The RabbitMQ Acknowledge Snap may be positioned anywhere in the pipeline following the Consumer Snap. The RabbitMQ Acknowledge Snap should be configured with necessary acknowledgement mode based on message processing. Options available are Acknowledge, Reject and Recover. 

If you select the PIPELINE_CONTROL option and the message is not acknowledged, by default, the Consumer Snap waits for 30 minutes to get the acknowledgment. If Snap does not get any acknowledgment for the message even after 30 minutes, Snap considers the status as unacknowledged and moves to the next message.

Acknowledge type

Default value: Acknowledge Message 
Example: Reject Message

Dropdown list

Appears on selecting Auto_Acknowledgment from the Message Acknowledge Mode dropdown list.

Select one of the following acknowledge types for the message.

  • Acknowledge Message: Successful acknowledgment of the message by the Snap to the RabbitMQ server and removal of that message from the queue.

  • Reject Message: Notifies the RabbitMQ server to mark the particular message as ‘dead lettered’ and route the message to the ‘Dead Lettered Exchange.’ However, if the ‘Dead Lettered Exchange’ is not configured on the RabbitMQ server, the message is dropped from the queue automatically.

  • Recover Message: Notifies the RabbitMQ server to requeue the message to be available in the queue.

x-match

Default Value: all
Example: any

Dropdown list

None

Specify the x-match value to be submitted for binding. The available options are:

  • any: If selected, even one matching header value is sufficient

  • all: If selected, mandates that all the values should match 

Select x-match only when the Exchange type is header

Header properties

Use this fieldset to define header properties to bind the queue with an exchange as arguments.  

Configure Header properties only when the Exchange type is headers.

Header key

Default Value: N/A
Example: org

String/Expression

None

Specify the name of the header that is being used for the binding.


Header value

Default Value: N/A
Example: snap

String/Expression

None

Specify the header value corresponding to the respective header key.

Argument properties

Use this field set to define custom argument properties to ensure all declarations for the queues use the same configuration/options/arguments.  

Argument key

Default value: N/A
Examplex-max-len-bytesx-max-priority

String/Expression

None

Specify the name of the argument used for declaration. Learn more about Optional Arguments in RabbitMQ documentation.

Argument value

Default value: N/A
Example1048576, 2

String/Expression

None

Specify the value corresponding to the argument key.

Message Count

Default value: -1
Example: 5


Integer/Expression

None

Controls the number of messages to be consumed from the destination queue before the consumer is stopped. Negative integer value, that is the default value of -1 makes the consumer run indefinitely.

  • -1: The consumer Snap runs in a never ending loop and keeps consuming messages from the destination. This is the default behavior. For example, if you set this field to the default value of -1, the Pipeline runs infinitely and reads all the messages until you stop that pipeline.

  • 0: The consumer Snap reads all the messages from the destination and then stops. For example, if there are 10 messages in the destination queue, all the 10 messages are read before it stops.

  • >0: The consumer Snap reads the specified number of messages before stopping. For example, if there are 20 messages in the queue and you have specified 4 as the value, then only 4 messages are read before it stops.

Maximum connection attempts

Default Value: 3
Example: 4

Integer

None

Specify the maximum number of connection attempts in case of a connection failure. The Snap retries for the configured number of attempts for establishing the connection.

If it exceeds the configured value, and the Route connection errors property is enabled, the messages are routed to the error view with the error information along with the original.

When the Route connection errors property is disabled, the Snap displays the number of published messages along with the error information, however, the Snap fails.

Connection retry interval

Default Value: 10
Example: 5

Integer

None

Specify the time taken in seconds to wait before retrying for another connection.

Route connection errors

Default Value: Deselected

Checkbox

None

Select this checkbox to route the connection errors to the error view displaying the error information along with the original. If you deselect this checkbox, the Snap fails to execute.

Snap Execution

Default Value: Validate & Execute
Example: Execute only

Dropdown list

None

Indicates how the Snap must be executed. Available options are:

  • Validate & Execute: Performs limited execution of the Snap (up to 50 records) during Pipeline validation; performs full execution of the Snap (unlimited records) during Pipeline execution.

  • Execute only: Performs full execution of the Snap during Pipeline execution; does not execute the Snap during Pipeline validation.

  • Disabled: Disables the Snap and, by extension, its downstream Snaps.

Examples

 RabbitMQ Consumer Snap consumes the messages from a specified queue

This example Pipeline demonstrates how to consume messages from a specified queue and write them to the output view.

The RabbitMQ Consumer Snap, consumes the messages from the queue DemoQueue either in synchronous or asynchronous mode.

Note that the Consumer creates the Exchange and Queue if they are not available in the RabbitMQ server, and binds them based on the properties configured such as Routing keyHeader Properties, x-match etc., and then starts consuming the messages from the server. Successful execution of the Pipeline displays the message properties in the output preview:



 RabbitMQ Consumer Snap reads and sends messages to the CRM instance

In the below Pipeline, the RabbitMQ Consumer Snap reads the messages from a queue and sends them to the CRM instance. The JSON Parser parses the messages and maps the messages to the CRM instance. The Script Snap is configured for a time delay so all the messages configured in the queue are processed.



 Click to view/expand
Release Snap Pack VersionDateType  Updates

May 2024

main26341

 

Stable

Updated and certified against the current SnapLogic Platform release.

February 2024main25112 StableUpdated and certified against the current SnapLogic Platform release.
November 2023main23721 StableUpdated and certified against the current SnapLogic Platform release.

August 2023

main22460

 

Stable

Updated and certified against the current SnapLogic Platform release.

May 2023

main21015 

Stable

Upgraded with the latest SnapLogic Platform release.

February 2023main19844 StableUpgraded with the latest SnapLogic Platform release.
November 2022main18944 StableUpgraded with the latest SnapLogic Platform release.
September 2022430patches18079 Latest

The RabbitMQ Consumer and RabbitMQ Producer Snaps now display an informative error message when you do not provide the Queue type attribute in the Quorum queue or provide conflicting Queue types for Classic and Quorum queues.

August 2022430patches17462 LatestThe RabbitMQ Account now successfully validates even when the port is out of range.
August 2022main17386 Stable

The RabbitMQ Consumer and RabbitMQ Producer Snaps can consume and produce messages respectively from the quorum queue by setting the Argument key field as x-queue-type and Argument value as quorum to consume or publish messages from/to the quorum queue.

4.30main17386 StableUpgraded with the latest SnapLogic Platform release.
4.29 Patch429patches17049 Latest

Enhanced the RabbitMQ Consumer Snap with:

  • Acknowledgment Type the dropdown list that allows you to set the acknowledgment type, which is available only when you select Auto_Acknowledgement in the Message Acknowledge Mode field.

  • A default 30-minute acknowledgment timeout—the Snap waits only for 30 minutes to get an acknowledgment. When there is no acknowledgment even after 30 minutes, Snap considers the status as unacknowledged and moves to the subsequent message.

4.29main15993 StableUpgraded with the latest SnapLogic Platform release.
4.28main14627 StableUpgraded with the latest SnapLogic Platform release.

4.27

main12833

 

Stable

Upgraded with the latest SnapLogic Platform release.
4.26 Patch426patches11770LatestEnhanced the Rabbit MQ Account with a new checkbox Cluster Support that enables multiple cluster node support for RabbitMQ Consumer and RabbitMQ Producer Snaps. You can select this checkbox to enable high availability of clusters by adding multiple hosts (Host name and Port) for each cluster node.
4.26main11181 StableUpgraded with the latest SnapLogic Platform release.
4.25main9554
 
StableUpgraded with the latest SnapLogic Platform release.
4.24 Patch 424 patches8750 Latest

Fixed the handling of RabbitMQ Headers in the RabbitMQ Consumer Snap, including the x-death header.

4.24main8556
StableUpgraded with the latest SnapLogic Platform release.
4.23main7430
 
StableUpgraded with the latest SnapLogic Platform release.
4.22main6403
 
Stable

Enhances the RabbitMQ Consumer and RabbitMQ Producer Snaps to allow defining custom arguments to ensure that all declarations for the queues use the same configuration/options/arguments. 

4.21snapsmrc542

 

StableUpgraded with the latest SnapLogic Platform release.
4.20snapsmrc535
 
StableUpgraded with the latest SnapLogic Platform release.
4.19snaprsmrc528
 
Stable

Added the Auto delete check box, which enables you to indicate that the RabbitMQ autoDelete property associated with the existing queue is set totrue.

4.18 Patch rabbitmq8058 Latest

Fixed an issue with RabbitMQ Snaps wherein the Snaps throw an error when the auto-delete setting is set to true.

4.18snapsmrc523
 
StableUpgraded with the latest SnapLogic Platform release.
4.17ALL7402
 
Latest

Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers.

4.17snapsmrc515
 
Latest

Added the Snap Execution field to all Standard-mode Snaps. In some Snaps, this field replaces the existing Execute during preview check box.

4.16snapsmrc508
 
StableUpgraded with the latest SnapLogic Platform release.
4.15snapsmrc500
 
StableUpgraded with the latest SnapLogic Platform release.
4.14snapsmrc490
 
StableUpgraded with the latest SnapLogic Platform release.
4.13 Patch rabbitmq5296 Latest

Fixed the RabbitMQ Acknowledge Snap that returns a NullPointerException error when no DeliveryTag is sent during pipeline execution. 

4.13

snapsmrc486

 
Stable
  • Added a new Snap, the RabbitMQ Acknowledge Snap to perform an acknowledge, reject or a recover action on the received message from the RabbitMQ server.
  • Added a new property “Message Acknowledge Mode” in the RabbitMQ Consumer Snap to specify the mode of message acknowledgement -- Auto Acknowledge or Pipeline Control (if the RabbitMQ Acknowledge Snap is used).
4.12

snapsmrc480

 
Stable
  • Added a new RabbitMQ Producer Snap setting to customize the wait-for-confirms timeout window.
  • Fixed an issue that prevented a clean connection recovery in the RabbitMQ Producer when a previous connection attempt failed and exhausted the retry attempts.
4.11 Patch rabbitmq4653 

Fixed an issue with the RabbitMQ Retry Mechanism that fails to process after failure.

4.11 Patch rabbitmq4613 

Added the RabbitMQ Producer Snap with property 'Wait for confirms timeout (ms)' to enable the user to configure the publish timeout in milliseconds.

4.11 Patch rabbitmq4535 

Enhanced the RabbitMQ Producer and Consumer for connection retry logic when running in Ultra Pipelines.

4.11 Patch  rabbitmq4465 

Fixed an issue with the RabbitMQ Producer where dynamic queues were created for each instance of an Ultra Pipeline. The property Create dynamic queue is added as a part of the fix.

4.11 Patch rabbitmq4423 

Fixed an issue with the RabbitMQ Producer Snap that now creates dynamic queue for each message received.

4.11 Patch rabbitmq4324 

Fixed the following issues:

  • The RabbitMQ Snap, which failed, throwing a connection timeout error instead of trying to reconnect.
  • Issue with the Consumer Snap errors not going through error view.
  • Ultra Task Snap loaded the same document onto the RabbitMQ queue five times.
  • Ultra Task Snap, on an error logic retry, loaded the same document onto the RabbitMQ Producer Snap.
  • Inability to specify input variable for Routing key.
4.11snapsmrc465
 
StableUpgraded with the latest SnapLogic Platform release.

4.10 Patch 

rabbitmq4094

 

Fixed an issue related to messages being processed multiple times under certain failure conditions with Ultra pipelines. Added additional timeout conditions to the retry/error view handling.

4.10 Patch 

rabbitmq3860

 

Fixed an issue in Retrying.The configuration for retrying can be provided in Producer and Consumer Snaps individually.

4.10

snapsmrc414

 
Stable
  • The Message count property updated to a default value of -1.
  • Added the Producer Snap with Retry request interval and Maximum request attempts fields to support the retry logic in case of a network failure.
  • Added support to the RabbitMQ Account with the Use TLSv1.2,Trust all certificates, Key Store, TrustStore, Key/Truststore password and Key alias properties to support the SSL/TLS for secured communications.
4.9snapsmrc405
 
StableUpgraded with the latest SnapLogic Platform release.
4.8 Patch rabbitmq2731 Latest
  • Provided the Enhance Exchange Header support for RabbitMQ Snap Pack.
  • x-match and Header properties added to support different exchange headers. 
4.8 Patch rabbitmq2720 Latest
  • Addressed the following issues:
    • RabbitMQ Snap Pack - Thrown by RESTEasy when HTTP Not Acceptable (406) is encountered.
    • Unable to configure RabbitMQ - Producer to work with existing Topic Exchange
    • Exchange type and Durable Exchange added to support different exchange types. 
4.8

snapsmrc398

 
Stable

Info tab added to accounts.

4.7

snapsmrc382

 
StableUpgraded with the latest SnapLogic Platform release.
4.6snapsmrc362
 
StableUpgraded with the latest SnapLogic Platform release.
4.5.1

snapsmrc344

 
Stable

Resolved an issue with the Producer and Consumer Snaps not releasing connections.

4.4.1
 Stable

NEW! This Snap Pack contains a Consumer and a Producer Snap.

  • No labels