ASB Consumer

In this article

Overview

ASB Consumer is a Read-type Snap that reads messages from Azure Service Bus (ASB).

Prerequisites

Support for Ultra Pipelines 

Works in Ultra Pipelines. 

Limitations and Known Issues

None.

Snap Views

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input 

Document

 

  • Min: 0

  • Max: 1

  • Mapper

  • CSV Formatter

  • JSON Formatter

  • XML Formatter

A document containing a message or topic.

Output

Document

 

  • Min: 1

  • Max: 1

  • Mapper

  • Copy

  • JSON Formatter and File Writer

The message data with key or value parameters and metadata.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab. The available options are:

  • Stop Pipeline Execution: Stops the current pipeline execution when the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the rest of the records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

  • Asterisk (*): Indicates a mandatory field.

  • Suggestion icon ( ): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon (): Indicates whether the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon ( ): Indicates that you can add fields in the field set.

  • Remove icon (): Indicates that you can remove fields from the field set.

Field Name

Field Type

Field Dependency

Description

Field Name

Field Type

Field Dependency

Description

Label*

 

Default Value: ASB Consumer
Example: ASB Consumer

String

N/A

Specify the name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

 

Destination Type

 

Default Value: QUEUE
Example: TOPIC

Dropdown list

Enables Queue, Topic, and Subscription fields.

Choose the destination type to which the message must be sent. Available options are:

  • QUEUE: Messages are sent to the queue

  • TOPIC: Messages are sent to the topic.

 

Queue

 

Default Value: N/A
Example: customers_queue

String/Expression/Suggestion

Appears when you select QUEUE for Destination Type.

Specify the queue from which the message should be consumed in ASB. Alternatively, click the Suggestion icon to fetch the list of queues corresponding to the Azure Service Bus account and select a queue.

 

Topic

 

Default Value: N/A
Example: XYZ

String/Expression/Suggestion

Appears when you select TOPIC for Destination Type.

Specify the topic from which the message should be consumed in ASB. Alternatively, click the Suggestion icon to fetch the list of topics corresponding to the Azure Service Bus account and select a topic.

 

Subscription

 

Default Value: N/A
Example: XYZ

String/Expression/Suggestion

Appears when you select TOPIC for Destination Type.

Specify the subscription for the topic. Alternatively, click the Suggestion icon to fetch the list of Subscriptions for the topic corresponding to the Azure Service Bus account and select a Subscription.

 

Processing Mode

 

Default Value: ASYNCHRONOUS
Example: SYNCHRONOUS

Dropdown list

Appears when you select QUEUE for Destination Type.

Select the processing mode to make calls to the Azure Service Bus service. The available options are:

  • ASYNCHRONOUS: This mode enables the consumer to read the message from the destination when it arrives until a 'STOP' is read or the Message Count is reached. 

  • SYNCHRONOUS: This mode enables the consumer to read messages from the destination every 5 seconds until a 'STOP' is read or the Message Count is reached. If the consumer reads the message within 5 seconds, it will process it and not wait every 5 seconds for the next message.

When you use the ASB Acknowledge Snap in the pipeline, you need to use SYNCHRONOUS Processing mode. Otherwise, we recommend that you use ASYNCHRONOUS mode for better performance.

Prefetch Count

 

Default Value: 0
Example: 10

Integer/Expression

Appears when you select ASYNCHRONOUS for Processing Mode.

Specify the number of messages to be read into the buffer before the Snap begins processing. If the field is left empty, the Snap sets it to 0.

Incrementing the prefetch count improves the overall performance of the ASB Consumer Snap. The prefetch count can be from 0 (disabled) to 5. If you enter a value that exceeds the recommended limit of 5, a Lint warning is displayed in the pipeline statistics.

Count Down Latch Timeout (in seconds)

 

Default Value: 1
Example: 5

Integer/Expression

Appears when you select ASYNCHRONOUS for Processing Mode.

Specify the time interval in seconds to receive the messages.

Acknowledgment Type

 

Default Value: Acknowledge Message
Example: Defer Message

Dropdown list

Appears when you select ASYNCHRONOUS for Processing Mode.

Select an option for the Acknowledgement type to determine the action on the received messages. Select the action to take on the received message The available options are:

  • Defer Message: Indicates that the message is a lower priority and can be processed after higher priority messages.

  • Abandon Message: Discards the message.

  • Acknowledge Message: Ensures that the receiver has read the message.

  • Deadletter Message: Holds the messages that cannot be delivered to the receiver.

Max Message Count

 

Default Value: -1
Example: 5

Integer/Expression

Appears when you select SYNCHRONOUS for Processing Mode.

Specify the maximum number of messages to read before the Consumer stops. If you provide a negative value, the Snap continuously reads messages.

 

Max Idle Time (in seconds)

 

Default Value: 120
Example: 100

Integer/Expression

Appears when you select SYNCHRONOUS for Processing Mode.

Specify the maximum idle time (in seconds) before the Snap stops to wait for messages.

 

Batch Pooling

 

Default Value: Deselected

Checkbox

Appears when you select SYNCHRONOUS for Processing Mode.

Select this checkbox to pool multiple messages in a batch.

 

Max Messages (per poll)

 

Default Value: 1
Example: 12

Integer/Expression

Appears when you select Batch Pooling.

Specify the maximum number of messages to read for each poll.

 

Polling Interval (in milli seconds)

 

Default Value: N/A
Example: 250

Integer/Expression

Appears when you select Batch Pooling.

Specify the time (in milliseconds) to wait between polls for messages.
We recommend you to leave this field empty.

 

Message Acknowledge Mode

 

Default Value: AUTO_ACKNOWLEDGE

Dropdown list

Appears when you select SYNCHRONOUS for Processing Mode.

Choose the mode for acknowledging messages in non-transaction sessions.

  • AUTO_ACKNOWLEDGE

  • PIPELINE_CONTROL

Acknowledgement wait time (in milliseconds)

 

Default Value: 10
Example: 5000

Integer

Appears when you select PIPELINE_CONTROL for Message Acknowledge Mode.

Specify the duration for which the Snap should wait to receive acknowledgments from the ASB Acknowledgement Snap. We recommend you to leave this field empty to ensure accurate acknowledgment of every message.

Max Auto-lock Renew Duration (in seconds)

 

Default Value: 60
Example: 70

Integer/Expression

Appears when you select QUEUE for Destination Type.

Specify the maximum time (in seconds) to renew the message lock’s expiration.

Snap Execution

 

Default Value: Validate & Execute
Example: Execute only

Dropdown list

N/A

Select one of the three modes in which the Snap executes. Available options are:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Troubleshooting

Error

Reason

Resolution

Error

Reason

Resolution

Exception occurred while processing the message.

Cannot configure more than one Consumer Snap within the same Pipeline with Message acknowledge mode as PIPELINE_CONTROL

Verify the connection settings and provide values for the input properties.

Examples

Consume Messages from Azure Service Bus

This example pipeline demonstrates how to read messages from ASB using the ASB Consumer Snap.

Step 1: Configure the ASB Consumer Snap to consume messages from the queue.

After validating the pipeline, you can see the details of the messages that are read in the output preview of the Snap as follows.

Step 2: Configure the Mapper Snap to extract the values of Message id, Sequence number, and Lock token (as highlighted in the screenshot) from the output of the upstream Snap.

The Mapper Snap returns the output as follows.

Step 3: Configure the ASB Acknowledge Snap to acknowledge the messages that are received.

Download this Pipeline. 

Consume Messages and Write to Topic

This example pipeline demonstrates how to use the ASB Consumer Snap to read and write messages to a topic.

Configure the ASB Consumer Snap to send messages to a topic.

After validating the Pipeline, you can see the output in the Snap’s preview. After executing the Pipeline, the consumed messages are written to the topic.

Download this Pipeline.

Downloads

 

  File Modified

File Example1_ASB Consumer.slp

Dec 01, 2021 by Kalpana Malladi

File Example2_ASB Consumer.slp

Dec 01, 2021 by Kalpana Malladi

Snap Pack History


Related Links