Google Cloud PubSub Subscriber

Google Cloud PubSub Subscriber

In this article

Overview

You can use this Snap to subscribe or pull messages from a Google Pubsub topic using the subscription name. This Snap uses the asynchronous streaming pull API.

pubsub-subscriber.png

 

Snap type

The Google PubSub Subscriber Snap is a Read-type Snap.

Limitation

You must create subscriptions for the topic on the Google console.

Behavior change

The output of the Subscriber Snap has been updated. Previously, the message properties were displayed in string format; now, they are represented as key-value pairs under Message Attributes.

subscriber-beh-change.png

If the downstream Snap uses Message Attributes for further processing, then you need to reconfigure your pipeline.

Snap views

0Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

0Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input 

Document

 

  • Min: 0

  • Max: 1

  • Mapper

  • JSON Generator

  • CSV Parser

Each document contains data in the form of JSON..

Output

Document

  • Min: 1

  • Max: 1

  • JSON Generator

Each document contains a corresponding JSON response.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab. The available options are:

  • Stop Pipeline Execution: Stops the current Pipeline execution when the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

  • Asterisk (*): Indicates a mandatory field.

  • Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon (): Indicates whether the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon (): Indicates that you can add fields in the field set.

  • Remove icon (): Indicates that you can remove fields from the field set.

Field Name

Field Type

Description

Field Name

Field Type

Description

Label*

 

String

Specify the name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

 

Default Value: PubSub Subscriber
Example: PubSub Subscriber

Subscription name*

 

String/Expression/Suggestion

Specify the subscription name from which messages are to be pulled, The property is suggestable and will retrieve available subscription names based on the Project ID from the service account.

 

Default Value: N/A
Example: my-topic-subscription

Messages in buffer*

 

Integer/Expression

Specify the maximum number of messages in the buffer at a given time until they are acknowledged.

The default batch size for each pull request is 1000. If the requested messages in the buffer are less than 1000, the batch size will be set to the message count before subscribing to the messages.

Default Value: 1000
Example: 500

Auto  Acknowledge

 

Checkbox/Expression

Select this checkbox to auto-acknowledge the received messages. If selected, the Snap acknowledges the receipt of a message when the message listener returns successfully.

If you deselect this checkbox, the same data can be pulled, which means duplicate data is displayed in the output.

Default state: Deselected

Subscriber timeout

 

String/Expression

Specify the maximum time in seconds the Snap will be receiving messages. A zero wait timeout means that fetching will be performed without waiting.

 

Default Value: 30
Example: 50

Snap Execution

 

Dropdown list

Select one of the three modes in which the Snap executes. Available options are:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

 

Default Value: Validate & Execute
Example: Execute only

 

Snap Pack History

 


Related Content