Kafka Consumer

Owned by Kalpana Malladi
Last updated: May 10, 2024 by Gouri Bhagchandani

In this article

Overview

You can use the Kafka Consumer Snap to read or consume documents from a Kafka topic.

Kafka Snap Settings

Prerequisites

  • A valid account with the required permissions.
  • A Kafka topic with parameters that are valid for consumption (reading). 

In order for the notifications to work properly, all the Consumer and Acknowledge Snaps that belong to one consumer group should execute in the same node of the Snaplex. You can either use Pipeline Execute Snap to execute different Pipelines in the same node, or place all the Consumer and Acknowledge Snaps into one Pipeline.

Support for Ultra Pipelines

Works in Ultra Pipelines.

Limitations and Known Issues

None.

Snap Input and Output

Input/OutputType of ViewNumber of ViewsExamples of Upstream and Downstream SnapsDescription
InputDocument
  • Min:0
  • Max:1
  • Mapper
  • CSV Formatter
  • JSON Formatter
  • XML Formatter
A document containing a Kafka topic, a consumer group, a partition, and an offset.
OutputDocument
  • Min:1
  • Max:1
  • Mapper
  • Copy
  • JSON Formatter and File Writer
The message data with key/value parameters and metadata.

Snap Settings

Parameter NameData TypeDescriptionDefault ValueExample
LabelString

Required. The name for the Snap. You can modify the default name to be more specific and meaningful, especially if you have more than one of the same Snaps in your Pipeline.

Kafka Consumer