On this Page
Table of Contents | ||||
---|---|---|---|---|
|
Multiexcerpt include macro | ||||
---|---|---|---|---|
|
Snap type: | Read | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Description: | This Snap fetches and consumes data from Kafka under specific topic or topics, and feed the data to the downstream pipeline.
| |||||||||||||
Prerequisites: | [None] | |||||||||||||
Support and limitations: | Works in Ultra Task Pipelines. | |||||||||||||
Account: | This Snap uses account references created on the Accounts page of SnapLogic Manager to handle access to this endpoint. See Configuring Kafka Accounts (Deprecated) for information on setting up this type of account. | |||||||||||||
Views: |
| |||||||||||||
Settings | ||||||||||||||
Label | Required. The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline. | |||||||||||||
Topic | Required. Topics this reader should subscribe to. Each reader can subscribe to one or more topics. Topics should be separated by comma. This attribute can be from a text field, pipeline parameters, or upstream document. Example: topic1,topic2 Default value: [None] | |||||||||||||
Group ID | Required. The ID of the consumer group that this reader belongs to. All readers in a consumer group will share the message offset in the Kafka broker. They’ll consume messages cooperatively. This attribute can be from text field, pipeline parameters or upstream document. Default value: consumer_group_1 | |||||||||||||
Message count | The number of message to read before consuming stop. If it is equal to 0, it'll wait for some time specified in Max Waiting Time and stop. If it is less than 0, it will wait indefinitely. This attribute can be from text field, pipeline parameters or upstream document. Default value: 0 | |||||||||||||
Fetching timeout | Max milliseconds the reader should wait before it gets a message from Kafka. This is the timeout for each fetching request. This attribute can be from text field, pipeline parameters or upstream document. Default value: 100
| |||||||||||||
Pass through header | Select this option if pass through the input document as the header of the binary output. This option only works when there is an input view. Default value: Not selected | |||||||||||||
Commit synchronously | Commit offset of the message synchronously. If it is checked, the consumer will commit the offset after each message sending. Default value: Not selected | |||||||||||||
|
|
Examples
See Also
Insert excerpt | ||||||
---|---|---|---|---|---|---|
|