- Created by Kalpana Malladi, last modified on Jul 20, 2023
You are viewing an old version of this page. View the current version.
Compare with Current View Page History
« Previous Version 9 Next »
In this article
Overview
You can use the Kafka Kerberos account type to connect the Confluent Kafka Snaps with data sources that use Kafka Kerberos accounts.
Prerequisites
None.
Limitations
None.
Known Issues
None.
Account Settings
Parameter | Data Type | Description | Default Value | Example |
---|---|---|---|---|
Label | String | Required. Specify a unique label for the account. | N/A | Kafka Kerberos Account_89 |
Bootstrap Servers | Use this field set to configure the bootstrap servers. Click to add a new row in this table for configuring bootstrap servers. This field set contains the Bootstrap Server field. | |||
Bootstrap Server | String/Expression | Specify an ordered list of host:port pairs to use for establishing the initial connection to the Kafka cluster. | N/A | ec2-54-234-24-58.compute-1.amazonaws.com:9092 |
Schema Registry URL | String/Expression | Specify the schema registry server URL. | N/A | http://ec2-54-234-24-58.compute-1.amazonaws.com:8081 |
Advanced Kafka Properties | Use this field set to specify any additional Kafka properties that are not explicitly provided in the Snaps for connecting to the Kafka server. The properties that you provide here overwrites any values defined by the Snap. All Kafka properties are automatically defined by the Snap. Advanced Kafka Properties field enables you to to define additional Kafka properties. These properties are passed directly to the server and are not tested by SnapLogic, Inc. This field set contains the following fields:
| |||
Key | String/Expression | Specify the key for any Kafka parameters that are not explicitly supported by the Snaps. | N/A | max.message.size |
Value | String/Expression | Specify the value for the corresponding key that are not explicitly supported by the Snaps. | N/A | 5MB |
Security Protocol | String/Expression | Choose a security protocol that GSSAPI/Kerberos authentication supports. The available options are:
| SASL_SSL | SASL_PLAINTEXT |
Principal | String/Expression | Required. Specify a unique name of a user or service for authentication. | N/A | User: testuser Service: kafka/testhost.example.com. |
Keytab | String/Expression | Required. Specify the path of the Kerberos Keytab file that includes the Principals. | N/A | /etc/security/keytabs/nn.service.keytab |
Truststore Filepath | String/Expression | Provide the location of the Truststore file that is used to authenticate the server. Provide the location if the Security protocol is SASL_SSL and the certificate is not signed by a Certificate Authority in the system's Truststore. | N/A | kafka.net.ssl.truststore |
Truststore Password | String | Specify the password to access the Truststore file, if used. | N/A | test1234 |
Snap Pack History
Release | Snap Pack Version | Date | Type | Updates |
---|---|---|---|---|
May 2024 | main26341 | Stable | Kafka Snap Pack: Enhanced the following fields to support expressions through pipeline parameters:
| |
February 2024 | main25112 | Stable | Updated and certified against the current SnapLogic Platform release. | |
November 2023 | main23721 |
| Stable | Updated and certified against the current SnapLogic Platform release. |
August 2023 | main22460 | | Stable | Updated and certified against the current SnapLogic Platform release. |
May 2023 | 433patches21070 | Latest | Fixed an issue with the Kafka Consumer Snap that caused it to skip records in a partition when a pipeline failed without successfully processing any records from that partition. This was an edge case that was only possible with a specific configuration of the Snap and a specific ordering of events during termination of the pipeline. | |
May 2023 | main21015 | Stable | Upgraded with the latest SnapLogic Platform release. | |
February 2023 | 432patches20143 | Latest/Stable | Fixed an issue with the Kafka Producer Snap to avoid an error about delivery.timeout.ms . The Snap now adjusts the value of the Message Publish Timeout setting automatically and display a warning in the Pipeline Execution Statistics. | |
February 2023 | main19844 | Stable | Upgraded with the latest SnapLogic Platform release. | |
November 2022 | 431patches19770 | Latest | Fixed a memory issue that occurred when the Kafka Producer Snap was used to produce many records in a long-running pipeline. | |
November 2022 | 431patches19211 | Latest | Enhanced the Kafka Consumer Snap with the new Max Poll Interval (ms) field, which allows you to specify the time limit between subsequent calls to poll. This field corresponds to the Kafka max.poll.interval.ms property. | |
November 2022 | main18944 | Stable | Upgraded with the latest SnapLogic Platform release. | |
August 2022 | main17386 | Stable |
| |
4.29 Patch | 429patches17109 | Latest |
| |
4.29 | main15993 | Stable | The Kafka Snap Pack now supports Azure Event Hubs streaming service. Learn more: Azure Event Hubs configuration in Azure portal for Kafka SSL Account. | |
4.28 Patch | 428patches14904 | Latest | Fixed an issue with Kafka Snaps where the Snaps failed with null pointer exceptions when logging the cluster information with the controller’s identity not known. | |
4.28 | main14627 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.27 | main12833 | | Stable | Upgraded with the latest SnapLogic Platform release. |
4.26 | main11181 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.25 Patch | 425patches10543 | Latest |
| |
4.25 | main9554 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.24 Patch | 424patches8805 | Latest | Fixed an issue in the Kafka Producer Snap by removing the validation of the account when the Snap initialized in a Pipeline execution. | |
4.24 | main8556 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.23 Patch | 423patches7900 | Latest |
| |
4.23 | main7430 | Stable | Fixed an intermittent issue of Confluent Kafka Consumer Snap stopping when Message Count is 0 while the records are available. The Snap will now stop only if a poll returns no records twice in a row, and has partitions assigned before and after each poll. | |
4.22 | main6403 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.21 Patch | 421patches6136 | Latest | Enhanced the Confluent Kafka Consumer Snap with an option to allow reprocessing of records that remain unacknowledged due to acknowledge timeouts. This ensures that all records from Kafka are processed. | |
4.21 Patch | 421patches5862 | Latest | Fixes the Confluent Kafka Consumer Snap that aborts abruptly upon getting a | |
4.21 | snapsmrc542 |
| Stable | Enhanced the Confluent Kafka Consumer Snap to significantly reduce the volume of commit requests for offsets when using the Record acknowledgement mode (Wait after each record), and thereby, improve performance and stability. |
4.20 Patch | confluentkafka8747 | Latest | Updated the Confluent Kafka SSL Account settings to make the Truststore filepath and Truststore password properties optional. | |
4.20 | snapsmrc535 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.19 | snaprsmrc528 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.18 Patch | confluentkafka8111 | Latest | Fixed an issue with the Confluent Kafka Producer Snap wherein the Topic property doesn't evaluate expression against input documents. | |
4.18 Patch | confluentkafka8006 | Latest | Fixed an exception that occurs while consuming log-compacted topics, in the Confluent Kafka Consumer Snap. | |
4.18 Patch | confluentkafka7784 | Latest | Fixed the Consumer Snap to stop quickly and cleanly on CLOSE and STOP lifecycle events. | |
4.18 Patch | confluentkafka7732 | Latest | Added support for Kerberos-based authentication to enable connecting to Kerberos-enabled Kafka servers. | |
4.18 | snapsmrc523 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.17 Patch | confluentkafka7537 | Latest | Updated the following Snaps:
| |
4.17 | ALL7402 | Latest | Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers. | |
4.17 | snapsmrc515 | Latest |
| |
4.16 Patch | confluentkafka7118 | Latest | Fixed an issue with the Confluent Kafka Producer Snap wherein the Snap fails to produce the upstream Snap's input schema for Partition Number property in the output preview. | |
4.16 Patch | confluentkafka6891 | Latest | Certified Confluent Platform version 5.x. | |
4.16 | snapsmrc508 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.15 Patch | confluentkafka6446 | Latest | Fixed an issue with the Confluent Kafka Consumer Snap wherein some connections were not closed after a Pipeline is aborted. | |
4.15 | snapsmrc500 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.14 Patch | confluentkafka5737 | Latest | Added a new property to the Confluent Kafka Producer Snap, Message publish timeout (milliseconds), that lets you specify a timeout value. | |
4.14 Patch | confluentkafka5663 | Latest | Fixed the Confluent Kafka Snaps that do not delete temp files after pipeline execution. | |
4.14 | snapsmrc490 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.13 Patch | confluentkafka5614 | Latest | Fixed the Confluent Kafka Snap Pack that does not delete the temp files after executing pipelines. | |
4.13 Patch | confluentkafka5330 | Latest | Added support in Confluent Kafka Producer Snap the ability to write data to multiple partitions, in a round robin fashion. | |
4.13 | snapsmrc486 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.12 Patch | confluentkafka5087 | Latest | Fixed an issue with the Consumer Snap that fails when the specified timeout value is less than the pipeline execution time. | |
4.12 | snapsmrc480 | Stable | Enhanced the performance of the Confluent Kafka Producer Snap in writing incoming documents into the Confluent Kafka Cluster. | |
4.11 Patch | confluentkafka4626 | Latest | Fixed an issue with the Confluent Kafka Consumer Snap wherein the Consumer session is not closing properly when manually stopping a pipeline, which causes the subsequent pipeline execution to wait for the previous session to die and not consume messages. | |
4.11 Patch | confluentkafka4302 | Latest | Fixed an issue with the Confluent Kafka Consumer Snap that fetched 500 messages only, when message count set to 0 caused due to driver default value change. | |
4.11 | snapsmrc465 | Stable | Updated the Confluent Kafka SSL Account with SCRAM_SASL support. | |
4.10 | snapsmrc414 | Stable | Confluent v3.2 support for Confluent Kafka Snap Pack. | |
4.9 Patch | confluentkafka3156 | Latest | Fixed an issue regarding account validation failure due to unauthorized topics; Enhance the error message for empty topic suggestions | |
4.9 | snapsmrc405 | Stable |
| |
4.8 Patch | confluentkafka2804 | Latest | Addressed an issue with Confluent Kafka Producer not including input document schema in Message Key and Value expression properties | |
4.8 | snapsmrc398 | Stable | New Snap Pack for 4.8. It consists of Confluent Kafka Consumer and Confluent Kafka Producer. |
Additional Information
When connecting to a Kerberos-enabled Kafka server, you must enable the User Datagram Protocol (UDP) connections to Port 88 of the Key Distribution Center (KDC) service. To do so, follow these steps:
Connect to the Snaplex node (on Windows / Linux).
Navigate to the krb5.conf file.
Edit the krb5.conf file using any text editor.
Note: Before you edit, take a backup of the current file.Navigate to the [libdefaults] section.
Add the following entry in the krb5.conf file under [libdefaults].
udp_preference_limit = 1
Save and close the krb5.conf file.
Troubleshooting
Error | Reason | Resolution |
---|---|---|
Read timed out (read timeout = 300) | The Kafka Kerberos account validation fails when you do not enable the UDP connections. | Enable UDP connections by adding the following entry in the krb5.conf file under [libdefaults]. udp_preference_limit = 1 Refer to the Additional Information section for details. |
Snap Pack History
Release | Snap Pack Version | Date | Type | Updates |
---|---|---|---|---|
May 2024 | main26341 | Stable | Kafka Snap Pack: Enhanced the following fields to support expressions through pipeline parameters:
| |
February 2024 | main25112 | Stable | Updated and certified against the current SnapLogic Platform release. | |
November 2023 | main23721 |
| Stable | Updated and certified against the current SnapLogic Platform release. |
August 2023 | main22460 | | Stable | Updated and certified against the current SnapLogic Platform release. |
May 2023 | 433patches21070 | Latest | Fixed an issue with the Kafka Consumer Snap that caused it to skip records in a partition when a pipeline failed without successfully processing any records from that partition. This was an edge case that was only possible with a specific configuration of the Snap and a specific ordering of events during termination of the pipeline. | |
May 2023 | main21015 | Stable | Upgraded with the latest SnapLogic Platform release. | |
February 2023 | 432patches20143 | Latest/Stable | Fixed an issue with the Kafka Producer Snap to avoid an error about delivery.timeout.ms . The Snap now adjusts the value of the Message Publish Timeout setting automatically and display a warning in the Pipeline Execution Statistics. | |
February 2023 | main19844 | Stable | Upgraded with the latest SnapLogic Platform release. | |
November 2022 | 431patches19770 | Latest | Fixed a memory issue that occurred when the Kafka Producer Snap was used to produce many records in a long-running pipeline. | |
November 2022 | 431patches19211 | Latest | Enhanced the Kafka Consumer Snap with the new Max Poll Interval (ms) field, which allows you to specify the time limit between subsequent calls to poll. This field corresponds to the Kafka max.poll.interval.ms property. | |
November 2022 | main18944 | Stable | Upgraded with the latest SnapLogic Platform release. | |
August 2022 | main17386 | Stable |
| |
4.29 Patch | 429patches17109 | Latest |
| |
4.29 | main15993 | Stable | The Kafka Snap Pack now supports Azure Event Hubs streaming service. Learn more: Azure Event Hubs configuration in Azure portal for Kafka SSL Account. | |
4.28 Patch | 428patches14904 | Latest | Fixed an issue with Kafka Snaps where the Snaps failed with null pointer exceptions when logging the cluster information with the controller’s identity not known. | |
4.28 | main14627 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.27 | main12833 | | Stable | Upgraded with the latest SnapLogic Platform release. |
4.26 | main11181 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.25 Patch | 425patches10543 | Latest |
| |
4.25 | main9554 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.24 Patch | 424patches8805 | Latest | Fixed an issue in the Kafka Producer Snap by removing the validation of the account when the Snap initialized in a Pipeline execution. | |
4.24 | main8556 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.23 Patch | 423patches7900 | Latest |
| |
4.23 | main7430 | Stable | Fixed an intermittent issue of Confluent Kafka Consumer Snap stopping when Message Count is 0 while the records are available. The Snap will now stop only if a poll returns no records twice in a row, and has partitions assigned before and after each poll. | |
4.22 | main6403 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.21 Patch | 421patches6136 | Latest | Enhanced the Confluent Kafka Consumer Snap with an option to allow reprocessing of records that remain unacknowledged due to acknowledge timeouts. This ensures that all records from Kafka are processed. | |
4.21 Patch | 421patches5862 | Latest | Fixes the Confluent Kafka Consumer Snap that aborts abruptly upon getting a | |
4.21 | snapsmrc542 |
| Stable | Enhanced the Confluent Kafka Consumer Snap to significantly reduce the volume of commit requests for offsets when using the Record acknowledgement mode (Wait after each record), and thereby, improve performance and stability. |
4.20 Patch | confluentkafka8747 | Latest | Updated the Confluent Kafka SSL Account settings to make the Truststore filepath and Truststore password properties optional. | |
4.20 | snapsmrc535 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.19 | snaprsmrc528 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.18 Patch | confluentkafka8111 | Latest | Fixed an issue with the Confluent Kafka Producer Snap wherein the Topic property doesn't evaluate expression against input documents. | |
4.18 Patch | confluentkafka8006 | Latest | Fixed an exception that occurs while consuming log-compacted topics, in the Confluent Kafka Consumer Snap. | |
4.18 Patch | confluentkafka7784 | Latest | Fixed the Consumer Snap to stop quickly and cleanly on CLOSE and STOP lifecycle events. | |
4.18 Patch | confluentkafka7732 | Latest | Added support for Kerberos-based authentication to enable connecting to Kerberos-enabled Kafka servers. | |
4.18 | snapsmrc523 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.17 Patch | confluentkafka7537 | Latest | Updated the following Snaps:
| |
4.17 | ALL7402 | Latest | Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers. | |
4.17 | snapsmrc515 | Latest |
| |
4.16 Patch | confluentkafka7118 | Latest | Fixed an issue with the Confluent Kafka Producer Snap wherein the Snap fails to produce the upstream Snap's input schema for Partition Number property in the output preview. | |
4.16 Patch | confluentkafka6891 | Latest | Certified Confluent Platform version 5.x. | |
4.16 | snapsmrc508 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.15 Patch | confluentkafka6446 | Latest | Fixed an issue with the Confluent Kafka Consumer Snap wherein some connections were not closed after a Pipeline is aborted. | |
4.15 | snapsmrc500 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.14 Patch | confluentkafka5737 | Latest | Added a new property to the Confluent Kafka Producer Snap, Message publish timeout (milliseconds), that lets you specify a timeout value. | |
4.14 Patch | confluentkafka5663 | Latest | Fixed the Confluent Kafka Snaps that do not delete temp files after pipeline execution. | |
4.14 | snapsmrc490 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.13 Patch | confluentkafka5614 | Latest | Fixed the Confluent Kafka Snap Pack that does not delete the temp files after executing pipelines. | |
4.13 Patch | confluentkafka5330 | Latest | Added support in Confluent Kafka Producer Snap the ability to write data to multiple partitions, in a round robin fashion. | |
4.13 | snapsmrc486 | Stable | Upgraded with the latest SnapLogic Platform release. | |
4.12 Patch | confluentkafka5087 | Latest | Fixed an issue with the Consumer Snap that fails when the specified timeout value is less than the pipeline execution time. | |
4.12 | snapsmrc480 | Stable | Enhanced the performance of the Confluent Kafka Producer Snap in writing incoming documents into the Confluent Kafka Cluster. | |
4.11 Patch | confluentkafka4626 | Latest | Fixed an issue with the Confluent Kafka Consumer Snap wherein the Consumer session is not closing properly when manually stopping a pipeline, which causes the subsequent pipeline execution to wait for the previous session to die and not consume messages. | |
4.11 Patch | confluentkafka4302 | Latest | Fixed an issue with the Confluent Kafka Consumer Snap that fetched 500 messages only, when message count set to 0 caused due to driver default value change. | |
4.11 | snapsmrc465 | Stable | Updated the Confluent Kafka SSL Account with SCRAM_SASL support. | |
4.10 | snapsmrc414 | Stable | Confluent v3.2 support for Confluent Kafka Snap Pack. | |
4.9 Patch | confluentkafka3156 | Latest | Fixed an issue regarding account validation failure due to unauthorized topics; Enhance the error message for empty topic suggestions | |
4.9 | snapsmrc405 | Stable |
| |
4.8 Patch | confluentkafka2804 | Latest | Addressed an issue with Confluent Kafka Producer not including input document schema in Message Key and Value expression properties | |
4.8 | snapsmrc398 | Stable | New Snap Pack for 4.8. It consists of Confluent Kafka Consumer and Confluent Kafka Producer. |
Related Content
- No labels