You must create Confluent Kafka accounts to connect to data sources that you want to use in your Pipelines.
Snap | Kafka Account | Kafka Kerberos Account | Kafka SSL Account | Protocols Supported | No Account |
---|---|---|---|---|---|
Kafka Acknowledge | ✖ | ✖ | ✖ | TLS/SSL | ✔ |
Kafka Consumer | ✔ | ✔ | ✔ | TLS/SSL | ✖ |
Kafka Producer | ✔ | ✔ | ✔ | TLS/SSL | ✖ |
You can configure your Confluent Kafka accounts in SnapLogic using either the Designer or Manager.
Drag a Kafka Snap to the Canvas and click the Snap to open its settings. Click the Account tab. You can now either use an existing account or create a new one.
SnapLogic organizes and displays all accounts to which you have access, sorting them by account type and location. To select an existing account:
Enter additional information on this account in the Notes field of the Info tab. This will help you–and other users–understand the purpose of the account, especially if there are multiple accounts of the same type. |
Click Validate to verify the account, if the account type supports validation.
Click Apply to complete configuring the Kafka account.
Use Manager to create accounts without associating them immediately with Pipelines.
Accounts in SnapLogic are associated with projects. You can use accounts created in other projects only if you have at least Read access to them. |
Repeat the steps numbered 3 through 5 in the Creating an account section.
Avoid updating account credentials while Pipelines using that account are executing. This may lead to unexpected results, including locking your account. |
Standard Encryption | If you use Standard Encryption, the High sensitivity settings under Enhanced Encryption are used. |
Enhanced Encryption | If the account you are using has the Enhanced Encryption feature, the account fields are encrypted for each sensitivity level as shown below:
|
Error | Reason | Resolution |
---|---|---|
The message size exceeds the default message limit. | By default, the message size limit is 1 MB. To increase this limit, you must change a few settings in the Confluent Kafka broker and Snap account. On the Confluent Kafka broker side, set the server.properties and producer.properties to a size larger than your message size. | In the Advanced Kafka Properties field of the applicable Snap account settings, you must add For example, you can increase the message size limit to 15 MB in any one of the following ways:
|