Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In this article

Table of Contents
maxLevel3
absoluteUrltrue
excludeOlder Versions|Additional Resources|Related Links|Related Information


Articles in this section

Panel
bgColor#ebf7e1

Child pages (Children Display)


Overview

You must create Confluent Kafka accounts to connect to data sources that you want to use in your Pipelines.

Snap-Account Compatibility

SnapKafka AccountKafka Kerberos AccountKafka SSL AccountProtocols SupportedNo Account
Kafka Acknowledge

TLS/SSL
Kafka ConsumerTLS/SSL
Kafka ProducerTLS/SSL

Configuring Confluent Kafka Accounts

You can configure your Confluent Kafka accounts in SnapLogic using either the Designer or Manager.

Using SnapLogic Designer

Drag a Kafka Snap to the Canvas and click the Snap to open its settings. Click the Account tab. You can now either use an existing account or create a new one.

Selecting an existing account

SnapLogic organizes and displays all accounts to which you have access, sorting them by account type and location. To select an existing account:

  1. Click the  icon to view the accounts to which you have access and select the account that you want to use.
  2. Click Image Modified to save the Snap settings.
    Image Modified

Creating an account

  1. Click Add Account in the Account Reference dialog.
  2. Select the Location in which you want to create the account, select the account type, and click ContinueThe Add Account dialog associated with the account type appears.
    Image Modified

  3. Enter the required account details. For detailed guidance on how to provide information associated with each account type, use the following links:
    • Kafka Account
    • Kafka Kerberos Account
    • Kafka SSL Account

      Info

      Enter additional information on this account in the Notes field of the Info tab. This will help you–and other users–understand the purpose of the account, especially if there are multiple accounts of the same type.


  4. Click Validate to verify the account, if the account type supports validation.

  5. Click Apply to complete configuring the Kafka account.

Using SnapLogic Manager

Use Manager to create accounts without associating them immediately with Pipelines.

Note

Accounts in SnapLogic are associated with projects. You can use accounts created in other projects only if you have at least Read access to them.

  1. In the left pane, browse to the project in which you want to create the account and click Create Account Confluent Kafka, followed by the appropriate account type.
    Image Modified

  2. Repeat the steps numbered 3 through 5 in the Creating an account section.

    Note

    Avoid updating account credentials while Pipelines using that account are executing. This may lead to unexpected results, including locking your account.


Account Encryption

Standard EncryptionIf you use Standard Encryption, the High sensitivity settings under Enhanced Encryption are used.
Enhanced Encryption

If the account you are using has the Enhanced Encryption feature, the account fields are encrypted for each sensitivity level as shown below:

  • High: [None]
  • Medium + High: [None]
  • Low + Medium + High[None]

Troubleshooting

ErrorReasonResolution
The message size exceeds the default message limit.By default, the message size limit is 1 MB. To increase this limit, you must change a few settings in the Confluent Kafka broker and Snap account. On the Confluent Kafka broker side, set the server.properties and producer.properties to a size larger than your message size.

In the  Advanced Kafka Properties field of the applicable Snap account settings, you must add  max.request.size and set it to a size larger than your message size. 

For example, you can increase the message size limit to 15 MB in any one of the following ways:

  1. Assign the following values to properties in Confluent Kafka broker > server.properties:
    • message.max.bytes = 15728640
    • replica.fetch.max.bytes = 15728640
    • max.request.size = 15728640
    • fetch.message.max.bytes = 15728640
  2. In Confluent Kafka broker > producer.properties, set:
    • max.request.size = 15728640
  3. In the required Snap account settings, add the following Key/Value pair in the Advanced Kafka Properties field:
    • max.request.size = 15728640


Insert excerpt
Kafka Snap Pack
Kafka Snap Pack
nopaneltrue