Versions Compared

Version Old Version 10 New Version Current
Changes made by

Kalpana Malladi

Kalpana Malladi

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

You can use this account type to connect Kafka Snaps with data sources that use the Kafka MSK IAM Account.

Prerequisites

...

Kafka MSK Cluster Setup.

...

IAM access control

...

This account detects an IAM role in the environment and uses an IAM role assigned to an EC2 Groundplex. It also supports Amazon Elastic Container Service (ECS) and Amazon Elastic Kubernetes Service (EKS), both of which require additional configuration. Learn more. The IAM role either must have sufficient permissions to access the MSK cluster or can assume another IAM role with these permissions. If the role has sufficient permissions, then none of the Cross Account IAM fields need to be set.

Prerequisites

  • IAM access

Limitations and Known Issues

...

Role

 

Default Value: Deselected

Select this checkbox to use the IAM role associated with the EC2 instance . Learn more - Access the MSK cluster from inside AWS but outside the cluster's Amazon VPC.

The IAM role applies only to EC2-type Groundplexes. For the required configuration, refer to the IAM Access Control for Amazon Managed Streaming for Apache Kafka.

Cross account IAM propertiesSession duration (seconds)

Default Value: N/A
Example: 900

Field Name

Field Type

Description

Label*

 

Default Value: Kafka MSK IAM Account
Example: Kafka_Client_Auth_MSK_IAM

String

Specify a unique label for the account.

 

Bootstrap servers*

Use this field set to specify the ordered list of host-port pairs to establish an initial connection to the Kafka cluster.

Bootstrap server

 

Default Value: N/A
Example: 

b-1.kafka-cluster-name.abcde.c5.kafka.us-west-2.amazonaws.com:9092

String/Expression

Specify a host-port pair that you use to establish an initial connection to the Kafka cluster.

 

Schema registry URL

 

Default Value: N/A
Example: http://localhost:8081

String/Expression

Specify the URL for the schema registry server.

Advanced Kafka properties

Use this field set to specify any additional properties to connect to the Kafka server not explicitly provided in the Snap.

These properties are directly passed to the Kafka server and not tested by SnapLogic, Inc.

Key

 

Default Value: N/A
Example: session.timeout.ms

String/Expression

Specify the key for the Kafka property that the Snap does not explicitly support.

 

Value

 

Default Value: N/A
Example: 10000

Integer/Expression

Specify the value for the Kafka property that Snap does not explicitly support.

 

Security protocol

 

Default Value: SASL_SSL
Example: SSL

  String/Expression

Select one of the following security protocols from the suggestions:

  • SSL

  • SASL_SSL

  • SASL_PLAINTEXT

Cross account IAM

Checkbox

properties

The IAM role either must have sufficient permissions to access the MSK cluster

or can assume another IAM role with these permissions. If the role has sufficient permissions, then none of the Cross Account IAM fields need to be set.

AWS role ARN

 

Default Value: N/A
Example: arn:aws:iam::12345678929:role/snaptest-msk-cluster-read-write-role

String/Expression

Specify the ARN of the cross-account IAM role. This ARN defines the permissions and trust policies for assuming the role. Learn more about ARNs.

External ID

 

Default Value: N/A
Example: my-external-id-12345

String/Expression

Specify the external ID to add an extra layer of security by preventing ‘confused deputy’ attacks. Learn more - confused deputy attacks.

You must configure this field when a third party assumes the role.

AWS region

 

Default Value: N/A
Example: us-west-2

String/Expression

Specify the AWS region where the application is running.

  • Using a region-specific endpoint that matches the MSK cluster’s region can improve the performance.

  • If you do not specify a region, the AWS global default region is used.

 

 

String/Expression

Specify the duration in seconds for which the assumed role session is valid.

  • The session duration can range from 900 seconds (15 minutes) to the maximum session duration set for the field.

  • By default, the maximum session duration is 1 hour, but you can set it to a maximum of 12 hours.

    • If the maximum session duration exceeds the maximum duration configured for the role, the request is denied.

    Session name

     

    Default Value: N/A
    Example: kafka-access-session-2024-09-24

    String/Expression

    Specify an identifier for the assumed role session. This identifier helps to uniquely identify a session when different entities assume the same role.

    ...

    Error

    Reason

    Resolution

    Error assuming the role with roleArn.

    The Role ARN, External ID, or the session duration has incorrect information.

    Ensure that the role configuration, ARN, external ID, and session duration are correct, check AWS service status, handle exceptions properly, and consult customer support if needed.contact Customer support if needed.

    Debug IAM Permissions

    The Kafka MSK IAM Account supports debugging for IAM configuration. To enable IAM debugging, you must update the Snaplex configuration, specifically the logging level, and define a JVM system property. When you enable IAM debugging (Global properties and Logging Level) in the Snaplex configuration, the account logs the IAM credential identity in the Snaplex log.

    Note

    Enabling the debug property might have a potential performance implication. Therefore, we recommend that you use this feature in your development environment instead of the production environment. If you need to enable this property in your production environment, we advise you to leave it enabled only until the issue is resolved and then disable it afterward.

    1. On the Logging tab, set the logging level to Debug.

      update-snaplex-logging.pngImage Added

    2. On the Node Properties tab, define a Global Property for JVM system properties as shown below or append a value to an existing property.
      Key: jcc.jvm_options
      Value: -Daws.msk.iam.debug=true

      update-snaplex-node-properties.pngImage Added

    3. Restart the Snaplex node.

    Info

    Executing or validating a pipeline with a Kafka Snap that uses the MSK IAM Account logs the credential identity. In the example log entry below, the IAM role mcb-msk-role-2 is assigned to an EC2, which can assume the IAM role mcb-msk-role-1, which the debug log entry confirms.

    ...

    Insert excerpt
    Kafka Snap Pack
    Kafka Snap Pack
    nopaneltrue

    ...