In this article

Overview

SnapLogic supports Kubernetes orchestration on your Groundplex instances. You can deploy Snaplex nodes in your Kubernetes environment by setting up a Helm Chart that defines the node configuration for discoverability in the Kubernetes environment. This article explains how you can deploy and configure SnapLogic Snaplex nodes in a Kubernetes environment and contains an attached Helm Chart that you can use.

Workflow

Prerequisites

• You must be an Org-level administrator.

• You are familiar with the SnapLogic Snaplex installation process.

Downloading the Configuration File from SnapLogic Manager

1. Open an existing Snaplex in the Org:

   1. Navigate to the target Snaplex in Manager.

   2. Click on the Snaplex name to display the Update Snaplex dialog.
      Alternatively, if none exists, Create a Snaplex. 

2. On the Downloads tab, download the Configuration file using the option or use the API api/1/rest/public/snaplex/config/<path_to_groundplex> to download the slpropz file.

   

3. Click Cancel to exit the dialog.

4. Use the API to retrieve the information needed to configure a node on a Groundplex.
   GET https://{controlplane_path}/api/1/rest/public/snaplex/config/{plex_path}?{query_parameters}.
   An example would be similar to the path provided below: https://elastic.snaplogic.com/api/1/rest/public/snaplex/config/snaplogic/shared/MyGround

   1. Use the path retrieved from the API in the snaplogic_config_link parameter of the Helm chart. Learn more at Retrieve config info for a Groundplex and Download the slpropz configuration file

5. Set the parameter snaplogic_secret in the Helm chart YAML file to the name of the Kubernetes secret you create, as described in the Deploy the Helm Chart section.

Running the Snaplex with Org Credentials

You can associate Org admin credentials with the SnapLogic secret created when enabling enhanced encryption. This makes it easier to share the Snaplex service with your organization's users. The SnapLogic Org admin credentials and the enhanced encryption secret are in JSON format as key/value pairs.

1. Generate a key and encode it for each value.
   NOTE: We recommend that you use Base64 to encode the values.

2. To create the SnapLogic secret:

   1. Create the YAML file with the following two keys: username and password.

      Example YAML File

      apiVersion: v1
      kind: Secret
      metadata:
        name: mysecret
      type: Opaque
      data:
        username: <Base64 username>
        password: <Base64 password>

      TIP: Run the following command to encode your username/password into the text of the secret:
      $ echo -n "snaplogic_username_or_password" | Base64

      IMPORTANT: If your password includes any of the following characters, you must escape the character with a backslash (\) in the string that you pass to the encoder:
      \ (backslash)
      $ (dollar sign)
      ' (apostrophe or single-quote)
      ` (backtick)
      " (double-quotes)
      & (ampersand)
      | (pipe symbol)
      ; (semicolon)
      ! (exclamation mark)
      For example, if your password is mypa$$word, pass the string mypa\$\$word to the Base64 encoder.
      

   2. Run the following command:

      $ kubectl apply -f snaplogic_secret.yaml

    

3. (Optional) If Enhanced Encryption is enabled for your Org, create the Enhanced Encryption secret by running the following commands:
   $ kubectl create secret generic enhanced-encryption-secret --from-file=keystore_jks --from-file=keystore_pass
$ kubectl apply -f enhanced_encryption_secret.yaml
   

4. After the secret is created, delete the YAML file because it is no longer needed.

You can now deploy the Helm Chart.

Deploy the Helm chart

About the Helm Chart

The helm chart defines the values for your Groundplex nodes in your Kubernetes environment.

You can download the Helm Chart package, which contains the following:

• values.yaml: This file is the helm chart.

• templates folder: Boilerplate charts based on parameters.

• Chart.yaml: This file contains metadata about the artifact repository.

Helm Chart Fields

The following list describes each field parameter in the values.yaml file:

Example Helm Chart

# Default values for snaplogic-snaplex.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
# Regular nodes count
jccCount: 1
# Feedmaster nodes count
feedmasterCount: 1
# Docker image of SnapLogic snaplex
image:
  repository: snaplogic/snaplex
  tag: latest
# SnapLogic configuration link
snaplogic_config_link:
# SnapLogic Org admin credential
#snaplogic_secret:
# Enhanced encryption secret
#enhanced_encrypt_secret:
# CPU and memory limits/requests for the nodes
limits:
  memory: 8Gi
  cpu: 2000m
requests:
  memory: 8Gi
  cpu: 2000m
# Default file ulimit and process ulimit
sl_file_ulimit: 8192
sl_process_ulimit: 4096
# Enable/disable startup, liveness and readiness probes
probes:
  enabled: true
# JCC HPA
autoscaling:
  enabled: false
  minReplicas:
  maxReplicas:
  # Average count of Snaplex queued pipelines (e.g. targetPlexQueueSize: 5), leave empty to disable
  # To enable this metric, Prometheus and Prometheus-Adapter are required to install.
  targetPlexQueueSize:
  # Average CPU utilization (e.g. targetAvgCPUUtilization: 50 means 50%), leave empty to disable.
  # To enable this metric, Kubernetes Metrics Server is required to install.
  targetAvgCPUUtilization:
  # Average memory utilization (e.g. targetAvgMemoryUtilization: 50 means 50%), leave empty to disable.
  # To enable this metric, Kubernetes Metrics Server is required to install.
  targetAvgMemoryUtilization:
  # window to consider waiting while scaling up. default is 0s if empty.
  scaleUpStabilizationWindowSeconds:
  # window to consider waiting while scaling down. default is 300s if empty.
  scaleDownStabilizationWindowSeconds:
# grace period seconds after JCC termination signal before force shutdown, default is 30s if empty.
terminationGracePeriodSeconds: 900
# Enable IPv6 service for DNS routing to pods
enableIPv6: false

Steps

1. Configure the following parameters in the Helm Chart and name the file values.yaml.

2. In the Helm Chart console, run the following command: 

   $ helm install snaplogic <helm_chart_folder>
   Where <helm_chart_folder> is the Helm Chart zip file, which you can download from this document.

3. Run the helm list command to determine the status of the deployment.
   The following sample output shows a successful deployment:

   

   If you deploy the Snaplex JCC nodes but resources in the Kubernetes environment are not available to fulfill the Helm Chart, then your deployment goes into a pending state until sufficient resources are available. The following sample output shows the pending status of resources:
   

   

After you deploy your Helm Chart, you can deploy a load balancer.

Configuring IPv6

To set up IPv6 on Kubernetes:

1. In the values.yaml file of the Helm Chart, set the value enableIPv6: true.

2. Set the global property for the jcc.k8s_subdomain_service in this format -- <Helm Release Name>-snaplogic-snaplex-ipv6.

Deploy a load balancer

To add load balancers to your JCC and FeedMaster nodes:

1. In the Helm console, run the helm list command to list the services. 

2. In SnapLogic Manager, navigate to the target Project folder, then click the target Snaplex; the Update Snaplex dialog appears. 

3. On the Settings tab of the Update Snaplex dialog, enter the corresponding values in the following fields:

   • Load balancer. Enter the protocol and port number of the Snaplex JCC node. Refer to PORT(s) associated with snaplogic-snaplex-regular.

   • Ultra load balancer. Enter the protocol and port number of the FeedMaster node. Refer to PORT(s) associated with snaplogic-snaplex-feed.

4. Review the information, then click Update.

After your Snaplex and FeedMaster nodes are deployed, you can start designing and running Pipelines and Tasks.

Installing utilities

For Kubernetes-based deployments, users have to build images/containers that install the utilities in appropriate locations. Those images can use the official Snaplogic image as the base image. When deploying the Snaplex to Kubernetes, the users would then use that image/container for deployment and have necessary dependencies and utilities in place.

Disk sizing guidelines

By default, Kubernetes pods use the disk space of the node they run on, called ephemeral disk. If ephemeral disk runs low, the Kubernetes pod taking the most disk space on the node will be evicted (e.g. restarted), and that disk space will be freed up. Kubernetes pods do not retain ephemeral disk space across restarts, so each time a pod restarts, its filesystem will be essentially cleared.

The amount of ephemeral disk space a Kubernetes worker node needs is dependent on the workloads running on that node, e.g., the number of ground plexes on K8s, the number of other pods, etc.

If more disk space is needed, a Persistent Volume can be used. In cloud environments like AWS, this is often EBS storage. Persistent volumes can be mounted to pods, and they retain data across restarts.

Best Practices

• Avoid running processes in the same container as the JCC so that the JCC has the maximum amount of memory available, as requested.

• Do not overwrite the global properties options unless you are working with your CSM to customize your Groundplex.

• Request resources upfront. The requests determine the minimum required resources, while limits set the maximum resources a Container can consume. Setting them to the same amount ensures stability and exact resource usage. To do this, set the pod’s request and limit to the same value, as shown in the image below:

  

FAQ

Downloads

Download and extract the following files, using the values.yaml file as the basis for your Helm Chart.

See Also

• What is a Container?

• The CentOS Project

• On-premises Snaplex/Groundplex