In this article
| Table of Contents | ||
|---|---|---|
|
Overview
You can deploy Snaplex instances in a Kubernetes environment. In this deployment, you host the resources, and the Snaplex type is a Groundplex, which you also administer, taking advantage of the orchestration that Kubernetes provides. This support extends to the Kubernetes Horizontal Pod Autoscaler (HPA) resources.
...
| Note |
|---|
This solution is distinct from the previous one supported in prior releases—the Helm Charts, methodology, and deployment steps are completely different. Even though the previous solution is still supported, we recommend that you deploy your Groundplex nodes in Kubernetes using the Helm Charts in this article. To take advantage of Elastic Scaling on existing Groundplex nodes, you must redeploy them using the Helm Charts contained in this article. |
Key Features
Graceful termination: Resources for the JCC nodes in the Groundplex now gracefully terminate when Pipeline executions complete.
HPA support in Helm Chart: You can use the Horizontal Pod Autoscale algorithm to define custom performance metrics.
CPU/Memory utilization metrics: You can now view the average of these metrics observed on Snaplex nodes can be collected through the Metrics Server.
Dashboard Support: You can set up the Grafana application monitoring for the scaling up and down of resources.
Summary of Implementation
Elastic Scaling is based on the Kubernetes Horizontal Pod Autoscaling, which operates on the ratio between the desired metric value and current metric value. The implementation requires the K8 Metrics Server and Prometheus adapters, both open source software common in Kubernetes. Grafana is an optional component that provides the dashboard monitoring for the scale up and scale down activities.
- To deploy the Elastic Scaling, see How to Deploy Deploying a Groundplex in Kubernetes with Elastic Scaling.
- To monitor your Groundplex nodes in Kubernetes, see Monitoring your Snaplexes in a Kubernetes Environment.
Workflow
- Org admin configures Groundplex nodes.
- Kubernetes Admin deploys Helm Chart.
- Groundplex node resources spin up and down depending on need.
- Org admin monitors Pipelines and Groundplex node resources through SnapLogic Dashboard.
- Kubernetes Admin monitors Elastic Scaling through analytics dashboard like Grafana.
Prerequisites
- You must install the following software:
- Metrics Server from GitHub
- Prometheus (Recommend Helm Chart version: 14.0.1)
- Prometheus Adapter (Recommend Helm Chart version: 2.12.1)
- Grafana
- You should have familiarity with the following topics:
- Provisioning Snaplex Nodes in Kubernetes using Helm Charts
- Horizontal Autoscaling
- Prometheus
Support and Limitations
The customized metric for queued Pipelines may not autoscale properly when Snaplex-based Scheduler is enabled for the Org.
Known Issues
None