In this Page
Table of Contents | ||
---|---|---|
|
Install a Snaplex on Linux
This article describes the procedures for installing an on-premise Snaplex on Linux OS. An on-premise Snaplex is also known as a Groundplex. This document uses the term Groundplex to distinguish this type of Snaplex (on-premise) from other types, such as a In this Page
Table of Contents | ||
---|---|---|
|
Install a Snaplex on Linux
This article describes the procedures for installing an on-premise Snaplex on Linux OS. An on-premise Snaplex is also known as a Groundplex. This document uses the term Groundplex to distinguish this type of Snaplex (on-premise) from other types, such as a Clouplex or an eXtremeplex. Note that in command syntax and throughout the UI, the generic term Snaplex is used.
...
- Log into SnapLogic Manager as an Org admin.
- Navigate to the project where you want to create your Groundplex, then do one of the following:
- Click the Snaplex tab, then click to display the Snaplex popup.
OR
- Click to display the Assets drop-down list, then select Snaplex; the Snaplex popup appears.
- Click the Snaplex tab, then click to display the Snaplex popup.
- Enter the required information on the Create Snaplex form. Once completed, the Downloads tab on the Snaplex popup is displayedDownloads tab on the Snaplex popup appears. The Downloads tab has links to the installer and configuration files.
- Download the RPM/DEB and the configuration file onto a Linux machine. The Downloads panel has links to the installer and configuration files.
For CentOS (or Redhat) 6.3 or newer, run the following command
$ sudo rpm -i <filename>.rpm
For Ubuntu 14.04 or newer, run the following command:
$ sudo dpkg -i <filename>.deb
.
where Where <filename> is the name of the current installer file.
After the software is installed, place the downloaded configuration file in the
/opt/snaplogic/etc
directory and make sure the file name ends with.slpropz
. Change the slpropz file so that snapuser owns it by running the following commands:Code Block $ sudo chown snapuser:snapuser /opt/snaplogic/etc/myplex.slpropz $ sudo chmod 600 /opt/snaplogic/etc/myplex.slpropz
- To start the Snaplex service, run:
$ sudo /opt/snaplogic/bin/jcc.sh start
- To verify the Snaplex has started, visit https://elastic.snaplogic.com/sl/dashboard.html#Health. The newly installed Snaplex node should show up in the list of nodes for the Snaplex.
...
Info | ||
---|---|---|
For Snaplex downloads prior to Fall 2016 (4.7):
The RPM installation needs to be performed as the root user. The service startup needs to be done as the root user, the JCC process itself runs as the user snapuser. This allows enhanced security by allowing the keys stored in /etc/snaplogic to be protected from access by regular users. If enhanced encryption is not being used, it is possible to enable the service startup to be done directly as snapuser. To do that, perform steps 1 to 4 as above, then do:
The service start and stop can then be done as snapuser without needing root access.
|
Setting up a FeedMaster
|
Setting up a FeedMaster
A FeedMaster is a type of node that you use when running tasks based on Ultra Pipelines. You can designate a Groundplex as a FeedMaster by setting the Snaplex node types on the Node Properties panel in the Update Snaplex dialog. For complete instructions, see Deploying a FeedMaster Node.
Info |
---|
Prior to Fall 2016 (4.7)Setting up a FeedMaster should be the same as a JCC (installing the rpm, RPM or copying keys, etc). The only configuration change is to add the following to the global.properties file:
That will switch switches to the FeedMaster mode instead of the standard JCC node. The FeedMaster does not have has the same outbound websocket requirement as the JCC, but it still connectivity requirements as a regular JCC and it needs to contact elastic.snaplogic.com over HTTPS and WSS. |
Import a Certificate to the SnapLogic JCC
While configuring accounts for various Snap Packs, you might encounter the following error message:
"Failed to validate account: Invalid credentials Cause: Could not send Message. (Reason: unable to find valid certification path to requested target; Resolution: Please provide valid credentials.)"
If this error occurs, update the cacert trust store in the SnapLogic JCCs to enable a successful SSL handshake to the endpoint.
...
JCC
To generate and import a self-signed certificate on all the JCC nodes in the Snaplex:
- Obtain the certificate (.pem) file by using SSH.
Import the file by using the following command in the JCC node:
Paste code macro /opt/snaplogic/pkgs/jre1.8.0_45/bin/keytool -import -file <path_to_pem_file_from_step_1> -alias <alias_name_optional> -keystore /opt/snaplogic/pkgs/
...
jre1.8.0_45/lib/security/cacerts
...
To import a certificate to the SnapLogic JCC:
- Get the certificate (.pem) file by using SSH.
- Import the file by using the following command in the JCC node
-vEnter keystore password: changeit
Note This example is an excerpt from 1.8.0_45; the directory path might vary based on the Java installation directory.
- Restart the JCC node process to confirm the import.
Run the following command to list certificates from the CAcerts file:
Paste code macro /opt/snaplogic/pkgs/jre1.8.0_45/bin/keytool -import -file <path_to_pem_file_from_step_1> -alias <alias_name_optional> .8.0_45/bin/keytool -list --keystore /opt/snaplogic/pkgs/jre1.8.0_45/lib/security/cacerts -vEnter keystore password: changeit
Note This example is an excerpt from 1.8.0_45; the directory path may vary based on the Java installation directory.
- Restart the JCC process to confirm the import.
Run the following command to list certificates from the cacerts file:
Paste code macro |
---|
/opt/snaplogic/pkgs/jre1.8.0_45/bin/keytool -list --keystore /opt/snaplogic/pkgs/jre1.8.0_45/lib/security/cacerts -v+ |
Note |
---|
The steps to import the certificate to the JCC can vary based on the certificate format and the OS. |
Automatically Start and Stop an On-premises Snaplex on Linux
You can use either the systemd
or init.d
utilty to start and stop the Snaplex. Procedures for both are included below. The procedures vary depending on which Linux distribution is installed on the Snaplex host.
Start/Stop the Snaplex by Using Systems
Red Hat-like distributions (Red Hat, Fedora Core, CentOS, and Suse)
To add the Snaplex as a Service:
...
Create the startup service file as follows:
Code Block |
---|
touch /etc/systemd/system/snaplogic.service |
Change the permissions on the file as follows:
Code Block |
---|
chmod 664 /etc/systemd/system/snaplogic.service |
...
Open the file wiht a text editor. For example, the vim
editor:
Code Block |
---|
vim /etc/systemd/system/snaplogic.service |
Add the following text to the file:
Paste code macro |
---|
[Unit]
Description=SnapLogic JVM
After=network.target
[Service]
Type=forking
ExecStart=/opt/snaplogic/bin/jcc.sh start
ExecReload=/opt/snaplogic/bin/jcc.sh restart
ExecStop=/opt/snaplogic/bin/jcc.sh stop
[Install]
WantedBy=default.target |
...
Enable the service as follows – the service will start automatically when the host reboots.
Code Block |
---|
systemctl enable snaplogic.service |
Start the service as follows:
Code Block |
---|
systemctl start snaplogic.service |
To delete the Snaplex as a service:
Disable the service as follows:
Code Block |
---|
systemctl disable snaplogic.service |
Start/Stop the Snaplex by Using init.d
RedHat-like Distribution (Red Hat, Fedora Core, CentOS, Suse)
To add the Snaplex as a Service:
- Login to the Linux machine as root.
Change directories as follows:
Code Block cd /etc/init.d/
Create a softlink to the jcc.sh file as follows:
Code Block ln -s /opt/snaplogic/bin/jcc.sh snaplex
Add the softlink to
chkconfig
management as follows:Code Block chkconfig --add snaplex
Note It is recommended to reboot the machine to verify whether the Snaplex service is restarting automatically on machine reboot. One issue which has been seen to cause the restart to fail is that the symlink resolution can fail when the machine is starting up. In that case, the Snaplex startup script can be changed to be a file instead of a symlink. To do that, run the following as root user:
rm /etc/init.d/snaplex cpCode Block .0_45/lib/security/cacerts -v+
Note |
---|
The steps to import the certificate to the JCC node can vary based on the certificate format and the OS. |
Info | ||
---|---|---|
| ||
While configuring accounts for various Snap Packs, you might encounter the following error message:
If this error occurs, update the CAcert trust store in the SnapLogic JCC nodes to enable a successful SSL handshake to the target endpoint. Find the trust store at |
Automatically Start and Stop a Groundplex on Linux
You can use either the systemd
or init.d
utility to start and stop the Snaplex. Procedures for both are included below. The procedures vary depending on which Linux distribution is installed on the Snaplex host.
Starting and Stopping the Groundplex by Using the Linux Systems Call
Use this procedure for Red Hat-like Linux distributions such as Red Hat, Fedora CoreOS, CentOS, and SuSE.
To add the Groundplex as a Service:
- Log into the Groundplex host as a root or sudo user.
Create the startup service file:
Code Block touch /etc/systemd/system/snaplogic.service
Change the permissions on the file:
Code Block chmod 664 /etc/systemd/system/snaplogic.service
This change provides read and write permissions for the owner and group, and read permission for others.
Open the file with a text editor. For example, the using a vim editor:
Code Block vim /etc/systemd/system/snaplogic.service
Add the following text to the file:
Paste code macro [Unit] Description=SnapLogic JVM After=network.target [Service] Type=forking ExecStart=/opt/snaplogic/bin/jcc.sh start ExecReload=/opt/snaplogic/bin/jcc.sh restart ExecStop=/opt/snaplogic/bin/jcc.sh stop [Install] WantedBy=default.target
- Save and exit the file.
Enable the service by running the following command:
Code Block systemctl enable snaplogic.service
The service will start automatically when the host reboots.
Start the service:
Code Block systemctl start snaplogic.service
To stop the Snaplex as a service, run the following command:
Code Block systemctl disable snaplogic.service
Starting and Stopping the Groundplex by Using the init.d Utility
Use this procedure for Red Hat-like Linux distributions such as Red Hat, Fedora CoreOS, CentOS, and SuSE.
To add the Groundplex as a service:
- Login to the Linux machine as root.
Change directories:
Code Block cd /etc/init.d/
Create a softlink to the
jcc.sh
file:Code Block ln -s /opt/snaplogic/bin/jcc.sh
/etc/init.d/snaplex
echo "export SL_ROOT=/opt/snaplogic" >> /etc/sysconfig/jcc
To delete the Snaplex as a service:
...
Add the softlink to
chkconfig
management:Code Block chkconfig --
...
add snaplex
Debian-like Distribution (Debian, Ubuntu)
To add the Snaplex as a Service:
...
Change directories as follows:
Code Block |
---|
cd /etc/init.d/ |
Create a soft-link to the jcc.sh file as follows:
...
Note | ||
---|---|---|
We recommend that you reboot the machine to verify if the Snaplex service is restarting automatically on machine reboot. Under some conditions, the symlink resolution might fail when the machine is starting up. In this case, you can change the Snaplex startup script to be a file instead of a symlink. Run the following commands a root user:
|
...
Install the service using update-rc.d. For example:
Code Block |
---|
sudo update-rc.d snaplex defaults 98 02 |
...
It is recommended to reboot the machine to verify whether the Snaplex service is restarting automatically on machine reboot. One issue which has been seen to cause the restart to fail is that the symlink resolution can fail when the machine is starting up. In that case, the Snaplex startup script can be changed to be a file instead of a symlink. To do that, run the following as root user:
...
|
To delete the Snaplex as a service, remove the service from the chkconfig
management:
Code Block |
---|
chkconfig --del snaplex |
Debian-like Distribution (Debian and Ubuntu)
To add the Snaplex as a Service:
...
- Log in to the Linux machine as root.
Change directories:
Code Block cd /etc/init.d/
snaplex cpCreate a soft-link to the
jcc.sh
file:Code Block ln -s /opt/snaplogic/bin/jcc.sh
/etc/init.d/snaplex echo "export SL_ROOT=/opt/snaplogic" >> /etc/sysconfig/jccOn a Debian system, the /etc/sysconfig directory would need to be created if not already present.
To delete the Snaplex as a Service:
Remove the service from update-rc.d management as follows:
Code Block |
---|
update-rc.d -f snaplex remove |
System Limits
Some Linux installations have system ulimit settings set to low values. This can cause errors when running higher pipeline load on the node. The error usually seen is "java.lang.OutOfMemoryError: unable to create new native thread". To fix this, system limits for the Snapuser user need to be increased. The below can be added in the /etc/security/limits.conf to increase the file and process limits.
Note |
---|
The JCC process needs to be restarted after the limits are updated. |
snaplex
Install the service using
update-rc.d
. For example:Code Block sudo update-rc.d snaplex defaults 98 02
Note title Troubleshooting if the machine reboot fails We recommend that you reboot the machine to verify whether the Snaplex service is restarting automatically on machine reboot. Under some conditions, the symlink resolution might fail when the machine is starting up. In this case, you can change the Snaplex startup script to be a file instead of a symlink. Run the following commands a root user:
Code Block rm /etc/init.d/snaplex cp /opt/snaplogic/bin/jcc.sh /etc/init.d/snaplex echo "export SL_ROOT=/opt/snaplogic" >> /etc/sysconfig/jcc
On a Debian system, the
/etc/sysconfig
directory would need to be created if not already present.To stop the Snaplex as a Service, remove the service from
update-rc.d
management:Code Block update-rc.d -f snaplex remove
Configuring Java 11 on Linux
...
- Stop the existing JCC node by running the following command:
$ sudo /opt/snaplogic/bin/jcc.sh stop
- Download the new Snaplex installer and install the Groundplex, running the RPM, DEB, or Docker installers as appropriate.
- For RPM systems, run the following command:
$
rpm -U snaplogic-snaplex.rpm
- For DEB systems, run the following command:
$
dpkg -i snaplogic-snaplex.deb
- For Docker, stop the existing container and start a new container using the latest image.
- For RPM systems, run the following command:
- Add the following entry to the
/etc/sysconfig/jcc
directory. You must create this directory and file if neither are present.export SL_JAVA_HOME=/opt/snaplogic/pkgs/jdk-11.0.1/
- Start the JCC node by running the following command:
$ sudo /opt/snaplogic/bin/jcc.sh start
.
...
- Run the following commands after installing the RPM/DEB package:
sudo mv /opt/snaplogic /myopt/myroot
sudo chown -R myuser /myopt/myroot
- Add the following entities in the
/etc/sysconfig/jcc
file. If this file does not exist, create it.export SL_USER=myuser
export SL_ROOT=/myopt/myroot
- Restart the service with the following command:
sudo /myopt/myroot/bin/jcc.sh restart
- To make a service using init.d, make the
/etc/init.d/snaplex
file a symlink to the/myopt/myroot/bin/jcc.sh
file.
Groundplex on Azure
When running a Linux Groundplex on Azure, the default TCP keep alive keepalive settings on Azure have been seen to might cause connectivity issues between the Groundplex and the SnapLogic control plane. The workaround is to disable keep alive the keepalive settings in the JCC configuration.
Add the following
...
entities in
global.properties:
Code Block jcc.jvm_options = -Dhttp.keepAlive=False -Dcom.ning.http.client.AsyncHttpClientConfig.allowPoolingConnection=false -Dcom.ning.http.client.AsyncHttpClientConfig.allowSslConnectionPool=false
- Then restart the JCC using
...
- the
/opt/snaplogic/bin/jcc.sh restart
...
- command.
Code Block |
---|
jcc.jvm_options = -Dhttp.keepAlive=False -Dcom.ning.http.client.AsyncHttpClientConfig.allowPoolingConnection=false -Dcom.ning.http.client.AsyncHttpClientConfig.allowSslConnectionPool=false |
Note |
---|
Note |
If you are unable to create an SLDB file using international language characters (such as æ, Æ, Ø) in the file name, update the |
System Limits
Some Linux installations have system ulimit settings that are set to low values. This low setting can cause errors when running higher Pipeline loads on the Grounplex node, such as: java.lang.OutOfMemoryError: unable to create new native thread
. To fix this issue, you need to increase the system limits for the Snapuser user. You can add the following in the /etc/security/limits.conf
folder to increase the file and process limits.
Note |
---|
The JCC process needs to be restarted after the limits are updated. |
Code Block |
---|
snapuser soft nproc 8192
snapuser hard nproc 65536
snapuser soft nofile 8192
snapuser hard nofile 65536 |