Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In this Page

Table of Contents
maxLevel2

Note

The Linux installer comes bundled with the JRE necessary for this Snaplex.

Install a Snaplex On-Premises on Linux

To install a Snaplex on-premises on Linux:

...

For CentOS (or Redhat) 6.3 or newer, do sudo rpm -i <filename>.rpm.
For Ubuntu 14.04 or newer, do sudo dpkg -i <filename>.deb.
where <filename> is the name of the current installer file.  

After the software has been installed, place the downloaded configuration file in the "/opt/snaplogic/etc" directory and make sure the file name ends with .slpropz. Change the slpropz file to be owned by snapuser user, by running:

Code Block
sudo chown snapuser:snapuser /opt/snaplogic/etc/myplex.slpropz
sudo chmod 600 /opt/snaplogic/etc/myplex.slpropz

...

In this Page

Table of Contents
maxLevel2

Note

The Linux installer comes bundled with the JRE necessary for this Snaplex.

Install a Snaplex On-Premises on Linux

To install a Snaplex on-premises on Linux:

  1. As an org admin, log into Manager.
  2. Under Project Spaces on the left, locate the project where you want to create your Groundplex (or the global "shared" project), then select the Snaplexes tab on the right.
  3. Click the Create button and complete the Create dialog (later known as the Updating Snaplex dialog). Once completed, the dialog reopens to the Downloads panel.
  4. Download the RPM/DEB and the configuration file onto a Linux machine. The Downloads panel has links to the installer and config files.
  5. For CentOS (or Redhat) 6.3 or newer, do sudo rpm -i <filename>.rpm.
    For Ubuntu 14.04 or newer, do sudo dpkg -i <filename>.deb.
    where <filename> is the name of the current installer file.  

  6. After the software has been installed, place the downloaded configuration file in the "/opt/snaplogic/etc" directory and make sure the file name ends with .slpropz. Change the slpropz file to be owned by snapuser user, by running:

    Code Block
    sudo chown snapuser:snapuser /opt/snaplogic/etc/myplex.slpropz
    sudo chmod 600 /opt/snaplogic/etc/myplex.slpropz


  7. To start the on-premises service, type:  sudo /opt/snaplogic/bin/jcc.sh start

  8. To verify the Snaplex has started, visit https://elastic.snaplogic.com/sl/dashboard.html#Health. The newly install node should show up in the list of nodes for the Snaplex.


Info

For Snaplex downloads prior to Fall 2016 (4.7):

  1. Download the RPM/DEB provided by SnapLogic onto a Linux machine.

  2. For CentOS (or Redhat) 6.3 or newer, do sudo rpm -i <filename>.rpm.
    For Ubuntu 14.04 or newer, do sudo dpkg -i <filename>.deb.
    where <filename> is the name of the current installer file.

  3. Copy the keys.properties file provided by SnapLogic onto /opt/snaplogic/etc/keys.properties

  4. Edit /opt/snaplogic/etc/global.properties.
    Update jcc.subscriber_id and jcc.environment to the values provided by SnapLogic.
    You may also change the entry for the Java heap size (jcc.heap.max_size) to whatever the maximum JVM heap space on the machine should be set to. A reasonable heuristic is to set it to 1GB less than the amount of RAM available on the machine.  If you don’t know, leave it as it is; in the future, you may want to check with the SnapLogic team for possible adjustment. 

  5. To start the on-premises service, type:  sudo /opt/snaplogic/bin/jcc.sh start 

  6. To verify the Snaplex has started, visit https://elastic.snaplogic.com/sl/dashboard.html#Health. The newly install node should show up in the list of nodes for the Snaplex.

The RPM installation needs to be performed as the root user. The service startup needs to be done as the root user, the JCC process itself runs as the user snapuser. This allows enhanced security by allowing the keys stored in /etc/snaplogic to be protected from access by regular users. If enhanced encryption is not being used, it is possible to enable the service startup to be done directly as snapuser. To do that, perform steps 1 to 4 as above, then do:

  1. sudo /opt/snaplogic/bin/jcc.sh

...

To verify the Snaplex has started, visit https://elastic.snaplogic.com/sl/dashboard.html#Health. The newly install node should show up in the list of nodes for the Snaplex.

...

For Snaplex downloads prior to Fall 2016 (4.7):

  1. Download the RPM/DEB provided by SnapLogic onto a Linux machine.

  2. For CentOS (or Redhat) 6.3 or newer, do sudo rpm -i <filename>.rpm.
    For Ubuntu 14.04 or newer, do sudo dpkg -i <filename>.deb.
    where <filename> is the name of the current installer file.

  3. Copy the keys.properties file provided by SnapLogic onto /opt/snaplogic/etc/keys.properties

  4. Edit /opt/snaplogic/etc/global.properties.
    Update jcc.subscriber_id and jcc.environment to the values provided by SnapLogic.
    You may also change the entry for the Java heap size (jcc.heap.max_size) to whatever the maximum JVM heap space on the machine should be set to. A reasonable heuristic is to set it to 1GB less than the amount of RAM available on the machine.  If you don’t know, leave it as it is; in the future, you may want to check with the SnapLogic team for possible adjustment. 

  5. To start the on-premises service, type:  sudo /opt/snaplogic/bin/jcc.sh start 

  6. To verify the Snaplex has started, visit https://elastic.snaplogic.com/sl/dashboard.html#Health. The newly install node should show up in the list of nodes for the Snaplex.

The RPM installation needs to be performed as the root user. The service startup needs to be done as the root user, the JCC process itself runs as the user snapuser. This allows enhanced security by allowing the keys stored in /etc/snaplogic to be protected from access by regular users. If enhanced encryption is not being used, it is possible to enable the service startup to be done directly as snapuser. To do that, perform steps 1 to 4 as above, then do:

  1. sudo /opt/snaplogic/bin/jcc.sh genkey
  2. sudo chown -R snapuser:snapuser /etc/snaplogic
  3. sudo su - snapuser
  4. /opt/snaplogic/bin/jcc.sh start

The service start and stop can then be done as snapuser without needing root access.

Note

By default, the Snaplex does not automatically restart in the event that the server is rebooted. Your system administrator would need to restart the Snaplex manually per the steps above. Alternatively, you can configure the Snaplex to run as a service to run automatically on startup or reboot.

FeedMaster

You can designate a Groundplex as a FeedMaster by setting the Snaplex node types on the Node Properties panel in the Update Snaplex dialog.

Info

Prior to Fall 2016 (4.7)

Setting up a FeedMaster should be the same as a JCC (installing the rpm, copying keys, etc). The only configuration change is to add the following to the global.properties file:

server_type = feed_master

That will switch to the FeedMaster mode instead of JCC. The FeedMaster does not have the same outbound websocket requirement as the JCC, but it still needs to contact elastic.snaplogic.com.

Import a Certificate to the SnapLogic JCC

While configuring accounts for various Snap Packs, you might encounter the following error message:

"Failed to validate account: Invalid credentials Cause: Could not send Message. (Reason: unable to find valid certification path to requested target; Resolution: Please provide valid credentials.)"

If this error occurs, update the cacert trust store in the SnapLogic JCCs to enable a successful SSL handshake to the endpoint.

Find the trust store at /opt/snaplogic/pkgs/java version/lib/security/cacerts, and confirm that every node in the Groundplex has the certificate imported. Perform the following task (for Linux) to generate and import a self-signed certificate on all the nodes in the Snaplex.

To import a certificate to the SnapLogic JCC:

...

Import the file by using the following command in the JCC node:

Paste code macro
 
/opt/snaplogic/pkgs/jre1.8.0_45/bin/keytool -import -file <path_to_pem_file_from_step_1> -alias <alias_name_optional> -keystore /opt/snaplogic/pkgs/jre1.8.0_45/lib/security/cacerts -vEnter keystore password: changeit
Note

This example is an excerpt from 1.8.0_45; the directory path may vary based on the Java installation directory.

...

Run the following command to list certificates from the cacerts file:

Paste code macro
 
/opt/snaplogic/pkgs/jre1.8.0_45/bin/keytool -list --keystore /opt/snaplogic/pkgs/jre1.8.0_45/lib/security/cacerts -v+
Note

The steps to import the certificate to the JCC can vary based on the certificate format and the OS.

Automatically Start and Stop an On-premises Snaplex on Linux

You can use either the systemd or init.d utilty to start and stop the Snaplex. Procedures for both are included below. The procedures vary depending on which Linux distribution is installed on the Snaplex host.

Start/Stop the Snaplex by Using Systems

Red Hat-like distributions (Red Hat, Fedora Core, CentOS, and Suse)

To add the Snaplex as a Service:

...

Create the startup service file as follows:

Code Block
touch /etc/systemd/system/snaplogic.service

Change the permissions on the file as follows:

Code Block
chmod 664 /etc/systemd/system/snaplogic.service

...

Open the file wiht a text editor. For example, the vim editor:

Code Block
 vim /etc/systemd/system/snaplogic.service

Add the following text to the file:

Paste code macro
[Unit]
Description=SnapLogic JVM
After=network.target

[Service]
Type=forking
ExecStart=/opt/snaplogic/bin/jcc.sh start
ExecReload=/opt/snaplogic/bin/jcc.sh restart
ExecStop=/opt/snaplogic/bin/jcc.sh stop

[Install]
WantedBy=default.target

...

Enable the service as follows – the service will start automatically when the host reboots.

Code Block
 systemctl enable snaplogic.service

Start the service as follows:

...

  1. genkey
  2. sudo chown -R snapuser:snapuser /etc/snaplogic
  3. sudo su - snapuser
  4. /opt/snaplogic/bin/jcc.sh start

The service start and stop can then be done as snapuser without needing root access.

Note

By default, the Snaplex does not automatically restart in the event that the server is rebooted. Your system administrator would need to restart the Snaplex manually per the steps above. Alternatively, you can configure the Snaplex to run as a service to run automatically on startup or reboot.


FeedMaster

You can designate a Groundplex as a FeedMaster by setting the Snaplex node types on the Node Properties panel in the Update Snaplex dialog.

Info

Prior to Fall 2016 (4.7)

Setting up a FeedMaster should be the same as a JCC (installing the rpm, copying keys, etc). The only configuration change is to add the following to the global.properties file:

server_type = feed_master

That will switch to the FeedMaster mode instead of JCC. The FeedMaster does not have the same outbound websocket requirement as the JCC, but it still needs to contact elastic.snaplogic.com.

Import a Certificate to the SnapLogic JCC

While configuring accounts for various Snap Packs, you might encounter the following error message:

"Failed to validate account: Invalid credentials Cause: Could not send Message. (Reason: unable to find valid certification path to requested target; Resolution: Please provide valid credentials.)"

If this error occurs, update the cacert trust store in the SnapLogic JCCs to enable a successful SSL handshake to the endpoint.

Find the trust store at /opt/snaplogic/pkgs/java version/lib/security/cacerts, and confirm that every node in the Groundplex has the certificate imported. Perform the following task (for Linux) to generate and import a self-signed certificate on all the nodes in the Snaplex.

To import a certificate to the SnapLogic JCC:

  1. Get the certificate (.pem) file by using SSH.
  2. Import the file by using the following command in the JCC node:

    Paste code macro
     
    /opt/snaplogic/pkgs/jre1.8.0_45/bin/keytool -import -file <path_to_pem_file_from_step_1> -alias <alias_name_optional> -keystore /opt/snaplogic/pkgs/jre1.8.0_45/lib/security/cacerts -vEnter keystore password: changeit


    Note

    This example is an excerpt from 1.8.0_45; the directory path may vary based on the Java installation directory.


  3. Restart the JCC process to confirm the import.
  4. Run the following command to list certificates from the cacerts file:

    Paste code macro
     
    /opt/snaplogic/pkgs/jre1.8.0_45/bin/keytool -list --keystore /opt/snaplogic/pkgs/jre1.8.0_45/lib/security/cacerts -v+


Note

The steps to import the certificate to the JCC can vary based on the certificate format and the OS.

Automatically Start and Stop an On-premises Snaplex on Linux

You can use either the systemd or init.d utilty to start and stop the Snaplex. Procedures for both are included below. The procedures vary depending on which Linux distribution is installed on the Snaplex host.

Start/Stop the Snaplex by Using Systems

Red Hat-like distributions (Red Hat, Fedora Core, CentOS, and Suse)

To add the Snaplex as a Service:

  1. Log in to the Snaplex host as a root or sudo user.
  2. Create the startup service file as follows:

    Code Block
    touch /etc/systemd/system/snaplogic.service


  3. Change the permissions on the file as follows:

    Code Block
    chmod 664 /etc/systemd/system/snaplogic.service

    This change provides read and write permissions for the owner and group, and read permission for others.

  4. Open the file wiht a text editor. For example, the vim editor:

    Code Block
     vim /etc/systemd/system/snaplogic.service


  5. Add the following text to the file:

    Paste code macro
    [Unit]
    Description=SnapLogic JVM
    After=network.target
    
    [Service]
    Type=forking
    ExecStart=/opt/snaplogic/bin/jcc.sh start
    ExecReload=/opt/snaplogic/bin/jcc.sh restart
    ExecStop=/opt/snaplogic/bin/jcc.sh stop
    
    [Install]
    WantedBy=default.target


  6. Save and exit the file.
  7. Enable the service as follows – the service will start automatically when the host reboots.

    Code Block
     systemctl enable snaplogic.service


  8. Start the service as follows:

    Code Block
    systemctl start snaplogic.service


To delete the Snaplex as a service:

Disable the service as follows:

Code Block
 systemctl disable snaplogic.service

Start/Stop the Snaplex by Using init.d

RedHat-like Distribution (Red Hat, Fedora Core, CentOS, Suse)

To add the Snaplex as a Service:

  1. Login to the Linux machine as root.
  2. Change directories as follows:

    Code Block
    cd /etc/init.d/


  3. Create a softlink to the jcc.sh file as follows: 

    Code Block
    ln -s /opt/snaplogic/bin/jcc.sh snaplex


  4. Add the softlink to chkconfig management as follows:

    Code Block
    chkconfig --add snaplex


    Note

    It is recommended to reboot the machine to verify whether the Snaplex service is restarting automatically on machine reboot. One issue which has been seen to cause the restart to fail is that the symlink resolution can fail when the machine is starting up. In that case, the Snaplex startup script can be changed to be a file instead of a symlink. To do that, run the following as root user:

    Code Block
    rm /etc/init.d/snaplex
    cp /opt/snaplogic/bin/jcc.sh /etc/init.d/snaplex
    echo "export SL_ROOT=/opt/snaplogic" >> /etc/sysconfig/jcc



To delete the Snaplex as a service:

Disable the service Remove the service from chkconfig management as follows:

Code Block
 systemctl disable snaplogic.service

Start/Stop the Snaplex by Using init.d

...

chkconfig --del snaplex

Debian-like Distribution (Debian, Ubuntu)

To add the Snaplex as the Snaplex as a Service:

  1. Login Log in to the Linux machine as root.
  2. Change directories as follows:

    Code Block
    cd /etc/init.d/


  3. Create a softlink to a soft-link to the jcc.sh file as follows: 

    Code Block
    ln -s /opt/snaplogic/bin/jcc.sh snaplex


  4. Add the softlink to chkconfig management as follows:Install the service using update-rc.d. For example: 

    Code Block
    chkconfigsudo update--addrc.d snaplex defaults 98 02


    Note

    It is recommended to reboot the machine to verify whether the Snaplex service is restarting automatically on machine reboot. One issue which has been seen to cause the restart to fail is that the symlink resolution can fail when the machine is starting up. In that case, the Snaplex startup script can be changed to be a file instead of a symlink. To do that, run the following as root user:

    Code Block
    rm /etc/init.d/snaplex
    cp /opt/snaplogic/bin/jcc.sh /etc/init.d/snaplex
    echo "export SL_ROOT=/opt/snaplogic" >> /etc/sysconfig/jcc

    On a Debian system, the /etc/sysconfig directory would need to be created if not already present.


To delete the Snaplex as the Snaplex as a serviceService:

Remove the service from chkconfig the service from update-rc.d management as follows:

Code Block
chkconfigupdate-rc.d --delf snaplex

Debian-like Distribution (Debian, Ubuntu)

To add the Snaplex as a Service:

...

Change directories as follows:

Code Block
cd /etc/init.d/

Create a soft-link to the jcc.sh file as follows: 

Code Block
ln -s /opt/snaplogic/bin/jcc.sh snaplex

Install the service using update-rc.d. For example: 

Code Block
sudo update-rc.d snaplex defaults 98 02

...

It is recommended to reboot the machine to verify whether the Snaplex service is restarting automatically on machine reboot. One issue which has been seen to cause the restart to fail is that the symlink resolution can fail when the machine is starting up. In that case, the Snaplex startup script can be changed to be a file instead of a symlink. To do that, run the following as root user:

...

 remove

System Limits

Some Linux installations have system ulimit settings set to low values. This can cause errors when running higher pipeline load on the node. The error usually seen is "java.lang.OutOfMemoryError: unable to create new native thread". To fix this, system limits for the Snapuser user need to be increased. The below can be added in the /etc/security/limits.conf to increase the file and process limits.

Note

The JCC process needs to be restarted after the limits are updated.


Code Block
snapuser soft nproc 8192
snapuser hard nproc 65536
snapuser soft nofile 8192
snapuser hard nofile 65536

Configuring Java 11 on Linux

To switch existing nodes to OpenJDK Java 11:

  1. Stop the existing JCC node by running the following command:
    $ sudo /opt/snaplogic/bin/jcc.sh /etc/init.d/snaplex echo "export SL_ROOT=/opt/snaplogic" >> /etc/sysconfig/jcc

    On a Debian system, the /etc/sysconfig directory would need to be created if not already present.

To delete the Snaplex as a Service:

Remove the service from update-rc.d management as follows:

Code Block
update-rc.d -f snaplex remove

System Limits

Some Linux installations have system ulimit settings set to low values. This can cause errors when running higher pipeline load on the node. The error usually seen is "java.lang.OutOfMemoryError: unable to create new native thread". To fix this, system limits for the Snapuser user need to be increased. The below can be added in the /etc/security/limits.conf to increase the file and process limits.

Note

The JCC process needs to be restarted after the limits are updated.

...

  1. stop

  2. Download the new Snaplex installer and install the Groundplex, running the RPM, DEB, or Docker installers as appropriate.
    • For RPM systems, run the following command:
      rpm -U snaplogic-snaplex.rpm
    • For DEB systems, run  the following command:
      dpkg -i snaplogic-snaplex.deb
    • For Docker, stop the existing container and start a new container using the latest image.

  3. Add the following entry to the /etc/sysconfig/jcc directory. You must create this directory and file if neither are present. 
    export SL_JAVA_HOME=/opt/snaplogic/pkgs/jdk-11.0.1/

  4. Start the JCC node by running the following command:
    $ sudo /opt/snaplogic/bin/jcc.sh start.

Groundplex on Azure

When running a Linux Groundplex on Azure, the default TCP keep alive settings on Azure have been seen to cause connectivity issues between the Groundplex and the SnapLogic control plane. The workaround is to disable keep alive in the JCC configuration. Add the following in global.properties and restart the JCC using "/opt/snaplogic/bin/jcc.sh restart"

Code Block
jcc.jvm_options = -Dhttp.keepAlive=False -Dcom.ning.http.client.AsyncHttpClientConfig.allowPoolingConnection=false -Dcom.ning.http.client.AsyncHttpClientConfig.allowSslConnectionPool=false


Note

If you are unable to create an SLDB file using international language characters (such as æ, Æ, Ø) in the file name, update the 'jcc.use_lease_urls' property in the Snaplex's Global Properties to False. This workaround works for all UTF-8 characters, and hence supports all global languages.