In this article
You can start a SnapLogic Snaplex on your server as a Groundplex. This allows access to endpoints that are within your network firewall. The Groundplex does not need inbound network connectivity. The only requirement is outbound connectivity to the SnapLogic control plane over the HTTPS port 443. Your Snaplex makes outbound HTTPS requests to the SnapLogic control plane. The Snaplex also makes WebSocket Secure (WSS) outbound connections with the control plane, which uses this connection to send inbound control messages to the Snaplex. WSS is an extension of HTTPS that provides a standards compliant and secure message passing mechanism.
In addition to the SnapLogic control plane, the Snaplex makes outbound connections to any endpoints referenced in a Pipeline. So if the Pipeline running on the Snaplex communicates with Salesforce and Redshift endpoints, then the Snaplex makes outbound connections with Salesforce and Redshift databases. The protocol depends upon the endpoint. Salesforce Snaps use HTTPS connection, while Redshift Snaps use a TCP connection, over TLS (SSL) optionally, to establish the JDBC connection.
Most Groundplex instances run on nodes with outbound access to the Internet enabled. Hence, your Snaplex can start without any specific configuration. If your Snaplex is running on a node with restricted outbound access, use one of the following two methods to configure your Snaplex:
- HTTP Proxy: Configure the Snaplex to communicate with the SnapLogic control plane and other endpoints through an HTTP forward proxy.
- IP Address Allowlist: Open an outbound firewall rule to the SnapLogic control plane and other endpoints as necessary.
We recommend using HTTP proxy, since it enables communication with any endpoints using the HTTP protocol.
The IP address allowlist method requires you to open the firewall rules for each endpoint you need the Snaplex to communicate with. Many endpoints do not have a single IP address to allowlist. Also, the Snaplex communicates directly with Amazon S3 for file operations, so those requests need to proxy through the control plane, causing the requests to run slowly. Some operations cannot be supported when direct access to S3 is disabled. The HTTP proxy method described above does not have these limitations.
HTTP Proxy Configuration
You can obtain the proxy configuration settings from your network operations team. Launch the Snaplex properties in SnapLogic Manager and configure the HTTP proxy settings (for details, see Updating a Snaplex).
By default, a proxy routes all outbound HTTP and HTTPS requests. To restrict the proxy for requests routing outside your firewall, you need to change the proxy settings.
Non-Proxy Hosts
You need to configure the Non-proxy Hosts field to restrict the endpoints the proxy uses. Disable the proxy to enable communication among the nodes in a Snaplex; else, it can result in neighbor connectivity check failures for your Snaplex.
In the following example, the field restricts HTTP requests from routing to the local host or host in the example.com domain.
jcc.http.nonProxyHosts = localhost|127.*|[::1]|MYHOSTNAME|*.example.com
The proxy settings are configured per the standard JRE settings. These are exposed in the Node Proxies tab of your Snaplex in SnapLogic Manager.
jcc.http.proxyHost = myproxy.example.com jcc.http.proxyPort = 3128 jcc.http.nonProxyHosts = localhost|127.*|[::1]|MYHOSTNAME|*.example.com jcc.https.proxyHost = myproxy.example.com jcc.https.proxyPort = 3128 jcc.https.nonProxyHosts = localhost|127.*|[::1]|MYHOSTNAME|*.example.com jcc.http.proxyUser=proxyuser jcc.http.proxyPassword=proxypass
Configuring the Script Snap to Use a HTTP Proxy
HTTP-compatible Snap Packs can use a HTTP proxy configured in the Node Proxies configuration tab of a Snaplex within SnapLogic Manager. However, the Script Snap allows you to write scripts that call external processes (for example: curl) and these scripts will not be aware of proxy configurations within the SnapLogic application.
You can use curl to configure a proxy directly via the --proxy argument. To enforce proxy usage across all usages of the Script Snap, set the http_proxy and/or https_proxy environment variables within the following file:
/etc/sysconfig/jcc.
Environment variables declared within the /etc/sysconfig/jcc file are visible to the Snaplex application (OS-level environment variables are not visible). If the /etc/sysconfig directory and /etc/sysconfig/jcc file does not exist in your Snaplex, run the following command with your own username/password (if authentication is required), proxy-ip-address, and port (you could also add https_proxy) to create them:
sudo mkdir -p /etc/sysconfig; sudo sh -c"echo 'export http_proxy=username:password@proxy-ip-address:port' >> /etc/sysconfig/jcc"
After the file and its directory are created, run one of the following commands to restart the Snaplex application:
/opt/snaplogic/bin/jcc.sh restart c:\opt\snaplogic\bin\jcc.bat restart
The http_proxy/https_proxy environment variable is now active within the SnapLogic product. You can now run your script to call the external process.
For example, the following Script Snap uses the subprocess library to execute curl and adds the response body to the output document:
# Import the interface required by the Script snap.
from com.snaplogic.scripting.language import ScriptHook
import subprocess
class TransformScript(ScriptHook):
def __init__(self, input, output, error, log):
self.input = input
self.output = output
self.error = error
self.log = log
# The "execute()" method is called once when the pipeline is started
# and allowed to process its inputs or just send data to its outputs.
def execute(self):
self.log.info("Executing Transform script")
while self.input.hasNext():
try:
# Read the next input document, store it in a new dictionary, and write this as an output document.
inDoc = self.input.next()
proc = subprocess.Popen(['curl','https://www.snaplogic.com'], stdout=subprocess.PIPE)
(out, err) = proc.communicate()
outDoc = {
'original' : out
}
self.output.write(inDoc, outDoc)
except Exception as e:
errDoc = {
'error' : str(e)
}
self.log.error("Error in python script")
self.error.write(errDoc)
self.log.info("Script executed")
# The "cleanup()" method is called after the snap has exited the execute() method
def cleanup(self):
self.log.info("Cleaning up")
# The Script Snap will look for a ScriptHook object in the "hook" variable. The snap will then call the hook's "execute" method.
hook = TransformScript(input, output, error, log)
On execution, the proxy access log shows the request being routed through the proxy.
Troubleshooting
To verify if outbound requests are permitted from the Snaplex node, run:
curl https://elastic.snaplogic.com/status
A response with status OK indicates successful completion. If this requests hangs or fails, then a proxy is required. Request the HTTP proxy information from your network operations team. To check access through a proxy, run:
curl -x myproxy.mydomain.com:3128 https://elastic.snaplogic.com/status
If this request fails with a 407 (Proxy Authentication Required), then you need to specify the authentication information in the proxy. To check the proxy authentication, run:
curl -x myproxy.mydomain.com:3128 --proxy-user "proxyuser:proxypasswd" https://elastic.snaplogic.com/status
The -v option can be added to curl to get detailed messages. For Windows, download the TLS (SSL) enabled curl binary to verify your configuration.
Reverse Proxy Configuration
The above configuration is the typical HTTP Proxy configuration, a forward proxy, which can forward requests to any endpoint. You can use the same HTTP proxy for connecting with the SnapLogic control plane and also for connecting to other REST endpoints, such as Salesforce. Forward HTTP proxy type is the most flexible method for integrating multiple endpoints. In some scenarios, your network operations team can configure a reverse proxy instead of a forward proxy. In that case, all requests to the proxy are directly sent to the SnapLogic control plane. For example, if https://myproxy.test.com/ is the proxy server, a request will return the status from the SnapLogic control plane.
For example:
curl https://myproxy.test.com/status
To enable the Snaplex to work with the reverse proxy, add the following information to your Snaplex properties:
jcc.sldb_uri = https://myproxy.test.com:443 jcc.host_header = elastic.snaplogic.com
Since setting up a reverse proxy is uncommon, your network operations team has to allow access as explained above to enable SnapLogic to access your S3 instance.