...
in the etc/global.properties by adding it to the Update Snaplex dialog, Node Properties tab, Global properties table.
JCC Node Communication within a Groundplex
We recommend that you set up JCC nodes in a Snaplex within the same network and data center. Communication between JCC nodes in the same Snaplex is required for the following reasons:
The Pipeline Execute Snap communicates directly with neighboring JCC nodes in a Snaplex to start child Pipeline executions and send documents between parent and child Pipelines.
The data displayed in Data Preview is written to and read from neighboring JCC nodes in the Snaplex.
The requests and responses made in Ultra Pipelines are exchanged between a FeedMaster node and all of the JCC nodes in a Snaplex.
A Ground Triggered Task (invoked from a Groundplex) can be executed on a neighboring JCC node due to load-balancing—in which case, the prepare request and the bodies of the request and response are transferred between nodes.
Therefore, any extra latency or network hops between neighboring JCC nodes can introduce performance and reliability problems.
Name
Every Snaplex requires a name, for example, ground-dev or ground-prod. In the SnapLogic Designer, you can choose on which Snaplex Pipelines are executed. The Snaplex configuration also has an Environment variable associated with it, for example, dev or prod. When you configure the nodes for the Snaplex, you must set the jcc.environment to the Environment value that you have configured for the Snaplex.
...
The nodes need to communicate to each other on the following ports: 8081, 8084, and 8090.
The nodes should have a reliable, low-latency network connection between them.
The nodes should be homogeneous in that they should have similar CPU and memory configurations, as well as access the same network endpoints.
Temporary Folder
This section explains what data traversing the SnapLogic platform is encrypted or unencrypted. The temporary folder stores unencrypted data.
Encrypted data:
Transit data – Is always encrypted, assuming the endpoint supports encryption.
Preview and Account data – Is always encrypted.
Unencrypted data:
Snaplex – Data processing on Groundplex, Cloudplex, and eXtremeplex nodes occur principally in-memory as streaming, which is unencrypted.
Larger dataset – When larger datasets are processed that exceed the available compute memory, some Snaps like Sort and Join, which process multiple documents, write Pipeline data to the local disk as unencrypted during Pipeline execution to help optimize the performance.
These temporary files are deleted when the Snap/Pipeline execution completes. You can update your Snaplex to point to a different temporary location in the Global properties table of the Node Properties tab in the Update Snaplex dialog:
...
| Code Block |
|---|
jcc.jvm_options = -Djava.io.tmpdir=/new/tmp/folder |
The following Snaps write to temporary files on your local disk:
...
Anaplan: Upload, Write
...
Binary: Sort, Join
...
Box: Read, Write
...
Confluent Kafka: All Snaps that use either Kerberos and SSL accounts
...
Database: When using local disk staging for read-type Snaps
...
Email: Sender
...
Hadoop: Read, Write (Parquet and ORC formats)
...
JMS: When the user provides a JAR file
...
Salesforce: Bulk Query, Snaps that process CSV data
...
Script: PySpark
...
Snowflake: When using internal staging
...
Teradata: TPT FastExport
...
Transform: Aggregate, Avro Parser, Excel Parser, Join, Unique, Sort
...
Vertica: Bulk Load
...
Workday Prism Analytics: Bulk Load
...
JCC Node Communication within a Groundplex
We recommend that you set up JCC nodes in a Snaplex within the same network and data center. Communication between JCC nodes in the same Snaplex is required for the following reasons:
The Pipeline Execute Snap communicates directly with neighboring JCC nodes in a Snaplex to start child Pipeline executions and send documents between parent and child Pipelines.
The data displayed in Data Preview is written to and read from neighboring JCC nodes in the Snaplex.
The requests and responses made in Ultra Pipelines are exchanged between a FeedMaster node and all of the JCC nodes in a Snaplex.
A Ground Triggered Task (invoked from a Groundplex) can be executed on a neighboring JCC node due to load-balancing—in which case, the prepare request and the bodies of the request and response are transferred between nodes.
Therefore, any extra latency or network hops between neighboring JCC nodes can introduce performance and reliability problems.