Per the SnapLogic Release Process, all Snaplex instances across Orgs are upgraded to the 4.18 GA release at 9 pm PT on September 14, 2019.
The Snaplex version for the 4.18 release is master-5771, and the Snaps version is snapsmrc523.
This release also includes the patches made to SnapLogic after the 4.17 GA release, as listed in the 4.17 Dot Releases page.
We recommend that you upgrade to the latest Chrome or Firefox browser version and clear your browser cache before logging into the SnapLogic Intelligent Integration Platform (IIP).
Platform Enhancements
Panel
General
Introducing the following major enhancements to the API Management service:
API Manager: Enables API Developers to control access, discoverability, and approval of Proxies. It also enables users to add and delete other users for consuming APIs created in SnapLogic.
Developer Portal: Enables non-SnapLogic users to consume APIs created in the SnapLogic platform.
Single Sign Out (SSO): Enhances SSO options with the capability to enforce logging out users.
Pipeline Runtime ID: After the Snaplex instances update to 4.18, the runtime ID for Pipelines will have the following changes:
The Pipeline Runtime ID increases in length from 36 characters to 61 characters. If the ID is saved externally, the storage size must increase.
The status API for Snaplex nodes returns limited information unless called with user authentication information.
Snaplex IP Address Whitelist Update
To support in-built High Availability (HA) and Disaster Recovery (DR) capabilities for customer organizations that are subscribed to Cloudplex instance in North America, SnapLogic is migrating its Cloud infrastructure. In addition, Groundplex connectivity IP address will also change. Therefore, it is critical that you read the Snaplex IP Address Whitelist Update document.
Warning
title
IP Address Whitelist Required
If you are affected by this update, you must take action or else your production Pipelines might stop running.
Upgrading Your Groundplex to Java 11
Starting from 4.18 release, SnapLogic supports Java 11 for Groundplex instances. We recommend customers to migrate to Java 11, since Oracle no longer releases security updates for Java 8.
Info
title
Java 11 support highlights
Starting from the 4.18 release, Snaplex Linux installers are bundled with the Java 11 distribution, which includes the Java Runtime Engine (JRE 11).
The 4.18 release supports Java 8, and so will the 4.19 release.
Current Groundplex installations will continue to work with Java 8 until the Snaplex is re-installed using the new installer.
All Groundplex instances prior to 4.18 release must reinstall the Groundplex using the new installer.
The new Groundplex installs with Java 11. However, the current default version is Java 8.
Support for Java 8 version will cease with the SnapLogic 4.20 release.
End of support for Nested Pipelines: Starting from the 4.18 release, we no longer support the guaranteed delivery option for Nested Pipelines (also known as sub-Pipelines). We recommend that you use the Pipeline Execute Snap instead of sub-pipelines. You can use the Pipeline Execute Snap with Resumable child pipelines to handle scenarios where exactly-once-delivery is required. Contact your SnapLogic Account Manager for further details.
SFTP connection for Snaps: We are reverting the default value formaxSftpChannelsto 1. In the 4.17 release, the maxSftpChannels default value was set to 2. However, to keep only one open channel, the value is reverted to 1. Having only one open channel avoids caching of open sessions if all the channels are closed.
Dot Releases in 4.18 - Platform
The following 4.17 Platform Dot Release updates are now a part of the 4.18 release:
Update Date
Build
Updates
master-5492 Control Plane Only
Enhances the SSO Login functionality to support additional authentication methods RequestedAuthnContext and RequestedAuthnContext Comparison at the Org-level. As an Org admin, you can configure these additional options for user logins across your Org.
master-5508 - Patch 4 (Recommended)
Fixes an issue that can cause Pipeline Execute child pipelines to fail while loading accounts.
master-5472 - Patch 3
Fixes an issue that causes Groundplex triggered Pipelines comprising Pipeline Execute Snap with child Pipelines and performing an SLFS operation to fail with a 401 error.
Fixes an issue that causes expression verification to generate warnings for the Pipeline suggest feature.
master-5396 - Patch 2
Fixes a performance issue in Tasks triggered on a Groundplex, where the Pipeline uses the Pipeline Execute Snap to invoke a child Pipeline that references expression libraries.
master-5365 - Patch 1
Adds a retry handler to manage network failures during Pipeline information retrieval.
Enhances performance optimization for Groundplex Triggered Pipelines that use the Pipeline Execute Snap.
Fixes an issue which causes some Pipelines, typically with Union Snaps, to hang during execution.
Fixes an issue which causes some Pipeline creation times to show as Invalid date.
N/A
Fixes an issue causing Groundplex Triggered Task invocations to perform permission checks as the invoker instead of the Task owner.
Fixes an issue causing Groundplex Triggered Tasks to fail if the CC user is present as a member of the organization.
Fixes an issue causing runtime pushes to fail if a large number of Groundplex triggers are done concurrently.
UI Enhancements
Panel
Designer
Data Preview: We are enhancing the Data Preview functionality in Pipelines with new types of graphs.
Anchor
Pipeline_Recommendations
Pipeline_Recommendations
Pipeline Recommendation: We are enhancing the suggestion capabilities of Iris to include Pipeline Recommendations. You can now switch from Snap to Pipeline suggestions at a single click. Iris' new functionalities enable you to do the following when designing your Pipelines:
Based on the Pipeline label and the Snaps on the canvas, you can view a gallery of Pipeline shapes, and can choose a Pipeline to open.
Iris suggests Pipelines from 3 sources -- existing Pipelines that you have access to, Pattern Pipelines from SnapLogic Cloud Pattern Catalog, and artificially synthesized Pipelines.
You can open one of the Pipeline shapes as a new Pipeline.
You can also replace the partially configured Pipeline with the Pipeline shape from the gallery.
Iris Assistant: You can now use a new option to search for Pipelines using keywords, while also benefiting from improved Snap search capabilities.
Snap Search: When you search for a keyword that does not match any Snap name, Iris provides suggestions based on the content on the SnapLogic documentation site.
Pipeline Search: When you search for a Pipeline, Iris displays a list of project spaces and projects that contain Pipelines that match the search terms. The Pipelines are indexed on an hourly basis and new Pipelines are not displayed in the search until the next refresh happens.
Manager
Anchor
API_Manager
API_Manager
API Manager: We introduce a new page in Settings for API developers with Org-level admin access to control access to APIs and their discoverability type. From API Manager, you can also publish APIs to the Developer Portal. The API Manager puts all your Proxies for an Org in one place. Besides Proxy details, you can also view users, access types, and web applications that consume your Proxies.
Snaplex Node Disk Notification: We are introducing a Snaplex notification for JCC (Java Containerized Component) node disk usage. As an Org-level admin, you can create a notification that alerts users when the threshold for available capacity has been exceeded.
Dashboard
Anchor
Data_Lineage
Data_Lineage
Data Lineage: Introducing a new tab in Dashboard that provides a visualization of all the data sources to which Pipelines connect. You can now select one or more Pipelines and fetch the data sources for those Pipelines. Conversely, you can start with a data source, and view the list of Pipelines that interact with it.
Anchor
Developer_Portal
Developer_Portal
Developer Portal
Introducing an API Catalog to which API Developers can publish API Proxies. From the API Manager, you can access the Developer Portal, a new UI that requires a unique set of credentials. The Developer Portal includes the following functionalities:
Allows non-SnapLogic users to search and explore APIs created in SnapLogic.
Enables consumers to consume APIs.
Known Issue
Certain output of JSON Parser Snap may display truncated or inaccurate data. For example, a numerical output value 0.19331011717644054 displaying as 0.19331011717644053.
Dot Releases in 4.18 - UI
The following 4.17 UI Dot Release updates are now a part of the 4.18 release:
Update Date
Build
Updates
N/A
Fixes an issue with the Dependent Keys that causes the DataViz data preview to crash intermittently.
Fixes an issue with the Visualization Key drop-down that does not let you scroll all the column values if they overflow the given dropdown size.
Fixes issues with the Expression Builder for a Snap wherein you cannot undo an action; the cursor defaults to the beginning position while editing, and the scroll bar is missing.
Fixes an issue with the Snap properties view that does not close when clicking the esc key on the keyboard.
N/A
Fixes an issue where a script error appears when saving a Snap with a file input field (such as File Writer) in a Pipeline while at the same time another Snap with a file input field (like File Reader) is open in a different Pipeline.
Fixes an issue where the JSON and Table View displays the new line character (\n) incorrectly in the Snap output preview.
Fixes an issue with Snap recommendations when checking the compatibility of view type of the Mapper and Pipeline Execute Snaps because both Snaps support binary and document input/output.
Snap Enhancements
Panel
Info
Pipelines that start before a Snap update continue to run with the older version of the Snap. All Pipelines executed after a Snap update run with the new Snaps.
For Ultra Pipelines, the currently running instances continue to run with the older Snap Packs. Editing and saving the task instance in Manager causes a rolling restart of the Ultra Pipeline instances, and the new Snap Packs get picked up.
Customers using SQL Server with Windows authentication and SAP need to restart their Groundplex and Cloudplex instances.
New Snap Packs appear in the SnapLogic Designer only after all the JCC nodes in an Org are upgraded to the latest version.
Adds a new account type, Google Service Account, which enables you to isolate workflows from individual users. This support applies only to the Google Storage file protocol.
Projection Condition: Specifies which fields must be returned in the matching documents.
Sort Condition: Specifies the sort order for documents in the result set.
Offset: Specifies the starting document for the FIND query.
Limit: Specifies the number of documents the FIND query must return.
Group result: Groups the find query results in one field named result, instead of an array.
MongoDB Group: Adds a new Snap that enables you to group input documents by a specified expression, and output to the next stage, one document for each distinct grouping.
Mongo DB Update: Adds two new fields, Update operation and Exclude list, which enable you to update operations and exclude a list of JSON properties before sending the updated documents to MongoDB.
Reverts a 4.17-patch (db/mysql7395)update regarding how BIT and BOOLEAN column data type values are displayed. They now display as true or false (4.17 GA behavior now in snapsmrc523) rather than converting to 0 or 1 values (4.17-patch behavior).
Adds support for SQL Server account Active Directory authentication for SQL server for driver JAR version mssql-jdbc-6.2.2-jre8.jar.
Reverts a 4.17-patch (db/sqlserver7394) update regarding how BIT and BOOLEAN column data type values are displayed. They now display as true or false (4.17 GA behavior now in snapsmrc523) rather than converting to 0 or 1 values (4.17-patch behavior).
Sort: Enhances the sort feature to support specifying sort order at the field level. Adds two new fields, Sort Path and Sort Order, to the Sort paths property; renames the Sort order property to Sort order (Global).
Group by N: Enhances the Group Size property to be expression enabled and removes the upper bound threshold (of 10000).
Enhances all the Snaps in this Snap Pack with the capability to create, delete, and update Snaplex directly inside the Pipeline. This feature facilitates administrative tasks if you are running your Pipelines on Groundplex. You can now create Snaplex programmatically by doing the following:
Building a Pipeline with a Create Snap where you specify Snaplex as the Asset.
Creating a Triggered Task in which the URL invokes the Pipeline.
Sharing the URL with users in your Org so that when they run the Pipeline, the Snaplex is automatically created.
Furthermore, you can use the Update Snap to update the node or logging properties in existing Snaplex.
Known Issues
Pipelines configured with an on-premises Microsoft Dynamics CRM Account fails with Java 11 displaying this error –com.sun.xml.internal.messaging.saaj.soap.SAAJMetaFactoryImpl. Hence, you need to remain on Java 8 on the associated Snaplex for the said Pipelines to execute.
A NullPointerException issue in the AWS SDK impacts the Binary, Redshift, Hadoop, Snowflake, and DynamoDB Snap Packs. Users leveraging AWS functionality and whose Snaplex has either a system configuration that uses an unauthenticated HTTP/HTTP proxy, or who have the same defined via Environment Variables are impacted. If a proxy was not used, or a proxy username and password was used, then customers are not affected. You can expect this to resolve via a patch release that includes AWS SDK version 1.11.610, which has a fix for this issue. This issue was resolved in the 4.20 release.
Preview output for Parquet Parser Snap does not match in Standard mode vis-a-vis the eXtreme mode, which displays the correct preview.
File Writer and ZipFile Write Snaps do not create an output file when using the input from SAS Generator Snap configured with only the DELETE SAS permission. This is not the case when the target file exists.
Dot Releases in 4.18 - Snap
The following 4.17 Snap Dot Release updates are now a part of the 4.18 release:
Snap Pack
Update Date
Build
Updates
All Snap Packs
ALL7402
Pushes automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers.
Fields added: Max poll records, Acknowledge mode, Key deserializer, Value deserializer, and Add 1 to offsets.
Fields removed: Auto commit interval (msec) and Standardize error output.
Diagnostic fields are added in the metadata object in the Snap's output:
client_id: The unique ID assigned to a single Kafka consumer client instance. All documents written by a Consumer Snap during a single pipeline execution or validation will have the same value for this field.
batch_size: The number of messages in the batch of messages that the current message came from. Note that “batch” isn’t Kafka terminology; we’re using it to mean the set of messages received from the broker in a single call to Kafka’s poll method.
batch_index: The current batch number relative to a single consumer instance. This number starts at 0 for the first batch and is only incremented for batches which contain at least one message.
record_index: The current record number relative to its batch. The first message in a batch has an index of 0; the last has an index of batch_size - 1.
Confluent Kafka Acknowledge Snap
Field added: Metadata path
Confluent Kafka Producer Snap
Fields added: Key serializer and Value serializer.
Fixes an issue with the Data Validator Snap wherein the output fails to report all but one of the violations when there are multiple constraints of the Required type.
Fixes an issue with the Redshift Bulk Load Snap wherein the Snap fails to copy the entire data from source to the Redshift table without any statements being aborted.
Adds Order by clause and Limit clause properties to the Salesforce Reader Snap to fix an issue wherein the Snap throws an error while fetching records.
Adds AWS Server-Side Encryption support for AWS S3 and AWS KMS (Key Management Service) for Snowflake Bulk Load, Snowflake Bulk Upsert, and Snowflake Unload Snaps.
Adds a new field, Format as canonical XML, to the XSLT Snap that enables canonical XML formatting.
eXtreme Big Data Enhancements
Panel
General
Anchor
Azure_Databricks_Cluster_Support
Azure_Databricks_Cluster_Support
eXtremeplex: Adds the ability to configure an eXtremeplex on Azure Databricks. You can now run your Spark jobs in eXtreme on Azure Databricks clusters in addition to AWS EMR clusters.
Parquet Formatter Snap: Adds partition support. You can now partition the input tabular data by specifying the target partitioning keys.
New Snaps
Redshift Select: Extracts data from your Redshift Databases. You can now build eXtreme Pipelines that query and pull data from your Redshift databases.
Redshift Insert: Inserts data in your Redshift Databases. You can now build eXtreme Pipelines that write data to your Redshift databases.
Known Issue
After an eXtremeplex terminates on Azure Databricks, the terminate event does not display on the Dashboard.
4.18 UAT Delta
Panel
This section tracks the changes made during the iterative pushes to the UAT server and the GA release. The expected schedule is:
UAT #1: Friday, July 26, 2019 (Release updates are published above)
UAT #2: Friday, August 2, 2019
GA: Saturday, August 10, 2019 (9 pm PT)
UAT #2 Updates
UI
Fixed an issue that allows you to create duplicate Proxy assets.
Fixed an issue that prevents API consumers from searching for web applications on the Access and OAuth tabs of the API Manager.
Fixed an issue where default value for SSO option is missing.
Fixed an issue where Data Lineage display is not clearing previous results upon subsequent inquiries.
Fixed an issue where clicking on Replace Current Pipe and Suggest Pipeline in Pipeline Recommendations results in a script error.
Fixed an issue where the Pipeline Execution Logs for Azure Databricks does not display any information.
Fixed an issue where the Developer Portal page does not work on Mozilla FireFox.
Fixed an issue where a very long Proxy name breaks the Developer Portal landing page.
Fixed an issue where disabling an Ultra Pipeline results in an empty error message.
Fixed an issue where, if you cancel the deletion of a Pipeline in SnapLogic Designer, the pop-up closes, but the UI displays the busy icon interminably.
Fixed an issue where, if you log in as an Admin viewing an Org of which you are not a member, the SnapLogic Dashboard does not display Snaplex instances correctly.
Fixed an issue with Pipeline Recommendations where clicking Replace Current Pipe then deleting a Snap results in an error, and the Pipeline is not saved.
Platform
Added support for DerbyDB when running with JRE 11.
Fixed an issue with SSO redirect login.
Fixed an issue with schema caching when running with older versions of Snaplex.
Fixed multiple issues with Developer Portal related APIs.
Snaps
Fixed an issue with ORC Writer, Parquet Writer, HDFS Writer, and HDFS Zip File Writer Snaps wherein the Snaps throw an error and there is no output file for double slash after the container in the directory path.
Fixed an issue with the HDFS Reader Snap where the Snap fails after rerouting error to the error view.
Fixed an issue with the eXtreme Redshift Snaps pointing to incorrect documentation.
Upgraded JRuby version to 9.1.17.0 that enables Script Snap to work on JDK 11 Snaplex.
Improved the error message in Hadoop Snap Pack in the case of an incorrect dirpath format for ABFS protocol.
GA Updates
UI
Fixed an issue where adding a Snap on the canvas when the Suggested Pipeline panel is open results in a script error.
Fixed an issue where adding a Snap in a replaced suggested Pipeline and then saving it results in a script error.
Fixed an issue where Snaps inserted in the middle of suggested Synthesis Pipelines does not have input/output views.
Fixed an issue where clicking a Pipeline in Suggested Pipelines panel displays an incorrect message.
Big Data
Fixed an issue with cluster monitoring in the eXtremeplex configuration.
4.18 Dot Releases
Panel
Platform
Update Date
Build
Updates
master-6135 - 4.18 Patch 4
Fixed an issue that causes the parent Pipeline to not include the correct error message when its child Pipeline having a Pipeline Execute Snap fails.
master-5912 - 4.18 Patch 3 (Recommended)
Fixed an issue with error handling when the child pipeline with Pipeline Execute Snap fails.
Fixed an error when loading error Pipelines that use dynamic expressions.
Fixed an issue with Pipeline execution stalling when child Pipeline uses binary views.
Data lineage information is now throttled to prevent growth in runtime data size that may cause some Pipelines to fail.
Optimized loading metadata from a control plane by using compression.
UI
Update Date
Build
Updates
N/A
Fixed an issue where Data Preview does not work in eXtreme Pipelines.
Anchor
4.18 Dot
4.18 Dot
Fixed an issue where Snap Suggestions do not appear when you open a second tab or subsequently on the initial tab displaying the Snap Suggestions.
Fixed an issue where clicking a Project folder named Alerts misdirects you to the Alerts page.
Fixed usability issues with Pipeline Recommendations.
Reverted a fix where certain output data with JSON Parser Snap is inaccurate, fixing which results in a regression issue in Data Preview.
Updated the UI content and buttons for Suggested Pipelines/Snaps panel in the Designer's canvas to improve usability.
N/A
Fixed an issue where you could only view the preview data after Pipeline validation completes.
Fixed an issue where editing Snap settings in the Pipeline Execute Snap results in the following error message: Type error: Cannot set property ‘dialed’ of undefined.
Fixed an issue where File Previewer does not refetch the full HTML report.
Enhanced Pipeline Recommendations by enabling a wizard that allows users to populate the Pattern tab with recommended Pipelines from the Patterns Catalog.
Fixed an issue with Snap recommendations where adding a Snap in a replaced suggested Pipeline and saving it results in an error.
Fixed an issue where selecting a recommended Pipeline through Pipeline Synthesis results in the middle snaps of the recommended synthesis Pipeline not having any input or output views.
Fixed an error where adding a Snap on the canvas when the Suggested Pipeline panel is open results in an error.
Fixes the File Reader, File Writer, and Directory Browser Snaps to resolve 0x103 (STATUS_PENDING) and 0xc000022d (STATUS_RETRY) errors for SMB file operations.
Fixes an issue in the SQL Server Merge Snap wherein identity columns are not handled correctly. Also, adds a field Enable Identity Insert that enables you to insert values into an identity column.
Adds properties named Earliest Relative, Latest Relative, and Preset Relative Search to the Splunk Search Snap to fix an issue wherein the Snap returns inaccurate and inconsistent results regarding last-30-day and year-to-date searches.
Introduces a field Advanced properties in the File Reader, File Writer, File Delete, ZipFile Read, ZipFile Write, Directory Browser, Multi File Reader, File Poller, and File Operation Snaps. Use this field to configure the SAS URI for an access to Azure Storage. The supported SAS types are Service SAS on container, Service SAS on blob, and Account SAS.
Introduces a Snap SAS Generator. This Snap enables you to generate Shared Access Storage (SAS) URIs for accessing Azure Storage containers and blobs.
Adds a property Validate full JSON path to the Data Validator Snap to fix an issue wherein a field validation failure against a constraint does not produce an error.
Fixes an issue by using the NVARCHAR2 datatype in Oracle databases when the character set is not AL32UTF8. However, a known issue is that special characters appear as .
Fixes an issue with the NetSuite Update Snap and the NetSuite Create Snap wherein the Snaps are unable to properly handle null values for the standard and the custom fields.
Fixes an issue with the Excel Parser Snap by upgrading Apache POI to version 3.14 wherein the Snap is unable to parse an excel file with custom namespaces.
Fixes an issue with the Excel Parser Snap wherein the Snap converts real numbers to two decimal places when formatted for currency. A new property Cell formatting now supports unformatted outputs.
Fixes an issue with the ZipFile Read Snap wherein the Snap fails to extract files from a zip file larger than 100MB when the number of retries is greater than zero.
.