May 2020, 4.21 Release Notes
In this article:
Important
About SnapLogic Releases
- Per the SnapLogic Release Process, all remaining Snaplex instances across customer Orgs are upgraded to the recommended 4.21 version (master-7451 - 4.21 Patch 1) at 9 p.m. PT on June 13, 2020. If you upgraded your Snaplex to a 4.21 version prior to the auto-upgrade event, then your Snaplex version does not change.
- To use the new features and Snaps in the 4.21 release, ensure that your Snaplex nodes are upgraded to the recommended 4.21 patch version (master-7451 - 4.21 Patch 1).
- This 4.21 GA release includes the patches deployed to SnapLogic after the 4.20 GA release, as listed in the 4.20 Dot Releases page.
- We recommend that you upgrade to the latest Chrome or Firefox browser version and clear your browser cache before logging into the SnapLogic Intelligent Integration Platform (IIP).
Java 11 Support
Starting from the 4.18 August release, SnapLogic supports Java 11 for Groundplex instances. We recommend all customers to migrate to Java 11, since Oracle no longer releases security updates for Java 8. For details, see Configuring the Java Version on Your Groundplex.
Java 11 support highlights
Groundplex:
- The 4.21 Snaplex installer uses Java 11 by default. Java 8 is also supported as a rollback option, if required.
- Current Groundplex installations will continue to work with Java 8 until the Snaplex is reinstalled using the new installer.
- You are required to upgrade their Groundplex instances to Java 11 using the new installer. Support for Java 8 will cease from the August 2020 release.
Cloudplex:
- During the current 4.21 release, all your existing Cloudplex instances hosted in the United States will be migrated to Java 11. Currently, this is being planned for Friday, June 26, 2020. For more details, see Migrating Your Cloudplex Nodes to Java 11.
New IP Whitelisting Requirements Reminder
We remind all customers of the new IP Whitelisting requirements that have been in effect since the August 2019, 4.18 release. Ensure that your network administrators review the following:
Platform
New Features and Enhancements
- Insights dashboard: Revamps the Insights dashboard to provide data visualization of Org-level Pipeline-execution statistics for a specific date range.
- Task monitoring: Adds a new tab for Tasks under Dashboard that does data visualization of triggered and scheduled tasks for a selected time range. The new tab displays the number of failed and completed executions for each task type along with their details.
- Snap auto-configuration: Enhances the suggestion capabilities of Iris to include Snap auto-configuration in Snaps with suggestible properties. Iris Expression recommends the configuration of these properties based on similar Snap settings from Pipelines that you can access.
Sticky Notes in SnapLogic Designer: Introduces the ability to add notes to a Pipeline in SnapLogic Designer. This enables you to collaborate with multiple Pipeline developers or add important notes for complex Pipelines. The notes are included in the Pipeline metadata and are immediately visible to other users accessing the Pipeline.
Known Issue
When you copy a Pipeline with Sticky Notes, the notes are also copied with the Pipeline. Additionally, when you modify or add Sticky Notes in the copied Pipeline, the notes in the original Pipeline are also updated.
- Target recommendation: Enhances the Iris Expression recommendation capabilities to include Target path suggestions in Mapper Snap. Iris recommends Target paths based on exact matches between Expression and Target paths from other Pipelines in your Org.
Known Issues
- A Triggered Task on a Groundplex with an underlying Pipeline that contains child Pipelines might stall as the Data Validator Snap is executing in the parent Pipeline.
- An Ultra Pipeline sometimes processes the same message twice even though the message was processed successfully during the first attempt.
Updates to Expression Behavior
Starting in the 4.21 Release (May 2020), the escaping required in the replacement text of the ReplaceAll function is identical to that of the Replace function. Prior to this release, you had to use additional escaping in the replacement text with ReplaceAll to get the desired output.
Previous Behavior New Behavior Example:
$msg.
replaceAll("-", "\\\\-")
where $msg contains mother-in-law
Result: mother\-in\-law
Example:
$msg.replaceAll("-", "\\-")
where $msg contains mother-in-law
Result: mother\-in\-law
Starting in the 4.21 Release (May 2020), the Date.parse() function returns the response text Not a Number when you enter a non-number or infinity. Prior to this release, doing either resulted in an error, even though the formatting of the string was correct.
Previous Behavior New Behavior Example: $
date.parse(Math.abs(1/0))
where $date is infinity
Result: 1970-01-01T00:00:00.000 UTC
Example: $
date.parse(“foo”, “yyyymmdd”)
Result: Error
Example:
$
date.parse(Math.abs(1/0))
where $date is infinity
Result: Not a number
Example: $
date.parse(“foo”, “yyyymmdd”)
Result: Not a number
Standard-mode Snaps
New Snaps
- SharePoint Online Snap Pack: This Snap Pack enables organizations to share and manage content and collaborate across an organization using the following SharePoint Online Snaps:
- SharePoint Online - Get Item: Retrieves metadata of an item (files/folders) from SharePoint Online document library.
- SharePoint Online - Create Item Permission: Creates permissions for an item and shares the same via email to the configured set of recipients.
- SharePoint Online - Delete Item Permission: Deletes permissions for a specific SharePoint Online item.
- SharePoint Online - Download File: Enables downloading the content of the primary stream (file) of an item.
- SharePoint Online - Search Items: Searches for the hierarchy of items matching the search query within a folder hierarchy, a whole drive, or files shared with the user.
- SharePoint Online - Item Children: Retrieves metadata of items from the selected SharePoint Online document library.
- SharePoint Online - Copy Item: Copies the item resource path that represents a folder or file stored in a document library.
- SharePoint Online - Update Item: Updates a name or moves an item (file or folder) within SharePoint Online.
- SharePoint Online - Create Folder: Creates a folder in a SharePoint Online document library and stores the metadata in a file.
- SharePoint Online - Delete Item: Deletes items from the SharePoint Online document library.
- SharePoint Online - Document Library: Retrieves the document library metadata of a site.
- SharePoint Online - Get Item Permissions: Retrieves permissions of an item from the SharePoint Online document library.
- SharePoint Online - Update Item Permission: Updates an item permission associated with an item/folder of SharePoint Online document library.
- SharePoint Online - List DeleteItem: Deletes a particular list item from a SharePoint Online site.
- SharePoint Online - List UpdateItem: Updates the values of an existing list item in a SharePoint Online site.
- SharePoint Online - Create List: Creates a list or list item in the SharePoint Online site.
- SharePoint Online - List Read: Reads metadata of a list along with its columns and items from the SharePoint Online site.
- SharePoint Online - Upload File: Uploads the contents of a new file or updates the contents of an existing file in a document library.
- Adobe Experience Platform Snap Pack: This Snap Pack enables you to integrate with the new Adobe Experience Platform and includes the following Snaps:
- Adobe Experience Platform File Generator: Generates an Adobe Enterprise Platform (AEP)-compatible file by restructuring upstream data to match a specific schema stored in AEP.
- Adobe Experience Platform S3 Connector:
- Enhances the Snap by removing the Create New Dataset checkbox. The Snap now automatically creates a new dataset if it does not exist.
- Retrieves data from AWS S3 and writes it into AEP.
- Cassandra: Extends support to Cassandra version 3.11.5.
- Adobe Experience Platform Write: Writes data from upstream Snaps into AEP.
- SAP S/4HANA Snap Pack: This Snap Pack supports the SAP S/4 HANA on-premise environment and includes the following Snaps:
- SAP S/4HANA Read Snap: Retrieves data from SAP S/4 HANA applications.
- SAP S/4HANA Create Snap: Creates data objects in SAP S/4 HANA applications.
- SAP S/4HANA Update Snap: Updates data in SAP S/4HANA applications.
- SAP S/4HANA Delete Snap: Removes data objects from SAP S/4HANA applications.
- ML Data Preparation: Introduces the Mask Snap that enables you to hide sensitive information in your dataset before exporting the dataset for analytics or writing the dataset to a target file.
Updated Snap Packs
- Adobe Experience Platform Snap Pack: Enhances the Adobe Experience Platform S3 Connector Snap by removing the Create New Dataset checkbox. The Snap now automatically creates a new dataset if it does not exist.
- Cassandra: Extends support to Cassandra version 3.11.5.
- Confluent Kafka: Enhances the Kafka Consumer Snap to significantly reduce the volume of commit requests for offsets when using the record acknowledgment mode (Wait after each record), and thereby, improve performance and stability.
- ML Data Preparation:
- Enhances the Match Snap to add a new field, Match all, which matches one record from the first input with multiple records in the second input. Also, enhances the Comparator field in the Snap by adding one more option, Exact, which identifies and classifies a match as either an exact match or not a match at all.
- Enhances the Deduplicate Snap to add a new field, Group ID, which includes the Group ID for each record in the output. Also, enhances the Comparator field in the Snap by adding one more option, Exact, which identifies and classifies a match as either an exact match or not a match at all.
- Enhances the Sample Snap by adding a second output view which displays data that is not in the first output. Also, a new algorithm type, Linear Split, which enables you to split the dataset based on the pass-through percentage.
- NetSuite
Removes the obsolete Sandbox Account check box from the NetSuite Token Account.
Removes the now redundant Custom WSDL URL Domain field from the NetSuite Token Account to support NetSuite's transition to automatic account-specific domain identification.
Oracle: Updates the Oracle Thin Account and Oracle Thin Dynamic Account, enabling them to connect via Oracle Active Data Guard (ADG).
- SAP: Enhances the SAP Execute Snap to add a new field, Output date time as Text, which enables you to output date and time as a String data type instead of a DateTime object.
Script Snap Pack: Adds a new Cleanup method in the ScriptHook interface associated with the Script Snap. This method enables the Script Snap to automatically execute a clean-up once the configured script completes executing.
- ServiceNow: Adds support for ServiceNow Madrid, New York, and Orlando versions.
- Transform: Adds support in the Mapper Snap to display schemas with complex nesting. For example, if Snaps downstream from the Mapper Snap have custom fields that contain multiple keys, then these keys automatically display in the Mapper Snap's Target Schema dropdown list when you toggle-off Expression Builder and select the field in the Mapping Table.
Breaking Change
The HTTP header field's behavior in the SOAP Execute Snap has been changed. Specifying key alone is not sufficient. Both key and value must be provided when invoking a task.
Known Issues
- In some Snaps, if you leave any expression-enabled mandatory fields empty, a null pointer exception error message might display instead of the usual error message on validation.
The Cassandra Snap Pack does not support the following data types introduced in Apache Cassandra V3.x:
DateRange
Duration
Geo-spatial data types such as Point, Polygon and LineString
Snaps in this Snap Pack display an exception when you query a map column that has a timestamp as the key.
- Google Big Query Snap Pack: The Snaps no longer return the
affectedRowCount
field with a null value for DML (data manipulation language) queries. The said field no longer exists. - MySQL: When MySQL Execute Snaps (MySQL - Execute and MySQL - Multi Execute) are followed by MySQL Snaps, such as MySQL - Insert, MySQL - Merge, they display the following error when executed:
Table definition has changed, please retry transaction.
This happens due to a known issue in the MySql Database. For more information, see MySQL Bug #65378.
Parquet Writer (Behavior Change): In the 4.20 Snap version (snapsmrc535 and hadoop8776) when a value for a field defined as
repeated
(for example,devops
in the below example) is not present in the input document, the Parquet Writer Snap displays"The required field 'devops' is missing from a document"
error. However, in the 4.21 release, the Parquet Writer Snap outputs a null value when there is no input for the repeated fields. If this behavior change impacts your existing Pipelines having the Parquet Writer Snap, then you might want to consider potentially redesigning your impacted Pipelines.repeated group devops { required binary lan (UTF8); optional binary lan2 (UTF8); }
Custom Snaps
During the May2020, 4.21 release cycle, we did multiple enhancements to the third-party dependencies used by the SnapLogic platform and the Snaps. Hence, we strongly recommend updating and redeploying all your custom Snap Packs to ensure compatibility with this release. For details, see Updating Custom Snap Packs in the SnapLogic Developer Documentation.
Prior 4.20 Snap Dot Releases in 4.21 GA
The following 4.20 Snap Dot Release updates are now a part of the current 4.21 GA release.
eXtreme (Big Data)
New Features and Enhancements
Cross Account IAM Role Support: Users can now specify cross-account IAM role for an AWS account to spin up an eXtremeplex on AWS EMR without needing to share secret and access keys of their AWS accounts.
- Enhanced Account Encryption: Introduces Enhanced Account Encryption (EAE) support for AWS accounts. This enables you to encrypt and secure account credentials that access endpoints from SnapLogic using a private key/public key model. The data is encrypted with a public key before it leaves the browser, then decrypted with a private key on the eXtremeplex.
- Publicly Available Datasets: Provides free limited datasets in our public AWS S3 bucket and Azure Databricks clusters. You can use these datasets to try out the eXtreme use case Pipelines. Alternatively, you can also generate your own TPC-DS benchmark datasets to use with these Pipelines.
New Snaps
- Spark SQL 2.X Snap Pack: Enhances the Snap Pack to support Java Database Connectivity (JDBC). This enhancement adds the following Snaps and account type:
- JDBC Insert: Inserts data into a target table through a JDBC connection.
- JDBC Select: Fetches data from a target table through a JDBC connection.
- JDBC Storage Account: Enables you to connect to databases that support JDBC.
Known Issue
- eXtreme Execute Snap Pack: The Spark job continues being in the running state even after the SnapLogic Designer/Dashboard displays the Pipeline status as failed on AWS EMR.
Prior 4.20 eXtreme Dot Releases in 4.21 GA
The following 4.20 eXtreme (Big Data) Dot Release update is now a part of the 4.21 GA release.
4.21 UAT Delta
This section tracks the changes made during the iterative pushes to the UAT server and the GA release. The schedule is:
- UAT #1: Friday, April 24, 2020 (Release updates are published above)
- UAT #2: Friday, May 01, 2020
- GA: Saturday, May 09, 2020 (9 p.m. PT)
The SnapLogic GA build on the production server is also deployed on the UAT server.
UAT #2 Fixes
Platform
Fixes an issue where a Triggered Task invoked with a Groundplex URL fails to execute under the following two conditions:
The Snaplex contains a FeedMaster node.
The Do not start new execution if one is already active option in the Create Task dialog is selected.
Fixes an issue with a Scheduled Task where a Snaplex node is put into maintenance mode, and the Scheduled Task, which should be executed on a different node in the Snaplex, does not execute.
Fixes an issue where the expression toggle display does not render correctly for some users.
Fixes an issue where the Help hook for Pipeline execution statistics redirects to an incorrect documentation page.
- Updates the Task monitoring display to make today's date default.
- Fixes an issue where the Task Monitoring Dashboard does not display all metrics for a specified hour when seeking metrics for Tasks older than 24 hours.
Standard-mode Snaps
- None
eXtreme-mode Snaps
- None
4.21 Dot Releases
Platform
Date of Update | Snaplex Build | Updates |
---|---|---|
master-7588 - 4.21 Patch 3 | This patch is applicable to only select customers and is not recommended. | |
master-7513 - 4.21 Patch 2 (Restricted) | This patch is applicable to only select customers and gives the option of reverting the ReplaceAll expression functionality to the 4.20 release. Contact Customer Support for more information. | |
master-7451 - 4.21 Patch 1 (Recommended) |
|
UI
Date of Update | Updates |
---|---|
Enhances Task and Snaplex error message responses. | |
|
Standard-mode Snaps
Snap Pack | Update Date | Build | Updates |
---|---|---|---|
Binary | 421patches6136 | Fixes the issue with Multi File Reader Snap that continues to look for a directory even when FileSystem information is unavailable. | |
Azure SQL, Cassandra, Hive, JDBC, MongoDB, MySQL, Oracle, PostgreSQL, Redshift, SAP HANA, Snowflake, SQL Server, Teradata, Vertica | 421patches6272 | Fixes the issue where Snowflake SCD2 Snap generates two output documents despite no changes to Cause-historization fields with DATE, TIME and TIMESTAMP Snowflake data types, and with Ignore unchanged rows field selected. | |
Binary | 421patches6090 | Fixes the 'Connection pool shut down' error in S3 File Writer Snap by keeping the AmazonS3Client class object open until all the documents are processed. | |
SOAP | 421patches6040 | Requires the use of a generic envelope while executing a create request through an NTLM account in the SOAP Execute Snap to ensure the output is rendered as expected. | |
Azure SQL, Cassandra, Hive, JDBC, MongoDB, MySQL, Oracle, PostgreSQL, Redshift, SAP HANA, Snowflake, SQL Server, Teradata, Vertica | 421patches6144 | Fixes the issue with DB Snaps where a connection thread waits indefinitely causing the subsequent connection requests to become unresponsive. | |
Fixes the issue due to connection leaks during Pipeline execution, across DB Snaps. | |||
Fixes the exception | |||
Confluent Kafka | 421patches6136 | Enhances the Kafka Consumer Snap with an option to allow reprocessing of records that remain unacknowledged due to acknowledge timeouts. This ensures that all records from Kafka are processed. | |
Transform | 421patches5901 | Enhances the JSON Generator Snap to include pass-through functionality where the Snap embeds the upstream input document under the original field of the output document along with other records. | |
Binary | 421patches6077 | Fixes the UnknownHostException error in Binary Snaps due to intermittent failures in FileSystemProvider registration, by replacing ServiceLoader with direct instantiations of FileSystemProvider class objects. | |
Binary | 421patches5994 | Fixes an error in Multi File Reader and File Delete Snaps during Pipeline execution where the Snaps fail to retrieve the temporary files from the given S3 location. | |
Binary | 421patches5963 | Enhances the Directory Browser Snap by making the File filter field optional. | |
SharePoint Online | 421patches6011 (Latest and Stable) | Introduces the productized version of SharePoint Online Snap Pack, which enables organizations to share and manage content and collaborate across an organization using the following SharePoint Online Snaps:
| |
Binary | 421patches5975 | Fixes the Directory Browser Snap that fails with an | |
Transform | 421patches5848 | Fixes the Sort Snap that fails while performing sorting, displaying a NoClassDefFoundError . | |
Azure SQL | 421patches5864 | Adds support for UTF_8 characters with BCP (bulk copy program) command to the Azure SQL Bulk Extract Snap. | |
Splunk | 421patches5851 | Fixes the Splunk Snaps that fail to route connection errors to error view, thus aborting the Snaps. | |
Confluent Kafka | 421patches5862 | Fixes the Confluent Kafka Consumer Snap that aborts abruptly upon getting a CommitFailedException error. Upgraded the Kafka client library from version 2.2 to 2.5. | |
Transform | 421patches5848 | Fixes the Excel Formatter Snap that fails to create an empty worksheet when you do not select the Ignore Empty Stream checkbox. | |
Microsoft Dynamics AX | 421patches5851 | Fixes an issue in the Microsoft Dynamics AX Execute Snap that fails when using the Apache HTTPComponents library API if the request XML has Unicode characters. | |
Binary | 421patches5871 | When configuring the SSH Auth account type, SSH passphrase is now optional, whereas it was a required field earlier. | |
Anaplan | 421patches5851 | Fixes the Anaplan Action Snap that outputs the same value for TaskID and TaskName parameters. | |
Coupa | 421patches5851 | Fixes the Coupa Read Snap that is unable to read all the available records. | |
DynamoDB | 421patches5851 | Removes the Snap timeout setting while waiting for the threads to complete processing. | |
Hive | 421patches5851 | Fixes the Hive Execute Snap that fails with a java.lang.NullPointerException error. | |
REST | rest8858 | Fixes the REST Get and REST Post Snaps that does not parse Content-Encoding and Content-Type headers for gzipped responses. | |
Snowflake | db/snowflake8860 | Adds a new field, Handle Timestamp and Date Time Data, to Snowflake Lookup Snap. This field enables you to decide whether the Snap should translate UTC time to your local time and the format of the Date Time data. | |
MS Dynamics 365 for Sales | dynamics365forsales8856 | Fixes Pipeline validation error that occurs when writing data from Dynamics 365 for Sales Update Snap to output documents. | |
Flow | flow8855 | Fixes the Binary Router Snap's failure in Ultra Pipelines by preventing a deep-copy (performing a copy of the original objects recursively) of binary data headers. | |
DynamoDB | dynamodb8854 | Fixes the DynamoDB Scan Snap pagination issue where the second page of output and beyond cannot be accessed during execution and validation. | |
Hadoop | hadoop8853 | Updates the Parquet Writer and Parquet Reader Snaps to support the yyyy-MM-dd format for the DATE logical type. | |
Binary | binary8849 | Enhances the Directory Browser Snap by adding a new advanced property Use fast file-type check for WASB/WASBS protocols for quick Snap execution. | |
binary8843 | Fixes the issue in PGP Encrypt Snap where decrypting its output file using a third-party PGP software adds a random suffix and TMP extension in the file name. This issue occurs only when using the PGP Encrypt Snap in a Windows-based Groundplex. | ||
MySQL | mysql8842 |
| |
Azure SQL, | MULTIPLE8841 | Fixes the connection issue in Database Snaps by detecting and closing open connections after the Snap execution ends. | |
SQL Server | db/sqlserver8837 | Fixes the SQL Server - Stored Procedure Snap to support the UUID datatype. | |
Salesforce | salesforce8829 | Fixes the Salesforce Create, Salesforce Update, Salesforce Bulk Create, Salesforce Bulk Update, and Salesforce Bulk Upsert Snaps, enabling you to upload files containing duplicate values across multiple fields. |
eXtreme (Big Data)
Snap Pack | Update Date | Build | Updates |
---|---|---|---|
421patches5928 |
| ||
421patches5851 | Optimizes Spark engine execution on AWS EMR, requiring lesser compute resources. | ||
N/A | master-7451 – recommended | Fixes the following issues in the DataViz feature:
|
Have feedback? Email documentation@snaplogic.com | Ask a question in the SnapLogic Community
© 2017-2024 SnapLogic, Inc.