May 2019, 4.17 Release Notes

On this page:

  • Per the SnapLogic Release Process, all Snaplexes across Orgs are upgraded to the 4.17 release at 9 pm PT on June 15, 2019. In case of Hadooplex, you have to manually upgrade the Snaplex. 
  • This release also includes the patches made to SnapLogic after the 4.16 GA release, as listed in the 4.16 Dot Releases page. 
  • We recommend that you upgrade to the latest Chrome or Firefox browser version and clear your browser cache before logging into the SnapLogic Intelligent Integration Platform (IIP).

Release Highlights

General Platform Enhancements

  • Disabled Snap Functionality: Introduces a fundamental shift in the modes in which your Pipelines are executed. You can now disable a Snap, preventing it–and all downstream Snaps–from executing. This change takes a major complexity out of iterative development of Pipelines. If you have Pipelines with multiple branches, you can now test each branch in isolation without having to copy and import the branch into a new canvas.



  • Triggered Task Updates: Introduces the following enhancements for running Pipelines as Triggered Tasks:
    • You can access custom HTTP headers during execution through Pipeline parameters.
    • Performance optimizations resulting in improved latency when triggering Tasks through On-premise and Local Override URLs, reducing latency.
  • API Management EnhancementsThe API Management layer is enhanced to handle at-scale management of APIs (Triggered tasks) configured to run on a Groundplex, as well as add support for OAuth2 authentication flow for APIs. APIM Proxies now support Tasks that are triggered from Groundplexes.
  • Enhanced Support for Data Catalog: Enhances data catalog usability by enabling you to infer the Schema of your data using the JSON Formatter Snap. Previously, only metadata from Parquet-formatted files could be written to the Data Catalog. You can now use the following Snap pattern to populate the Data Catalog: JSON Formatter → File Writer → Catalog Insert.
  • SFTP connection update for Snaps: Changed the default value for maxSftpChannels field to 2.
  • New Expressions: Enhance the SnapLogic Expressions Language by adding the following expressions:

UI

General

  • The session timeout behavior has changed. Until 4.16, if you belonged to multiple Orgs, the shortest idle timeout set in any of the Orgs was applied. With the 4.17 release, the session timeout of the current Org is applied instead.
  • Insights data is now automatically aggregated on a daily basis. As such, the Insights Schedule setting has been removed from SnapLogic Manager. 

Designer

  • Data Preview Enhancement: Adds the ability to display your data in pie, bar, line, and scatter charts. You can see the entire dataset while navigating through your data records in the paginated tabular view. Apart from offering better controls for navigating through your previewed data Snap by Snap, the enhanced Data Preview form provides multiple options for plotting the various categories of data. You can also select the number of records (up to 2000) that you want to preview through a drop-down menu available on some Snaps.

    The Data Preview charts only appear if you are on the latest Snaplex version.




  • Snaplex Deprecated Version Indicators: Alerts you about using an older Snap Catalog if any Snaplex node is running an older version of SnapLogic. Starting with the May 2019 release, a warning appears if any of your Snaplexes are deprecated. Additionally, in the Snaplex drop-down menu, the deprecated Snaplexes appear in italics, shaded in gray.

Manager

  • Files: Enhances SnapLogic Manager to support the download of certain file types, such as HTML, JPEG, and PNG files. Additionally, you can now upload multiple files at a time to Manager.
  • Notification Enhancements: When you log in as an Org Admin, you can configure notifications in Manager for CPU and Memory usage based on thresholds that you set in Manager.

Dashboard

  • Snaplex Deprecated Version Indicators: When you hover over a Snaplex that contains old versions of JCC nodes, a warning message appears indicating the deprecated version of the JCC node. When you click the Snaplex, the Details pop-up for that Snaplex displays a note recommending the latest SnapLogic version.

Snap Enhancements

  • Pipelines that start before a Snap update continue to run with the older version of the Snap. All Pipelines executed after a Snap update run with the new Snaps.
  • For Ultra Pipelines, the currently running instances continue to run with the older Snap Packs. Editing and saving the task instance in Manager causes a rolling restart of the Ultra Pipeline instances, and the new Snap Packs get picked up.
  • Customers using SQL Server with Windows authentication and SAP need to restart their Groundplex and Cloudplex instances.
  • New Snap Packs appear in the SnapLogic Designer only after all the JCCs in an org upgrade to the latest version.

Updated Snap Packs

Binary Snap Pack

Confluent Kafka Snap Pack and Hadoop Snap Pack

  • The Snap Packs are tested and certified against CDH 6.1.

Database Snap Packs

Hive Snap Pack

  • Added a new authentication method, User ID and Password with SSL, for Hive SSL Accounts which allows SSL connections for valid user name and password credentials.
  • Certified and tested the Snap Pack against CDH 6.1.

ML Analytics Snap Pack

  • ProfileAdded an optional second output view. When enabled, this outputs a graphical representation of the Snap's first output as a HTML file.

ML Core Snap Pack

  • AutoML: Enhanced the AutoML Snap. You can now:
    • Select algorithms to derive the top models.
    • Input the best model generated by another AutoML Snap from a previous execution.
    • View an interactive HTML report that contains statistics of up to 10 models.
  • Clustering: New Snap that performs exploratory data analysis to find hidden patterns or groupings in data.

ML Data Preparation Snap Pack

  • Feature Synthesis: New Snap that automatically creates features out of multiple datasets that share a one-to-one or one-to-many relationship with each other. Features are measurements of data points. For example, height, mean, mode, min, max, etc.
  • Match: New Snap that identifies matched records across datasets that do not have a common key field.

NetSuite Snap Pack

  • NetSuite is enforcing two-factor authentication (2FA) for credentials-based accounts and advises that all web-based integrations transition to token-based authentication. To support this transition, NetSuite Credentials account will be deprecated, and we recommend that you either create or switch to NetSuite Token account immediately, so that all SnapLogic Pipelines continue to work without any disruption.

REST Snap Pack

  • Added support for OAuth 1.0 accounts, which enable you to connect to applications that require OAuth 1.0.

ServiceNow Snap Pack 

  • ServiceNow Insert and ServiceNow Query: Added Batch Size field that enables you to configure the number of documents that you want to process per batch. You can configure the Batch Size for Ultra Tasks as well.

Transform Snap Pack

  • JSON Formatter: Added the Derive schema from sample size menu option, whereby you select the sampling size of the schema from the data source.
  • Mapper: Added the capability to select either document (previously supported) or binary data (new) for your input and output views.

Snap Execution Enhancement Impacts all Snaps

This release renames the Execute During Preview field as Snap Execution for those Snaps where the field exists and introduces a drop-down menu called Snap Execution for all other Snaps. This field allows you to enable (full or limited) or disable Snap execution.

Backwards Compatibility

Disabled Snaps are not supported in Snap Packs prior to the 4.17 release. If you decide to roll back a Snap Pack to an earlier version, change any Snap Execution Type from Disabled to Execute Only or Execute & Validate before you initiate the rolled-back.

Known Issues

Platform/UI

  • If you click a Pipeline link, SnapLogic Designer opens the Pipeline only if you are in the Org that corresponds to the Pipeline. Otherwise, an Org change is implemented, and an empty workspace appears. The workaround is to paste the link in the browser URL bar to open the Pipeline.
  • You might experience performance issues running ground triggered Pipelines that have Pipeline Execute Snaps when the child Pipeline runs on a different node.
  • The JSON View and Table View are incorrect for new line character (\n) in the Snap output preview.
  • When the output document contains too many columns, the dropdown list in DataViz view of the Data Preview fails to show all of the columns when you should be able to scroll through all of them.
  • If you create a Pipeline with a File Reader Snap left open while creating a second Pipeline with the File Writer Snap open, you receive the following error message when saving the second Pipeline: 


Try again or refresh the page. If error continues to occur, please contact customer support.
sl.ibm.saveAll() Threw an unexpected error: "TypeError: Converting circular structure to JSON".

  Clicking OK results in the following error:

Refresh the page to try reloading your workspace. If the error continues to occur, please contact customer support.
Uncaught TypeError: Cannot read property 'class_id' of undefined.

Snaps

  • In the Email Reader Snap, when you specify the full file path in the Attachment Folder field and you select Rename in the File Action field, you receive the following error message:  Failed to retrieve messages.

  • Soap Execute Snap does not evaluate expressions for the Attachments properties, even when the expressions are enabled.
  • ServiceNow Snaps do not work with Ultra Pipelines/Tasks.
  • Snowflake Lookup Snap fails to retrieve table column details in Lookup column name when the same table name exists in another database in Snowflake.
  • Redshift Insert Snap fails to create a table with a 2nd input view when the input documents contain NCHAR/BPCHAR (fixed length character string) data type column data.
  • MS Dynamics 365 for Sales Read Snap fails with a NullPointerException error.
  • Auto-refresh does not work for Box Accounts.
  • Parquet Writer Snap is unable to write Parquet files to S3 when executing Pipelines on HDP Hadooplex.
  • File Delete Snap fails to validate even when the SFTP file is successfully deleted.
  • Google Worksheet Writer Snap is having an issue where the number of valid records in the output should not include the header row. 
  • JDBC Insert Snap connected with Derby database when create table is enabled, fails with the length, precision, scale attribute for column, or type mapping VARCHAR (variable character field) error.
  • File Reader Snap is accepting zip files, which should not the expected behavior.
  • Azure SQL - Bulk Extract Snap is  unable to parse big integer calculations. 

  • OpenAir Read Snap is missing ResourceRequest and ResourceRequestQueue read types.
  • Anaplan Read Snap pulls archived and deleted models.
  • The Snowflake Bulk Upsert Snap does not support the file format options that the Snowflake Bulk Load Snap supports.
  • CSV Formatter Snap fails when no value is entered in the Quote character field. A possible workaround is to ensure a null value or in Manager choose the 4.16 release Snap version – snapsmrc508.

UAT Delta

This section tracks the changes made during the iterative pushes to the UAT server and the GA release. The expected schedule is:

  • UAT #1: Friday, April 19, 2019 (Release updates are published above)
  • UAT #2: Friday, April 26, 2019
  • UAT #3: Friday, May 3, 2019
  • GA: Saturday, May 11, 2019 (9 pm PT)

UAT #2 Updates

UI

  • Fixed an issue where the Pie Chart in Data Preview does not display when the document count is over 500 because there are too many sectors.
  • Fixed an issue where the Alert Notification Settings dialog for Snaplex Node Usage Event Type does not have a field for the Snaplex name.
  • Fixed an issue where the Visualization Key drop-down menu does not display correctly for the Data Preview.
  • Fixed an issue where infoboxes are not displaying properly in Match, Clean Missing Values, and Numeric to Categorical Snaps.
  • Fixed an issue where Data Preview crashes after selecting remaining Dependent Keys when the values are already specified.
  • Fixed an issue with Profile and AutoML where HTML rendering did not display correctly in Firefox browsers.
  • Fixed an issue where you have to reload Tasks to see them in SnapLogic Manager.
  • Fixed an issue to make the table in Data Preview display correctly.

Platform

  • Fixed an issue that causes preview data to be unavailable after 15 minutes.
  • Fixed an issue with the Snap version used when triggering Pipeline execution from a Groundplex.
  • Fixed an issue that causes the certificate warning to show up when using signed certificates.
  • Fixed an issue that causes the SnapLogic Designer canvas to be cleared when switching organizations.
  • Fixed an error during Dynamic Validation for the Mapper Snap when a binary input view is selected.
  • Fixed a property value with Dynamic Accounts wherein properties in these Snaps did not validate properly, resulting in the error: java.lang.String cannot be cast to java.math.BigInteger.
  • Fixed an issue where after creating an alert for node usage, the notification is not actually sent to the recipients.
  • Fixed a visualization issue in the new Data Previewer that does not work if the previewer is opened before the Pipeline validation is complete. 
  • Fixed an issue with The Profile and AutoML report that do not work in Firefox because of a security policy. 

Big Data

  • Cluster events for eXtreme such as cluster_initiated and cluster_terminated are not displaying on the Dashboard

Snaps

  • Fixed an issue with the ForEach Snap where no preview output displays when the Validate and Execute option is selected as the Execution Mode.
  • Fixed an issue with the Directory Browser and File Poller Snaps that fails getting a list of files when using an SMB server account.
  • Fixed an issue with the HDFS ZipFile Read Snap that writes the encoded name to the output view despite the Prevent URL Encoding property enabled.
  • Fixed an issue with the File Poller Snap that fails to access the sldb:/// Directory, displaying a "URLDecoder: Illegal hex characters in escape (%) pattern" error.
  • Fixed an issue with the MongoDB Find Snap that fails with Query Expression with the error "Failed to parse MongoDB query expression."
  • Fixed an issue where the Microsoft Dynamics CRM Account fails to validate after the first time as the ADFS Mex URL property does not work as expected.

UAT #3 Updates

UI

  • Fixed an issue where Table and JSON Views display the new line character (\n) incorrectly in the Data Preview.

  • Fixed an issue where File Preview did not display binary files or a file name given in an incorrect file path.

  • Fixed an issue where ML Showcase, some tutorials, and the reporting feature of Profile and AutoML Snaps when loading the scripts for Bootstrap, JQuery, and Highcharts display unreliably. 

Platform

  • Fixed an issue where Snaplex alerts generate regardless of the Snaplex for which the alert is configured.

  • Fixed an issue where runtime information and execution details are missing from the Dashboard for some Triggered Tasks.

  • Fixed an issue where If a child Pipeline has an unconnected output view, the Pipeline Execute Snap in the parent Pipeline has to have an output view, or else the main Pipeline fails with an Invalid Output View Configuration error.

Big Data

  • Fixed an issue where Extreme Snowflake Pipelines fail due to the Snowflake Account not decrypting.
  • Fixed an issue where Extreme Pipelines fail when using private Snap Packs.
  • Fixed an issue when downloading Pipeline Execution statistics on Hadooplex gives a 500 error.

Snaps

  • Fixed an issue with the Match Snaps and Feature Synthesis Snaps where the suggested schema works for only one of the input views.
  • Fixed an issue with the ServiceNow Query Snap that retrieves only 250 records at a time. 
  • Fixed an issue with the Preview doc count value when it is hardcoded to 50 for some Snaps and cannot be set to any other value.
  • Fixed an issue with the REST OAuth2 accounts experiencing an intermittent issue where a Pipeline fails displaying an "Invalid JWT token. The token is expired." error even when the token is valid.
  • Fixed an issue with the Catalog Writer Snap (Spark SQL 2.x) where the expressions are not evaluated correctly.
  • Fixed an issue with the Parquet Reader, ORC Reader, HDFS Reader, HDFS ZipFile Reader, and Hadoop Directory Browser Snaps that do not accept a full path in the File and Directory fields.
  • Fixed an issue with the Match Snap where incorrect input values in Left field and Right field outputs all the records from the upstream Snap.
  • Fixed an issue with the Directory Browser, File Poller, and Multi File Reader Snaps where their performance slows down when using using SMBv1 Account, taking over 2 hours to process.  
  • Fixed an issue with the Pipeline Execute Snap that fails when not having an output open view after receiving a document from a child Pipeline. 
  • Fixed an issue with the Match Snap that fails when the number of documents in the first input view is more than the ones in the second input view.  
  • Fixed an issue with the File Poller Snap not polling indefinitely when Polling timeout value is -1.
  • Fixed an issue with the File Delete Snap for S3 fails with "The file server returned success on the delete request, but the file still exists" error when the Validate deletion option is selected.
  • Fixed an issue with the REST Snaps where auto refresh does not work for OAuth2 accounts.

GA Updates

Platform

  • Fixed an issue that caused authentication errors in some scenarios when running ground triggered Pipelines that have Pipeline Execute Snaps.
  • Fixed an issue where the pipe.flags.is_suggest flag is not initialized for ground trigger Pipelines.
  • Fixed an issue where the property label for Execute During Preview was set incorrectly for eXtreme mode and Spark mode Snaps.
  • Fixed an issue that caused File not found errors during SLFS file operations.

4.17 Dot Releases

Snap Packs

Snap PackUpdate DateBuildUpdates
MongoDB7/29/2019db/mongo7665Fixed an issue with the MongoDB Find Snap wherein the Snap fails to query using UUIDs.
Flow7/22/2019flow7637Fixed an issue with the Data Validator Snap wherein the output fails to report all but one of the violations when there are multiple constraints of the Required type.
SQL Server7/18/2019MULTIPLE7479

Fixed an issue with the SQL Server - Lookup Snap wherein it fails a UUID search.

PostgreSQL7/16/2019db/postgres7588Fixed an issue with tables sharing an overlapping column name wherein Pipeline execution fails due to the table collision.
Binary7/16/2019binary7589

Fixed an issue with the File Reader Snap wherein SMB fails to provide multiple Snaps with concurrent access.

Salesforce7/15/2019salesforce7474

Added Order by clause and Limit clause properties to the Salesforce Reader Snap to fix an issue wherein the Snap throws an error while fetching records.

Confluent Kafka07/12/2019confluentkafka7537

Updated the following Snaps:

  • Confluent Kafka Consumer Snap
    • Properties added: Max poll records, Acknowledge mode, Key deserializer, Value deserializer, Add 1 to offsets.
    • Properties removed: Auto commit interval (msec), Standardize error output.
    • Diagnostic fields added in the metadata object in the Snap's output:
      • client_id:  The unique ID assigned to a single Kafka consumer client instance. All documents written by a Consumer Snap during a single pipeline execution or validation will have the same value for this field.
      • batch_size: The number of messages in the batch of messages that the current message came from. Note that “batch” isn’t Kafka terminology; we’re using it to mean the set of messages received from the broker in a single call to Kafka’s poll method.
      • batch_index: The current batch number relative to a single consumer instance.  This number starts at 0 for the first batch and is only incremented for batches which contain at least one message.
      • record_index: The current record number relative to its batch.  The first message in a batch has an index of 0; the last has an index of batch_size - 1.
  • Confluent Kafka Acknowledge Snap
    • Property added: Metadata path
  • Confluent Kafka Producer Snap
    • Properties added: Key serializer, Value serializer.
Binary07/02/2019binary7487Fixed an issue with the File Writer Snap wherein the Overwrite File action for SMB files fails on subsequent executions.
Oracle06/26/2019db/oracle7459Fixed an issue with the Oracle Stored Procedure Snap wherein the Pipeline execution fails with an error when the input data type is CLOB.
Transform06/26/2019transform7431

Added a new field, Ignore empty stream, to the Avro Formatter Snap that writes an empty array in the output in case of no input documents.

Redshift6/20/2019db/redshift7433Fixed an issue with the Redshift Bulk Load Snap wherein the Snap fails to copy the entire data from source to the Redshift table without any statements being aborted.
MongoDB6/18/2019db/mongo7427Fixed a Null Pointer Exception for old MongoDB accounts that did not have driver jars.
Anaplan6/18/2019Anaplan7422Fixed an issue with the Anaplan Read Snap wherein the Snap pulls archived and deleted models.
Transform6/17/2019Transform7417Added a new field, Format as canonical XML, to the XSLT Snap that enables canonical XML formatting.
All Snap Packs6/11/2019ALL7402Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers.
SQL Server6/10/2019db/sqlserver7394Fixed an issue wherein bit data types in the SQL Server - Select table convert to true or false instead of 0 or 1.
MySQL6/10/2019db/mysql7395Fixed an issue wherein bit data types in the MySQL - Select table convert to true or false instead of 0 or 1.
Snowflake6/10/2019db/snowflake7396Fixed an issue wherein bit data types in the Snowflake - Select table convert to true or false instead of 0 or 1.
Binary6/7/2019binary7381Fixed an issue with SMB accounts wherein login to the server fails if password contains a plus (+) sign.
Salesforce6/7/2019salesforce7352

Fixed an issue with the Read and SOQL Snaps wherein Pipeline validation fails because of a dynamic account opening multiple file descriptors.

SumoLogic6/5/2019sumologic7369Fixed an issue with the Execute Search Job Snap wherein it fails to return the results correctly.
Binary6/4/2019binary7366

Added support for multiple slashes as a prefix for SMB file URLs.

MySQL6/3/2019db/mysql7357Fixed an issue with the MySQL Execute snap wherein the Retry property for connection fails after the database connection is severed.
Snowflake 5/31/2019db/snowflake7334

Added AWS Server-Side Encryption support for AWS S3 and AWS KMS (Key Management Service) for Snowflake Bulk Load, Snowflake Bulk Upsert, and Snowflake Unload Snaps.

MongoDB5/24/2019db/mongo7331Fixed an issue with the MongoDB - Update Snap wherein the Snap converts all non-updated integer and float data types to string data type.
Google Spreadsheet5/15/2019google/spreadsheet7283

Fixed an issue with the Worksheet Writer Snap wherein the No of valid records written field in the output document would count the header row.

Binary5/15/2019binary7280

Added a new field, File delete action, to the File Delete Snap that handles missing target files.

Platform

Update DateBuildUpdates
7/2/2019

master-5492
Control Plane Only

Enhanced SSO Login functionality to support additional authentication methods RequestedAuthnContext and RequestedAuthnContext Comparison at the Org-level. As an Org admin, you can configure these additional options for user logins across your Org.
6/28/2019master-5508 - Patch 4 
(Recommended)
  • Fixed an issue that can cause Pipeline Execute child pipelines to fail while loading accounts.
6/13/2019master-5472 - Patch 3 
  • Fixed an issue that causes Groundplex triggered Pipelines comprising Pipeline Execute Snap with child Pipelines and performing an SLFS operation to fail with a 401 error.
  • Fixed an issue that causes expression verification to generate warnings for the Pipeline suggest feature.
6/4/2019master-5396 - Patch 2

Fixed a performance issue in Tasks triggered on a Groundplex, where the Pipeline uses the Pipeline Execute Snap to invoke a child Pipeline that references expression libraries.

5/22/2019master-5365 - Patch 1
  • Added a retry handler to manage network failures during Pipeline information retrieval.
  • Enhanced performance optimization for Groundplex Triggered Pipelines that use the Pipeline Execute Snap.
  • Fixed an issue which causes some Pipelines, typically with Union Snaps, to hang during execution.
  • Fixed an issue which causes some Pipeline creation times to show as Invalid date.
5/20/2019N/A
  • Fixed an issue causing Groundplex Triggered Task invocations to perform permission checks as the invoker instead of the Task owner.
  • Fixed an issue causing Groundplex Triggered Tasks to fail if the CC user is present as a member of the organization.
  • Fixed an issue causing runtime pushes to fail if a large number of Groundplex triggers are done concurrently.

UI

Update DateBuildUpdates
6/13/2019N/A
  • Fixed an issue with the Dependent Keys that causes the DataViz data preview to crash intermittently. 
  • Fixed an issue with the Visualization Key dropdown that does not let you scroll all the column values if they overflow the given dropdown size.
  • Fixed issues with the Expression Builder for a Snap wherein you cannot undo an action; the cursor defaults to the beginning position while editing; and the scroll bar is missing.
  • Fixed an issue with the Snap properties view that does not close when clicking the esc key on the keyboard.
5/20/2019N/A
  • Fixed an issue where a script error appears when saving a Snap with a file input field (such as File Writer) in a Pipeline while at the same time another Snap with a file input field (like File Reader) is open in a different Pipeline.
  • Fixed an issue where the JSON and Table View displays the new line character (\n) incorrectly in the Snap output preview.
  • Fixed an issue with Snap recommendations when checking the compatibility of view type of the Mapper and Pipeline Execute Snaps because both Snaps support binary and document input/output.