In this article:

Platform Enhancements

General

• Kubernetes Support: You can now deploy your Snaplex instances in a Kubernetes environment using Helm Charts. This support enables you to benefit from containerized technology through the resource efficiency and automated operations that Kubernetes container-orchestration system provides.
  
  
• Scheduled Task Notifications: Improving the mechanism for triggering email notifications when an error occurs in Scheduled Tasks. Previously, even if you configured a notification, Scheduled Task failures did not trigger email notifications under all failure conditions. With this release, you might see an increase in failure notifications because of this improvement. For example, if permissions are modified after a Scheduled Task is set up, then the Pipeline called by the Task can no longer execute, triggering an email notification.
  
  

• Modifying Shared Objects: Enhancing Pipelines Executions Statistics to display a warning message when you attempt to modify an object shared between multiple documents. Sharing primarily occurs with the $original object when a Snap outputs multiple documents for a single input. For example, an SQL Select Snap that matches multiple rows for a single input document produces documents where, for each entry, $original refers to the same object. With this release, if you follow that SQL Select Snap with a Mapper Snap that writes to a property in $original, a warning message appears in the Execution Statistics window. Since modifying a shared object can lead to inconsistent results, you must copy the object. 

  Benefit to Sharing $original Objects

  The $original objects are shared for performance reasons. Copying them for every output document can be resource intensive.

• Ensuring Backward Compatibility with JDK 9 and Upwards: If your Snaplex is using JDK 8u181 or higher, your existing Active Directory Pipelines are likely to break with a Failed to validate account error. This is because Java's Improved LDAP Support security enhancement enables endpoint identification on LDAPS connections by default.

  Workaround:

  Set the JVM option com.sun.jndi.ldap.object.disableEndpointIdentification to true in your Snaplex's node properties (Global properties). This disables endpoint identification. You may have to contact your SnapLogic Organization Admin to include this setting. 

API Management Enhancements

Enhancing the SnapLogic API Management with the following API proxy features:

• Request Size Limiting: Support for blocking API requests that attempt to pass a payload size beyond a specified limit. Such requests can affect critical backend systems. This policy enables you to restrict the payload size in the API response. 
  
  
• Update Proxies: Support for editing the following configuration properties in a Proxy:
  
  • Modify Policies
  • Modify the APIs included in the Proxy

The update functionality makes it easier for you to modify the endpoints and policies in your Proxies while retaining the same Proxy name. Previously, to update a Proxy, you would have had to create a new Proxy with a different name.

Known Issues

• Importing a synthesized Pipeline through Pipeline recommendations may cause an error. Use the Replace option instead.
  

Previous Dot Releases in 4.19 - Platform

The following 4.18 Platform Dot Release updates are now a part of the 4.19 release:

UI Enhancements

Designer

• New UI Form: Introducing a new UI form as an opt-in feature for Snap configuration. Besides a new look, the new UI form offers an improved user experience and reduced latency. To enable the new UI form, select the New Form UI checkbox on the User Settings menu in SnapLogic Designer. Additionally, in the Mapper Snap integrates the AutoLink feature (formerly, SmartLink) in the same Infobox popup.
  

• Data Preview: Enhancing the Data Preview feature with the addition of the Formatted checkbox. This functionality enables you to format document data with line-feeds that would otherwise be ignored, and increase the readability of this data in table/JSON modes. When viewing error documents that have a stack trace column in JSON/table mode, you can select Formatted to view this information easier.

• SnapLogic Cloud Pattern Search and Preview

  • Introducing SnapLogic Cloud Pattern Catalog Search. You can now search the extensive set of Pattern Pipelines from SnapLogic and preview Pattern Pipelines that match your search criteria. You can either use a Pattern Pipeline to create a new Pipeline or copy its contents into an existing Pipeline.
    

  • Creating Branched Pipelines Using Pipeline Recommendations

    You can now create complex branched Pipelines using SnapLogic's Pipeline Recommendations feature. Just provide a start Snap, an end Snap, and an intermediary Snap and let Iris AI recommend Pipelines that connect all the three Snaps in a specified order.

    

Manager

• Snaplex Notifications: Adding a Snaplex drop-down list for Notifications for ease of use. You can now set notifications for your Snaplex instances without having to look up the name in the Designer or Dashboard. When you configure a Snaplex notification, a drop-down list of Snaplex names appears.

  

Dashboard

• Error Messages and Logs: Enhancing the troubleshooting experience in the SnapLogic Dashboard. Logging for JCC nodes is optimized so that duplicate or similar logs are filtered out, enabling you to find unique errors more quickly when you click View Details on the Snap Statistics tab in the Pipeline Executions Statistics popup.

Developer Portal

• Logo and Banner Customization: Introducing the ability to customize your API Developer Portal. You can now add your own organizational logo and banner text to the Developer Portal from API Manager. Since the Developer Portal serves as the public catalog for your users to consume APIs, you can now customize the look to reflect your organization's branding.

  

Known Issues

General

• Clicking the Execute button in the Try it section on the API tab of a Proxy might cause the Proxy window to abruptly close.

• When configuring an account for a database Snap, the Search field is not enabled in the the drop-down list.

New UI Form

• The Workday Reader Snap does not work with the New UI Form enabled because the accounts cannot be validated.

• Leaving mandatory fields blank while configuring a Snap or its account, and then clicking Save should display an error message but does not when the New UI Form enabled.

Previous Dot Releases in 4.19 - UI

The following 4.18 UI Dot Release updates are now a part of the 4.19 release:

Snap Enhancements

General

• AutoLink (formerly, SmartLink): Enhances the AutoLink feature to simplify the mapping of data by suggesting Input Schema mapping to the Target Schema in the Mapper Snap. The SnapLogic platform uses a combination of algorithms to derive mapping suggestions. AutoLink feature enables you to:
  • Filter mapping suggestions based on a score threshold.
  • Get the best mapping recommendations.
  • Map nested schemas easily as the machine learning model has higher flexibility for mapping nested schemas.
  • Preview and refine the matches.

• SnapLogic B2B Integration: Leverages SnapLogic's capabilities to integrate the EDI gateway of 1 EDI Source with another EDI gateway or B2B solution. You can integrate with multiple applications supported by SnapLogic, such as NetSuite, SAP, and Oracle.

New Snap Pack

• Coupa: We now integrate with Coupa's business spend management application. This Snap Pack includes the following Snaps:
  
  • Coupa Create: Create Coupa resource records.
  • Coupa Read: Fetch Coupa resource records.
  • Coupa Update: Update Coupa resource records. You can also deactivate Coupa resources using this Snap. 

Updated Snap Packs

• SOAP 
  • Introduces the Library API field to the SOAP Execute Snap to enable execution over HTTP. The supported libraries are Apache HttpComponents and Apache CXF, and both the libraries support the Message Transmission Optimization Mechanism (MTOM). 
    
  • Adds a new field, Allow duplicate headers, in the SOAP Execute Snap. Enabling this field allows duplicate headers value; if disabled, the last value of the duplicate header key is used.
    
    

• Azure SQL: Enhances the error handling in PolyBase Bulk Load Snap when writing to a data warehouse. The Snap writes a new blob in the Azure container. This new blob highlights the first invalid row that caused the bulk load operation to fail.
  
  
• Binary
  • The Key Passphrase field is now a mandatory field for the SSH Auth account. This update does not impact existing accounts and Pipelines. However, if you update existing SSH accounts or create a new account, then the passphrase is required. 
  • Raises the minimum buffer size in the S3 File Writer Snap to 6 MB.
    
    
• SnapLogic Metadata: The SnapLogic Read Snap now features the Normalization checkbox, which, when selected, makes Pipeline comparisons easier.
  
  
• ML Data Preparation: Introducing the Deduplicate Snap. Use this Snap to remove duplicate records from input documents. When you use multiple matching criteria to deduplicate your data, it is evaluated using each criterion separately, and then aggregated to give the final result.
  
  
• Transform Snap Pack: The output of the AVG function in the Aggregate Snap now rounds up all numeric values that have more than 16 digits.
  
  
• NetSuite: Supports NetSuite version 2019.1. While this does not impact any of your existing Pipelines, you can use the following WSDL configurations in your SnapLogic NetSuite accounts to benefit from this version's capabilities:
  • WSDL Year: 2019
  • WSDL Release: 1
  • WSDL Patch: 0
    
    
• PostgreSQL: Adds new Snap, PostgreSQL Bulk Load, to support bulk load operations.
  
  
• Salesforce: Introduces an account type Salesforce.com OAuth2 and the following Snaps that support the Bulk API v2.0:
  • Salesforce Bulk Create: Adds new Salesforce records.
  • Salesforce Bulk Update: Updates Salesforce records.
  • Salesforce Bulk Upsert: Updates or inserts Salesforce records.
  • Salesforce Bulk Delete: Deletes Salesforce records.
    
    
• Snowflake
  • Enhances the Snowflake SCD2 Snap to support auto-historization of column data by adding a new field-set, Auto Historization Query. You can now detect whether the incoming record is a historical event or a current event.
  • Raises the minimum buffer size in the Snowflake - Bulk Upsert and Snowflake - Bulk Load Snaps to 6 MB.
    
    
• Workday: The Workday Write Snap now provides a Validate Only Load check box, which enables users to upload–and then manually validate–data before importing it into Workday.
  
  
• ServiceNow: The Snap Pack supports the Madrid release of ServiceNow.
  
  
• PostgreSQL Snap Pack: Added a new PostgreSQL Dynamic Account.
  
  
• RabbitMQ Snap Pack: Added the Auto delete check box, which enables you to indicate that the RabbitMQ autoDelete property associated with the existing queue is set to true.
  
  
• Anaplan Snap Pack: Removed the suggestion button from the Workspace and Model fields in the Snap Settings dialog. Hence, you need to enter the field values manually.  

Breaking Change: Fix to the Active Directory List Users Snap Might Fail Pipelines

We found an issue where the Active Directory List Users Snap inserts a space before every record of each field in the output. While we are fixing this issue in the 4.19 release, if you have Pipelines that have the Active Directory List Users Snap that assumes a space in the first character, then this fix might break your Pipelines. 

Known Issues

• Generic JDBC - Execute Snap: If a SELECT query in the SQL statement field in the Snap Settings contains duplicate column names, then the query result displays the column name twice, with the latter mention prefixed with the table name. If the column name occurs more than twice, the second entry is still prefixed with the table name but it displays the value of the column mentioned last in the query. The example below illustrates this behavior. 

   Example: Incorrect results for SELECT queries with duplicate column names

  Example

  Consider the following query, where the column name "name" is repeated three times: 

  select id, name, city, location, id as name, city as name, location as name from address

  Expected output:

  The  Generic JDBC - Execute Snap's output should reflect the correct query result containing all the six fields, as shown in the image below:

  

  Actual output:

  The Snap's output displays the repeat occurrence of the column "name" only once, prefixed with the table name "address" as shown in the image below. Further, it holds the value of the table column "location", which was mentioned last in the SELECT query. 

  

  
  

  To avoid this issue, we recommend that you give unique column names in the query.

• The ZipFile Write Snap with Use input view label enabled produces incorrect file names – different from the file names assigned by users.
  
  
• The REST GET Snap Link HTTP response header maybe be malformed compared to other third-party applications like POSTMAN.
  

• The File Delete Snap, when configured with a Google Service account, intermittently throws a NoClassDefFoundError: com/google/common/base/MoreObjects error. 
  
  

• The following NetSuite Snaps currently do not support custom records: NetSuite Get, NetSuite Get List, NetSuite Async Get, and NetSuite Delete.
  
  

• The Sandbox Account setting, available while creating a NetSuite account, does not work as expected. Do not select this check box while creating a sandbox NetSuite account.
  
  

• The CSV Formatter Snap ignores escape characters when used along with any quote characters. When this happens, the resultant data cannot be parsed correctly using CSV Parser Snap which can cause an error. See Apache CSV Format issue for details.

Previous Dot Releases in 4.19 - Snaps

The following 4.18 Snap Dot Release updates are now a part of the 4.19 release:

Snap Pack	Update Date	Build	Updates
Confluent Kafka	23 Oct 2019	confluentkafka8111	Fixes an issue with the Confluent Kafka Producer Snap wherein the Topic property doesn't evaluate expression against input documents.
Salesforce	16 Oct 2019	salesforce8066	Fixes an issue with the Salesforce SOQL Snap wherein the Snap fails with NullPointerException.
Hadoop	15 Oct 2019	hadoop8033	Fixes an issue with the Parquet Writer Snap wherein the Snap throws an error when working with WASB protocol.
RabbitMQ	15 Oct 2019	rabbitmq8058	Fixes an issue with RabbitMQ Snaps wherein the Snaps throw an error when the auto-delete setting is set to true.
Snowflake	14 Oct 2019	db/snowflake8044	Fixes an issue with the Snowflake - Select Snap wherein the Snap converts the Snowflake-provided timestamp value to the local timezone of the account.
MS Dynamics 365 for Sales	14 Oct 2019	dynamics365forsales7957	Fixes a NullPointerException issue with the Dynamics 365 for Sales Delete Snap wherein the Object ID is invalid. Fixes a concurrency issue with multiple Dynamics 365 for Sales Snaps when running in parallel.
Binary	14 Oct 2019	binary8046	Enhances the Snap Pack to support AWS SDK 1.11.634 to fix the NullPointerException issue in the AWS SDK. This issue occurs in AWS-related Snaps that have HTTP or HTTPS proxy configured without a username and/or password.
Snowflake		db/snowflake8044
Redshift		db/redshift8043
Confluent Kafka	10 Oct 2019	confluentkafka8006	Fixes an exception that occurs when consuming log-compacted topics, in the Confluent Kafka Consumer Snap.
PostgreSQL	10 Oct 2019	postgres8021	Fixes an issue with the PostgreSQL grammar to better handle the single quote characters.
Binary	09 Oct 2019	MULTIPLE7884	Fixes an issue with the PostgreSQL grammar to better handle the single quote characters.
Snowflake
Redshift
Transform	07 Oct 2019	transform7994	Adds a field Round dates to the Excel Parser Snap, which enables you to round numeric excel data values to the closest second.
SQL Server	07 Oct 2019	db/sqlserver7989	Fixes an issue in the SQL Server Merge Snap wherein identity columns are not handled correctly. Also, adds a field Enable Identity Insert that enables you to insert values into an identity column.
MS Dynamics 365 for Sales	01 Oct 2019	dynamics365forsales7952	Adds support for lookup type in the input document to be formatted per MS Dynamics 365 for Sales syntax.
JMS	01 Oct 2019	jms7865	Fixes an issue with the JMS Consumer Snap that reinserts failed messages in the JMS queue without checking for duplicate messages.
Binary	25 Sep 2019	binary7860	Fixes an issue with the File Writer Snap that is intermittently unable to set the correct permissions.
NetSuite	23 Sep 2019	netsuite7879	Fixes an issue with the NetSuite Snaps that intermittently fail to reach, cache, and view the NetSuite WSDL and XSD files.
MySQL	23 Sep 2019	db/mysql7846	Fixes an issue with the MySQL Snaps that exhibit degraded performance with snapsmrc523.
Splunk	20 Sep 2019	splunk7812	Adds properties named Earliest Relative, Latest Relative, and Preset Relative Search to the Splunk Search Snap to fix an issue wherein the Snap returns inaccurate and inconsistent results regarding last-30-day and year-to-date searches.
Snowflake	16 Sep 2019	db/snowflake7821	Fixes an issue with the Snowflake - Execute Snap wherein the Snap is unable to support the '$' character in query syntax.
Binary	16 Sep 2019	binary7851	Introduces a field Advanced properties in the File Reader, File Writer, File Delete, ZipFile Read, ZipFile Write, Directory Browser, Multi File Reader, File Poller, and File Operation Snaps. Use this field to configure the SAS URI for an access to Azure Storage. The supported SAS types are Service SAS on container, Service SAS on blob, and Account SAS. Introduces a Snap SAS Generator. This Snap enables you to generate Shared Access Storage (SAS) URIs for accessing Azure Storage containers and blobs.
Workday	11 Sep 2019	workday7837	Fixes an issue with the Workday Read Snap wherein the Snap is unable to log SOAP calls.
Salesforce	11 Sep 2019	salesforce7832	Enhances the Salesforce Subscriber Snap to capture Change Data Capture (CDC) events.
Flow	09 Sep 2019	flow7743	Adds a property Validate full JSON path to the Data Validator Snap to fix an issue wherein a field validation failure against a constraint does not produce an error.
Confluent Kafka	30 Aug 2019	confluentkafka7784	Fixes the Consumer Snap to stop quickly and cleanly on CLOSE and STOP lifecycle events.
Oracle	30 Aug 2019	db/oracle7786	Fixes an issue by using the NVARCHAR2 datatype in Oracle databases when the character set is not AL32UTF8. However, a known issue is that special characters appear as .
NetSuite	29 Aug 2019	netsuite7779	Fixes an issue with the NetSuite Update Snap and the NetSuite Create Snap wherein the Snaps are unable to properly handle null values for the standard and the custom fields.
Transform	29 Aug 2019	transform7780	Fixes an issue with the Excel Parser Snap by upgrading Apache POI to version 3.14 wherein the Snap is unable to parse an excel file with custom namespaces. Fixes an issue with the Excel Parser Snap wherein the Snap converts real numbers to two decimal places when formatted for currency. A new property Cell formatting now supports unformatted outputs.
Binary	28 Aug 2019	MULTIPLE7778	Updates the AWS SDK library version to default to the Signature Version 4 Signing process for API requests across regions.
DynamoDB
Redshift
Snowflake
SQS
Transform	26 Aug 2019	transform7741	Fixes an issue with the Sort and Join Snaps wherein the platform removes all temp files at the end of Pipeline execution.
Binary	21 Aug 2019	binary7746	Fixes an issue with the SMB file protocol wherein Binary Snaps are unable to create a native thread using an SMB file path.
Snowflake	20 Aug 2019	db/snowflake7739	Fixes an issue with the Snowflake - Bulk Upsert Snap wherein the Snap fails when using a pipeline parameter in Key columns. Fixes an issue with the Snowflake - Unload Snap wherein the Snap does not abort the query when you stop the Pipeline execution.
Snowflake	20 Aug 2019	db/snowflake7727	Fixes an issue with the Snowflake - Unload Snap wherein the Snap does not abort the query when the Pipeline is stopped.
Transform	19 Aug 2019	transform7711	Fixes an issue with the XML Parser Snap wherein a class cast exception occurs when the Snap is configured with a Splitter and Namespace Context.
Confluent Kafka	16 Aug 2019	confluentkafka7732	Adds support for Kerberos-based authentication to enable connecting to Kerberos-enabled Kafka servers.
Spark SQL	15 Aug 2019	sparksql7718	Fixes an issue with the Spark Pipeline wherein the Pipeline on Hadooplex fails on release 4.18 because of dependency change in the global POM file.
Binary	15 Aug 2019	binary7720	Fixes an issue with the ZipFile Read Snap wherein the Snap fails to extract files from a zip file larger than 100MB when the number of retries is greater than zero.

eXtreme (Big Data) Enhancements

General

• Adding the following new information for Pipeline Execution Statistics:
  

  • Total input and output bytes

  • Total input and output records
  • Bytes and records processed per Spark job ID
    
    
• Adding use cases that demonstrate how you can use SnapLogic eXtreme to solve big data integration and processing challenges on AWS EMR:
  • Analyzing Customer Returns for a Retail Chain: Use SnapLogic eXtreme to analyze customer returns for a retail chain.
  • Analyzing Online vs Stores Sales for a Retail Stores Chain: Analyze customer spend online vs. stores for a retail chain.
  • Analyzing Customer Sales for a Retail Stores Chain: Analyze customer sales for a retail chain.
  • Using eXtreme Delta Lake to Analyze Customer Returns for a Retail Chain: Use eXtreme Delta Lake Snaps to analyze customer returns for a retail chain.
  • Analyzing Google Books Ngrams Dataset: Analyze the Google Books Ngrams dataset to calculate decade-wise top trending topics.
    
    
• Adding support for shorter Directory Paths for Azure WASB and ABFS protocols to bring the functionality at par with SnapLogic's Standard-mode Snaps. For example, wasb:///<containername>/<path>.

New Snap Pack

• eXtreme Delta Lake: Introducing the /wiki/spaces/SD/pages/851837151. Use this Snap Pack to manage data in Delta Lake tables. Delta Lake provides ACID (Atomicity, Consistency, Isolation, and Durability) transactions, scalable metadata handling, and unifies the streaming and batch data processing. The eXtreme Delta Lake Snap Pack offers the following Snaps:
  
  • /wiki/spaces/SD/pages/851935418: Read data from Delta Lake tables.
  • /wiki/spaces/SD/pages/851935426: Insert data into Delta Lake tables.
  • /wiki/spaces/SD/pages/853967501: Merge (or upsert) data into Delta Lake tables.
  This Snap Pack also introduces the new /wiki/spaces/SD/pages/852331782, which enables you to connect your eXtreme Delta Lake Snaps to target Delta Lake tables.

Updated Snap Packs

• Spark SQL 2.x: Includes the following updates:
  • Supports ADLS Gen2 and Gen2 OAuth2 account types for the File Writer - Spark SQL 2.x and File Reader - Spark SQL 2.x Snaps, enabling you to connect the Snaps to your ADLS database.
  • The Catalog WriterSnap introduces a mandatory field File Action that enables various actions on the file when the specified path already exists. You can use this property to append, ignore, and overwrite the existing path.

Deprecated Feature

We are deprecating Hadooplex from 4.19 with a target sunset date during the 4.20 release. 

Fixes

After an eXtremeplex terminates on Azure Databricks, the terminate event now displays on the SnapLogic Dashboard.

Known Issues

• Executing an eXtreme Pipeline twice in the same (unterminated) cluster for Azure Databricks displays Pipeline statistics of both the executed Pipelines, as the SparkUI tab in Databricks' console does not segregate the data. The workaround is to execute the said Pipeline in a new cluster each time, to view the correct Pipeline statistics. 
  
  
• Reading and writing Avro format files currently fails for both AWS EMR and Azure Databricks.

4.19 UAT Delta

4.19 Dot Releases