General

• Introducing the following major enhancements to the API Management service:
  • API Manager: Enables API Developers to control access, discoverability, and approval of Proxies. It also enables users to add and delete other users for consuming APIs created in SnapLogic.
  • Developer Portal: Enables non-SnapLogic users to consume APIs created in the SnapLogic platform.
• Single Sign Out (SSO): Enhances SSO options with the capability to enforce logging out users.

• Pipeline Runtime ID: After the Snaplex instances update to 4.18, the runtime ID for Pipelines will have the following changes:

  • The Pipeline Runtime ID increases in length from 36 characters to 61 characters. If the ID is saved externally, the storage size must increase.
  • The status API for Snaplex nodes returns limited information unless called with user authentication information.

Snaplex IP Address Whitelist Update

To support in-built High Availability (HA) and Disaster Recovery (DR) capabilities for customer organizations that are subscribed to Cloudplex instance in North America, SnapLogic is migrating its Cloud infrastructure. In addition, Groundplex connectivity IP address will also change. Therefore, it is critical that you read the Snaplex IP Address Whitelist Update document.

Upgrading Your Groundplex to Java 11

Starting from 4.18 release, SnapLogic supports Java 11 for Groundplex instances. We recommend customers to migrate to Java 11, since Oracle no longer releases security updates for Java 8. 

For details, see Upgrading Your Groundplex to Java 11.

Deprecated Features

• End of support for Nested Pipelines: Starting from the 4.18 release, we no longer support the guaranteed delivery option for Nested Pipelines (also known as sub-Pipelines). We recommend that you use the Pipeline Execute Snap instead of sub-pipelines. You can use the Pipeline Execute Snap with Resumable child pipelines to handle scenarios where exactly-once-delivery is required. Contact your SnapLogic Account Manager for further details.

• SFTP connection for Snaps: We are reverting the default value for maxSftpChannels to 1. In the 4.17 release, the maxSftpChannels default value was set to 2. However, to keep only one open channel, the value is reverted to 1. Having only one open channel avoids caching of open sessions if all the channels are closed.

Dot Releases in 4.18 - Platform

The following 4.17 Platform Dot Release updates are now a part of the 4.18 release:

Designer

• Data Preview: We are enhancing the Data Preview functionality in Pipelines with new types of graphs.
  
  
• Pipeline Recommendation: We are enhancing the suggestion capabilities of Iris to include Pipeline Recommendations. You can now switch from Snap to Pipeline suggestions at a single click. Iris' new functionalities enable you to do the following when designing your Pipelines: 
  
  • Based on the Pipeline label and the Snaps on the canvas, you can view a gallery of Pipeline shapes, and can choose a Pipeline to open.
  • Iris suggests Pipelines from 3 sources -- existing Pipelines that you have access to, Pattern Pipelines from SnapLogic Cloud Pattern Catalog, and artificially synthesized Pipelines.
  • You can open one of the Pipeline shapes as a new Pipeline.
  • You can also replace the partially configured Pipeline with the Pipeline shape from the gallery. 
    
    
    
• Iris Assistant: You can now use a new option to search for Pipelines using keywords, while also benefiting from improved Snap search capabilities.
  • Snap Search: When you search for a keyword that does not match any Snap name, Iris provides suggestions based on the content on the SnapLogic documentation site. 
    
    
    
  • Pipeline Search: When you search for a Pipeline, Iris displays a list of project spaces and projects that contain Pipelines that match the search terms. The Pipelines are indexed on an hourly basis and new Pipelines are not displayed in the search until the next refresh happens.
     

Manager

• API Manager: We introduce a new page in Settings for API developers with Org-level admin access to control access to APIs and their discoverability type. From API Manager, you can also publish APIs to the Developer Portal. The API Manager puts all your Proxies for an Org in one place. Besides Proxy details, you can also view users, access types, and web applications that consume your Proxies.
  
  
  
• Snaplex Node Disk Notification: We are introducing a Snaplex notification for JCC (Java Containerized Component) node disk usage. As an Org-level admin, you can create a notification that alerts users when the threshold for available capacity has been exceeded.

Dashboard

• Data Lineage: Introducing a new tab in Dashboard that provides a visualization of all the data sources to which Pipelines connect. You can now select one or more Pipelines and fetch the data sources for those Pipelines. Conversely, you can start with a data source, and view the list of Pipelines that interact with it.
  
  

Developer Portal

Introducing an API Catalog to which API Developers can publish API Proxies. From the API Manager, you can access the Developer Portal, a new UI that requires a unique set of credentials.

The Developer Portal includes the following functionalities: 

• Allows non-SnapLogic users to search and explore APIs created in SnapLogic.
• Enables consumers to consume APIs.
  

Known Issue

• Certain output of JSON Parser Snap may display truncated or inaccurate data. For example, a numerical output value 0.19331011717644054 displaying as 0.19331011717644053.

Dot Releases in 4.18 - UI

The following 4.17 UI Dot Release updates are now a part of the 4.18 release:

Updated Snap Packs

Adobe Cloud Platform Snap Pack

• Enhances the Snap Pack to support the Map datatype in XSD-based datasets. 

Binary Snap Pack

• Adds a new account type, Google Service Account, which enables you to isolate workflows from individual users. This support applies only to the Google Storage file protocol.

Box Snap Pack

• Adds a new account type, Box Service Account, which enables you to create user accounts that can only be accessed programmatically through the APIs.
• Adds new properties, Number of retries and Retry interval (seconds), in the Box Write Snap to handle failed write attempts.

Hadoop Snap Pack

• Adds ADLS Gen2 support for ABFS (Azure Blob File System) and ABFSS protocols to the HDFS ZipFile Reader, HDFS ZipFile Writer, HDFS Reader, ORC Reader, Parquet Reader, and Hadoop Directory Browser Snaps. This support enables overriding account name and endpoint details in the Account Settings fields with those that are provided in the URL.

MongoDB Snap Pack

• Mongo DB Find: Adds the following new fields:
  • Projection Condition: Specifies which fields must be returned in the matching documents.
  • Sort Condition: Specifies the sort order for documents in the result set.
  • Offset: Specifies the starting document for the FIND query.
  • Limit: Specifies the number of documents the FIND query must return.
  • Group result: Groups the find query results in one field named result, instead of an array.
• MongoDB Group: Adds a new Snap that enables you to group input documents by a specified expression, and output to the next stage, one document for each distinct grouping.
• Mongo DB Update: Adds two new fields, Update operation and Exclude list, which enable you to update operations and exclude a list of JSON properties before sending the updated documents to MongoDB.

JDBC Snap Pack

• Adds support for JDBC account Active Directory authentication for SQL server for driver JAR version mssql-jdbc-6.2.2-jre8.jar.

MySQL Snap Pack

• Reverts a 4.17-patch (db/mysql7395) update regarding how BIT and BOOLEAN column data type values are displayed. They now display as true or false (4.17 GA behavior now in snapsmrc523) rather than converting to 0 or 1 values (4.17-patch behavior).

Script Snap Pack

• PySpark: Enhances to work in Kerberized Hadoop clusters, with the addition of a Kerberos account.

Snowflake Snap Pack

• Multi Execute: Adds a property, Use Result Query, which enables you to write results to an output view.

SQL Server Snap Pack

• Adds support for SQL Server account Active Directory authentication for SQL server for driver JAR version mssql-jdbc-6.2.2-jre8.jar.
• Reverts a 4.17-patch (db/sqlserver7394) update regarding how BIT and BOOLEAN column data type values are displayed. They now display as true or false (4.17 GA behavior now in snapsmrc523) rather than converting to 0 or 1 values (4.17-patch behavior).

Transform Snap Pack

• Sort: Enhances the sort feature to support specifying sort order at the field level. Adds two new fields, Sort Path and Sort Order, to the Sort paths property; renames the Sort order property to Sort order (Global).
• Group by N: Enhances the Group Size property to be expression enabled and removes the upper bound threshold (of 10000).

Metadata Snap Pack

• Enhances all the Snaps in this Snap Pack with the capability to create, delete, and update Snaplex directly inside the Pipeline.
  This feature facilitates administrative tasks if you are running your Pipelines on Groundplex. You can now create Snaplex programmatically by doing the following:
  
  1. Building a Pipeline with a Create Snap where you specify Snaplex as the Asset.
  2. Creating a Triggered Task in which the URL invokes the Pipeline.
  3. Sharing the URL with users in your Org so that when they run the Pipeline, the Snaplex is automatically created.
  Furthermore, you can use the Update Snap to update the node or logging properties in existing Snaplex.

Known Issues

• Pipelines configured with an on-premises Microsoft Dynamics CRM Account fails with Java 11 displaying this error – com.sun.xml.internal.messaging.saaj.soap.SAAJMetaFactoryImpl. Hence, you need to remain on Java 8 on the associated Snaplex for the said Pipelines to execute.
• A NullPointerException issue in the AWS SDK impacts the Binary, Redshift, Hadoop, Snowflake, and DynamoDB Snap Packs. Users leveraging AWS functionality and whose Snaplex has either a system configuration that uses an unauthenticated HTTP/HTTP proxy, or who have the same defined via Environment Variables are impacted. If a proxy was not used, or a proxy username and password was used, then customers are not affected. This issue was resolved in the 4.20 release.
• Preview output for Parquet Parser Snap does not match in Standard mode vis-a-vis the eXtreme mode, which displays the correct preview.
• File Writer and ZipFile Write Snaps do not create an output file when using the input from SAS Generator Snap configured with only the DELETE SAS permission. This is not the case when the target file exists.

Dot Releases in 4.18 - Snap

The following 4.17 Snap Dot Release updates are now a part of the 4.18 release:

General

• eXtremeplex: Adds the ability to configure an eXtremeplex on Azure Databricks. You can now run your Spark jobs in eXtreme on Azure Databricks clusters in addition to AWS EMR clusters.
• Parquet Formatter Snap: Adds partition support. You can now partition the input tabular data by specifying the target partitioning keys.

New Snaps

• Redshift Select: Extracts data from your Redshift Databases. You can now build eXtreme Pipelines that query and pull data from your Redshift databases.
• Redshift Insert: Inserts data in your Redshift Databases. You can now build eXtreme Pipelines that write data to your Redshift databases.

Known Issue

• After an eXtremeplex terminates on Azure Databricks, the terminate event does not display on the Dashboard.

This section tracks the changes made during the iterative pushes to the UAT server and the GA release. The expected schedule is:

• UAT #1: Friday, July 26, 2019 (Release updates are published above)
• UAT #2: Friday, August 2, 2019
• GA: Saturday, August 10, 2019 (9 pm PT)

UAT #2 Updates

UI

• Fixed an issue that allows you to create duplicate Proxy assets.
• Fixed an issue that prevents API consumers from searching for web applications on the Access and OAuth tabs of the API Manager.
• Fixed an issue where default value for SSO option is missing.
• Fixed an issue where Data Lineage display is not clearing previous results upon subsequent inquiries.
• Fixed an issue where clicking on Replace Current Pipe and Suggest Pipeline in Pipeline Recommendations results in a script error.
• Fixed an issue where the Pipeline Execution Logs for Azure Databricks does not display any information.
• Fixed an issue where the Developer Portal page does not work on Mozilla FireFox.
• Fixed an issue where a very long Proxy name breaks the Developer Portal landing page.
• Fixed an issue where disabling an Ultra Pipeline results in an empty error message.
• Fixed an issue where, if you cancel the deletion of a Pipeline in SnapLogic Designer, the pop-up closes, but the UI displays the busy icon interminably.
• Fixed an issue where, if you log in as an Admin viewing an Org of which you are not a member, the SnapLogic Dashboard does not display Snaplex instances correctly.
• Fixed an issue with Pipeline Recommendations where clicking Replace Current Pipe then deleting a Snap results in an error, and the Pipeline is not saved.

Platform

• Added support for DerbyDB when running with JRE 11.
• Fixed an issue with SSO redirect login.
• Fixed an issue with schema caching when running with older versions of Snaplex.
• Fixed multiple issues with Developer Portal related APIs.

Snaps

• Fixed an issue with ORC Writer, Parquet Writer, HDFS Writer, and HDFS Zip File Writer Snaps wherein the Snaps throw an error and there is no output file for double slash after the container in the directory path.
• Fixed an issue with the HDFS Reader Snap where the Snap fails after rerouting error to the error view.
• Fixed an issue with the eXtreme Redshift Snaps pointing to incorrect documentation.
• Upgraded JRuby version to 9.1.17.0 that enables Script Snap to work on JDK 11 Snaplex.
• Improved the error message in Hadoop Snap Pack in the case of an incorrect dirpath format for ABFS protocol.

GA Updates

UI

• Fixed an issue where adding a Snap on the canvas when the Suggested Pipeline panel is open results in a script error. 
• Fixed an issue where adding a Snap in a replaced suggested Pipeline and then saving it results in a script error.
• Fixed an issue where Snaps inserted in the middle of suggested Synthesis Pipelines does not have input/output views. 
• Fixed an issue where clicking a Pipeline in Suggested Pipelines panel displays an incorrect message.

Big Data

• Fixed an issue with cluster monitoring in the eXtremeplex configuration.

Platform

Update Date	Build	Updates
11 Oct 2019	master-6135 - 4.18 Patch 4	Fixed an issue that causes the parent Pipeline to not include the correct error message when its child Pipeline having a Pipeline Execute Snap fails.
06 Sep 2019	master-5912 - 4.18 Patch 3 (Recommended)	Fixed an issue with error handling when the child pipeline with Pipeline Execute Snap fails. Fixed an error when loading error Pipelines that use dynamic expressions. Fixed an issue with Pipeline execution stalling when child Pipeline uses binary views. Data lineage information is now throttled to prevent growth in runtime data size that may cause some Pipelines to fail. Optimized loading metadata from a control plane by using compression.

UI

Update Date	Build	Updates
03 Sep 2019	N/A	Fixed an issue where Data Preview does not work in eXtreme Pipelines. Fixed an issue where Snap Suggestions do not appear when you open a second tab or subsequently on the initial tab displaying the Snap Suggestions. Fixed an issue where clicking a Project folder named Alerts misdirects you to the Alerts page. Fixed usability issues with Pipeline Recommendations. Reverted a fix where certain output data with JSON Parser Snap is inaccurate, fixing which results in a regression issue in Data Preview.  Updated the UI content and buttons for Suggested Pipelines/Snaps panel in the Designer's canvas to improve usability.
22 Aug 2019	N/A	Fixed an issue where you could only view the preview data after Pipeline validation completes.  Fixed an issue where editing Snap settings in the Pipeline Execute Snap results in the following error message:  Type error: Cannot set property ‘dialed’ of undefined. Fixed an issue where File Previewer does not refetch the full HTML report. Enhanced Pipeline Recommendations by enabling a wizard that allows users to populate the Pattern tab with recommended Pipelines from the Patterns Catalog. Fixed an issue with Snap recommendations where adding a Snap in a replaced suggested Pipeline and saving it results in an error. Fixed an issue where selecting a recommended Pipeline through Pipeline Synthesis results in the middle snaps of the recommended synthesis Pipeline not having any input or output views. Fixed an error where adding a Snap on the canvas when the Suggested Pipeline panel is open results in an error.

Snaps

Snap Pack	Update Date	Build	Updates
Binary	18 Mar 2020	binary8768	Fixes the File Reader, File Writer, and Directory Browser Snaps to resolve 0x103 (STATUS_PENDING) and 0xc000022d (STATUS_RETRY) errors for SMB file operations.
Transform	07 Nov 2019	transform8199	Fixes an issue with the Excel Multi Sheet Formatter Snap wherein the Snap fails to create sheets in the expected order.
Confluent Kafka	23 Oct 2019	confluentkafka8111	Fixes an issue with the Confluent Kafka Producer Snap wherein the Topic property doesn't evaluate expression against input documents.
Salesforce	16 Oct 2019	salesforce8066	Fixes an issue with the Salesforce SOQL Snap wherein the Snap fails with NullPointerException.
Hadoop	15 Oct 2019	hadoop8033	Fixes an issue with the Parquet Writer Snap wherein the Snap throws an error when working with WASB protocol.
RabbitMQ	15 Oct 2019	rabbitmq8058	Fixes an issue with RabbitMQ Snaps wherein the Snaps throw an error when the auto-delete setting is set to true.
Snowflake	14 Oct 2019	db/snowflake8044	Fixes an issue with the Snowflake - Select Snap wherein the Snap converts the Snowflake-provided timestamp value to the local timezone of the account.
MS Dynamics 365 for Sales	14 Oct 2019	dynamics365forsales7957	Fixes a NullPointerException issue with the Dynamics 365 for Sales Delete Snap wherein the Object ID is invalid. Fixes a concurrency issue with multiple Dynamics 365 for Sales Snaps when running in parallel.
Binary	14 Oct 2019	binary8046	Enhances the Snap Pack to support AWS SDK 1.11.634 to fix the NullPointerException issue in the AWS SDK. This issue occurred in AWS-related Snaps that had HTTP or HTTPS proxy configured without a username and/or password.
Snowflake		db/snowflake8044
Redshift		db/redshift8043
Confluent Kafka	10 Oct 2019	confluentkafka8006	Fixes an exception that occurs when consuming log-compacted topics, in the Confluent Kafka Consumer Snap.
PostgreSQL	10 Oct 2019	postgres8021	Fixes an issue with the PostgreSQL grammar to better handle the single quote characters.
Binary	09 Oct 2019	MULTIPLE7884	Fixes an issue with the PostgreSQL grammar to better handle the single quote characters.
Snowflake
Redshift
Transform	07 Oct 2019	transform7994	Adds a field Round dates to the Excel Parser Snap, which enables you to round numeric excel data values to the closest second.
SQL Server	07 Oct 2019	db/sqlserver7989	Fixes an issue in the SQL Server Merge Snap wherein identity columns are not handled correctly. Also, adds a field Enable Identity Insert that enables you to insert values into an identity column.
MS Dynamics 365 for Sales	01 Oct 2019	dynamics365forsales7952	Adds support for lookup type in the input document to be formatted per MS Dynamics 365 for Sales syntax.
JMS	01 Oct 2019	jms7865	Fixes an issue with the JMS Consumer Snap that reinserts failed messages in the JMS queue without checking for duplicate messages.
Binary	25 Sep 2019	binary7860	Fixes an issue with the File Writer Snap that is intermittently unable to set the correct permissions.
NetSuite	23 Sep 2019	netsuite7879	Fixes an issue with the NetSuite Snaps that intermittently fail to reach, cache, and view the NetSuite WSDL and XSD files.
MySQL	23 Sep 2019	db/mysql7846	Fixes an issue with the MySQL Snaps that exhibit degraded performance with snapsmrc523.
Splunk	20 Sep 2019	splunk7812	Adds properties named Earliest Relative, Latest Relative, and Preset Relative Search to the Splunk Search Snap to fix an issue wherein the Snap returns inaccurate and inconsistent results regarding last-30-day and year-to-date searches.
Snowflake	16 Sep 2019	db/snowflake7821	Fixes an issue with the Snowflake - Execute Snap wherein the Snap is unable to support the '$' character in query syntax.
Binary	16 Sep 2019	binary7851	Introduces a field Advanced properties in the File Reader, File Writer, File Delete, ZipFile Read, ZipFile Write, Directory Browser, Multi File Reader, File Poller, and File Operation Snaps. Use this field to configure the SAS URI for an access to Azure Storage. The supported SAS types are Service SAS on container, Service SAS on blob, and Account SAS. Introduces a Snap SAS Generator. This Snap enables you to generate Shared Access Storage (SAS) URIs for accessing Azure Storage containers and blobs.
Workday	11 Sep 2019	workday7837	Fixes an issue with the Workday Read Snap wherein the Snap is unable to log SOAP calls.
Salesforce	11 Sep 2019	salesforce7832	Enhances the Salesforce Subscriber Snap to capture Change Data Capture (CDC) events.
Flow	09 Sep 2019	flow7743	Adds a property Validate full JSON path to the Data Validator Snap to fix an issue wherein a field validation failure against a constraint does not produce an error.
Confluent Kafka	30 Aug 2019	confluentkafka7784	Fixes the Consumer Snap to stop quickly and cleanly on CLOSE and STOP lifecycle events.
Oracle	30 Aug 2019	db/oracle7786	Fixes an issue by using the NVARCHAR2 datatype in Oracle databases when the character set is not AL32UTF8. However, a known issue is that special characters appear as .
NetSuite	29 Aug 2019	netsuite7779	Fixes an issue with the NetSuite Update Snap and the NetSuite Create Snap wherein the Snaps are unable to properly handle null values for the standard and the custom fields.
Transform	29 Aug 2019	transform7780	Fixes an issue with the Excel Parser Snap by upgrading Apache POI to version 3.14 wherein the Snap is unable to parse an excel file with custom namespaces. Fixes an issue with the Excel Parser Snap wherein the Snap converts real numbers to two decimal places when formatted for currency. A new property Cell formatting now supports unformatted outputs.
Binary	28 Aug 2019	MULTIPLE7778	Updates the AWS SDK library version to default to the Signature Version 4 Signing process for API requests across regions.
DynamoDB
Redshift
Snowflake
SQS
Transform	26 Aug 2019	transform7741	Fixes an issue with the Sort and Join Snaps wherein the platform removes all temp files at the end of Pipeline execution.
Binary	21 Aug 2019	binary7746	Fixes an issue with the SMB file protocol wherein Binary Snaps are unable to create a native thread using an SMB file path.
Snowflake	20 Aug 2019	db/snowflake7739	Fixes an issue with the Snowflake - Bulk Upsert Snap wherein the Snap fails when using a pipeline parameter in Key columns. Fixes an issue with the Snowflake - Unload Snap wherein the Snap does not abort the query when you stop the Pipeline execution.
Snowflake	20 Aug 2019	db/snowflake7727	Fixes an issue with the Snowflake - Unload Snap wherein the Snap does not abort the query when the Pipeline is stopped.
Transform	19 Aug 2019	transform7711	Fixes an issue with the XML Parser Snap wherein a class cast exception occurs when the Snap is configured with a Splitter and Namespace Context.
Confluent Kafka	16 Aug 2019	confluentkafka7732	Adds support for Kerberos-based authentication to enable connecting to Kerberos-enabled Kafka servers.
Spark SQL	15 Aug 2019	sparksql7718	Fixes an issue with the Spark Pipeline wherein the Pipeline on Hadooplex fails on release 4.18 because of dependency change in the global pom file.
Binary	15 Aug 2019	binary7720	Fixes an issue with the ZipFile Read Snap wherein the Snap fails to extract files from a zip file larger than 100MB when the number of retries is greater than zero.

eXtreme