March 2023 Release Notes

Highlights

The SnapLogic® March 2023 Release includes the following:

AutoSync

  • New release available in Studio

API Management

  • SQL Threat Detection API policy

  • Support for generating an OAS 3.0 API specification

  • YAML support extended to Developer Portal

Platform

  • Git Integration supports Azure Repos.

  • New Public APIs to do the following:

    • (API Management) Create an API version from an existing project.

    • Enable/Disable a task.

    • Update the owner of a task.

  • Enable or disable task operations added to the Task Details page.

Snaps

  • Microsoft Dynamics 365 Finance and SCM Snap Pack

    • Dynamics 365 Finance and SCM Read

    • Dynamics 365 Finance and SCM Search

Studio

  • Hover action over a Metrics chart tracks across all charts

 

Dot Releases - Platform

Date of Update

Component

Updates

Date of Update

Component

Updates

Mar 29, 2023

Snaplex build: main-16624 - 4.32 Patch 4

Launched AutoSync in Studio. You can view the documentation here.

Mar 17, 2023

Snaplex build: main-16442 - 4.32 Patch 3

Fixed an issue where an Ultra pipeline task fails to start for a pipeline that has no input views and multiple output views.

Mar 10, 2023

Snaplex build: main-16360 - 4.32 Patch 2

Fixed an issue where FeedMaster nodes would not start up after 4.32 GA upgrade under certain conditions.

Dot Releases - Snaps

Snap Pack

Date of Update

Snap Pack Version

Updates

Snap Pack

Date of Update

Snap Pack Version

Updates

Amazon SNS

Apr 12, 2023

432patches20327

The Amazon SNS Account now includes an Enable large message support checkbox to support publishing messages larger than 256 KB. When you select this checkbox, the Snap displays S3 related properties that enables you to store the messages in the S3 bucket.

JMS

Apr 12, 2023

432patches20479

Fixed an issue where the JMS Producer Snap was previously only writing headers to the WebLogic queue. Now the Snap writes the entire document.

Azure Synapse SQL

Apr 12, 2023

432patches20389

The Azure Synapse SQL Account and Azure Synapse SQL Active Directory Account now support the latest JDBC driver by default (version 12.2.0.jre11). You do not have to upload the JAR file in the JDBC Driver field manually.

Transform

Apr 11, 2023

432patches20446

The Join Snap is enhanced with the following:

  • The Pipeline Execution Statistics of the Join Snap now has a status message that displays the parameters - Free disk space, Available memory, and Average document size.

  • The internal sort buffer size is reduced to a minimum of 10MB when the available memory in the node becomes lower than 500MB to avoid the out-of-memory crash.

  • The internal sort buffer size is restored to its original size when the available memory becomes larger than 2GB.

  • We have improved the readability of the error message for the out of disk space on node error. The updated error message now provides clearer information and guidance for users, as shown below:
    Reason: Insufficient free disk space available to stage sort data into temporary files.
    Resolution:  Increase the amount of free disk space and try again.

Binary

Apr 11, 2023

432patches20431

  • Fixed an issue where the File Writer Snap would not retry on completing the writing of the file.

  • Added a configuration warning message when using the WASBS protocol with AzCopy, as it supports only the HTTPS protocol.

Salesforce

Apr 7, 2023

432patches20393

Amazon S3

Apr 5, 2023

432patches20385

The Amazon S3 Snap Pack now supports Ultra Pipelines.

PostgreSQL

Apr 5, 2023

432patches20409

The PostgreSQL - Bulk Load and PostgreSQL - Insert Snaps no longer fail with the message ERROR: type modifier is not allowed for type 'bytea' when creating a new table if Create table if not present is selected and the target table does not exist. This issue occurred when metadata from the second input view document contained columns of the bytea data type.

OpenAPI

Apr 5, 2023

432patches20357

OpenAPI Snaps now support reference components with . in the name of the component.

Snowflake

Apr 3, 2023

432patches20266

Fixed an issue with the Snowflake - Bulk Load Snap that resulted in lowercase (or mixed case) column names when creating a new table under specific conditions. The new Create table with uppercase column names checkbox addresses this issue.

Hadoop

Mar 31, 2023

432patches20209

The Apache Commons Compress library is upgraded to version 1.22.

Azure SQL

Mar 30, 2023

432patches20318

The Azure SQL - Bulk Extract Snap no longer fails with a java.lang.NumberFormatException error.

Binary

Mar 30, 2023

432patches20349

The JSCH library is upgraded to version 0.2.7.

Email

Mar 30, 2023

432patches20237

The JSCH library is upgraded to version 0.2.7.

Flow

Mar 29, 2023

432patches20279

  • The Pipeline Execute Snap now returns errors from child pipelines with a unique identifier.

  • Fixed an issue with the Pipeline Execute Snap that was causing Unable to create a deep copy of the object errors.

Google BigQuery

Mar 28, 2023

432patches20298

Transform

Mar 28, 2023

432patches20250

  • Fixed an issue with the JSON Splitter Snap that was causing errors when using multiple repeated dots in the JSON Path.

  • The Sort Snap includes the following improvements:

    • The Maximum memory % field is revised to Maximum memory.

    • The Maximum memory unit (new dropdown list) enables you to choose a unit, percentage (%), or MB for better memory control.

Workday

Mar 28, 2023

432patches20313

Fixed an issue with the Workday REST Snap where it failed with a null pointer exception when the input document was null.

Salesforce

Mar 28, 2023

432patches20145

Fixed an issue with the Salesforce Read Snap that was causing an error for the Time data type.

Redshift

Mar 28, 2023

432patches20166

Updated the description for S3 Security Token field as follows:

Specify the S3 security token part of AWS Security Token Service (STS) authentication. It is not required unless a particular S3 credential is configured to require it.

Azure SQL Snap Pack

Mar 28, 2023

432patches20219

Fixed an issue with the Azure SQL - Bulk Load Snap involving special characters in JDBC URL properties, such as passwords. The Snap escapes the special characters now.

Dynamics 365 for Sales

Mar 16, 2023

432patches20134

  • Fixed an issue with the Dynamics 365 for Sales Delete Snap that caused an Index Out of Bounds error when the Batch size was more than 1.

  • Fixed the stalled thread issue in the Dynamics 365 for Sales Upsert Snap—the Snap now works as expected and processes all the documents without getting stuck.

Kafka

Mar 10, 2023

432patches20143

Fixed an issue with the Kafka Producer Snap to avoid an error about delivery.timeout.ms. The Snap now automatically adjusts the value of the Message Publish Timeout setting and displays a warning in the Pipeline Execution Statistics.

Jira

Mar 10, 2023

432patches20123

Fixed an issue with the Jira Create Snap that caused custom attributes to not display in the target schema of Mapper Snaps.

Transform

Mar 10, 2023

432patches20151

Fixed an issue that occurred with the JSON Splitter Snap when used in an Ultra pipeline. The request was acknowledged before it was processed by the downstream Snaps, which caused a 400 Bad Request response.

AutoSync

As of March 29, 2023, a new AutoSync release is available in Studio. No changes were made to AutoSync V1.0 for the March release. Customers who purchased or are evaluating AutoSync V1.0 will still have access to it until they migrate their data pipelines to the new version. To evaluate the new AutoSync, contact your CSM.

This is what you need to know:

  • The AutoSync Studio release has a new user-friendly interface:

  • This release supports the synchronization of data from Salesforce to Snowflake. More endpoints are coming soon.

  • Both the new release and AutoSync V1.0 are available simultaneously for Orgs where they are enabled:

    • Access V1.0 from the pulldown menu on the IIP Designer tab:

    • Access the new AutoSync from the Studio navigation menu:

  • Org admins who enable the new AutoSync for new users must also give them access to the Intelligent Integration Platform (IIP).

  • SnapLogic provisions Snaplexes dedicated to AutoSync. For V1.0, the Snaplex name is AutoSyncPlex and for AutoSync in Studio, the Snaplex name is AutoSync. Only run AutoSync data pipelines, not pipelines created in the IIP, on these Snaplexes.

  • Orgs with AutoSync Snaplexes provisioned before March 29, 2023 need to upgrade to main-16624 - 4.32 Patch 4.

Learn more about the new AutoSync available in Studio.

Limitations for AutoSync in Studio

  • On some screens, including laptops, browsers with enhanced zoom, and screens using a non-standard resolution:

    • The text on AutoSync dashboard cards might overlap.

    • In the Studio Execution overview, all five status summary cards might not display. If this occurs reduce the zoom percentage to view them all:

  • In the Studio Execution overview, if you click a summary card to filter the execution list, child data pipeline executions do not show the associated parent. To find the parent, click on the execution. The last component of the Path is the parent pipeline name. For example, mydatapipeline:

    Learn more about the Execution overview.

  • If Auto synchronize is disabled for a data pipeline, you cannot enable it from the data pipeline details panel:

    Although the toggle appears to be on, the Next run value remains blank in the panel. Instead, use the Auto synchronize toggle on the card to enable synchronization.

  • When you edit a data pipeline, the screen with Sources and Destinations to choose from might be blank. Refresh the page to continue. Learn how to edit a data pipeline.

  • The AutoSync dashboard auto-refreshes periodically. However, it does not correctly update the Next run value on data pipeline cards.

New data pipeline wizard

  • A data pipeline unexpectedly runs twice if the synchronization schedule start time is in the past and you click Save and run at the end of the wizard.
    To avoid running it twice, either make sure the schedule time is in the future, or click Save instead of Save and run.

  • Changing the data pipeline name at the last step of the wizard causes an error. To avoid the error, change the name before the last step or edit it later.

Credentials and accounts

  • In IIP Manager, Org admins can add Accounts to a user group so that AutoSync users can share them. The dropdown list contains Account types that are not supported by AutoSync. For example, in the following screen, you should only select Salesforce or Snowflake, because those endpoints are supported in this release:

     

  • For Accounts created in the IIP, AutoSync only supports the following Account types. In AutoSync, only the supported types display in the dropdown.

    • Snowflake S3 Database and Snowflake S3 OAuth2

    • Salesforce.com and Salesforce OAuth2 Account types

  • AutoSync does not validate that an Amazon S3 account for staging data to Snowflake has write permission. Use an Amazon S3 account with write permission or the data pipeline will fail.

Include deleted records checkbox for Salesforce

AutoSync might incorrectly report new records on the data pipeline card and in the details panel. This can happen when:

  1. A data pipeline is edited to enable tracking of deleted records in Salesforce, with Include deleted records selected.

  2. Records were recently deleted from Salesforce before the data pipeline was edited.

On the next run, AutoSync might report the deleted records as if they were new records added to Snowflake. However, in the target table, these records are correctly marked as deleted. Learn more about how AutoSync tracks deleted records for Salesforce.

API Management

New Features

  • SQL Threat Detection API policy. To protect APIs against SQL commands injected in the header, path, query, and payload of incoming HTTP requests, apply this API policy.

  • Support for the OAS 3.0 specification. You can generate a specification in the Version Details tab and choose the JSON or YAML version in addition to the OAS version.

Enhancements

  • YAML support is now added to the following workflows:

  • Your Developer Portal users can download a specification in either JSON or YAML format when they explore an API.

  • The Developer Portal features a redesigned navigation menu.

  • In the Subscription Manager > User Access tab, the table for tracking the users of your Developer Portal now includes a column for user type. Portal indicates Developer Portal users who are not members of the SnapLogic Org. Org indicates users who are already members of the Org. When you add a new user to the Developer Portal, you receive a message if the user already exists.

Fixed Issues

You can now permanently delete APIs and API versions from the Recycle Bin in Manager. Previously, deleting these assets from the Recycle Bin resulted in an error.

Behavior Changes

All policy violation error messages are returned in JSON format for the following API policies:

Before this release, some policies returned policy violation error messages in plain text.

Platform

Ultra enhancement

Starting with the February release, we introduced an Ultra enhancement that will significantly speed up the creation, restart, and disabling of Ultra Tasks by reducing scheduling times. The Ultra Task is decentralized by moving it out of the Control Plane and into the production Snaplexes. Extensive internal and private beta testing for a few months shows startup times reduced from several minutes down to several seconds.

This change automatically applies when you upgrade your Org Snaplex to the 4.32 GA. This enhancement will have little to no impact on the resource use of Snaplex nodes.

New Features

Enhancements

  • Sensitive Account information is protected when you change an encrypted field to use an expression.

  • Label changes for the Copy Project Assets dialog correctly describe fields for the Target Org and Target Project or Folder:

     

  • Enable and disable Pipeline Validation at the project level when you create or edit a project. Previously, this was an Org-level setting.

  • The Task Details page allows you to enable/disable a task through a toggle switch. Previously, you could only disable or enable a task in the assets table in Manager.

  • The Project Import wizard now supports duplicate asset handling. When you upload a project ZIP with assets containing the same name as existing assets, you can choose to duplicate the asset, make a copy of the project, or select which assets to overwrite.

  • The Public API to import a project is updated with a new parameter to allow you to specify which asset types can be overwritten.

  • In line with the support for Git, the new icon for Git operations replaces the icon displayed prior to the March 2023 release.

Fixed Issues

  • Fixed an issue where the Insights tab View By Snaplex field displayed the Snaplex environment instead of the Snaplex name.

Snaps

The Snaps mentioned in this section are deployed on the Latest distribution type. To access the updated Snap Packs, your Org admin will need to select the mentioned Snap Pack version in Manager > Snap Packs.

New Snaps

Introduced the Microsoft Dynamics 365 Finance and SCM Snap Pack, which contains the following Snaps and accounts (432patches20035):

Enhancements

  • The Azure Active Directory OAuth2 account supports selecting a Grant type, which enables you to authorize your account with an authorization code or client credentials. (432patches20059)

  • Added support for SQS accounts to reauthenticate when authentication expires and the IAM Role checkbox is selected. (432patches19990)

  • The Kerberos Account that is available for a subset of Snaps in the Hadoop Snap Pack now supports a configuration that enables you to read from and write to the Hadoop Distributed File System (HDFS) managed by multiple Hadoop clusters. You can specify the location of the Hadoop configuration files in the Hadoop config directory field. The value in this field overrides the value that is set on the Snaplex system property for configuring a single cluster. (432patches20139)

  • Redshift

    • The JDBC driver class for Redshift accounts is bundled with the com.amazon.redshift.jdbc42.Driver as the default driver. This upgrade is backward compatible. The existing pipelines will continue to work as expected and the new pipelines will use the Redshift Driver as the default driver. SnapLogic will support providing fixes for the issues you might encounter with accounts that use the PostgreSQL driver only until November 2023.
      After November 2023, SnapLogic will not provide support for the issues with the PostgreSQL driver. Therefore, we recommend migrating from the PostgreSQL JDBC driver to the Redshift JDBC driver. Learn more about migrating from the PostgreSQL JDBC Driver to the Amazon Redshift Driver. (432patches20101)

    • The Instance type option in the Redshift Bulk Load Snap enables you to use the Amazon EC2 R6a instance. This property appears only when the parallelism value is greater than one. (432patches20101)

    • The Redshift Snaps that earlier supported only the Redshift Cluster now also support Redshift Serverless. With Redshift Serverless, you can avoid setting up and managing data warehouse infrastructure when you run or scale analytics (432patches20035).

  • The REST AWS SigV4 Account (REST Snap Pack) and the AWS Signature V4 Account (API Suite Snap Pack) now support DNS canonical names for S3 buckets. (432patches20054)

  • The Snowflake Bulk Load, Bulk Upsert, and Unload Snaps now support expressions for the Staging location field. (432patches20120)

  • The Salesforce Create, Update, and Upsert Snaps now include an Apply active assignment rules checkbox so you can control whether to use assignment rules for the selected object. (432patches20087)

  • The below Exchange Online patch got re-deployed as stable.
    The following permissions are no longer mandatory for the Exchange Online Snaps to work:

    You can choose to NOT assign these permissions in your Snap accounts. (432patches20035)

Fixed Issues

Intermittent connectivity issues no longer occur when using some Snaps in the Azure SQL Snap Pack. These issues caused the following message to display: The connection is broken and recovery is not possible. The connection is marked by the client driver as unrecoverable. No attempt was made to restore the connection. (432patches20049)

Behavior Changes

  • In the 4.27 (427patches13615) release and later, pipelines that truncated (or deleted and re-created) a table and then used the BigQuery Write or BigQuery Bulk Load (Streaming) Snaps were allowed to retry the load operation. This lead to potential data loss because Google BigQuery uses an eventually consistent backend where one server might respond that it’s okay to write, but other servers still truncate the table.
    In the 4.32 (432patches20298) release and later, we disabled retries on truncated tables in these two Snaps to prevent a potential loss of data. We recommend that you use the Google BigQuery Bulk Load (Cloud Storage) Snap instead.

  • Fixed an issue with the JSON Splitter Snap for some use cases where it was not backward compatible with the 4.31 GA version. These cases involved specific uses of either the Include scalar parents feature or the Include Paths feature.

  • In the Redshift Select Snap, when you configure Output fields and deselect the Match data types checkbox, the display of the label name for the timestamptz data type in the output preview varies with the Redshift JDBC and the PostgreSQL JDBC drivers. For the Redshift JDBC driver, the Snap prefixes Redshift to the Timestamp label. For the PostgreSQL JDBC driver, Snap displays the labels as configured in the Snap settings. This does not impact the performance of the Snap.
    Note: The behavior of the Snap remains the same when you select the Match data types checkbox, regardless of using the PostgreSQL or the Redshift driver—the label names are displayed as configured in the Snap settings.

Studio

Enhancements

  • The new ability to visually correlate metrics assists when you monitor and troubleshoot pipeline execution and system behavior. For example, to determine why memory use spiked, you might want to check the number of pipelines that were running at the same time. Do that on the Studio Metrics page by hovering over a chart to view data point values. The value indicator tracks the same point in time for all charts:


Documentation Enhancements

New

Enhancements

The following application configuration pages for the Coupa and Azure Active Directory Snap Packs are enhanced with the key steps, workflow, scenarios, and frequently asked questions, as applicable: