Highlights

The SnapLogic® March 2023 Release includes the following:

Dot Releases - Platform

Dot Releases - Snaps

AutoSync

As of March 29, 2023, a new AutoSync release is available in Studio. No changes were made to AutoSync V1.0 for the March release. Customers who purchased or are evaluating AutoSync V1.0 will still have access to it until they migrate their data pipelines to the new version. To evaluate the new AutoSync, contact your CSM.

This is what you need to know:

• The AutoSync Studio release has a new user-friendly interface:

  Open

  

• This release supports the synchronization of data from Salesforce to Snowflake. More endpoints are coming soon.

• Both the new release and AutoSync V1.0 are available simultaneously for Orgs where they are enabled:

  • Access V1.0 from the pulldown menu on the IIP Designer tab:

    Open

    

  • Access the new AutoSync from the Studio navigation menu:

• Org admins who enable the new AutoSync for new users must also give them access to the Intelligent Integration Platform (IIP).

• SnapLogic provisions Snaplexes dedicated to AutoSync. For V1.0, the Snaplex name is AutoSyncPlex and for AutoSync in Studio, the Snaplex name is AutoSync. Only run AutoSync data pipelines, not pipelines created in the IIP, on these Snaplexes.

• Orgs with AutoSync Snaplexes provisioned before March 29, 2023 need to upgrade to main-16624 - 4.32 Patch 4.

Learn more about the new AutoSync available in Studio.

Limitations for AutoSync in Studio

• On some screens, including laptops, browsers with enhanced zoom, and screens using a non-standard resolution:

  • The text on AutoSync dashboard cards might overlap.

  • In the Studio Execution overview, all five status summary cards might not display. If this occurs reduce the zoom percentage to view them all:

• In the Studio Execution overview, if you click a summary card to filter the execution list, child data pipeline executions do not show the associated parent. To find the parent, click on the execution. The last component of the Path is the parent pipeline name. For example, mydatapipeline:

  Learn more about the Execution overview.

• If Auto synchronize is disabled for a data pipeline, you cannot enable it from the data pipeline details panel:

  Although the toggle appears to be on, the Next run value remains blank in the panel. Instead, use the Auto synchronize toggle on the card to enable synchronization.

• When you edit a data pipeline, the screen with Sources and Destinations to choose from might be blank. Refresh the page to continue. Learn how to edit a data pipeline.

• The AutoSync dashboard auto-refreshes periodically. However, it does not correctly update the Next run value on data pipeline cards.

New data pipeline wizard

• A data pipeline unexpectedly runs twice if the synchronization schedule start time is in the past and you click Save and run at the end of the wizard.
  To avoid running it twice, either make sure the schedule time is in the future, or click Save instead of Save and run.

• Changing the data pipeline name at the last step of the wizard causes an error. To avoid the error, change the name before the last step or edit it later.

Credentials and accounts

• In IIP Manager, Org admins can add Accounts to a user group so that AutoSync users can share them. The dropdown list contains Account types that are not supported by AutoSync. For example, in the following screen, you should only select Salesforce or Snowflake, because those endpoints are supported in this release:

   

• For Accounts created in the IIP, AutoSync only supports the following Account types. In AutoSync, only the supported types display in the dropdown.

  • Snowflake S3 Database and Snowflake S3 OAuth2

  • Salesforce.com and Salesforce OAuth2 Account types

• AutoSync does not validate that an Amazon S3 account for staging data to Snowflake has write permission. Use an Amazon S3 account with write permission or the data pipeline will fail.

Include deleted records checkbox for Salesforce

AutoSync might incorrectly report new records on the data pipeline card and in the details panel. This can happen when:

1. A data pipeline is edited to enable tracking of deleted records in Salesforce, with Include deleted records selected.

2. Records were recently deleted from Salesforce before the data pipeline was edited.

On the next run, AutoSync might report the deleted records as if they were new records added to Snowflake. However, in the target table, these records are correctly marked as deleted. Learn more about how AutoSync tracks deleted records for Salesforce.

API Management

New Features

• SQL Threat Detection API policy. To protect APIs against SQL commands injected in the header, path, query, and payload of incoming HTTP requests, apply this API policy.

• Support for the OAS 3.0 specification. You can generate a specification in the Version Details tab and choose the JSON or YAML version in addition to the OAS version.
  
  

Enhancements

• YAML support is now added to the following workflows:

  • You can now reference the API specification YAML file from an existing version when you create a new version of that API.

  • You can specify the YAML format for an API specification when publishing an API to the Developer Portal.

• Your Developer Portal users can download a specification in either JSON or YAML format when they explore an API.
  
  

• The Developer Portal features a redesigned navigation menu.

• In the Subscription Manager > User Access tab, the table for tracking the users of your Developer Portal now includes a column for user type. Portal indicates Developer Portal users who are not members of the SnapLogic Org. Org indicates users who are already members of the Org. When you add a new user to the Developer Portal, you receive a message if the user already exists.

Fixed Issues

You can now permanently delete APIs and API versions from the Recycle Bin in Manager. Previously, deleting these assets from the Recycle Bin resulted in an error.

Behavior Changes

All policy violation error messages are returned in JSON format for the following API policies:

• Generic OAuth2

  • Client Throttling

  • IP Restriction 

  • Request Size Limit

  • JSON Validator

  • XML XSD Validator

  • XML DTD Validator

Before this release, some policies returned policy violation error messages in plain text.

Platform

New Features

• Git Integration supports Azure Repos. Learn how to configure Azure Repos for Git Integration in SnapLogic.

• New Public APIs:

  • (API Management) Create an API version from an existing project

  • Enable or disable a task

  • Update the owner of a task

Enhancements

• Sensitive Account information is protected when you change an encrypted field to use an expression.

• Label changes for the Copy Project Assets dialog correctly describe fields for the Target Org and Target Project or Folder:
  
  

   

• Enable and disable Pipeline Validation at the project level when you create or edit a project. Previously, this was an Org-level setting.

• The Task Details page allows you to enable/disable a task through a toggle switch. Previously, you could only disable or enable a task in the assets table in Manager.
  
  

• The Project Import wizard now supports duplicate asset handling. When you upload a project ZIP with assets containing the same name as existing assets, you can choose to duplicate the asset, make a copy of the project, or select which assets to overwrite.

• The Public API to import a project is updated with a new parameter to allow you to specify which asset types can be overwritten.

• In line with the support for Git, the new icon for Git operations replaces the icon displayed prior to the March 2023 release.

Fixed Issues

• Fixed an issue where the Insights tab View By Snaplex field displayed the Snaplex environment instead of the Snaplex name.

Snaps

New Snaps

Introduced the Microsoft Dynamics 365 Finance and SCM Snap Pack, which contains the following Snaps and accounts (432patches20035):

• Dynamics 365 Finance and SCM Read: Enables you to read the data from the specified entity.

• Dynamics 365 Finance and SCM Search: Enables you to search for details in the specified entity.

• Finance and SCM Access Token Account: Enables you to use an access token generated externally to authorize the account.

• Finance and SCM OAuth2 Account: Enables you to generate an access token by authorizing the account to use the Dynamics 365 Finance and SCM Snaps.

Enhancements

• The Azure Active Directory OAuth2 account supports selecting a Grant type, which enables you to authorize your account with an authorization code or client credentials. (432patches20059)

• Added support for SQS accounts to reauthenticate when authentication expires and the IAM Role checkbox is selected. (432patches19990)

• The Kerberos Account that is available for a subset of Snaps in the Hadoop Snap Pack now supports a configuration that enables you to read from and write to the Hadoop Distributed File System (HDFS) managed by multiple Hadoop clusters. You can specify the location of the Hadoop configuration files in the Hadoop config directory field. The value in this field overrides the value that is set on the Snaplex system property for configuring a single cluster. (432patches20139)

• Redshift

  • The JDBC driver class for Redshift accounts is bundled with the com.amazon.redshift.jdbc42.Driver as the default driver. This upgrade is backward compatible. The existing pipelines will continue to work as expected and the new pipelines will use the Redshift Driver as the default driver. SnapLogic will support providing fixes for the issues you might encounter with accounts that use the PostgreSQL driver only until November 2023.
    After November 2023, SnapLogic will not provide support for the issues with the PostgreSQL driver. Therefore, we recommend migrating from the PostgreSQL JDBC driver to the Redshift JDBC driver. Learn more about migrating from the PostgreSQL JDBC Driver to the Amazon Redshift Driver. (432patches20101)

  • The Instance type option in the Redshift Bulk Load Snap enables you to use the Amazon EC2 R6a instance. This property appears only when the parallelism value is greater than one. (432patches20101)

  • The Redshift Snaps that earlier supported only the Redshift Cluster now also support Redshift Serverless. With Redshift Serverless, you can avoid setting up and managing data warehouse infrastructure when you run or scale analytics (432patches20035).

• The REST AWS SigV4 Account (REST Snap Pack) and the AWS Signature V4 Account (API Suite Snap Pack) now support DNS canonical names for S3 buckets. (432patches20054)

• The Snowflake Bulk Load, Bulk Upsert, and Unload Snaps now support expressions for the Staging location field. (432patches20120)

• The Salesforce Create, Update, and Upsert Snaps now include an Apply active assignment rules checkbox so you can control whether to use assignment rules for the selected object. (432patches20087)

• The below Exchange Online patch got re-deployed as stable.
  The following permissions are no longer mandatory for the Exchange Online Snaps to work:

  • User.ReadAll permission for the Exchange Online OAuth2 Application account.

  • User.Read or User.ReadAll permission for the Exchange Online OAuth2 User account.

  You can choose to NOT assign these permissions in your Snap accounts. (432patches20035)

Fixed Issues

Intermittent connectivity issues no longer occur when using some Snaps in the Azure SQL Snap Pack. These issues caused the following message to display: The connection is broken and recovery is not possible. The connection is marked by the client driver as unrecoverable. No attempt was made to restore the connection. (432patches20049)

Behavior Changes

• In the 4.27 (427patches13615) release and later, pipelines that truncated (or deleted and re-created) a table and then used the BigQuery Write or BigQuery Bulk Load (Streaming) Snaps were allowed to retry the load operation. This lead to potential data loss because Google BigQuery uses an eventually consistent backend where one server might respond that it’s okay to write, but other servers still truncate the table.
  In the 4.32 (432patches20298) release and later, we disabled retries on truncated tables in these two Snaps to prevent a potential loss of data. We recommend that you use the Google BigQuery Bulk Load (Cloud Storage) Snap instead.

• Fixed an issue with the JSON Splitter Snap for some use cases where it was not backward compatible with the 4.31 GA version. These cases involved specific uses of either the Include scalar parents feature or the Include Paths feature.

• In the Redshift Select Snap, when you configure Output fields and deselect the Match data types checkbox, the display of the label name for the timestamptz data type in the output preview varies with the Redshift JDBC and the PostgreSQL JDBC drivers. For the Redshift JDBC driver, the Snap prefixes Redshift to the Timestamp label. For the PostgreSQL JDBC driver, Snap displays the labels as configured in the Snap settings. This does not impact the performance of the Snap.
  Note: The behavior of the Snap remains the same when you select the Match data types checkbox, regardless of using the PostgreSQL or the Redshift driver—the label names are displayed as configured in the Snap settings.

Studio

Enhancements

• The new ability to visually correlate metrics assists when you monitor and troubleshoot pipeline execution and system behavior. For example, to determine why memory use spiked, you might want to check the number of pipelines that were running at the same time. Do that on the Studio Metrics page by hovering over a chart to view data point values. The value indicator tracks the same point in time for all charts:

Documentation Enhancements

New

• Visit the new docs.snaplogic.com site to learn about the following:

  • AutoSync in Studio

  • Public APIs

  • Studio

  • The Git Integration

Enhancements

The following application configuration pages for the Coupa and Azure Active Directory Snap Packs are enhanced with the key steps, workflow, scenarios, and frequently asked questions, as applicable:

• Configuration of Dynamic OAuth2 Token Account and Configuration of OAuth2 Account.

• Application Configuration in Azure Active Directory.

• Configuring an EC2 role for IAM Role in AWS S3 Account and Configuring Cross Account IAM Role Support.