May 2022, 4.29 Release Notes

In this article

Key New Features in SnapLogic

Studio:

  • Introduced runtime details for the Execution overview when you click on a Task or Pipeline execution.

  • Added support for search in the table of executions on the Execution overview.

  • Introduced new icons for Pipeline execution status that show executions with errors and those that were completed with warnings.

Platform:

  • Added the ability to view child Pipeline execution status from the parent Pipeline in Dashboard or the Studio Execution overview.

  • Added Flow as an asset type when migrating Projects and Project Spaces.

  • Improved Snaplex network logging.

  • Updated Snaplex Helm chart.

API Management:

  • Added the ability to publish a Proxy to the Developer Portal.

  • Added conditional mapping rules between a Proxy endpoint and an upstream API.

  • Added support for Open API Specification (OAS) 3.0 in creating and publishing an API. Generated specifications still use OAS 2.0.

  • Added the ability to enable the Try It Out feature for all APIs in the Developer Portal.

  • Added the ability to specify custom subscription key parameters.

  • Replaced Token authorization with Bearer authorization for new API requests in the Developer Portal.

Snaps:

  • Introducing new SAP S/4HANA Cloud Snap Pack.

  • Added OAuth2 Account support for Coupa.

  • Added on-premises support for Microsoft Dynamics 365 For Sales and a new Dynamics 365 NTML Account type for this Snap Pack.

  • Added REST In-memory OAuth2 Account type for REST Snap Pack.

Data Automation:

  • ELT Snap Pack

    • Added the following Snaps:

      • ELT Router

      • ELT Cast Function

      • ELT String Function

    • Enhanced the ELT SCD2 Snap to improve its flexibility and efficiency.

  • FDL

    • Added Google Sheets as a source endpoint.

SnapLogic Flows:

  • Renamed the What’s New page to the Release Updates page.

  • Added Flows as an asset type when exporting/migrating projects in SnapLogic IIP.

  • Added new data collection types for the Marketo, Salesforce, and Jira endpoints.

Flows is available to you free with your current SnapLogic subscription, but only for a limited time. To get started, register for Flows.

4.29 UAT Delta

This section tracks the changes made during the iterative pushes to the UAT server and the GA release. The following is the release schedule:

  • UAT: Friday, May 6, 2022 – Snaplex version: 4.29_rc-12044 - 4.29 RC (Recommended)

  • GA: Saturday, May 14, 2022 – Snaplex version: 4.29_rc-12117 - 4.29 RC (Recommended)

The SnapLogic GA release build on the production server is also deployed on the UAT server.

Important

  • Per the SnapLogic Release Process, all remaining Snaplex instances across Orgs are auto-upgraded to the recommended version (main-12117) at 9 p.m. PT on Saturday, Jun 18, 2022. Orgs migrated to the 4.29 GA version prior to the auto-upgrade are not impacted. 
    The Snaplex upgrade process also upgrades the Java version to the latest version. After the upgrade, the Java version of your Cloudplex nodes may not be the same as your FeedMaster nodes. However, this does not impact the operations of your SnapLogic instance. We will be fixing this issue in a subsequent release.

  • To use the new SnapLogic features and Snaps in the 4.29 release, ensure that your Snaplex nodes are upgraded to the recommended version.

  • This 4.29 release includes the patches deployed to SnapLogic after the 4.28 release, as listed in the 4.28 Dot Releases section. 

  • SnapLogic TLS/SSL Digital Certificate Update: We updated the SSL/TLS certificates for the SnapLogic Platform servers (UAT and Elastic) at 9 p.m. PT on Tuesday, May 3, 2022, since the previous certificates were expiring. Customers who need to manually update their trust store to trust the new certificate must do so immediately. Learn more: https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1083998225

The SnapLogic Privacy Policy has been updated to remove all references to Privacy Shield certification as of April 2022.

4.29 Dot Releases

Snaps

Use Microsoft Dynamics 365 for Sales Snap Pack instead of Microsoft Dynamics CRM Snap Pack

As Microsoft has deprecated its following offerings, the Dynamics CRM Snaps configured with the Dynamics 365 Account For Online or Dynamics 365 Dynamic Account For Online fail with a Ws-Trust authentication error.

The Microsoft Dynamics CRM Snap Pack (CRM) will be deprecated in the upcoming August, 4.30 SnapLogic release after the on-premise support is added in the Microsoft Dynamics 365 for Sales Snap Pack. If you are using the Dynamics 365 Online Accounts for Dynamics CRM Pipelines, we recommend that you immediately update these Pipelines and Accounts with Microsoft Dynamics 365 for Sales that uses the REST-based API (Web API) recommended by Microsoft.

Before you begin your migration, contact your Customer Support Manager (CSM) or SnapLogic Support to subscribe to Microsoft Dynamics 365 for Sales Snap Pack.

Learn how to migrate MS Dynamics CRM Snap Pack Pipelines to MS Dynamics 365 For Sales Snap Pack.

Snap Pack

Date of Update

Snap Pack Version

Updates

Snap Pack

Date of Update

Snap Pack Version

Updates

SOAP

Aug 8, 2022

429patches17186

Fixed an issue with the SOAP Execute Snap where the Snap displayed an error when a valid WSDL getOrderStatus SOAP operation was uploaded.

Redshift

Jul 28, 2022

429patches16908

  • Enhanced the Redshift accounts with the following:

    • Expression enabler to pass values from Pipeline parameters.

    • Support for Security Token for S3 bucket external staging.

  • Fixed an issue with Redshift - Execute Snap where the Snap failed when the query contained comments with single or double quotes in it. Now the Pipeline executes without any error if the query contains a comment.

RabbitMQ

Jul 27, 2022

429patches17049

Enhanced the RabbitMQ Consumer Snap with:

  • Acknowledgment Type dropdown list that allows you to set the acknowledgment type, which is available only when you select Auto_Acknowledgement in the Message Acknowledge Mode field.

  • A default 30-minute acknowledgment timeout—the Snap waits only for 30 minutes to get an acknowledgment. When there is no acknowledgment even after 30 minutes, Snap considers the status as unacknowledged and moves to the subsequent message.

NetSuite

Jul 26, 2022

429patches16915

Fixed an issue with NetSuite Search Snap where the Snap displayed the Could not translate an element declaration element exception intermittently when multiple search Snaps are executed simultaneously.

MS Dynamics 365 for Sales

Jul 22, 2022

429patches17123

  • Dynamics 365 OAuth2 Account For Online now supports on-premise usage that allows you to access the latest version of the APIs supported in the Microsoft Dynamics 365 suite.

  • Fixed an issue with the Dynamics 365 For Sales Read Snap where the Snap displayed a Null Pointer error instead of the actual 429 HTTP status code - Too Many Requests error from the endpoint, and the Pipeline failed. Now Snap does not display a Null Pointer error if it encounters a 429 error and the Pipeline transitions to the completed state.

  • Fixed an issue with Dynamics 365 For Sales Upsert Snap where the Snap displayed a different error message in the error view when the batch size is greater than 1.

  • Fixed an issue with Dynamics 365 For Sales Update and Dynamics 365 For Sales Upsert Snaps where the Snaps wrote to both output view and error view when the update fails and batch size equals 1. Now the Snap writes only to the error view when the update fails.

  • Fixed an issue with Dynamics 365 For Sales Update and Dynamics 365 For Sales Upsert Snaps where the Snaps did not write to the output view when the response is null. Now the Snaps correctly write to the output view when the response is null.

  • Enhanced the Dynamics 365 For Sales Update and Dynamics 365 For Sales Upsert Snaps with Alternate Keys fieldset that enables you to refer to entities by unique or nonunique combinations of columns.

Alternate Keys: Use this field set to configure columns to use as the alternate key for entities. Click + to add Key Columns to set alternate keys. For example, CompanyName, EmailAddress.

Transform

Jul 22, 2022

429patches16990

  • Fixed an issue with the Aggregate Snap where the Snap failed to validate (after first successful validation) while using a field that may contain a date for MIN and MAX functions. The Snap now supports DATE-type fields.

  • Enhanced the Pivot Snap with the Treat selected fields as static checkbox that enables the Snap to treat the selected fields as static to preserve the structure of the selected fields while all other fields are pivoted.

Treat selected fields as static: Select this checkbox to treat selected fields as static in the output document. This checkbox is a conditional field that enables the following fields for you to apply the pivoting behavior on the target fields.

- No Pivots Target Field Behavior

- No Pivots Target Value Behavior

Transform

Jul 21, 2022

429patches16923

  • Enhanced the CSV Formatter and CSV Parser Snaps to support multiple characters or strings as delimiters.

  • Fixed an issue with the Join Snap where the Snap displayed an incorrect error if the Left path or Right path fields were expression-enabled or if you have specified properties other than the field name and irrespective of whether the Sorted streams field is Unsorted or not. For example, "$first + '2' . Now, the Snap runs properly if the Left path or Right path was expression enabled and you have specified properties other than the field name and the Sorted streams field is Sorted or Unsorted. The Snap now displays an informative error on encountering an issue while executing this Snap.

  • Fixed an issue with CSV Parser Snap where the Snap failed when more than six characters were used as delimiters. Now, the Snap executes properly when you use more than six characters as delimiters.

Teradata

Jul 21, 2022

429patches16235

Fixed an issue with Teradata Execute Snap where the Snap did not display a valid error message when the delete condition was invalid.

SQL Server

Jul 21, 2022

429patches16235

Fixed an issue with SQL Server Execute Snap where the Snap did not display a valid error message when the delete condition was invalid.

Snowflake

Jul 21, 2022

429patches16478

Fixed an issue with Snowflake - Bulk Load Snap where Snap failed with 301 status code - SnapDataException.

PostgreSQL

Jul 21, 2022

429patches17036

Enhanced the PostgreSQL Account and PostgreSQL Dynamic Account with SSH Tunneling configurations to encrypt the network connection between the client and the PostgreSQL Database server, thereby ensuring the secure network connection.

OpenAPI

Jul 21, 2022

429patches16939

  • Enhanced the OpenAPI Snap with the following:

    • Pass through checkbox that allows you to pass input data to the output under Original key.

    • Trust all certificates checkbox that allows you to use self-signed certificates.

    • Enable URL Encoding checkbox that allows automatic URL encoding of the request path.

    • Support for external files referenced in the OpenAPI specification field.

Kafka

Jul 21, 2022

429patches17109

  • Enhanced the Kafka Producer Snap with LZ4 and ZSTD Compression Types for compressing the messages that optimize Snap's performance.

  • Fixed an issue with the Kafka Producer Snap where th Snap wrote an output document even when there was an error.

  • Fixed an issue with Kafka Account where the expression buttons were enabled, by default, for the Bootstrap Server and Advanced Property Value fields.

  • Upgraded Kafka client libraries to Apache-Kafka Version 3.1.1 and Confluent-Kafka Version 7.1.1.

  • Enhanced the Kafka account, which is configured to connect to the Kafka Schema Registry, to use TLS v1.2 protocol instead of TLS that has security vulnerabilities.

Hadoop

Jul 21, 2022

429patches16630

  • Extended the AWS S3 Dynamic Account support to ORC Reader and ORC Writer Snaps to support AWS Security Token Service (STS) using temporary credentials.

  • Fixed an issue in the following Snaps that use AWS S3 dynamic account, where the Snaps displayed the security credentials like Access key, Secret key, and Security Token in the logs. Now, the security credentials in the logs are blurred for the Snaps that use AWS S3 dynamic account.

DynamoDB

Jul 21, 2022

429patches16169

Fixed an issue with the DynamoDB Snaps where the Snaps did not route the errors to the error view when the Snap encountered an exception and the Pipeline failed. Now the Snap route the errors to the error view and does not fail when it encounters an exception. If the error view is disabled, Snap stops the execution.

Azure SQL

Jul 21, 2022

429patches16545

Fixed an issue with the Azure SQL-Stored Procedure Snap where the Snap failed with an Invalid value type error when the stored procedure contained a NCHAR data type.

Google Directory

Jul 21, 2022

429patches16728

  • Enhanced the Google Directory Insert Snap where the Snap did not display the original payload in the error view. Now, the original payload is passed to the error view.

  • Enhanced the Google Directory Update Snap where the original payload is displayed in the error view, but not under the original tag. Now the original payload is displayed under the original tag.

Binary

Jul 21, 2022

429patches16569

  • Enhanced the File Operation Snap to support moving data from local node to Azure blob through the AZ Copy utility.

  • Enhanced Azure Storage Account with the Request Size (MB) field to set the buffer limit before writing to Azure storage to enhance the performance.

Request Size: Specify the number in MB for the buffer size to be used by the Binary Snap before writing to Azure storage.

ML Core

Jul 20, 2022

429patches16809

Removed the log4j dependency from the ML Core Snaps due to security vulnerabilities.

REST

Jul 19, 2022

429patches17084

  • Enhanced the REST Snaps (Delete, Get, Patch, Post, and Put) with Cookie Policy dropdown list to set the cookie policy specifications. This field is case-insensitive.

Cookie Policy options:

Browser Compatibility: This policy is compatible with different servers even if they are not completely standards-compliant. If you are facing issues while parsing cookies, you should try using this policy.

Ignore Cookies: This cookie policy ignores all cookies. You should use this policy to prevent HTTP Client from accepting and sending cookies.

RFC Strict: This policy uses the set-cookie header.

RFC Lax: The policy uses set-cookie and set-cookie2 for parsing.

  • Added Security Token field in the REST AWS Sig v4 Account to support AWS Security Token Service (STS) for using temporary credentials.

  • Enhanced the REST AWS Sig v4 Account to support expression values for Access-key ID and Secret Key fields.

Google Sheets

Jul 18, 2022

429patches16975

  • Fixed an issue with the Worksheet Writer Snap where the Snap failed to write data correctly after each batch write. 
    When the Write Mode is Create New Worksheet and Overwrite worksheet if it exists checkbox is selected, the cell data is shifted by five columns to the right after each batch in the resulting worksheet. Now, the data is written correctly in the worksheet.

  • Enhanced the following in the Worksheet Writer Snap:

    • Support of expressions for the Starting Cell Reference field.

    • Added a warning about potential overwriting of existing data for the Starting Cell Reference field when writing data in the same range of cells.

  • Introduced the following Snaps:

Azure Active Directory

Jul 12, 2022

429patches16884

Fixed the following issues in the Azure Active Directory OAuth2 account:

  • Removed the Header authenticated checkbox.

  • Modified the default values for OAuth2 Endpoint and Token values where the tenant-id was hardcoded. Now the tenant id is displayed as <tenant-id*> and not a static value.

Fixed the following issues in the Azure Active Directory Search Entries Snap:

  • The Snap retrieved only 1000 records and did not retrieve all the records from the directory. Now, Snap retrieves all the records from the directory.

  • The Snap failed with the 400 status code error during validation and aborted during execution when values were added in the Properties field. Now, Snap does not fail whether you add values to the Properties field or not.

Transform

Jul 7, 2022

429patches16521

  • Removed the default value for the Root element field in the XML Formatter Snap.

  • Fixed an issue with the Transcoder Snap where the Input character-set field was not displaying the suggestions properly.

SAP

Jul 7, 2022

429patches16637

  • Fixed an issue with the SAP IDoc Write Snap where the Snap wrote the same TID, Document_number, and so on for the different output documents though the I_DOCNUM array contained all the document numbers that are returned from SAP table. Now, the Snap writes different TID, Document_number for different output documents along with the I_DOCNUM array that contains all the document numbers.

  • Fixed an issue with SAP Execute and SAP IDoc Write Snaps where the output preview did not include the original document.

Transform

Jul 5, 2022

429patches16026

Enhanced the Excel Parser Snap with the Custom Locale dropdown list that allows you to select a user-defined locale to format numbers as per the selected locale.

Salesforce

Jun 29, 2022

429patches16708

  • Updates in Salesforce Subscriber Snap:

    • Upgraded the Cometd version from 5.0.9 to the latest 7.0.6 version.

    • Fixed an issue where the Snap reported an error and stopped only at the next resubscribe attempt when it encountered the daily limit exceeded error. Now, the Snap displays the error and stops immediately when it encounters the daily limit exceeded error.

    • Simplified the logging that provides useful diagnostic information without logging any sensitive data.

    • Enhanced the Snap design to make the callbacks or listeners asynchronously by the Cometd API when there is a specific event, and the Snap responds accordingly.

    • Enhanced the tracking of replay ID of the most recent message processed so that Snap can send that replay ID if there is a need to resubscribe. If the server responds that the replay ID is invalid, Snap automatically sets the Replay ID value to -2 to get all available messages.

  • Made the Null Setting with Bulk API checkbox in the Salesforce Update, Salesforce Create, and Salesforce Upsert Snaps visible to format a null value.

Oracle

Jun 29, 2022

429patches16603

  • Fixed an issue with Oracle Bulk Load Snap where the Snap encountered an error when trying to load data into Timestampwithtimezone datatype field.

  • Fixed an issue with Oracle Execute Snap where the Snap did not display a valid error message when the delete condition was invalid.

  • Fixed an issue with Oracle - Merge Snap where the Snap displayed an error when the RetryInterval and Number of Retries fields were expression enabled and the values were passed using the Pipeline parameters.

  • Fixed an issue with the Oracle Stored Procedure Snap where the Snap errored out due to an upgrade made to HikariCP that caused a conditional failure and prevented the downstream Snaps to supply the parameters to the stored procedure. 

  • Enhanced the Oracle - Merge and Oracle - Update Snaps with the Session parameters fieldset that provides National Language Support (NLS).

Session parameters: The fieldset that includes the Session parameter name and Session parameter value fields to set the National Language Support.

Learn more about Setting NLS Parameters.

Azure SQL

Jun 28, 2022

429patches16460

Fixed an issue with Azure SQL Bulk Load Snap where the Snaplex exited due to insufficient memory when a large number of rows are loaded into the target table and the input data contained a null value for a non-nullable column.

Snowflake

Jun 27, 2022

429patches16458

  • Fixed the following issues with the Snaps that use Snowflake Accounts Key Pair Authentication.

    • The Snaps were not able to read the private keys provided as Pipeline parameters, because the key was malformed by replacing the line breaks with whitespace characters. Now, the private keys are properly formed and read when provided as Pipeline parameters.

    • The Snap displayed a Null Point Exception error when you pass an invalid key into the Private Key field like test. Now, the Snap displays a valid error message.

  • Fixed an issue with the Snowflake Snaps due to an upgrade made to HikariCP that caused a conditional failure and prevented the downstream Snaps to supply the parameters to the stored procedure. 

  • Fixed an issue with Snowflake Execute and Snowflake Select Snaps where the Snap displayed an exception error when the FLOAT or DOUBLE datatype column has the value as NaN (Not a Number). Now the output value is written as-is (NaN) for FLOAT or DOUBLE datatype column and does not display an exception.

  • Enhanced the following Snowflake Accounts with support for Key Pair Authentication.

  • Removed support for unencrypted Private Keys to enforce security in Snowflake accounts.

JDBC

Jun 24, 2022

429patches16588

  • Fixed an issue with JDBC Select and JDBC Insert Snaps where the Snaps did not load suggestions for the Table property. Now the Snaps load the suggestions correctly.

  • Enhanced the Generic Database account to support dynamic values for Account properties, Advanced properties, and URL properties so that you can pass these values through Pipeline parameters.

MongoDB

Jun 23, 2022

429patches15807

Updated the expected output for the MongoDB - Update Snap that is changed due to the upgrade of Spring Core framework version.

ELT

Jun 17, 2022

429patches16665

  • Enhanced the ELT Snap Pack to support the latest JDBC drivers across CDWs—Azure Synapse, BigQuery, DLP, Redshift, and Snowflake. See Configuring ELT Database Accounts or the respective Account page for the exact versions.

  • Enhanced the ELT Pivot Snap to make the Value List field dynamic.

  • Enhanced the ELT Snowflake Account with support for Key Pair Authentication.

  • Enhanced the ELT DLP Account to configure S3 Bucket, Azure Storage, and DeltaLake Storage Gen2 mounts.

  • Enhanced the end Snap SQL query in the ELT Insert Select Snap’s preview output to display the CREATE TABLE... or the DELETE/DROP TABLE statements to be run before the query that inserts/loads data into a new table in the Snowflake target CDW.

  • Fixed the issue with ELT Load Snap where the Snap caused an SQL exception—[Simba][SparkJDBCDriver](500051) ERROR processing query/statement when reading from a CSV file in S3 mount point on DBFS in the case of a DLP target instance.

  • The ELT Insert Select, ELT Merge Into, ELT Load, and the ELT SCD2 Snaps now run successfully even when the specified target table does not exist. The Snaps now create a new target table if it does not exist during Pipeline validation.

The new table thus created will not be dropped in the event of a subsequent/downstream Snap failure during validation.

  • The ELT SCD2 Snap:

    • Upon Pipeline validation, displays the final SQL query to be executed on the target CDW in its output preview.

    • Can replace an existing target table with a new table and load the SCD2 entries from the source table/files into it. Use the new Overwrite existing table option in the Target Table action field to perform this action.

  • Updated the field names in ELT Aggregate, ELT Cast Function, ELT String Function, and ELT Transform Snaps to maintain consistency.

Marketo

Jun 9, 2022

429patches16430

  • Fixed an issue with Marketo OAuth2 Account where the Snaps failed with ACCESS TOKEN EXPIRED error. Now the Snaps that use Marketo OAuth2 Account work as expected and does not display an expired token error.

  • Fixed an issue with Marketo Bulk Lead Upsert Snap where the Snap processed only successfully submitted batch jobs. Now, the Snap processes the failed batch jobs and logs the response message for each status.

  • Fixed an issue with Marketo Bulk Lead Upsert Snap where the Snap hung when parsing large data.

  • Fixed an issue with Marketo Bulk Lead Upsert and Bulk Lead Extract Snaps where the Snaps did not display valid error messages for Timeout and Number of Retries property when the input is large, empty, or Null.

  • Fixed an issue with Marketo Bulk Lead Upsert and Bulk Lead Extract Snaps where the Snaps failed without any reason when the Timeout value was less than 10 seconds.

Azure Active Directory

Jun 9, 2022

429patches16391

Enhanced the Azure Active Directory Search Entries Snap with Properties field to display the attributes you want to view in the output.

Properties: The properties to be displayed in the output for the user or group.

Learn more about Advanced query capabilities on Azure AD directory objects.

Redshift

Jun 8, 2022

429patches15806

Fixed an issue with Redshift Account and Redshift SSL Account where the Redshift Snaps failed when the S3 Secret key or S3 Access-key ID contained special characters, such as +.

SAP

Jun 7, 2022

429patches16113

Fixed an issue with SAP Execute Snap where the Snap incorrectly handled connection exceptions for 104-JCO_ERROR_SYSTEM_FAILURE. Previously, the Snap captured only the output document without the error. Now, the Snap correctly captures both the output and error.

Slack

Jun 1, 2022

429patches16199

  • Fixed an issue with the Send Message Snap where the Snap displayed a Null Pointer Exception error when you select User Emails for the Target Type field and the user email id contained null, for example, @slackbot:null. Now, the Snap displays a valid error message.

  • Fixed an issue with the User ID field for the following Slack Snaps where the Snap failed with the User not found error when using @user.name in the User ID field.

ELT

May 28, 2022

4.29patches16287

Fixed an issue with the ELT SCD2 Snap where the Snap was rounding off decimal values to the nearest integer—the value 57.601000000000 in the source table was written to the target table as 58.000000000.

Binary

May 24, 2022

429patches15842

  • Improved the tooltip for the File action field in the ZipFile Write Snap.

  • Enhanced the SSH Auth Account with Expression enabler () for the following fields that allow the use of Pipeline parameters to populate the properties.

    • Username

    • Private key

    • Key passphrase

DynamoDB

May 20, 2022

429patches15954

  • Enhanced the DynamoDB Account with AWS Security Token field to pass the security token when making calls using temporary credentials.

AWS Security Token: The security token to get temporary access to AWS resources using temporary credentials.

Learn more about adding signature to the HTTP request - AWS General Reference.

REST

May 20, 2022

429patches16076

  • Enhanced the REST Snaps with support for auto-retry where the Snap retries automatically when the Snap encounters 429 HTTP status code - Too Many Requests error.

  • Improved the performance of REST Snaps when the Snaps encountered errors due to Unauthorized (401) or Forbidden (403) statuses though the user credentials are available in the organization and the account type is OAuth2.

SnapLogic Studio (Preview)

New Features

Execution Overview Includes Pipeline and Task Execution Details: You can now view details and execution statistics for your Pipelines and Tasks. When you click on a Pipeline or Task, SnapLogic Studio displays a fly-out panel with information about the invocation type, the name of the Snaplex, execution times and duration, as well as information about Snap statistics, Pipeline logs, and Pipeline parameters. The Pipeline and Task execution details enable you to survey Pipeline and Task executions in your Org and then drill deeper to understand individual Pipeline execution details.

New Execution Status Icons: New icons indicate Completed with Errors and Completed with Warnings statuses of your Pipeline executions when you select Route Error Data to Error View in Views > Error setting in a Snap.

Likewise, when Snaps encounter warnings or errors, the Snap displays an icon for each state respectively to indicate the status after Pipeline execution.

 


Search by Execution Status: You can now filter results by several attributes, including Pipeline execution status (for example, Started) and the person who ran the execution. You can also add and remove columns and rearrange them. This functionality enables you to quickly find Pipelines in specific states, as shown in the following screenshot:

Status icons

Platform 

As previously announced, SnapLogic has ceased support for Windows Server 2012 on Jan 1, 2022, since this version did not support strong ciphers. Ensure that you upgrade your Groundplex instances to Windows Server 2016 or 2019. 

The SnapLogic platform is going live with Cloudflare on Jul 14, 2022, wherein the SnapLogic UI will automatically redirect to the given Cloudflare CDN addresses. This will enhance SnapLogic platform’s response time, security, and performance. Only customers who restrict outbound IP addresses to a pre-identified list of IP addresses must extend their allowlist to add all the given IP addresses to ensure continued access to the SnapLogic UI. Learn more to understand if this impacts your organization.

Enhancements

  • Child Pipeline Execution States: Added the capability in both the SnapLogic Studio Execution overview and Dashboard to view a child Pipeline’s status by hovering over the parent Pipeline. You can quickly identify which child Pipeline execution failed in Pipelines with the Pipeline Execute Snap. The Pipeline execution displays the icon when you select when you select Route Error Data to Error View in Views > Error setting in the Pipeline Execute Snap and either the Snap or child Pipelines encounter errors.

  • Include Flows when Exporting and Migrating: Added Flows as an asset type in Manager. While exporting or migrating a project, you can now also select Flows as a separate asset type.

  • Logging and Monitoring: Improved the logging and monitoring of events and errors related to network conditions around Snaplex instances, enabling SnapLogic Support more information to troubleshoot.

  • Migration to shared Directory: The Project Migration Public API can migrate assets to the shared directory of the project space.

  • Updated Snaplex Helm chart: Version 3 of the Snaplex Helm chart includes the following enhancements:

    • Support for Liveness, Readiness, and Startup Probe.

    • Support for setting SL_FILE_ULIMIT and SL_PROCESS_ULIMIT.

    • Default Kubernetes Service type changed from NodePort to LoadBalancer.

Known Issue

In Studio, when an Ultra Pipeline has a Snap that has an execution status of Completed with errors, the Snap Statistics tab displays the error in red and yellow, instead of only yellow as the Dashboard version of the Snap Statistics tab does.

API Management

New Features

  • Proxy Mapping Rules: You can now create Proxy Mapping Rules to map your Proxy endpoints to the right target depending on the conditions. Learn more about Proxy Mapping Rules.

  • Proxy Publication: You can also publish Proxies in the API Manager. In the Developer Portal, they appear like other APIs. Learn more about publishing a Proxy.

  • Dynamic Search: Dynamic Search is now available in the API Manager.

  • CORS Org-wide Setting for “Try It Out”: Org Administrators can now enable the “Try It Out” feature for all APIs in the Developer Portal using a default CORS Restriction Policy. However, if a CORS Restriction Policy exists at the API level or API version level, it has precedence over the default Org-wide policy. Learn more about setting up the Developer Portal.

  • Auto-Approve Subscriptions: When you publish an API or Proxy, you can now choose to auto-approve requests for subscriptions to the API. Learn more about publishing an API version or Proxy.

  • Custom Subscription Key Parameters: When you publish an API or Proxy, you can specify the header or the query parameter that the API consumer must set to the subscription secret key. Learn more about publishing an API version or Proxy.

  • Fast Secret Copy: API consumers can also copy the application's secret key by clicking the key icon () on the application's card, instead of having to edit the application entry to copy the secret key.

Enhancements

  • Open API Specification (OAS) 3.0: The API Manager now accepts OAS 3.0 specifications for APIs; however, API consumers can download only OAS 2.0 specifications.

  • Bearer Authorization Header: When calling an API with an approved subscription, you must set the Authorization header as Authorization: Bearer <client secret>, instead of using Token.

  • Featured APIs: In the Grid View of the Developer Portal, featured APIs are now easier to find with the existence of the Featured pill at the top of the API card. Featured APIs appear at the top of the API list. The API's tags are also displayed as pills at the bottom of the card.

  • New Documentation Article: Using Long-running Triggered Tasks with an Enterprise Scheduler describes how to use a script to prevent timeout or lost connection errors when executing long-running tasks from an enterprise scheduler. The script starts Pipeline execution and polls continuously for completion status. This keeps the connection alive and prevents the scheduler from reporting an error.

Known Issues

  • API Dashboard:

    • The API Dashboard does not collect metrics for APIs that do not generate a response entity.

    • When the API Management feature is enabled for the Org, the API Dashboard does not automatically start collecting data. You must restart the Snaplex nodes and allow the Dashboard to start.

  • CORS Setting Delay: In the Portal Manager » Settings tab, changes to CORS Setting » Enable Try Me might not take effect for up to two hours due to caching. This occurs only in JCC nodes with Snaplex Version 4.28 and older.

  • No Notification About Modified Key Settings: If an API is republished with modified Key Settings, existing subscribers are not notified of the change.


Snaps

Read about the Snap features and fixes deployed in the prior 4.28 Snap dot releases, which are part of the current 4.29 GA version.

Use Microsoft Dynamics 365 for Sales Snap Pack instead of Microsoft Dynamics CRM Snap Pack

As Microsoft has deprecated its following offerings, the Dynamics CRM Snaps configured with the Dynamics 365 Account For Online or Dynamics 365 Dynamic Account For Online fail with a Ws-Trust authentication error.

The Microsoft Dynamics CRM Snap Pack (CRM) will be deprecated in the upcoming August, 4.30 SnapLogic release after the on-premise support is added in the Microsoft Dynamics 365 for Sales Snap Pack. If you are using the Dynamics 365 Online Accounts for Dynamics CRM Pipelines, we recommend that you immediately update these Pipelines and Accounts with Microsoft Dynamics 365 for Sales that uses the REST-based API (Web API) recommended by Microsoft.

Before you begin your migration, contact your Customer Support Manager (CSM) or SnapLogic Support to subscribe to Microsoft Dynamics 365 for Sales Snap Pack.

Learn how to migrate MS Dynamics CRM Snap Pack Pipelines to MS Dynamics 365 For Sales Snap Pack.

New Snaps

SAP S/4HANA Cloud: SAP S/4HANA Cloud is an enterprise resource planning (ERP) system with built-in intelligent technologies, such as AI, machine learning, and advanced analytics. It helps companies adopt new business models, manage business change, and organize internal and external resources. This Snap Pack offers the following Snaps:

Enhancements

Microsoft is deprecating its Azure Active Directory Graph APIs in the near future and migrating to Microsoft Graph APIs. Hence, SnapLogic has migrated from Azure AD Graph APIs to Microsoft Graph APIs for the Azure Active Directory Snaps.

Breaking Change:

  • If you have any existing Pipelines that use the accounts authenticated with the old URL (OAuth2), then you must update them using the new URLs (v2.0) for calling APIs and modify the existing accounts to use the Microsoft Graph APIs.

    • OAuth2 Endpoint: https://login.microsoftonline.com/2060aafa-89d9-423d-9514-eac46338ec05/oauth2/v2.0/authorize

    • OAuth2 Token: https://login.microsoftonline.com/2060aafa-89d9-423d-9514-eac46338ec05/oauth2/v2.0/token

  • If you have not configured Scope in the Auth endpoint config field set in your existing accounts, then you must set the default Scope as https://graph.microsoft.com/.default offline_access in the field set, else the Snap fails.

Learn more about Azure Active Directory OAuth2 account.

The Coupa Account will be deprecated after 4.29 GA, as Coupa will discontinue the support of the API-based authentication in their upcoming September 2022 Release (R34). We recommend that you migrate your existing Pipelines configured with Coupa Account to Coupa OAuth2 Account immediately. Learn more: Coupa API Keys Deprecation.

  • Google Sheets:

    • Enhanced the Spreadsheets Browser, Worksheet Reader, and Worksheet Writer Snaps with the Include Shared Drives checkbox to support working with the Google Shared Drives.

    • Enhanced the Worksheet Reader and Worksheet Writer Snaps with the Display Spreadsheet ID in Suggestions checkbox. You can select this checkbox to display both spreadsheet ID and the spreadsheet name in the suggestions. This helps you to choose among the spreadsheets having the same name.

    • Enhanced the Worksheet Writer Snap with the Write Mode dropdown list to perform the desired write option for the given worksheet—that is either create a new worksheet or append or edit an existing worksheet. You can write new data to the worksheet by specifying a cell reference as the starting point or append data to the worksheet at the last row in the worksheet containing content.

    • Enhanced the Spreadsheet Browser Snap with the Fetch Mode dropdown list to fetch the spreadsheets and worksheets. You can either fetch spreadsheets and worksheets within the specified Spreadsheet or only spreadsheets.

  • Hadoop:

  • JDBC:

    • Enhanced the Generic Database Account in the JDBC Snap pack to include Netsuite as an option for the Database name field to connect to the NetSuite database. While configuring the account, if you select Netsuite or Autodetect (if the Snap detects the target database as NetSuite) then the Limit rows field in the Generic JDBC - Select Snap is ignored.

  • Kafka:

  • MS Dynamics 365 for Sales:

  • OpenAPI:

    • Enhanced the Base URL field in the OpenAPI Snap to support suggestions associated with OpenAPI Specifications.

    • Added Dynamic OAuth2 account that enables OAuth authentication for OpenAPI Snap.

  • REST:

    • Added the REST In-memory OAuth2 Account type with Client Credentials Grant Type that supports and stores the access token in memory for reuse until it expires. When Snap encounters 401 or 403 errors, this account generates a new token after refreshing the cache.

  • Slack:

    • Enhanced the Slack - Send Message Snap with User Emails option for the Target Type dropdown list to send messages on the Slack app using the user’s email.

  • Snowflake:

    • Updated the Snowflake - Bulk Upsert Snap for the following:

      • The Snap displayed an incorrect resolution when the length of the value exceeded the value defined in a column.

      • The Snap failed with a NullPointer Exception when no value was provided for Error Limit field and Error percentage limit fields.

      • The Snap displayed an incorrect error message when S3 details were not provided.

      • Added the On Error dropdown list, where you can select an action to perform when the Snap encounters errors in a file.

    • The following enhancements were done for the Snowflake - Bulk Load Snap:

      • The Snap optimizes AWS account configuration by skipping account validation if the S3 Storage Integration field value is provided.

      • The Snap overrides the storage integration specified in the account settings with the storage integration specified in the Snap settings.

    • Enhanced the Snowflake - Unload Snap to skip AWS account validation if S3 Storage Integration property is provided.

  • SQL Server:

    • Enhanced the SQL Server - Bulk Load Snap with the Batch Size field to process records in a batch.

    • Enhanced the SQL Server - Select and SQL Server - Lookup Snaps with Option for Unique Identifier Column that allows you to convert the case to lower or upper while displaying the Unique Identifier output (display) value.

  • Teradata:

    • Enhanced the Teradata FastExport Snap with Character Set dropdown list to support encoding of data when you export data from the Teradata Database.

  • Transform:

    • Fixed an issue with CSV Formatter and CSV Generator Snaps that displayed an incorrect error when one or more header values in the CSV file were missing. This error is now routed to the error view.

    • Fixed an issue with the CSV Parser Snap, where the Snap stopped indefinitely while processing certain inputs.

    • Fixed an issue with the CSV Formatter Snap, where the escape character selected was not used.

    • Updated the CSV library used in the CSV Generator, CSV Parser, and CSV Formatter Snaps. This library update enforces values for each column in a parsed CSV header.

    • Enhanced the XML Formatter Snap with the Output Character Set field. You can specify or select the character set you want for your binary output. For example, UTF-8 or ISO860.

  • Workday:

    • This Snap Pack supports the latest version of Workday APIs (certified to be compatible with version 37.0).

    • Enhanced the Workday Cancel Snap with Number of Retries and Retry Interval (seconds) fields to support the retry mechanism.

Deprecated

  • The Twitter, Facebook, Foursquare Snap Packs are deprecated and are no longer available. Contact support@snaplogic.com, if your existing Pipelines use Snaps from the deprecated Snap Pack.

  • The Coupa Account will be deprecated after 4.29 GA, as Coupa will discontinue the support of the API-based authentication in their upcoming September 2022 Release (R34).

Known Issues

  • Worksheet Reader: When you provide a spreadsheet ID to fetch data, and if a spreadsheet with that ID already exists as a spreadsheet name, the Snap fails to fetch the data, instead it fetches the data of the spreadsheet whose name matches with the spreadsheet ID.
    Suggestion: If you want to fetch the data of the specified spreadsheet ID, ensure that you specify the spreadsheet name of that spreadsheet ID or both (spreadsheet name with spreadsheet ID).

  • Worksheet Writer: When you provide a spreadsheet ID to append or insert data, and if a spreadsheet with the same ID already exists as the spreadsheet name, the Snap fails to append or insert the data for the specified spreadsheet ID and creates a new worksheet for the spreadsheet name
    Suggestion: Ensure that you specify a unique spreadsheet name or spreadsheet name with spreadsheet ID.

Behavior Change

  • The CSV Formatter Snap now allows an empty value for the Escape Character field if no escape character is used in the input CSV data. Earlier, this Snap used a '\' (forward slash) in the input when the Escape Character field was left empty causing an invalid data error.

  • The expected output for the MongoDB - Update Snap is modified due to the upgrade of the Spring Core framework version and the matchedDocuments definition is introduced. Earlier, the updateOfExisting would be true if matchedDocuments was greater than 0. Now, updateOfExisting is true if updatedDocuments is greater than 0.

  • The Snowflake Execute and Snowflake Select Snaps now write the output value as-is for FLOAT or DOUBLE datatype columns if these columns have the value as NaN (Not a Number). Earlier, the Snaps displayed an exception error when the FLOAT or DOUBLE datatype column has the value as NaN. This behavior is not backward compatible.

  • The Salesforce Subscriber Snap now stops and reports the error immediately when it encounters the daily limit exceeded error. Earlier, Snap reported an error and stopped only at the next resubscribe attempt when it encountered the daily limit exceeded error.

  • The Dynamics 365 For Sales Update and Dynamics 365 For Sales Upsert Snaps now write the error to the error view and do not write to the output view when the Snaps encountered an error.


Data Automation

SnapLogic’s Data Automation solution for cloud data platforms accelerates the movement of data from legacy systems, applications, and many other sources into the cloud data warehouse (CDW). This flexible solution delivers traditional ETL integrations to achieve complex transformations of data in-flight, as well as ELT capabilities to transform data in-place, such that data does not move out of the CDW. Coupled with comprehensive API management capabilities, SnapLogic’s Data Automation solution drives full end-to-end automation of data processes, accelerates the loading/transformation of data into the CDW, and ultimately streamlines business processes for quicker data-to-decision delivery.

New ELT Snaps

  • Introduced a new Snap ELT Router to enable routing input SQL queries into multiple output views based on the given conditional expressions.

  • Introduced ELT Cast Function Snap to convert a data type of a column in the input SQL string into other supported data types.

  • Introduced ELT String Function Snap to support the various string functions supported by the different databases.

Enhancements

  • Enhanced the following Snaps to display the final SQL query in their output preview upon Pipeline validation.

    • ELT Load

    • ELT Merge Into

    • ELT Insert-Select

  • Enhanced the ELT Database Account to support OAuth2-based authentication on the target Snowflake database.

  • Enhanced the ELT Select, ELT Insert Select, ELT SCD2, ELT Merge Into, and ELT Load Snaps to display suggestions on the Schema Name field based on the Default Database Name provided in the Snap Account configuration when the Database Name is not specified in the respective Snap.

    • Improved usability of the suggestions features for these Snaps by making them case-insensitive. For example, typing default in the Schema Name field displays both default and DEFAULT, if they co-exist. You do not need to type DEFAULT to invoke and select the schema name DEFAULT from the suggestions list.

  • Enhanced the ELT SCD2 Snap to address different feature requests and issues raised by multiple customers. These changes provide more flexibility in configuring your SCD2 operations using this Snap.

    • Removed Check for nulls and duplicates in source field and added two dropdown lists - Null Value Behavior and Invalid Row Handling.

    • Refer to the ELT SCD2 scenarios to learn more.

    • Made the following items in the Meaning field of the Target Table Temporal Fields fieldset mandatory while making the Invalid historical rows flag optional.

      • Current row

      • Historical row

  • Enhanced the ELT Aggregate and ELT Window Functions Snaps to support the following functions across all supported CDWs:

    • KURTOSIS

    • MODE

    • SKEW

Although these functions are not natively supported in Redshift, Azure Synapse, DLP and BigQuery, SnapLogic enables you to use these functions on all the supported target CDWs.

  • Enhanced the ELT Aggregate Snap to support the following GROUP BY features across all supported CDWs:

    • Group by Cube

    • Group by Grouping Sets

    • Group by Rollup

    • Automatic GROUP BY for all input columns.

  • Fixed an issue with ELT Merge Into Snap where the Snap erroneously modified the target table column name when the column name contained the target table name.

  • Fixed an issue in ELT SCD2 Snap where the Snap causes incorrect results with Snowflake targets, when:

    • The Historical Row End Date value is provided.

    • Nulls and Invalid rows are recognized, but one or more start dates in the source are null.

  • Fixed the issue in ELT Transform Snap where the Output Schema of the Snap does not populate all the column names from its Input Schema.

Known Issues

  • [This issue is fixed in the July 2022 Release]: In the case of BigQuery, the ELT Load Snap fails to load data from files in S3 or Redshift when you specify more than one individual file in the File List fieldset. This is due to the API restrictions in Google BigQuery for loading separate files/tables from S3 and Redshift. Alternatively, you can use the File Name Pattern field to load data from multiple files in S3 or Redshift to your BigQuery instance.

For the best performance of your load operation, we recommend that you always use the File Name Pattern field to load data from multiple files at a source location (S3 or Redshift) instead of specifying the individual file names/paths in the File List fieldset.

  • If you have dropped one or more columns from your target table outside of the ELT Load Snap and then use this Snap to add these column(s) back and load data into the table, the Snap adds the new columns to the target table, but not the corresponding data from the source file.

  • When running without Sub-Query Pushdown Optimization (SPDO), ELT Pipelines that contain an ELT Aggregate Snap configured with one ore more GROUP BY ROLLUP fields do not verify the column data types while inserting the Snap output values in the target table. This may lead to incorrect data written to the target table. However, as long as SPDO is on, the same Pipeline runs without this issue.

  • In the case of Azure Synapse, if you are configuring the Start_Date column as a Target Table Natural Key in the ELT SCD2 Snap, the Snap fails in each of the following scenarios:

    • The source table/file contains one or more null values.

    • The target table is empty.

    • The End date of current row or End date of historical row has a static value, for example: '2021-01-01'.

  • [This issue is fixed in the June 2022 Release] When you are enclosing the column names within backticks (`<Column_Name>`) for creating a new target table in a DLP instance using the ELT Load Snap—typically to ensure that the column names are used verbatim—you may encounter the error: ERROR processing query/statement: no viable alternative at input 'FROM'(line X, pos XX). To prevent this error, edit the header in your source file to exclude any special characters, thereby avoiding the use of backticks in the target table column names.

  • [This issue is fixed in the July 2022 Release]: The ELT Select Snap does not honor the LIMIT clause included to query a BigQuery instance, during Pipeline validation. However, this issue does not occur during Pipeline runtime.

  • [This issue is fixed in the July 2022 Release]: In the case of Azure Synapse, BigQuery, DLP, and Redshift CDWs, the SKEW function available in the General Aggregate Functions List of the ELT Aggregate Snap returns null instead of the actual division-by-zero exception for certain ranges of values (for example, for a series of exactly same values).


Fast Data Loader

New Feature

Added Google Sheets as a source endpoint in Fast Data Loader (FDL). Google Sheets enables you to read data from the spreadsheet and write the data to the specified destination endpoint such as SAP-DWC or Snowflake.


Flows

Enhancements

  • The What’s New page has been renamed to the Release Updates page.

  • You can now select Flows assets when exporting or migrating projects in IIP.

  • Added two new data collection types for the Marketo endpoint:

    • Get Program by Name (source). Use this data collection type to find a Marketo program.

    • Clone Program (target). Use this data collection type to clone an existing Marketo program.

  • Added five new data delivery types for the Salesforce target endpoint:

    • Delete Account: Use this data delivery type to delete an account.

    • Delete Contact: Use this data delivery type to delete a contact.

    • Delete Lead: Use this data delivery type to delete a lead.

    • Delete Opportunity: Use this data delivery type to delete an opportunity

    • Delete another object: Use this data delivery type to delete another object.

  • Added the Transition an Issue data collection type for the Jira target endpoint. Use this delivery type to transition the status of a Jira issue.

  • Added the Dynamics 365 NTLM Account type for the Microsoft Dynamics 365 endpoint.

  • Added the REST In-memory OAuth2 Account type for the REST source endpoint.

Fixed Issues

  • Fixed an issue that prevented users from renaming Flows.

  • Fixed an issue that prevented users from loading a Flow from the Projects tab.