May 2024 Release Notes

Highlights

The SnapLogic May 2024 release includes the following:

Admin Manager

  • Collapsible navigation pane

API Management

  • New public API for Git Integration Operations for API version

  • API Metrics now available in Monitor

  • New public API to perform soft delete on the API and API version

  • A JWT Outbound policy is now available for Proxies

AutoSync

  • Merge columns transformation to concatenate data

  • Interface to update data pipelines to the latest AutoSync Snap Pack

  • New target endpoints: Databricks, Amazon S3, and Azure Data Lake Storage (ADLS) Gen2

Data Automation

  • Load flat and nested data sets from Parquet files to Snowflake with the ELT Load Snap

Documentation Updates

GenAI Builder

  • Added the Anthropic Claude on AWS Messages Snap to the Amazon Bedrock Snap Pack

Monitor

  • New pages:

    • Alert history

    • Integration catalog

Platform

  • New Git Integration public API to discard local changes

  • New public API to create, update, and delete a Groundplex

  • Modified public API for sending a notification email when a user is added

Snaps

  • The Azure SQL Bulk Extract Snap now supports the Azure SQL Entra and Azure SQL Entra Dynamic Accounts

  • Added support for Geospatial data types (Geography and Geometry) for the SQL Server Snap Pack

  • Added the MongoDB Execute and MongoDB - Atlas Vector Search Snaps to the MongoDB Snap Pack

  • Added the NetSuite Add List and NetSuite Async Add List Snaps to the NetSuite SOAP Snap Pack

  • Added Parquet Parser and Parquet Formatter Snaps to the Transform Snap Pack

SnapGPT

  • Improved UX

Dot Releases - Snaps

Snap Pack

Date of Update

Snap Pack Version

Updates

Snap Pack

Date of Update

Snap Pack Version

Updates

Google Analytics 4

Jun 10, 2024

437patches26687

  • Fixed an issue with the Google Analytics 4 Create Report and Create Realtime Report Snaps where the Snaps were unable to provide suggestions for the Metrics and Dimensions fields. 

  • Fixed an issue where the Google Analytics 4 Get Metadata Snap could not display an Unable to get metadata error during validation and execution.

SAP HANA

Jun 10, 2024

437patches26690

Fixed an issue in the SAP HANA Snap Pack where the downstream Snaps did not support the data type appropriately.

PostgreSQL

Redshift

Jun 10, 2024

437patches26634

Fixed an issue with PostgreSQL - Execute and Redshift - Execute Snaps that produced logs causing node crashes.

Teradata

Jun 10, 2024

437patches26471

The jOOQ library for Teradata Snap Pack is upgraded from v3.9.1 to v3.17.3. This upgrade will be part of the GA release on August 14, 2024 (Stable release). Pipelines using the Teradata Snaps are not impacted after the jOOQ upgrade.

AS2

Jun 6, 2024

main178

Fixed an issue with the AS2 Sender Snap that appended ‘3D’ in the XML document.

Azure Synapse SQL

Jun 3, 2024

437patches26597

Fixed an issue with the Azure Synapse SQL Select Snap that displayed an error and did not retry the connection when the serverless SQL pool was warming up.

Azure Service Bus

Jun 3, 2024

437patches26578

Fixed an issue with the Azure Service Bus Account that exposed the secret key when an illegal connection string exception occurred because of an incorrect string format.

PostgreSQL

Google AlloyDB

May 30, 2024

4postgresupgrade26570

  • Upgraded the PostgreSQL JDBC driver from v9.4.1207 to v42.7.2 (Java 8). This upgrade will be part of the GA release on August 14, 2024 (Stable release). As part of this upgrade, the 42.7.2 JDBC driver is bundled with the PostgreSQL Snap Pack as the default JDBC driver.

Breaking change:

Your existing PostgreSQL pipelines that use the default driver (bundled with the PostgreSQL Snap Pack) might break. This JDBC driver upgrade has resulted in specific behavior changes in errors, status codes, and success and failure messages. Learn more about the behavior changes to ensure your migration to the upgraded driver is seamless. 

  • The JDBC driver for the Google AlloyDB Snap Pack has been upgraded from v42.4.2 to v42.7.2 (Java 8). This upgrade will be part of the GA release on August 14, 2024 (Stable release). As part of this upgrade, the 42.7.2 JDBC driver is bundled with the AlloyDB Snap Pack as the default JDBC driver. Learn more about the behavior changes caused by the driver upgrade.

OpenSearch

May 30, 2024

437patches26566

Introduced the OpenSearch Snap Pack, which contains the following Snaps and Account:

  • OpenSearch Query: Executes a query on the specified OpenSearch index.

  • OpenSearch Upsert: Performs an operation that updates existing records if they exist, or inserts new records if they don't.

  • OpenSearch Account: Authenticates with OpenSearch services using the specified AWS account credentials.

Copybook

May 28, 2024

main174

Fixed an issue with the COBOL Copybook Parser Snap that incorrectly parsed packed decimal numbers.

Snowflake

May 28, 2024

437patches26508

Transform

May 24, 2024

437patches26453

  • Added expression support to the Skip lines field in the CSV Parser Snap to enable passing pipeline parameters and upstream values. 

  • Fixed an issue with the XML Parser Snap that caused an error when using the Splitter option in the Snap settings. 

API Suite

May 23, 2024

437patches26491

Fixed an issue with the GraphQL Client Snap that displayed an error when passing the Float data type query variables to the endpoint.

Hadoop

May 22, 2024

437patches26370

Enhanced the HDFS Writer Snap with the Write empty file checkbox which enables you to write an empty or a 0-byte file to all the supported protocols that are recognized and compatible with the target system or destination.

Databricks

May 15, 2024

437patches26400

Fixed an invalid session handle issue with the Databricks Snap Pack that intermittently triggered an error message when the Snaps failed to connect with Databricks to execute the SQL statement.

Oracle

May 14, 2024

437patches26346

Fixed an issue with the Oracle - Execute Snap that displayed an error when handling OffsetDateTime type bindings.

HTML Utilities

May 13, 2024

437patches26354

Fixed an issue with the HTML Parser and HTML to Markdown Converter Snaps that caused the Snaplex to stall when processing tables.

MySQL

May 9, 2024

437patches26298

Fixed an issue with the MySQL - Multi Execute Snap and the MySQL - Execute Snaps that displayed a null pointer exception when the Snap did not check  null table metadata.

Upgrading to this Release

  • To use the new SnapLogic features and Snaps in the current GA release, ensure that your Snaplex nodes are upgraded to the recommended version: main-24152 GA (Recommended).

  • This release includes stable versions of the monthly Snap patches deployed to the SnapLogic Platform after the February 2024 release, as listed in the Dot Releases section of the March and April Release Notes.

  • Per the SnapLogic Release calendar, all Snaplexes on older versions will be upgraded to the recommended version during the automatic upgrade, scheduled for 3 PM (Pacific Daylight Time) on Saturday, Jun 8, 2024. Orgs that were migrated to the current GA release before the automatic upgrade are not impacted. For information on how to upgrade your Snaplex, refer to the SnapLogic Release Process.

Admin Manager

New Features

  • The left navigation and header for Admin Manager have a new design. You can collapse the left navigation pane to increase space for the main panel and expand it to view page names:

    admin-manager-collapsed-highlight.png

  • AutoSync environment admins can now update the AutoSync Snap Pack version on the AutoSync settings page. Learn more.

    autosync-settings.png

  • The browser tab label now identifies Admin Manager and the name of the current page:

API Management

New Features

Enhancements

  • A new APIM function called request.roles enables API developers to invoke the list of existing roles for ease of use. Previously, you had to use IsUserinRole for each role type.

Fixed Issues

  • The Request Transformer, Authorize by Role, and other Proxy-only policies now support Basic Auth for authentication. Previously, any policy executed after authentication would fail with Basic Authentication.

  • Fixed an issue where the Client Throttle policy did not work when applied to an API Proxy running on a Snaplex with multiple nodes.

Deprecated

  • The Apigee integration solution is deprecated and the Apigee API Proxy selection will be removed from the Manager > Projects menu in July. This solution is no longer supported. We recommend that you use SnapLogic APIM for your integrations and APIs.

AutoSync

New Features

  • Concatenate data from multiple columns or fields of a string type with the new Merge columns feature. Merging columns can simplify complex data structures and result in more efficient storage and querying. Learn more.

  • Take advantage of new functionality and bug fixes by updating data pipelines to the latest AutoSync Snap Pack version. Environment admins can use the new Admin Manager AutoSync settings page to update multiple data pipelines at the same time. Learn more.

  • New target endpoints: Databricks, Amazon S3, and Azure Data Lake Storage (ADLS) Gen2.

  • You can now stop a data pipeline run mid-load. Because this can compromise target data integrity, we recommend stopping a data pipeline only when necessary. This might be, for example, with a data pipeline that takes a long time to load where you need to change the configuration or know that an issue with the source will result in errors. Learn more.

Enhancements

  • The text on the edit dialog for data pipelines with a CSV file endpoint has been enhanced. Because CSV loads can’t be scheduled, the former AutoSynchronize tab is now Load type. The Append load type clarifies that when you select a file that was previously loaded, the entire file will be appended to the existing table.

     

  • The AutoSync header has a new design:

     

  • The browser tab now identifies AutoSync:

     

  • The data preview table for transformations now shows 5 rows of data instead of 4.

Known Issues

  • For the ServiceNow endpoint, string and decimal values load to the target using the target’s default precision. MySQL has a default precision of 255, so values with a precision of more than 255 will fail to load to MySQL.

  • When creating a data pipeline with a CSV file source, the new Merge columns screen is skipped.

Fixed Issues

  • The Infer data types for CSV files checkbox now only displays for the S3 and SFTP endpoints for CSV files, not for JSON files.

  • For the ServiceNow endpoint, AutoSync ignores the maximum length defined by ServiceNow for string and decimal types and uses the default precision of the target.

Data Automation

Enhancements

Fixed Issue

  • Fixed an issue where the ELT Merge Into Snap failed to load data into the Google BigQuery target table because of the error: Number of source column expressions in the input SQL is greater than the target table columns (even after altering the target table to match the source table columns). The resolution was to modify an SQL statement that was used internally to achieve the merge action.

Documentation Updates

Content Published on New Documentation Site

You can find the following content on docs.snaplogic.com:

Video

Enhancements

  • Content about configuring Azure Key Vault as a secrets manager has been validated and improved.

GenAI Builder

The GenAI Snap Packs are part of the GenAI Builder product and cannot be purchased separately. To purchase GenAI Builder, contact your Customer Success Manager (CSM).

Monitor

New Features

  • The new Alert history displays all Snaplex node alerts to simplify review and analysis. You can tag events as open or closed to filter the list and you can download the log. Learn more.

  • The Integrations catalog is available by request for Enterprise customers. Contact either adm-support@snaplogic.com or your CSM. The catalog displays an inventory of integrations for pipelines with an associated task in a production environment. The table includes metadata and you can add custom metadata columns and values. Learn more.


    For environments with thousands of pipelines, the Integrations catalog offers:

    • Improved visibility of Integrations as data assets

    • Better governance for DataOps

    • Faster impact analysis for System Architects

    • Easier identification of reusable assets for Developers to reduce integration development time

Enhancements

  • Summary card counts on the Execution overview page now reflect any filters applied to the table in addition to the date range. For example, if two executions failed in the selected date range and were running on different Snaplexes, if you filter by one of the Snaplex names, the Failed card shows only one failure.

  • The left navigation and header for Monitor have a new design. You can collapse the left navigation to increase space for the main panel:

  • The browser tab now identifies Monitor and the name of the current page:

  • The Activity Log includes usernames and Snaplex version updates when users initiate Snaplex version changes. This also applies to the Activity Log in the Classic Manager. Learn more

Known Issues

  • With the Execution overview table grouped by Task or Pipeline, if you select a date range for more than one month's data, the execution table does not return results. The summary cards show the counts for executions by status.

  • The summary card counts do not update correctly when you select the following advanced search filters for Invocation type:

    • Manual

    • Nested Pipeline

    • Error Pipeline

    • For Each

    • Regression Test

Platform

New Features

  • The new public API for the Git Integration to discard local changes uses the project path and asset name to identify the assets to discard. Learn more.

  • The new public Snaplex APIs enable administrators to:

  • The public API for users sends a notification email after a user is added successfully to the organization. Learn more.

Enhancements

  • New support for AWS Secrets Manager to restrict secrets access by SnapLogic projects. To restrict access to specific projects, add role-based IAM aliases for them to a secrets-config.json file and restart your Groundplex nodes. Learn more.

  • The header for Designer, Manager, and Dashboard has a new look:

  • The user settings menu, which is available from a dropdown next to the user’s name, has a new design:

    • Settings are categorized as global or application-specific settings.

    • Non-admin users can’t change their username, they must ask an Environment admin to change it.
      The screenshot on the left shows the menu for Admin Manager, AutoSync, and Monitor. The screenshot on the right shows the menu for Dashboard and Manager:

  • Browser tab labels now identify the app and page name:
    tab-identifies.png

  • Open Telemetry reporting to third-party monitoring and logging tools now only sends messages labeled Error in severity level. Previously, messages marked Informational were also included. These Informational messages are still retained in the logs.

Fixed Issues

  • Fixed an issue for the toPrecision() function that converts the data incorrectly when processing a whole number. When you call the function with arguments greater than 100, it results in an error.

Upcoming behavior change

Project-level pipeline execution behavior change

Currently, tasks stored in a Project Space of an Org can be executed by pipelines in a different Project Space or even from requests from a different Org. There is an upcoming fix to stop this behavior in the July release. After the fix is deployed, tasks must belong to the same Project Space as the pipeline Project Space they are called from to be successfully executed.

Known Issues

  • When creating a Task and selecting the America/Mexico City time zone, the task runtime is off by one hour. Since Mexico City no longer observes daylight savings, the current implementation of CDT (aligned with CST) might assume that Mexico City changed time zones as it did in the past.
    Workaround: Upgrade your Snaplex to a JDK 11.0.18+ version.

Snaps

Upcoming

New Snaps

Introduced the following Snaps:

  • Added MongoDB Execute and MongoDB - Atlas Vector Search Snaps to the MongoDB Snap Pack:

    • MongoDB Execute: Executes all Data Definition Language (DDL) and Data Manipulation Language (DML) commands on the MongoDB database.

    • MongoDB - Atlas Vector Search: Performs advanced vector-based queries, such as Similarity searches, Approximate Nearest Neighbor (ANN) queries, and Range queries on vector data stored in MongoDB Atlas.

  • Added NetSuite Add List and NetSuite Async Add List Snaps to the NetSuite SOAP Snap Pack:

  • Added Parquet Parser and Parquet Formatter Snaps to the Transform Snap Pack:

    • Parquet Parser: Reads the binary Parquet data and writes document data to the output.

    • Parquet Formatter: Reads the document data and writes it to the output in binary Parquet format.

Enhancements

Previous name

New name

Previous name

New name

Azure Active Directory Snap Pack

Microsoft Entra ID Snap Pack

Azure Active Directory OAuth2 Account

Microsoft Entra ID OAuth2 Account

Azure SQL Active Directory Account

Azure SQL Entra Account

Azure SQL Active Directory Dynamic Account

Azure SQL Entra Dynamic Account

Azure Synapse SQL Active Directory Account

Azure Synapse SQL Entra Account

  • Enhanced the Google Directory Read Snap with an On empty result dropdown list that enables you to select the Snap behavior when no matching filter value is found.

  • Enhanced the JSON Splitter Snap to capture metadata and lineage information from the input document.

  • Kafka Snap Pack: Enhanced the following fields to support expressions through pipeline parameters:

    • Truststore Password

    • Keystore File Password

    • SSL Key Password

    • SASL Password or Secret

    • Registry Password or Secret

  • Enhanced the MongoDB Update Snap with the Array Filters field, which enables you to use array filters in the update operation. Additionally, the Update Query field is now modified into a text box for visibility and usability of input queries.

  • Increased the default Read timeout value for NetSuite REST Snaps from 60 to 300 seconds. This update supports longer-running operations and prevents early disconnections for complex data processing tasks.

  • Updated the Delete Condition (Truncates a Table if empty) field in the following Database Delete Snaps to Delete condition (deletes all records from a table if left blank) to indicate that all entries will be deleted from the table when this field is blank, but no truncate operation is performed.

  • Enhanced the HubSpot Snaps with the Retry policy field to handle retries for connection and error responses.

  • Updated the following NetSuite SOAP Snap names for usability based on the functionality of the Snaps:

Previous name

New name

Previous name

New name

NetSuite Create

NetSuite Add

NetSuite Async Delete

NetSuite Async Delete List

NetSuite Async Get

NetSuite Async Get List

NetSuite Async Upsert

NetSuite Async Upsert List

NetSuite Delete

NetSuite Delete List

NetSuite Upsert

NetSuite Upsert List

  • Enhanced the S3 Select Snap to capture metadata and lineage information from the input document.

  • Enhanced the Snowflake Bulk Load and Snowflake Insert Snaps to support iceberg tables. Iceberg tables partition and organize data across multiple nodes to distribute the workload and speed up data processing.

  • Enhanced the SQL Server Snap Pack to support Geospatial data types (Geography and Geometry), which enables you to seamlessly insert, read, and select Geospatial data in the SQL Server environment.

  • Upgraded Apache CXF from version 3.4.2 to 3.6.3 to prevent vulnerability issues for the following Snap Packs:

  • Upgraded Spring dependencies to the latest supported Java 11 version for the MongoDB Snap Pack.

Behavior Change

MySQL - Insert Snap: With the 4.37 GA release, when you pass a Date/DateTime value for the Year data type column, the MySQL Insert Snap does not accept those values and fails with an error. This is the expected behavior of the Snap; we recommend you to pass only year values for the Year data type. Previously, when you passed a Date/DateTime value for the Year data type column, the Insert Snap inserted the value as a Year data type.

Fixed Issues

  • Fixed an SMB (server message block) connectivity issue in the Binary Snap Pack, where the incorrect name provided by the SMB client did not match the Windows cluster virtual name, affecting the SPN (service principal name) connection.

  • Fixed a performance issue in the Copy Item Snap, which previously resulted in a nameAlreadyExists error when the Snap retried because of a connection reset.

  • Fixed an issue with the OpenAPI Snap where the upstream Mapper Snap did not display the full schema objects in the Target Schema.

  • Fixed an issue with the Oracle - Bulk Load Snap that intermittently displayed a null pointer exception.

  • Fixed the following issues with the SQL Server Snap Pack:

    • Snaps displayed an error when numeric values with scale 0 were inserted into the database with DECIMAL or NUMERIC type columns instead of inserting the value with decimal digits truncated.

    • Snaps displayed an error when fractional numeric values (numbers with a decimal point) were inserted into the database with an INT type column, instead of inserting the values with decimal digits truncated.

Known Issues

Deprecated

SnapGPT

  • Improved readability of Pipeline analysis summary and SnapGPT Analyze Pipeline Reports displays.

  • Each SnapGPT completion displays feedback icons and a copy icon in the following UI contexts:

    • Generate pipeline: Copies the pipeline to be pasted into a Designer canvas.

    • Describe pipeline: Copies the summary and pastes it as markdown formatted text.

    • Question and answer: Copies the SnapGPT response to the question and pastes it as markdown formatted text.

    • Analyze pipeline: Copies the pipeline analysis and pastes it as JSON.

    • Error: Copies error information.