May 2024 Release Notes
Highlights
The SnapLogic May 2024 release includes the following:
Collapsible navigation pane
New public API for Git Integration Operations for API version
API Metrics now available in Monitor
New public API to perform soft delete on the API and API version
A JWT Outbound policy is now available for Proxies
Merge columns transformation to concatenate data
Interface to update data pipelines to the latest AutoSync Snap Pack
New target endpoints: Databricks, Amazon S3, and Azure Data Lake Storage (ADLS) Gen2
Load flat and nested data sets from Parquet files to Snowflake with the ELT Load Snap
ML Analytics and ML Natural Language Processing Snap Packs are published on the new Documentation site.
Video on adding a Response Transformer Policy to a Proxy.
Added the Anthropic Claude on AWS Messages Snap to the Amazon Bedrock Snap Pack
New pages:
Alert history
Integration catalog
New Git Integration public API to discard local changes
New public API to create, update, and delete a Groundplex
Modified public API for sending a notification email when a user is added
The Azure SQL Bulk Extract Snap now supports the Azure SQL Entra and Azure SQL Entra Dynamic Accounts
Added support for Geospatial data types (Geography and Geometry) for the SQL Server Snap Pack
Added the MongoDB Execute and MongoDB - Atlas Vector Search Snaps to the MongoDB Snap Pack
Added the NetSuite Add List and NetSuite Async Add List Snaps to the NetSuite SOAP Snap Pack
Added Parquet Parser and Parquet Formatter Snaps to the Transform Snap Pack
Improved UX
Dot Releases - Snaps
Snap Pack | Date of Update | Snap Pack Version | Updates |
---|---|---|---|
Jun 10, 2024 | 437patches26687 |
| |
Jun 10, 2024 | 437patches26690 | Fixed an issue in the SAP HANA Snap Pack where the downstream Snaps did not support the data type appropriately. | |
Jun 10, 2024 | 437patches26634 | Fixed an issue with PostgreSQL - Execute and Redshift - Execute Snaps that produced logs causing node crashes. | |
Jun 10, 2024 | 437patches26471 | The jOOQ library for Teradata Snap Pack is upgraded from v3.9.1 to v3.17.3. This upgrade will be part of the GA release on August 14, 2024 (Stable release). Pipelines using the Teradata Snaps are not impacted after the jOOQ upgrade. | |
Jun 6, 2024 | main178 | Fixed an issue with the AS2 Sender Snap that appended ‘3D’ in the XML document. | |
Jun 3, 2024 | 437patches26597 | Fixed an issue with the Azure Synapse SQL Select Snap that displayed an error and did not retry the connection when the serverless SQL pool was warming up. | |
Jun 3, 2024 | 437patches26578 | Fixed an issue with the Azure Service Bus Account that exposed the secret key when an | |
May 30, 2024 | 4postgresupgrade26570 |
Breaking change: Your existing PostgreSQL pipelines that use the default driver (bundled with the PostgreSQL Snap Pack) might break. This JDBC driver upgrade has resulted in specific behavior changes in errors, status codes, and success and failure messages. Learn more about the behavior changes to ensure your migration to the upgraded driver is seamless.
| |
May 30, 2024 | 437patches26566 | Introduced the OpenSearch Snap Pack, which contains the following Snaps and Account:
| |
May 28, 2024 | main174 | Fixed an issue with the COBOL Copybook Parser Snap that incorrectly parsed packed decimal numbers. | |
May 28, 2024 | 437patches26508 |
| |
May 24, 2024 | 437patches26453 |
| |
May 23, 2024 | 437patches26491 | Fixed an issue with the GraphQL Client Snap that displayed an error when passing the | |
May 22, 2024 | 437patches26370 | Enhanced the HDFS Writer Snap with the Write empty file checkbox which enables you to write an empty or a 0-byte file to all the supported protocols that are recognized and compatible with the target system or destination. | |
May 15, 2024 | 437patches26400 | Fixed an invalid session handle issue with the Databricks Snap Pack that intermittently triggered an error message when the Snaps failed to connect with Databricks to execute the SQL statement. | |
May 14, 2024 | 437patches26346 | Fixed an issue with the Oracle - Execute Snap that displayed an error when handling | |
May 13, 2024 | 437patches26354 | Fixed an issue with the HTML Parser and HTML to Markdown Converter Snaps that caused the Snaplex to stall when processing tables. | |
May 9, 2024 | 437patches26298 | Fixed an issue with the MySQL - Multi Execute Snap and the MySQL - Execute Snaps that displayed a |
Upgrading to this Release
To use the new SnapLogic features and Snaps in the current GA release, ensure that your Snaplex nodes are upgraded to the recommended version:
main-24152 GA (Recommended)
.This release includes stable versions of the monthly Snap patches deployed to the SnapLogic Platform after the February 2024 release, as listed in the Dot Releases section of the March and April Release Notes.
Per the SnapLogic Release calendar, all Snaplexes on older versions will be upgraded to the recommended version during the automatic upgrade, scheduled for 3 PM (Pacific Daylight Time) on Saturday, Jun 8, 2024. Orgs that were migrated to the current GA release before the automatic upgrade are not impacted. For information on how to upgrade your Snaplex, refer to the SnapLogic Release Process.
Admin Manager
New Features
The left navigation and header for Admin Manager have a new design. You can collapse the left navigation pane to increase space for the main panel and expand it to view page names:
AutoSync environment admins can now update the AutoSync Snap Pack version on the AutoSync settings page. Learn more.
The browser tab label now identifies Admin Manager and the name of the current page:
API Management
New Features
The API metrics page is now in Monitor. You can view statistics and troubleshoot errors for your APIs.
New public APIs for APIM to perform Git Integration Operations include: Create Branch, Create API version, Checkout branch, Pull latest changes, Git tag, Check Repo-Status, Discard changes, and Untrack a branch.
New public API to perform a soft delete on APIs and API versions.
A JWT Outbound policy is now available for Proxies. Your Snaplex must be on the May version.
Enhancements
A new APIM function called
request.roles
enables API developers to invoke the list of existing roles for ease of use. Previously, you had to useIsUserinRole
for each role type.
Fixed Issues
The Request Transformer, Authorize by Role, and other Proxy-only policies now support Basic Auth for authentication. Previously, any policy executed after authentication would fail with Basic Authentication.
Fixed an issue where the Client Throttle policy did not work when applied to an API Proxy running on a Snaplex with multiple nodes.
Deprecated
The Apigee integration solution is deprecated and the Apigee API Proxy selection will be removed from the Manager > Projects menu in July. This solution is no longer supported. We recommend that you use SnapLogic APIM for your integrations and APIs.
AutoSync
New Features
Concatenate data from multiple columns or fields of a
string
type with the new Merge columns feature. Merging columns can simplify complex data structures and result in more efficient storage and querying. Learn more.Take advantage of new functionality and bug fixes by updating data pipelines to the latest AutoSync Snap Pack version. Environment admins can use the new Admin Manager AutoSync settings page to update multiple data pipelines at the same time. Learn more.
New target endpoints: Databricks, Amazon S3, and Azure Data Lake Storage (ADLS) Gen2.
You can now stop a data pipeline run mid-load. Because this can compromise target data integrity, we recommend stopping a data pipeline only when necessary. This might be, for example, with a data pipeline that takes a long time to load where you need to change the configuration or know that an issue with the source will result in errors. Learn more.
Enhancements
The text on the edit dialog for data pipelines with a CSV file endpoint has been enhanced. Because CSV loads can’t be scheduled, the former AutoSynchronize tab is now Load type. The Append load type clarifies that when you select a file that was previously loaded, the entire file will be appended to the existing table.
The AutoSync header has a new design:
The browser tab now identifies AutoSync:
The data preview table for transformations now shows 5 rows of data instead of 4.
Known Issues
For the ServiceNow endpoint, string and decimal values load to the target using the target’s default precision. MySQL has a default precision of 255, so values with a precision of more than 255 will fail to load to MySQL.
When creating a data pipeline with a CSV file source, the new Merge columns screen is skipped.
Fixed Issues
The Infer data types for CSV files checkbox now only displays for the S3 and SFTP endpoints for CSV files, not for JSON files.
For the ServiceNow endpoint, AutoSync ignores the maximum length defined by ServiceNow for string and decimal types and uses the default precision of the target.
Data Automation
Enhancements
Enhanced the ELT Load Snap’s capabilities to allow loading flat and nested data sets (from canonical and non-canonical formats) from your Parquet files to the target tables in Snowflake. Learn more about the usage of this feature at Load data from Parquet files and in the following example pipelines:
Fixed Issue
Fixed an issue where the ELT Merge Into Snap failed to load data into the Google BigQuery target table because of the error:
Number of source column expressions in the input SQL is greater than the target table columns
(even after altering the target table to match the source table columns). The resolution was to modify an SQL statement that was used internally to achieve the merge action.
Documentation Updates
Content Published on New Documentation Site
You can find the following content on docs.snaplogic.com:
Video
Enhancements
Content about configuring Azure Key Vault as a secrets manager has been validated and improved.
GenAI Builder
Added the Anthropic Claude on AWS Messages Snap to the Amazon Bedrock Snap Pack.
The GenAI Snap Packs are part of the GenAI Builder product and cannot be purchased separately. To purchase GenAI Builder, contact your Customer Success Manager (CSM).
Monitor
New Features
The new Alert history displays all Snaplex node alerts to simplify review and analysis. You can tag events as open or closed to filter the list and you can download the log. Learn more.
The Integrations catalog is available by request for Enterprise customers. Contact either adm-support@snaplogic.com or your CSM. The catalog displays an inventory of integrations for pipelines with an associated task in a production environment. The table includes metadata and you can add custom metadata columns and values. Learn more.
For environments with thousands of pipelines, the Integrations catalog offers:Improved visibility of Integrations as data assets
Better governance for DataOps
Faster impact analysis for System Architects
Easier identification of reusable assets for Developers to reduce integration development time
Enhancements
Summary card counts on the Execution overview page now reflect any filters applied to the table in addition to the date range. For example, if two executions failed in the selected date range and were running on different Snaplexes, if you filter by one of the Snaplex names, the Failed card shows only one failure.
The left navigation and header for Monitor have a new design. You can collapse the left navigation to increase space for the main panel:
The browser tab now identifies Monitor and the name of the current page:
The Activity Log includes usernames and Snaplex version updates when users initiate Snaplex version changes. This also applies to the Activity Log in the Classic Manager. Learn more
Known Issues
With the Execution overview table grouped by Task or Pipeline, if you select a date range for more than one month's data, the execution table does not return results. The summary cards show the counts for executions by status.
The summary card counts do not update correctly when you select the following advanced search filters for Invocation type:
Manual
Nested Pipeline
Error Pipeline
For Each
Regression Test
Platform
New Features
The new public API for the Git Integration to discard local changes uses the project path and asset name to identify the assets to discard. Learn more.
The new public Snaplex APIs enable administrators to:
The public API for users sends a notification email after a user is added successfully to the organization. Learn more.
Enhancements
New support for AWS Secrets Manager to restrict secrets access by SnapLogic projects. To restrict access to specific projects, add role-based IAM aliases for them to a
secrets-config.json
file and restart your Groundplex nodes. Learn more.The header for Designer, Manager, and Dashboard has a new look:
The user settings menu, which is available from a dropdown next to the user’s name, has a new design:
Settings are categorized as global or application-specific settings.
Non-admin users can’t change their username, they must ask an Environment admin to change it.
The screenshot on the left shows the menu for Admin Manager, AutoSync, and Monitor. The screenshot on the right shows the menu for Dashboard and Manager:
Browser tab labels now identify the app and page name:
Open Telemetry reporting to third-party monitoring and logging tools now only sends messages labeled Error in severity level. Previously, messages marked Informational were also included. These Informational messages are still retained in the logs.
Fixed Issues
Fixed an issue for the
toPrecision()
function that converts the data incorrectly when processing a whole number. When you call the function with arguments greater than 100, it results in an error.
Upcoming behavior change
Project-level pipeline execution behavior change
Currently, tasks stored in a Project Space of an Org can be executed by pipelines in a different Project Space or even from requests from a different Org. There is an upcoming fix to stop this behavior in the July release. After the fix is deployed, tasks must belong to the same Project Space as the pipeline Project Space they are called from to be successfully executed.
Known Issues
When creating a Task and selecting the America/Mexico City time zone, the task runtime is off by one hour. Since Mexico City no longer observes daylight savings, the current implementation of CDT (aligned with CST) might assume that Mexico City changed time zones as it did in the past.
Workaround: Upgrade your Snaplex to a JDK 11.0.18+ version.
Snaps
Upcoming
New Snaps
Introduced the following Snaps:
Added MongoDB Execute and MongoDB - Atlas Vector Search Snaps to the MongoDB Snap Pack:
MongoDB Execute: Executes all Data Definition Language (DDL) and Data Manipulation Language (DML) commands on the MongoDB database.
MongoDB - Atlas Vector Search: Performs advanced vector-based queries, such as Similarity searches, Approximate Nearest Neighbor (ANN) queries, and Range queries on vector data stored in MongoDB Atlas.
Added NetSuite Add List and NetSuite Async Add List Snaps to the NetSuite SOAP Snap Pack:
NetSuite Add List: Adds a list of records in bulk to NetSuite.
NetSuite Async Add List: Adds a list of records in NetSuite asynchronously.
Added Parquet Parser and Parquet Formatter Snaps to the Transform Snap Pack:
Parquet Parser: Reads the binary Parquet data and writes document data to the output.
Parquet Formatter: Reads the document data and writes it to the output in binary Parquet format.
Enhancements
The following Azure Active Directory-related Snaps and Accounts have been renamed because Microsoft has rebranded Azure Active Directory to Microsoft Entra ID.
Previous name | New name |
---|---|
Azure Active Directory Snap Pack | |
Azure Active Directory OAuth2 Account | |
Azure SQL Active Directory Account | |
Azure SQL Active Directory Dynamic Account | |
Azure Synapse SQL Active Directory Account |
Enhanced the Google Directory Read Snap with an On empty result dropdown list that enables you to select the Snap behavior when no matching filter value is found.
Enhanced the JSON Splitter Snap to capture metadata and lineage information from the input document.
Kafka Snap Pack: Enhanced the following fields to support expressions through pipeline parameters:
Truststore Password
Keystore File Password
SSL Key Password
SASL Password or Secret
Registry Password or Secret
Enhanced the MongoDB Update Snap with the Array Filters field, which enables you to use array filters in the update operation. Additionally, the Update Query field is now modified into a text box for visibility and usability of input queries.
Increased the default Read timeout value for NetSuite REST Snaps from 60 to 300 seconds. This update supports longer-running operations and prevents early disconnections for complex data processing tasks.
Updated the Delete Condition (Truncates a Table if empty) field in the following Database Delete Snaps to Delete condition (deletes all records from a table if left blank) to indicate that all entries will be deleted from the table when this field is blank, but no truncate operation is performed.
Enhanced the HubSpot Snaps with the Retry policy field to handle retries for connection and error responses.
Updated the following NetSuite SOAP Snap names for usability based on the functionality of the Snaps:
Previous name | New name |
---|---|
NetSuite Create | |
NetSuite Async Delete | |
NetSuite Async Get | |
NetSuite Async Upsert | |
NetSuite Delete | |
NetSuite Upsert |
Enhanced the S3 Select Snap to capture metadata and lineage information from the input document.
Enhanced the Snowflake Bulk Load and Snowflake Insert Snaps to support iceberg tables. Iceberg tables partition and organize data across multiple nodes to distribute the workload and speed up data processing.
Enhanced the SQL Server Snap Pack to support Geospatial data types (Geography and Geometry), which enables you to seamlessly insert, read, and select Geospatial data in the SQL Server environment.
Upgraded Apache CXF from version 3.4.2 to 3.6.3 to prevent vulnerability issues for the following Snap Packs:
Upgraded Spring dependencies to the latest supported Java 11 version for the MongoDB Snap Pack.
Behavior Change
MySQL - Insert Snap: With the 4.37 GA release, when you pass a Date/DateTime
value for the Year
data type column, the MySQL Insert Snap does not accept those values and fails with an error. This is the expected behavior of the Snap; we recommend you to pass only year values for the Year
data type. Previously, when you passed a Date/DateTime
value for the Year data type column, the Insert Snap inserted the value as a Year
data type.
Fixed Issues
Fixed an SMB (server message block) connectivity issue in the Binary Snap Pack, where the incorrect name provided by the SMB client did not match the Windows cluster virtual name, affecting the SPN (service principal name) connection.
Fixed a performance issue in the Copy Item Snap, which previously resulted in a
nameAlreadyExists
error when the Snap retried because of a connection reset.Fixed an issue with the OpenAPI Snap where the upstream Mapper Snap did not display the full schema objects in the Target Schema.
Fixed an issue with the Oracle - Bulk Load Snap that intermittently displayed a
null pointer exception
.Fixed the following issues with the SQL Server Snap Pack:
Snaps displayed an error when numeric values with scale 0 were inserted into the database with
DECIMAL
orNUMERIC
type columns instead of inserting the value with decimal digits truncated.Snaps displayed an error when fractional numeric values (numbers with a decimal point) were inserted into the database with an
INT type
column, instead of inserting the values with decimal digits truncated.
Known Issues
The SQL Server - Bulk Load Snap returns
0 rows copied
for tables containing spatial-type columns.
Deprecated
The Azure Data Lake Accounts have been removed from the Binary and Hadoop Snap Packs because Microsoft retired the Azure Data Lake Storage Gen1 protocol on February 29, 2024. We recommend replacing your existing Azure Data Lake Accounts (in Binary or Hadoop Snap Packs) with other Azure Accounts.
SnapGPT
Improved readability of Pipeline analysis summary and SnapGPT Analyze Pipeline Reports displays.
Each SnapGPT completion displays feedback icons and a copy icon in the following UI contexts:
Generate pipeline: Copies the pipeline to be pasted into a Designer canvas.
Describe pipeline: Copies the summary and pastes it as markdown formatted text.
Question and answer: Copies the SnapGPT response to the question and pastes it as markdown formatted text.
Analyze pipeline: Copies the pipeline analysis and pastes it as JSON.
Error: Copies error information.
Have feedback? Email documentation@snaplogic.com | Ask a question in the SnapLogic Community
© 2017-2024 SnapLogic, Inc.