July 2024 Release Notes

Highlights

The SnapLogic July 2024 Release includes the following highlights:

Admin Manager

  • Tag environments with an icon

AutoSync

  • New transformation to mask sensitive data

Documentation Updates

  • Use cases added for GenAI App Builder and the NetSuite SOAP Snap Pack

GenAI App Builder

  • Adobe PDF Snap Pack

  • Unstructured Snap Pack

  • Advanced prompt configuration support for OpenAI LLM, Azure OpenAI LLM, and Amazon Bedrock LLM Snap Packs

  • Vector query support for Google AlloyDB, PostgreSQL, and Snowflake Snap Packs

  • Added the Amazon OpenSearch Account to the OpenSearch Snap Pack

  • Added Azure OpenAI Entra ID Application, Azure OpenAI Entra ID User, and Azure OpenAI Entra Token Accounts to the Azure OpenAI LLM Snap Pack

Monitor

  • Additional information in pipeline execution details for individual Snaps

Platform

  • New public API to download and save the slpropz file

Snaps

  • The ELT Load Snap now supports loading data from nested AVRO, JSON Lines, ORC, or Parquet files

  • The SSL OAuth2 Account in the API Suite Snap Pack now supports sending client information as a header to the OAuth endpoint

  • The Pipeline Execute Snap now supports expression values via pipeline parameters for the Batch size field

  • The Snowflake Snap Pack now supports:

    • Grant types in OAuth2 Accounts

    • Vector data types

SnapGPT

  • Pipeline descriptions now utilize content in pipeline notes

July Snaplex version: main-25388 - 4.37 Patch 3.

Dot Releases - Platform

Date

Component

Version

Description

Date

Component

Version

Description

Aug 14, 2024

Platform

N/A

  • Fixed an issue with the OAuth2 password grant type to make sure that the Snap OAuth2 Account token refreshes successfully every 20 minutes.

Aug 5, 2024

Snaplex

4.37-26011 - 4.37 Patch 6 (Deprecated)

  • Fixed an issue where a FeedMaster node steadily consumed memory when using a load balancer for Groundplex Triggered Tasks.

  • Fixed an issue where certain OAuth account names would change during background maintenance.

  • Fixed an issue where private Snap Packs could not be accessed.

Dot Releases - Snaps

Snap Pack

Date of Update

Snap Pack Version

Updates

Snap Pack

Date of Update

Snap Pack Version

Updates

SQL Server

Aug 14, 2024

437patches27732

Fixed an issue with the SQL Server - Select Snap configured with an Ultra task to handle temporary database unavailability. The Snap now validates the database connection to ensure that it is reestablished after temporary database outages.

Microsoft Dynamics 365 for Sales

Aug 13, 2024

437patches27711

Fixed an issue with the Dynamics 365 for Sales Execute Snap that encountered a NullPointerException error for a 204 No Content return status. Now a custom output document is successfully returned for a 204 No Content status.

Confluent Kafka

Aug 13, 2024

437patches27730

Fixed an issue with the Kafka Consumer Snap where the Seek Type of Beginning or End (skip all existing messages) may not work correctly after a rebalance. 

API Suite

Aug 8, 2024

437patches27592

Updated the API Suite Snaps to process authentication information from incoming documents to support features such as Secrets manager-based credentials.

Databricks

Aug 8, 2024

437patches27246

Added the following Snap to the Databricks Snap Pack:

  • Databricks - Run Job: Executes a job, checks its status in Databricks, and, based on the job's status, completes or fails the pipeline.

Snowflake

Aug 7, 2024

437patches27531

Enhanced the Snowflake - Select Snap with Time Travel query support that enables you to access historical data at any point within a defined period. This includes restoring data-related objects, duplicating and backing up data, and analyzing data usage.

PostgreSQL

Aug 7, 2024

437patches27531

Enhanced the Postgres and Postgres Dynamic Accounts with support for IAM and Cross-account IAM role authentication.

API Suite

Aug 7, 2024

437patches27587

Fixed an issue with the HTTP Client Snap that caused NullPointerException when the Retry-After header contained a decimal value in the 429 (Too Many Requests) response.

RabbitMQ

Aug 6, 2024

437patches27616

Fixed the following issues with the RabbitMQ Producer Snap:

  • With no access to the virtual hosts, the Snap displayed a null pointer exception.

  • The Snap did not display the original payload when the error view was enabled.

Azure OpenAI LLM

Aug 1, 2024

437patches27527

Added the following accounts to the Azure OpenAI LLM Snap Pack:

OpenSearch

Aug 1, 2024

437patches27473

Added the Amazon OpenSearch Account to the OpenSearch Snap Pack to authenticate with OpenSearch services using the specified Access key ID and Secret access key. This account also provides Cross-account IAM role support.

PDF

Aug 1, 2024

437patches27596

Fixed an issue with the PDF Parser Snap that could not parse multiple incoming PDF documents and processed only the first document (without errors).

Hadoop

Jul 30, 2024

437patches27226

The upgrade of the Azure Storage library from v3.0.0 to v8.3.0 has impacted the Hadoop Snap Pack causing the following issue when using the WASB protocol.

Known Issue

When you use invalid credentials for the WASB protocol in Hadoop Snaps (HDFS Reader, HDFS Writer, ORC Reader, Parquet Reader, Parquet Writer), the pipeline does not fail immediately, instead it takes 13-14 minutes to display the following error:

reason=The request failed with error code null and HTTP code 0. , status_code=error

SnapLogic® is actively working with Microsoft®Support to resolve the issue.

Learn more about the Azure Storage Library Upgrade.

SQL Server

Jul 30, 2024

437patches27328

Fixed an issue with the SQL Server Insert Snap that caused an error when inserting the infinity value into a decimal-type column.

MongoDB

Jul 30, 2024

437patches27343

The MongoDB - Atlas Vector Search Snap now supports the following:

  • Suggestions for the Search index field that enables the Snap to populate the associated indices in the list.

  • The input schema displays the mandatory vector field and optional filter suggestion (if the Search index contains a filter type query) in alignment with the fields expected by the Snap.

Shopify

Jul 29, 2024

437patches27471

Fixed a resource leak issue with the Shopify Snaps, which involved too many stale instances of ProxyConnectionManager and significantly impacted memory utilization.

Facebook

Jul 29, 2024

437patches27471

Fixed a resource leak issue with the Facebook Snaps, which involved too many stale instances of ProxyConnectionManager and significantly impacted memory utilization.

Hadoop

Jul 29, 2024

437patches27471

Fixed a resource leak issue with the following Hadoop Snaps, which involved too many stale instances of ProxyConnectionManager and significantly impacted memory utilization:

JDBC

PostgreSQL

Jul 29, 2024

437patches27416

Fixed an issue with the Generic JDBC - Insert and PostgreSQL - Insert Snaps that exposed sensitive information in the error message when the batch operation was not successful.

PostgreSQL

Google AlloyDB

Jul 29, 2024

437patches27416

Fixed an issue with the PostgreSQL - Bulk Load and AlloyDB Bulk Load Snaps that caused incorrect or invalid binary data to be inserted when the column was of Binary type.

Kafka

Jul 26, 2024

437patches27404

Fixed an issue with the Kafka SSL Account that displayed an error when the Kafka Consumer Snap could not commit offsets during the rebalancing process. This behavior avoids unstable partition allocation. Now, the Snap logs a warning, allowing the Kafka Consumer to continue processing instead of displaying an exception.

Azure SQL

Jul 25, 2024

437patches27180

Fixed the following issues with the Azure SQL - Bulk Load Snap:

  • The Snap displayed an error when the DateTime was of the LocalDateTime type.

  • The Snap lost milliseconds when the DateTime was in the String data type because we were parsing the given DateTime format into a Date object and creating the timestamp object from that date.

ELT

Jul 23, 2024

437patches27372

  • Enhanced the pipeline execution statistics of the ELT Insert-Select Snap to be displayed in its output view and to allow downloading detailed stats as a JSON file that includes additional statistics (extraStats) on DML statement executions on target Databricks Lakehouse Platform (DLP) tables.

OpenAI LLM

Jul 22, 2024

437patches27319

  • Enhanced the OpenAI Prompt Generator Snap to output the entire prompt message array and user role selection for the prompt.

  • Enhanced the OpenAI Chat Completions Snap with message payload to generate responses based on a specified list of messages.

Azure OpenAI LLM

Jul 22, 2024

437patches27297

Amazon Bedrock LLM

Jul 22, 2024

437patches27322

Salesforce

Jul 19, 2024

437patches27307

Fixed authentication issues with the Salesforce Subscriber Snap in the Ultra Pipeline to ensure the Snap reauthenticates and reconnects as expected.

Anaplan

Jul 19, 2024

437patches27283

Fixed an intermittent connection reset issue with the Anaplan Snaps caused by the absence of the keep-alive header in the server response.

ELT

Jul 16, 2024

437patches27246

Enhanced the ELT Execute Snap to display statistics for all SQL statements executed. The statistics are shown in both the pipeline execution statistics and the output view of the Snap. The Snap also allows you to download detailed stats as a JSON file that includes additional statistics (extraStats) on DML statement executions on target Databricks Lakehouse Platform (DLP) tables.

Binary

Jul 15, 2024

437patches27146

Enhanced the Azure Storage Account with Managed Identity, which provides applications with an automatically managed identity for connecting to resources that support Microsoft Entra ID authentication.

Google Sheets

Jul 12, 2024

437patches27197

Fixed an issue with the Google Sheets Writer Snap where using cell names as worksheet names caused incorrect data writing behavior.

Admin Manager

Enhancements

  • Admin Manager categories and pages have been re-organized for clarity. The Environment settings category was removed and the left navigation pane now includes the following categories:

    • Application settings include the Designer page for validation and data preview configuration. The SnapGPT and AutoSync pages moved to this category from the bottom of the menu.

    • General settings: The Environment page contains settings moved from the General settings page and the Snaplexes page moved from the Environment settings category.

    • Security: Allowlists moved from Environment settings.

      nav-comp-new.png

  • From the new Environment tag tab on the Environment page, customize environments with icons that show up in the environment selector. Learn more.

    environment-tag.png
  • The Snaplex configuration file (.slpropz) is no longer available as a URL to enhance security. It can now only be downloaded directly.

API Management

Enhancements

  • The Trust all certificates checkbox is now available in the Generic OAuth2 and the Callout Authenticator Policies. Enable this functionality to bypass the certificate validation process and allow the request to proceed successfully.

  • Session management for the Generic OAuth2 policy returns the session ID and user or client details when the API call is made. Learn More.

Known Issues

The July Release Snaplex versions have the issues listed below:

Snaplex Version: main-25388 - 4.37 Patch 3

  • The Request Transformer policy does not get applied during the API call.

  • Headers are not modified by the API Proxy’s mapping rules.

Snaplex Version: main-25654 - 4.37 Patch 4

  • When the outbound account uses the Authorization Header with scheme type bearer it adds an extra space between the scheme and the bearer token making some upstream servers recognize the token as invalid, including Task authentication

To avoid these known issues, we recommend that you upgrade your Snaplex versions for APIM to 4.37-26011 Patch 6.

AutoSync

New Features

  • Use the new Mask data transformation to hide sensitive data in the target table. In contrast with the Filter columns transformation, Mask data retains the column so that the source and target schema are identical. Learn more.

Fixed Issues

  • The Merge column transformation now enforces rules for targets that require specific column name casing. For example, Oracle requires all uppercase names, and Amazon Redshift requires all lowercase. When you enter a name for the new column, AutoSync only accepts the correct case.

Known Issues

  • When you save a new merged column without changing the name and then change it and save again, the changed name might not be preserved.
    Workaround: Change the merged column name before saving the merge.

Documentation Updates

New Content

New Videos

  • In this video, let us see how SnapGPT analyzes your pipeline and provides a comprehensive list of critical errors, warnings, and informational messages for your pipeline and Snaps.

Content Published on New Doc Site

GenAI App Builder

New Features

  • Adobe PDF Snap Pack: Enables you to integrate PDF functionalities into applications using Adobe’s REST APIs. It contains the following Snaps and Account:

    • Extract: Extracts text, table, and figures from a PDF file.

    • OCR: Adds a scannable layer on top of an image-based PDF file.

    • Adobe Account: Authenticates with Adobe APIs using the specified Client ID and Client secret.

  • Unstructured Snap Pack: Enables seamless integration with the Unstructured API to extract structured data from raw unstructured documents and transform that data to inject into your application, data, and other generative AI needs. It contains the following Snap and Account:

    • Partition API: Extracts structured data from raw unstructured documents using the Unstructured API and transforms it into structured data.

    • Unstructured API Account: Authenticates with the Unstructured API using the specified credentials.

  • Vector Search Snaps: The following Snaps enable advanced vector-based queries using the SELECT statement.

Enhancements

Monitor

Enhancements

  • Improved performance for the group by Pipeline and Task aggregations in the Monitor Execution overview.

  • Monitor now saves specific selections across sessions. When you switch pages or log out and log in across different user sessions, Monitor saves some selections that you made on the initial page. For example, the list view/map view in the Infrastructure page and Column Preferences in the Execution overview are saved across sessions. Additionally, Monitor saves selections in the current session for the time window.

  • In the pipeline details Snap Statistics tab, hover over a Snap to see additional information such as the status, Snap ID, and Snap Pack Build:

Platform

New Features

  • New public API to download and save the slpropz file. Learn more.

Enhancements

  • Users can now configure sending Slack and email notifications when a task is delayed.

Change in Behavior

  • Project-level pipeline execution: Before this release, tasks stored in a Project Space could execute pipelines in a different Project Space or by requests from a different Org. With the current release, tasks must belong to the same Project Space as the pipeline to execute successfully. Learn more.

  • The Snaplex configuration file (.slpropz) is no longer available as a URL in the user interface to enhance security. Instead, you can download it from the Snaplex > Downloads tab by clicking or with the API GET /api/1/rest/plex/config/<path_to_groundplex>. Learn more.

Known Issues

  • When you call a Triggered Task on a Groundplex using the Cloud URL with a payload of 26 MB or larger, the pipeline processing can exceed the WebSocket connection timeout.
    While the pipeline succeeds in delivering the payload, the bytes after the 26MB limit might be lost on the response side.
    If the payload is on the request side, then only the first 26 MB of bytes are sent to the pipeline, causing the possibility of pipeline failure.

Snaps

Upcoming

Enhancements

  • Enhanced the ELT Load Snap to support loading data from nested AVRO, JSON Lines, ORC, or Parquet files in Azure storage to the target tables in a Databricks Lakehouse Platform (DLP) instance. Only two load actions are supported: Drop and create table and Append table.

  • The JDBC driver for the Google AlloyDB Snap Pack is upgraded from v42.4.2 to v42.7.2 (Java 8). This upgrade will be part of the GA release on August 14, 2024 (Stable release). As part of this upgrade, the 42.7.2 JDBC driver is bundled with the AlloyDB Snap Pack as the default JDBC driver. Learn more.

  • Enhanced the SSL OAuth2 Account (API Suite Snap Pack) with the Send Client Data as Basic Auth header checkbox to enable you to send the client information as a header to the OAuth endpoint.

  • Enhanced the MongoDB Execute Snap with the Timezone Offset field set to enable you to apply the timezone offset on the date fields.

  • Enhanced the Pipeline Execute Snap to support expression values via pipeline parameters for the Batch size field that enables you to use intermediate pipelines for generic use cases.

  • Upgraded the PostgreSQL JDBC driver from v9.4.1207 to v42.7.2 (Java 8). This upgrade will be part of the GA release on August 14, 2024 (Stable release). Learn more.

Behavior Changes

The JDBC driver upgrade for the PostgreSQL and the Google AlloyDB Snap Packs resulted in the following changes:

  • Some error messages will now include the data type, for example:

    • Old: "...where (badId = 1) was aborted:..."

    • New: "...where (badId = ('1'::numeric)) was aborted:..."

  • There are some minor text changes to a few of the error messages, for example:

    • Old: "Hint: No operator matches the given name and argument types."

    • New: "Hint: No operator matches the given name and argument type(s)."

This upgrade has also resulted in specific behavior changes in errors, status codes, and success and failure messages. Learn more.

Fixed Issues

  • The SELECT SQL statement in the ELT Merge Into Snap could not fetch the information for the target table from the PG_TABLE_DEF catalog table of the Amazon Redshift instance. To fetch the values from the catalog table, the SELECT SQL statement now uses lowercase for both non-delimited and delimited table names and excludes double quotes from delimited table names.

  • Conflicts between internally used Azure libraries prevented the ELT Load Snap from reading files when using the ADLS Gen 2 Account.

  • The File Poller Snap applied a case-sensitive filter for the Windows operating system.

  • In the HTTP Client Snap, URLs with spaces that were not properly encoded caused runtime errors.

  • Inconsistency in ObjectId and Date representation in the output preview between MongoDB - Execute and MongoDB - Find Snaps.

  • The SharePoint Online Snap Pack failed to delete temporary files after Snap completion.

  • The SharePoint Online - Read List Snap displayed a null pointer exception when handling pagination.

  • The case-sensitivity issue with the SQL Server—Bulk Load Snap caused a NullPointerException when retrieving the PATH environment variable in a Windows environment. This fix is applicable only on the Windows operating system, because environment variables are case-sensitive in Linux.

SnapGPT

Enhancements

  • The responses generated from the Describe this Pipeline feature now use content from pipeline notes.

  • The placeholder text in the prompt field changes according to the selected SnapGPT operation.

  • Generating a pipeline now chooses relevant pipelines in your workspace to use as examples for improved quality.