Highlights

The SnapLogic July 2024 Release includes the following highlights:

Admin Manager

• Tag environments with an icon

AutoSync

• New transformation to mask sensitive data

Documentation Updates

• Use cases added for GenAI App Builder and the NetSuite SOAP Snap Pack

GenAI App Builder

• Adobe PDF Snap Pack

• Unstructured Snap Pack

• Advanced prompt configuration support for OpenAI LLM, Azure OpenAI LLM, and Amazon Bedrock LLM Snap Packs

• Vector query support for Google AlloyDB, PostgreSQL, and Snowflake Snap Packs

• Added the Amazon OpenSearch Account to the OpenSearch Snap Pack

• Added Azure OpenAI Entra ID Application, Azure OpenAI Entra ID User, and Azure OpenAI Entra Token Accounts to the Azure OpenAI LLM Snap Pack

Monitor

• Additional information in pipeline execution details for individual Snaps

Platform

• New public API to download and save the slpropz file

Snaps

• The ELT Load Snap now supports loading data from nested AVRO, JSON Lines, ORC, or Parquet files

• The SSL OAuth2 Account in the API Suite Snap Pack now supports sending client information as a header to the OAuth endpoint

• The Pipeline Execute Snap now supports expression values via pipeline parameters for the Batch size field

• The Snowflake Snap Pack now supports:

  • Grant types in OAuth2 Accounts

  • Vector data types

SnapGPT

• Pipeline descriptions now utilize content in pipeline notes

Dot Releases - Platform

Dot Releases - Snaps

Admin Manager

Enhancements

• Admin Manager categories and pages have been re-organized for clarity. The Environment settings category was removed and the left navigation pane now includes the following categories:

  • Application settings include the Designer page for validation and data preview configuration. The SnapGPT and AutoSync pages moved to this category from the bottom of the menu.

  • General settings: The Environment page contains settings moved from the General settings page and the Snaplexes page moved from the Environment settings category.

  • Security: Allowlists moved from Environment settings.
    
    

    Open

    

• From the new Environment tag tab on the Environment page, customize environments with icons that show up in the environment selector. Learn more.
  
  

  Open

  

• The Snaplex configuration file (.slpropz) is no longer available as a URL to enhance security. It can now only be downloaded directly.
  
  

API Management

Enhancements

• The Trust all certificates checkbox is now available in the Generic OAuth2 and the Callout Authenticator Policies. Enable this functionality to bypass the certificate validation process and allow the request to proceed successfully.

• Session management for the Generic OAuth2 policy returns the session ID and user or client details when the API call is made. Learn More.

Known Issues

AutoSync

New Features

• Use the new Mask data transformation to hide sensitive data in the target table. In contrast with the Filter columns transformation, Mask data retains the column so that the source and target schema are identical. Learn more.
  
  

Fixed Issues

• The Merge column transformation now enforces rules for targets that require specific column name casing. For example, Oracle requires all uppercase names, and Amazon Redshift requires all lowercase. When you enter a name for the new column, AutoSync only accepts the correct case.

Known Issues

• When you save a new merged column without changing the name and then change it and save again, the changed name might not be preserved.
  Workaround: Change the merged column name before saving the merge.

Documentation Updates

New Content

• GenAI App Builder Use Case: Legal Contract Analysis

• Snaps Use Case: Transfer data from Salesforce to NetSuite

• Deploy OTEL as a Daemonset on Kubernetes to log Groundplex metrics and alerts

New Videos

• In this video, let us see how SnapGPT analyzes your pipeline and provides a comprehensive list of critical errors, warnings, and informational messages for your pipeline and Snaps.

Content Published on New Doc Site

• Groundplex Node Management

GenAI App Builder

New Features

• Adobe PDF Snap Pack: Enables you to integrate PDF functionalities into applications using Adobe’s REST APIs. It contains the following Snaps and Account:

  • Extract: Extracts text, table, and figures from a PDF file.

  • OCR: Adds a scannable layer on top of an image-based PDF file.

  • Adobe Account: Authenticates with Adobe APIs using the specified Client ID and Client secret.

• Unstructured Snap Pack: Enables seamless integration with the Unstructured API to extract structured data from raw unstructured documents and transform that data to inject into your application, data, and other generative AI needs. It contains the following Snap and Account:

  • Partition API: Extracts structured data from raw unstructured documents using the Unstructured API and transforms it into structured data.

  • Unstructured API Account: Authenticates with the Unstructured API using the specified credentials.

• Vector Search Snaps: The following Snaps enable advanced vector-based queries using the SELECT statement.

  • AlloyDB - Vector Search

  • PostgreSQL - Vector Search

  • Snowflake - Vector Search

Enhancements

• Enhanced the Anthropic Claude on AWS Messages, Azure OpenAI Chat Completions, and OpenAI Chat Completions Snaps with the Advanced prompt configuration field set that enables the Snap to parse JSON objects to include them in the output and configure system prompts to guide the responses or actions.

• Added vector data type support for the PostgreSQL - Insert and PostgreSQL - Update Snaps.

• Added the Amazon OpenSearch Account to the OpenSearch Snap Pack to authenticate with OpenSearch services using the specified Access key ID and Secret access key. This account also provides Cross-account IAM role support.

• Added the following accounts to the Azure OpenAI LLM Snap Pack:

  • Azure OpenAI Entra ID Application Account: Uses applications to access Azure resources securely using application-level permissions for service principal authentication.

  • Azure OpenAI Entra ID User Account: Uses user/delegate level permissions associated with the user profile logged in during authorization.

  • Azure OpenAI Entra Token Account: Uses token-based authentication to interact with Azure OpenAI APIs.

Monitor

Enhancements

• Improved performance for the group by Pipeline and Task aggregations in the Monitor Execution overview.

• Monitor now saves specific selections across sessions. When you switch pages or log out and log in across different user sessions, Monitor saves some selections that you made on the initial page. For example, the list view/map view in the Infrastructure page and Column Preferences in the Execution overview are saved across sessions. Additionally, Monitor saves selections in the current session for the time window.

• In the pipeline details Snap Statistics tab, hover over a Snap to see additional information such as the status, Snap ID, and Snap Pack Build:
  
  

Platform

New Features

• New public API to download and save the slpropz file. Learn more.

Enhancements

• Users can now configure sending Slack and email notifications when a task is delayed.

Change in Behavior

• Project-level pipeline execution: Before this release, tasks stored in a Project Space could execute pipelines in a different Project Space or by requests from a different Org. With the current release, tasks must belong to the same Project Space as the pipeline to execute successfully. Learn more.

• The Snaplex configuration file (.slpropz) is no longer available as a URL in the user interface to enhance security. Instead, you can download it from the Snaplex > Downloads tab by clicking or with the API GET /api/1/rest/plex/config/<path_to_groundplex>. Learn more.

Known Issues

• When you call a Triggered Task on a Groundplex using the Cloud URL with a payload of 26 MB or larger, the pipeline processing can exceed the WebSocket connection timeout.
  While the pipeline succeeds in delivering the payload, the bytes after the 26MB limit might be lost on the response side.
  If the payload is on the request side, then only the first 26 MB of bytes are sent to the pipeline, causing the possibility of pipeline failure.

Snaps

Upcoming

Enhancements

• Enhanced the ELT Load Snap to support loading data from nested AVRO, JSON Lines, ORC, or Parquet files in Azure storage to the target tables in a Databricks Lakehouse Platform (DLP) instance. Only two load actions are supported: Drop and create table and Append table.

• The JDBC driver for the Google AlloyDB Snap Pack is upgraded from v42.4.2 to v42.7.2 (Java 8). This upgrade will be part of the GA release on August 14, 2024 (Stable release). As part of this upgrade, the 42.7.2 JDBC driver is bundled with the AlloyDB Snap Pack as the default JDBC driver. Learn more.

• Enhanced the SSL OAuth2 Account (API Suite Snap Pack) with the Send Client Data as Basic Auth header checkbox to enable you to send the client information as a header to the OAuth endpoint.

• Enhanced the MongoDB Execute Snap with the Timezone Offset field set to enable you to apply the timezone offset on the date fields.

• Enhanced the Pipeline Execute Snap to support expression values via pipeline parameters for the Batch size field that enables you to use intermediate pipelines for generic use cases.

• Upgraded the PostgreSQL JDBC driver from v9.4.1207 to v42.7.2 (Java 8). This upgrade will be part of the GA release on August 14, 2024 (Stable release). Learn more.

• Snowflake:

  • Enhanced the Snowflake Azure OAuth2, Snowflake Google Storage OAuth2, and Snowflake S3 OAuth2 accounts with the Grant type dropdown list that enables you to authorize your account using the available (Authorization code, Client credentials, Password) grant types.

  • Updated the tooltip for the Encrypted Private Key Passphrase property in the Snowflake Azure Database Account for improved clarity.

  • Enhanced the Snowflake Snap Pack with full vector support, including schema generation and output compatibility with vector data types.

  • Enhanced the Snowflake Bulk Load and Bulk Upsert Snaps with full vector support that enables you to load and upsert data in vector data types.

  • Upgraded the JDBC driver to v3.16.0.

• The jOOQ library for the Vertica Snap Pack is upgraded from v3.9.1 to v3.17.x. Learn more. 

Behavior Changes

The JDBC driver upgrade for the PostgreSQL and the Google AlloyDB Snap Packs resulted in the following changes:

• Some error messages will now include the data type, for example:

  • Old: "...where (badId = 1) was aborted:..."

  • New: "...where (badId = ('1'::numeric)) was aborted:..."

• There are some minor text changes to a few of the error messages, for example:

  • Old: "Hint: No operator matches the given name and argument types."

  • New: "Hint: No operator matches the given name and argument type(s)."

This upgrade has also resulted in specific behavior changes in errors, status codes, and success and failure messages. Learn more.

Fixed Issues

• The SELECT SQL statement in the ELT Merge Into Snap could not fetch the information for the target table from the PG_TABLE_DEF catalog table of the Amazon Redshift instance. To fetch the values from the catalog table, the SELECT SQL statement now uses lowercase for both non-delimited and delimited table names and excludes double quotes from delimited table names.

• Conflicts between internally used Azure libraries prevented the ELT Load Snap from reading files when using the ADLS Gen 2 Account.

• The File Poller Snap applied a case-sensitive filter for the Windows operating system.

• In the HTTP Client Snap, URLs with spaces that were not properly encoded caused runtime errors.

• Inconsistency in ObjectId and Date representation in the output preview between MongoDB - Execute and MongoDB - Find Snaps.

• The SharePoint Online Snap Pack failed to delete temporary files after Snap completion.

• The SharePoint Online - Read List Snap displayed a null pointer exception when handling pagination.

• The case-sensitivity issue with the SQL Server—Bulk Load Snap caused a NullPointerException when retrieving the PATH environment variable in a Windows environment. This fix is applicable only on the Windows operating system, because environment variables are case-sensitive in Linux.

SnapGPT

Enhancements

• The responses generated from the Describe this Pipeline feature now use content from pipeline notes.

• The placeholder text in the prompt field changes according to the selected SnapGPT operation.

• Generating a pipeline now chooses relevant pipelines in your workspace to use as examples for improved quality.