Highlights

The SnapLogic October 2024 Release includes the following highlights:

GenAI App Builder

• Added the Amazon Bedrock Converse API Snap

• Added the OpenAI Add Vector Store File, OpenAI List Files, OpenAI List Vector Store Files, OpenAI Remove Vector Store File, and OpenAI Upload File Snaps to the OpenAI LLM Snap Pack

Monitor

• The Execution overview now displays up to 500 child pipeline executions

• Enhanced the System overview list view with information on CPU, disk, and memory use

Snaps

• Added the Kafka MSK IAM Account to the Kafka Snap Pack

• Enhanced the GraphQL Client Snap with the Include extensions checkbox and improved the error handling in the Snap

Dot Releases: Platform

To take advantage of the following fixes, upgrade the Snaplexes in your environment (Org) to the designated version.

Dot Releases: Snaps

Admin Manager

Enhancements

• The note in the UI for the Update Snaplex dialog box in the Downloads tab was updated to clarify that only the latest version download links are available for first-time installation.

API Management

Known Issues

• Managing API keys with Secrets Management is temporarily unavailable, and we will be re-enabling it in a future release

Fixed Issues

• URL encoding did not work when applied to a Proxy’s endpoint query parameters. For this fix, your Snaplex version should be main-27456 - 4.38 Patch 2.

AutoSync

Known Issues

• The AutoSync Snap Pack (438patches28270 and newer) now correctly maps Zendesk and Generic JDBC source NVARCHAR fields or columns to NVARCHAR. However, for existing target tables, change capture does not detect that existing VARCHAR fields and columns should be changed to NVARCHAR. To apply the fix to existing tables, follow the steps described in the second fixed issue in the next section.

Fixed Issues

• The data preview for the Replace empty values screen no longer displays data from the wrong table when the selected table has no nullable columns.

• For Zendesk and Generic JDBC sources, NVARCHAR fields or columns now load in the target correctly. To apply this fix:

  1. Upgrade data pipelines with Zendesk or Generic JDBC sources to use the 438patches28270 (or newer) AutoSync Snap Pack.

  2. Do one of the following:

     1. Edit the appropriate data pipelines to use Full load, which drops the tables and reloads them. If necessary, change to Incremental or SCD2 after the first load.

     2. Leave the load type as Incremental or SCD2 and manually drop the appropriate tables in the target.

Documentation Enhancements

New Videos

• Dashboard to Monitor - Checking Snaplex health

  Open

  

• Generate a Pipeline with SnapGPT

  Open

  

New Content

• A new article describes the workflow for troubleshooting API request issues by performing deep dive analysis from the API Metric page.

• This new tutorial explains how to assess the quality of your GenAI applications with our evaluation pipeline patterns:

  • GenAI App Builder - Evaluation Pipeline - Amazon Bedrock Edition

  • GenAI App Builder - Evaluation Pipeline - Azure OpenAI

GenAI App Builder

New Features

• Added the Amazon Bedrock Converse API Snap that enables you to generate message responses using the specified Converse API model and model parameters. This Snap supports all foundation models available in Amazon Bedrock.

• Added the following Snaps to the OpenAI LLM Snap Pack:

  • OpenAI Add Vector Store File: Adds an existing file from OpenAI storage into the specified vector store using the vector store and file IDs.

  • OpenAI List Files: Retrieves and lists all files associated with the OpenAI account.

  • OpenAI List Vector Store Files: Retrieves and lists all vector store files based on the specified vector store ID.

  • OpenAI Remove Vector Store File: Removes a file from the specified vector store using the file ID.

  • OpenAI Upload File: Uploads a file to the specified vector store using the vector store and file IDs.

Enhancements

• Enhanced the Amazon Bedrock Account with the Read timeout (seconds) field that enables the Snap to wait for a response from Amazon Bedrock.

• Enhanced the following Snaps with the Stop sequences field to support stop sequences as a model parameter. You can specify the sequence of tokens or text to indicate to the model when to stop generating text:

  • Anthropic Claude on AWS Chat Completions

  • Anthropic Claude on AWS Messages

  • Azure OpenAI Chat Completions

  • Google Gemini Generate

  • OpenAI Chat Completions

Known Issues

• For OpenAI List Vector Store Files, OpenAI documentation states that cancelled is a supported value for the Filter field; however, using this value might result in a status code 400 error from the endpoint.

Workaround: Use only the following supported values: completed, in_progress, and failed.

• OpenAI Remove Vector Store File Snap may indicate a successful deletion in the output; however, the file might still appear in the output of the OpenAI List Vector Store Files Snap. This is a known bug from OpenAI. Learn more.

Monitor

Enhancements

• The Execution overview table now displays up to 500 child pipeline executions. The previous limit in the ungrouped view was 100.

• The System overview list view for nodes now includes information on CPU, disk, and memory usage:

• The System overview list view now allows you to choose between all Snaplexes, all nodes, active nodes, or inactive nodes, as shown above. The graph view can display all Snaplexes, active nodes, or inactive nodes, as shown in the following screenshot:

• The Node diagnostics table in the node details panel now organizes information to be easier to scan and interpret. The columns contain the current, recommended, minimum, and maximum values and the values out of the recommended range display in red in the top rows:

Known Issues

• The Node status and Node type filters are missing from the System overview node view.

Platform

Enhancements

• Task notifications now identify the environment (Org), Project Space, and project that contains the pipeline. When a pipeline is deleted before it runs, the notification displays the Project Space and project that contained the pipeline with a pipeline deleted message.

Fixed Issues

Behavior Change

• Task names no longer support commas (,). Starting with this release, entering a comma in the Name field in the Task dialog results in a UI error message. Tasks existing before the October 2024 release are not affected, unless you modify the Task.

RFC compliance enforcement

Snaps

Upcoming

New Snaps/Accounts

• Added the Kafka MSK IAM Account, which enables you to use IAM access control for secure communication with the MSK cluster. You can securely integrate your Kafka pipelines with the MSK cluster.

Enhancements

• API Suite:

  • Enhanced the GraphQL Client Snap with the Include extensions checkbox to support extensions that enable you to access query content and extension data in a structure, similar to the GraphQL response. The non-standard fields in the GraphQL response are now ignored to avoid a possible error.

  • The GraphQL Client Snap now includes an improved error-handling mechanism that emulates how errors are reported in the raw GraphQL response. This enhancement provides additional context and pagination to the error output, which was previously limited to one page. Learn more.

• Amazon S3:

  • Enhanced the S3 Download Snap with an Enable Staging checkbox that enables you to download an S3 object into a local temporary file. This enhancement addresses the cases where certain downstream Snaps take longer to process large volumes of data, potentially causing a connection reset error and pipeline failure.

Breaking Changes

Behavior Changes

• The GraphQL Client Snap now includes an improved error-handling mechanism that emulates how errors are reported in the raw GraphQL response. This enhancement provides additional context and pagination to the error output, which was previously limited to one page. Learn more.

Known Issues

• When the Generic JDBC - Select and Generic JDBC - Execute Snaps connect to the Sybase database to retrieve BigTime type data, the Snaps display both date and time for the data type.

• When you insert or update multiple records to the derive views using the Generic JDBC-Insert and Generic JDBC-Update with the Denodo instance, the Snaps fail with [Failure: Batch operation failed, Reason: Batch operation failed]error.
  Workaround:

  • You must set the Batch size to 1 in the Generic Database Account when inserting multiple records. 

Deprecated

• The Reltio Account is deprecated and no longer available. Any existing Reltio pipelines that use the Reltio Account for API access will not be supported by Reltio. To ensure the continuity of your pipelines, we recommend that you use the Reltio OAuth2 Account.

SnapGPT

Enhancements

• Improved the feedback mechanism for interactions.

• Better quality and increased accuracy of the SnapGPT Q&A skill through applying a chunking strategy to the reference documents.

Fixed Issues

• The Resize SnapGPT panel would not wrap properly around the Knowledge Skills window.

Behavior Change

• If SnapGPT responses cannot provide an accurate response to a prompt, you receive the following message:
  I’m not 100%  confident about this answer but here are some related resources.