| Table of Contents | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
...
Added the Snowflake - Snowpipe Streaming Snap
The Kafka MSK IAM Account now supports Cross-Account IAM Access via Transit Gateway and multi-VPC Peering
Added Personal Access Token (PAT) authentication for the Tableau Account and Tableau REST Account to improve flexibility and security of Tableau integrations.
Upgrading to this Release
...
...
Dot Releases: Snaps
Snap Pack | Date | Snap Pack Version | Updates | ||
|---|---|---|---|---|---|
| 439patches29265 |
| |||
| 439patches29230 | Fixed the vulnerabilities in CXF library dependencies within the Salesforce Snap Pack to enhance security and ensure reliable functionality. | |||
| 439patches29164 |
| |||
| 439patches29063 | Enhanced the following accounts in the Snap Pack with the Read timeout (seconds) field that enables the Snap to wait for a response from Azure: | |||
| 439patches29181 | Enhanced the SAP Execute Snap with the Round to SAP decimal checkbox that enables the Snap to round the decimal value to match the number of Decimal Places defined in the SAP target field. This property only applies to the SAP packed decimal fields, which store data in the BCD (Binary Coded Decimal) format. | |||
| 439patches29008 |
Behavior change: Earlier, the ROWID columns were displayed in binary (Base64 encoded data) form in the output. With the | |||
| 439patches29124 |
| |||
| 439patches29078 | Fixed an issue with the CSV Parser Snap that introduced unexpected characters into the records and output data because of incorrect handling of the delimiter. | |||
| 439patches29014 | Fixed an issue with the Salesforce Lookup, SOQL, and SOSL Snaps, where errors were not routed to the error pipeline when Route Error Data to Error View was enabled, and a referenced expression variable was missing in the upstream or pipeline parameters. | |||
| 439patches29008 | Fixed an issue with Oracle Execute, Select, and Stored Procedure Snaps where the data was displayed in an unexpected structure and without the actual row address value when querying table columns of
| |||
AutoSync | Nov 19, 2024 | 439patches29077 438patches29076 | Fixed an issue loading to Databricks. To consume this fix:
|
Upgrading to this Release
| Info |
|---|
To use the new SnapLogic features and Snaps in the current GA release, ensure that your Snaplexes are upgraded to the recommended version: All Snaplexes on older versions will be upgraded to the recommended version during the mandatory automatic upgrade, which is scheduled for 3 PM (PDT) on Saturday, . For other Snaplex versions released this year, view the SnapLogic Release calendar. |
...
System Configuration now includes the Git Integration page for selecting a Git provider. This supports continuous integration and continuous development (CI/CD) across different environments. Learn more.
In the Security section, configure Enhanced Account Encryption (EAE) from the Account encryption page. With a subscription to EAE and use of a Groundplex, sensitive account data is encrypted with a public key in the SnapLogic domain control plane and decryped decrypted with a private key on the Groundplex. Learn more.
In the Application settings section, configure Asset catalog visibility. The Asset catalog is available to enterprise SnapLogic customers by request. Learn more.
...
Added the following Snaps to the Azure OpenAI Snap Pack :
Add Message: Adds messages to an existing thread.
Create Thread: Creates a thread by making an HTTP request to the Assistant API.
Create and Run Thread: Creates a thread and retrieves the result on a specified assistant.
Run Thread: Executes the specified thread and retrieves the result on a specified assistant.
Azure OpenAI Tool Calling: Provides external tools for the model to call, supplying internal data and information for the model's responses.
Azure OpenAI Function Result Generator: Formats the results generated by user-run functions into a unified structure.
Azure OpenAI Function Generator: Generates a tool definition that can be used for tool calling in the chat completions endpoint.
Azure OpenAI Multimodal Content Generator: Generates multimodal content for the user prompt with Azure OpenAI.
Flow:
Added the PipeLoop Snap that enables you to loop on pipeline executions based on a stop condition or iteration limit. The execution of this Snap enables the parent pipeline to control and manage iterative processes using the child pipeline for scenarios where repeated processing is required until a specific stop condition is met.
Added the following Snaps to the Amazon Bedrock LLM Snap Pack:
Amazon Bedrock Converse API Tool Calling: Provides external tools for the model to call, supplying internal data and information for the model's responses.
Amazon Bedrock Function Result Generator: Formats the results generated by user-run functions into a unified structure.
Amazon Bedrock Function Generator: Generates a tool definition in the output to inform the model about the tools that you want to call.
Amazon Bedrock Multimodal Content Generator Snap: Generates multimodal content for the user prompt with Amazon Bedrock.
Added the Google GenAI Multimodal Content Generator Snap that generates multimodal content for the user prompt with Google GenAI.
Added the Message Appender Snap to the LLM Utilities Snap Pack that enables you to collect all the results from the Result Generator Snaps and append them to the
messagesorcontentslist in the output of the Tool Calling Snap.
Added the following Snaps to the OpenAI LLM Snap Pack:
OpenAI Add Vector Store File: Adds an existing file from OpenAI storage into the specified vector store using the vector store and file IDs.
OpenAI List Files: Retrieves and lists all files associated with the OpenAI account.
OpenAI List Vector Store Files: Retrieves and lists all vector store files based on the specified vector store ID.
OpenAI Remove Vector Store File: Removes a file from the specified vector store using the file ID.
OpenAI Upload File: Uploads a file to the specified vector store using the vector store and file IDs.
OpenAI Tool Calling: Provides external tools for the model to call, supplying internal data and information for the model's responses.
OpenAI Function Result Generator: Formats the results generated by user-run functions into a unified structure.
OpenAI Function Generator: Generates tool definitions and informs the model about the tools that you can call.
OpenAI Multimodal Content Generator: Generates multimodal content for the user prompt with OpenAI.
...
Enhanced the following Snaps with Advanced response configurations. These configurations generate precise customization of response behavior, enabling finely tuned outputs that align with specific content requirements and user intent:
Enhanced the Amazon Bedrock Account and Unstructured API Account with the Read timeout (seconds) field that enables the Snap to wait for a response from Amazon Bedrock and the Unstructured API.
Enhanced the OpenAI API Key Account with the Project ID field that enables you to connect to the OpenAI APIs, ensuring that the correct organization and project are being accessed for API usage.
Enhanced the following Snaps with the Stop sequences field to support stop sequences as a model parameter. You can specify the sequence of tokens or text to indicate to the model when to stop generating text:
Enhanced the following Snaps with the support of multimodal inputs:
Enhanced the following Snaps to support incoming content from their respective Multimodal Content Generator Snaps:
...
On the APIM Metrics/API Dashboard, Alerts no longer display the error,
unable to instantiate policy.
AutoSync
New Features
The new SQL Server Bulk Load endpoint provides better performance for data pipelines that synchronize large datasets. This endpoint requires:
...
The data pipeline must run on a Groundplex configured to use the Microsoft SQL Server Bulk Copy Utility, bcp. Before using this endpoint in an AutoSync data pipeline, you must install the utility on each Groundplex node.
...
New Features
The new SQL Server Bulk Load endpoint provides better performance for data pipelines that synchronize large datasets. This endpoint requires:
The data pipeline must run on a Groundplex configured to use the Microsoft SQL Server Bulk Copy Utility, bcp. Before using this endpoint in an AutoSync data pipeline, you must install the utility on each Groundplex node.
The
438patches28952AutoSync Snap Pack. Refer to AutoSync settings for information on selecting a Snap Pack for a data pipeline.
Fixed Issues
Fixed an issue loading to Databricks. To consume this fix:
For a version 4.39 AutoSync Snaplex or Groundplex, upgrade data pipelines with Databricks as a target to use the AutoSync 439patches29077 Snap Pack.
For a version 4.38 AutoSync Snaplex or Groundplex, upgrade data pipelines with Databricks as a target to use the AutoSync 438patches29076 Snap Pack.
Documentation Enhancements
...
Learn how to create an APIM API version using assets in the Git Repository
Compare the interface for node metrics in Dashboard with the new one in Monitor: Dashboard to Monitor - Node Metrics
New Content
Enhanced the OAuth 2.0 Client Credentials article with an Okta integration example.Node Metrics
Monitor
Enhancements
The Integration catalog, available to enterprise SnapLogic customers by request, is now the Asset catalog. It includes the following enhancements:
With the new API described in the Platform section, you can add custom metadata columns to the table of tasks and add, update, or delete existing custom metadata values from a task.
With the new configuration described in the Admin Manager section, you can allow all users view-only access to the Asset catalog or restrict it to admins.
The new Pipelines tab displays standard pipeline metadata:
Task details are easier to read, with important information moved to the header and execution statistics split out into a new tab:
The Custom tab on the Task details pane includes the ability to edit custom metadata values:
...
The Asset catalog Edit metadata functionality in the Custom tab of task details has an issue. When editing, if you click Clear to remove an existing value and then click Cancel, the value is deleted instead of being retained as expected.
Workaround: Re-enter the cleared value and click Save.The average number of documents processed for pipelines that contain a BigQuery Execute Snap is being incorrectly reported as zero. This impacts both the Asset catalog and the Execution overview.
In the Asset catalog Pipelines tab, when you filter by the Path column, a search might not return all pipelines that match.
The Asset catalog Execution statistics tab for task and pipeline details currently displays no data.
Fixed Issues
The Node status and Node type filters are no longer missing from the System overview node view.
The documentation Graph descriptions page clarifies which graphs represent total values and which show the average percentage of the value.
...
The Third-party Observability documentation now provides detailed instructions for configuring OpenTelemetry and a Groundplex deployed on Docker to provide logging information to New Relic.
Fixed Issues
The runtime pusher (the mechanism by which the pipelines are initialized before they process data) was improved for better node performancebetter node performance.
Pipelines were failing because the Snaplex JCC ran out of the connections, resulting in the error:
Timeout waiting for connection from pool.Pipeline executions were taking longer due to the slow preparation of the pipelines.
On the APIM Metrics/API Dashboard, alerts were displaying the error:
unable to instantiate policy.
RFC Compliance Enforcement
...
Important Updates from Snowflake
Deprecated Drivers:
As of February 1, 2024, Snowflake has ceased support of older driver versions in compliance with its Support Policy. You must upgrade to the latest driver version to access important security, performance, and functionality improvements. Ensure that your Snowflake pipelines use the Snowflake JDBC Driver v3.13.27 or a higher version. Learn how to check your current driver version. The Snowflake Snap Pack is bundled with the 3.16.0 JDBC driver, but if you are using your own version, it may be deprecated in accordance with Snowflake Guidelines.
Security Policies:
Snowflake recently signed the Cybersecurity and Infrastructure Security Agency (CISA) Secure By Design Pledge. In line with CISA’s Secure By Design principles, Snowflake recently announced several security enhancements in their platform — most notably the general availability of Trust Center and a new Multi-Factor Authentication (MFA) policy.
Effective October 2024, MFA will be enforced by default for all human users in any new Snowflake account. Service users (accounts designed for service-to-service communication) will not be subject to this MFA requirement.
Basic Authentication Accounts:
If you currently use Basic authentication to access Snowflake, we recommend one of the following:
Update your password to a maximum of 14 characters and a minimum of 8, and ensure it does not match any of the previous five passwords. Learn more.
Migrate to Key-Pair authentication: This method involves generating a public and private key pair, where the private key is securely stored and the public key is registered with your Snowflake account.
Migrate to OAuth2 authentication: This is a widely adopted standard for secure delegated access. It allows applications to obtain limited access to user accounts on an HTTP service, facilitating safer and more manageable authentication flows.
Implementing the above recommendations will secure your data and protect sensitive data, ensuring compliance with Snowflake's updated security policies. Learn more. For questions, contact support@snaplogic.com.
Breaking Changes
| Note |
|---|
JDBC Snap Pack:
Reltio Account deprecation by the end of October 2024
|
...
Enhanced the GraphQL Client Snap with the Include extensions checkbox to support extensions that enable you to access query content and extension data in a structure similar to the GraphQL response. The non-standard fields in the GraphQL response are now ignored to avoid a possible error.
The GraphQL Client Snap now includes an improved error-handling mechanism that emulates how errors are reported in the raw GraphQL response. This enhancement provides additional context and pagination to the error output, which was previously limited to one page. Learn more.
Enhanced the S3 Download Snap with an Enable Staging checkbox that enables you to download an S3 object into a local temporary file. This enhancement addresses the cases where some downstream Snaps take longer to process large volumes of data, potentially causing a connection reset error and pipeline failure.
Flow:
Added the PipeLoop Snap that enables you to loop on pipeline executions based on a stop condition or iteration limit. The execution of this Snap enables the parent pipeline to control and manage iterative processes using the child pipeline for scenarios where repeated processing is needed until a specific stop condition is met.
Enhanced the Kafka MSK IAM Account to support Cross-Account IAM Access on Amazon MSK Cluster configured through the Transit Gateway and multi VPC Peering.
Enhanced the Kafka MSK IAM Account to support debugging for the IAM configuration. When you enable the IAM debugging (Global properties and Logging Level) in the Snaplex configuration, the account logs the IAM credential identity in the Snaplex log. Learn more.
Enhanced the MongoDB Replica and Mongo ReplicaSet Dynamic Accounts to define read preference options when querying data. The default option is Primary, so you cannot allocate the read load to the secondary node. Note that the Secondary preferred mode is not supported for the MongoDB Execute Snap.
Removed the Route connection errors checkbox from the RabbitMQ Producer and RabbitMQ Consumer Snaps. As a result, any existing RabbitMQ pipelines that are configured with Route connection errors checkbox enabled will stop functioning as before.
Suggested workaround
To run your pipelines successfully, you must manually Route the errors to the error view under the Views tabThis change should not impact your existing pipelines because the Route Error Data to Error View is now enabled by default.
Added 5672, the default RabbitMQ server value, for the Port field in the RabbitMQ Account.
Enhanced the Tableau Account and Tableau REST Account with Personal Access Token (PAT) authentication for flexibility and security of Tableau integrations. This enhancement ensures compatibility with MFA (Multi-Factor Authentication) or SSO (Single Sign-on) environments. PAT authentication is the default authentication type for the Tableau REST Account.
...