You can now build your pipelines using simple text prompts.
August UAT Delta
This section tracks the changes made during the iterative pushes to the UAT server and the GA release. The following is the release schedule:
UAT: Friday, August 11, 2023 – Snaplex version: main-19739 - 4.34 GA (Recommended)
GA: Wednesday, August 16, 2023 – Snaplex version: main-19739 - 4.34 GA (Recommended)
The SnapLogic GA release build on the production server is also deployed on the UAT server.
About Upgrading to this Release
To use the new SnapLogic features and Snaps in the current GA release, ensure that your Snaplex nodes are upgraded to the recommended version.
This release includes stable versions of the monthly Snap patches deployed to the SnapLogic platform after the May 2023 release, as listed in the Dot Releases section of the July Release Notes.
Per the SnapLogic Release calendar, all remaining Snaplexes across Orgs are automatically upgraded to the recommended version. The automatic upgrade ends approximately at 4 PM (Pacific Standard Time) on Saturday, Sep 16, 2023. Orgs migrated to the current GA release before the automatic upgrade are not impacted. We strongly recommend that you use the recommended version for your Snaplexes.
Fixed an issue with the SQL Server Stored Procedure and Table List Snaps that caused no retries to be executed even though the user had selected a number of retries in the UI when a custom JDBC driver was used.
Fixed an issue with the Oracle-Bulk Load Snap that was not resilient to the errors previously when trying to auto-discover the existing SQLLDR utility paths in the node. The Snap is now robust to those errors.
Fixed an issue with the Microsoft OneDrive Snap Pack where the Snaps displayed Duplicate key OneDrive error. The Snaps now hide the drive details stored in the PreservationHoldLibrary folder when displaying suggestions. You can use the Drive ID to access the PreservationHoldLibrary folder.
Fixed an issue with the Mircosoft OneDrive Snaps that displayed a JsonNull error when the shared library list was empty. Now, it displays the message No Suggestions available.
The JSON key field in the Binary Google Service Account supports JSON strings. You can upload the JSON key either from SLDB or dynamically pass the value using a pipeline parameter or access values from Secrets Manager.
Enhanced the Parquet Writer Snap with the support for LocalDate and DateTime. The Snap now displays the schema suggestions for LocalDate and DateTime correctly.
Enhanced the Parquet Reader Snap with the Use datetime types checkboxthat supports LocalDate and DateTime datatypes.
When you select the Use datetime types checkbox in the Parquet Reader Snap, the Snap displays the LocalDate and DateTime in the output for INT32 (DATE) and INT64 (TIMESTAMP_MILLIS) columns. When you deselect this checkbox, the columns retain the previous datatypes and display string and integer values in the output.
Fixed an issue with the HTTP Client Snap that displayed incorrect cookie information, with a null value in the cookies when you have duplicate cookies in the header. The Snap now displays the cookie information correctly.
Fixed an issue with the Pipeline Execute Snap that caused a null pointer exception during the retry when the input view was a Binary type. Retry is not supported for a Binary input view with this Snap.
Fixed an issue with the Google Analytics 4 Create Report and Create Realtime Report Snaps that improperly rounded off decimal data. You can now view the precise value in the report.
Admin Manager is a new interface for administrators with an updated user experience based on feedback from you. The classic IIP Manager will be phased out when all settings become available in Admin Manager.
Open Admin Manager from the application menu:
This release introduces a subset of features. Manage users, groups, service accounts, and roles from Admin Manager. Learn more.
Admin Manager introduces new concepts and terminology:
Orgs are called environments, and an Org admin is called an Environment admin.
Roles are used instead of groups to differentiate admins from regular users.
Groups provide a way to share assets such as AutoSync data pipelines, AutoSync credentials, and IIP Account assets.
On the Roles screen, the count of users for each role does not match the number shown on the details page. The details page opens when you click a role.
When you delete a user and get a confirmation notice, the deleted user might display in the list for a short time while their assets are being deleted.
The APIM dashboard displays more data for Ultra requests. The dashboard displays the number of requests waiting in the queue for each minute, every FeedMaster node, and the total number of requests across all FeedMaster nodes. You can now view the average wait time, average processing time, and average total time for each Ultra Task.
The API dashboard also introduces a new Queue Size widget that displays the number of requests inline to be processed per time segment.
In the Response Mapper and Response Transformer policies, APIM functions display in the expressions dropdown menu but do not actually work.
AutoSync now supports the following endpoints:
Google Sheets as a source
Snowflake as a source
Google BigQuery as a source
Detection of changes to source data types: Now AutoSync detects changes to source column or field data types and updates the destination table with a compatible type. Supported for Amazon Redshift, Google BigQuery, and Snowflake destinations. Learn more.
You can now append data to a table created from a CSV file by splitting the file. This is useful, for example, when the data exceeds the 100 MB size limit or when a system periodically generates CSV files and you want them to load to the same table. Learn more.
New experience for sharing data pipelines. Learn more.
The Snowflake destination endpoint now uses internal Snowflake staging, which removes the need to have and configure a separate staging environment.
The Amazon Redshift endpoint now supports using an S3 bucket for staging that is in a different AWS region. If the S3 bucket is in the same region as the Redshift instance, you can leave the region unspecified.
When a data pipeline with Snowflake as a destination encounters an error, it stops loading. It should instead log the error record and continue.
Upgraded the JDBC driver support for Snowflake to snowflake-jdbc-3.13.33.jar.
Updated the ELT Snowflake Account to append or replace (if it exists) the connection parameter application in its JDBC URL with the value SnapLogic_iPaaS.
This change complies with Snowflake’s latest Connection Identifier standard.
The Studio dashboard displays nodes that are missing a heartbeat in a gray color and saves the details for a period of 45 days. Learn more.
Introducing Secrets Cache support for Secrets Management that minimizes latency resulting from fluctuating requests to all the Supported Secrets Management providers.
Support for Git Lab Integration with your Org.
Connectivity to on-premises GitHub Enterprise Servers (GHES). You can now configure the new app gateway to connect your Groundplex to your on-premises GHES host. Previously, connectivity required IP filtering on requests to GHES from the SnapLogic Control Plane or endpoint restrictions, where a limited subset of API endpoints was available.
Secrets stored in the Azure Key Vault – and not keys and certificates.
Managed identity-based authentication on the Azure Active Directory (Azure AD): System-assigned identities and User-assigned identities.
Snaplex node crashes are now captured as events in the Activity Log.
Git repositories now link with Project Spaces. All Git operations supported in projects are also supported at the Project Space level.
Project Spaces Git Operations Menu - Left Navigation Tree
GitHub Operations Menu - Project Asset Toolbar
Project Spaces Git Operations Menu - Left Navigation Tree
GitHub Operations Menu - Project Asset Toolbar
When you associate a repository with a Project Space, all the Git folders are brought in as Projects, and any assets at the root level are placed in a logical directory labeled as root. This label does not interfere with the corresponding Git folder structure. Learn more.
You can add each individual untracked asset to the repository for tracking or remove one from the repository using the Remove from Repository option on that asset’s name.
The IIP toolbar no longer collapses and the Export Pipeline icon has been updated. The following shows the Designer toolbar:
The Import a Pipeline icon has been updated:
LDAP authentication to HashiCorp Vaults. This enhancement might affect existing accounts based on HashiCorp Vaults. Refer to the following breaking change and the action needed.
With the new support for LDAP authentication to HashiCorp Vaults, the Snaplex node configuration file secrets-config.json requires a new key, auth_method, with a value approle or ldap.
You must update each of your existing Snaplex node configuration files to include this key-value pair: "auth_method": "approle", as prescribed in the Step 1a-Approle, for the approle-based authentication to succeed.
The Runtime Archive does not display content.
The API response for Snaplex Configuration REST API gives the incorrect Snaplex version to download. Instead of the recommended Snaplex version, the response gets the latest version. Workaround: Use the Snaplex dialog Downloads tab for the recommended version.
In GitHub, you cannot create a branch doing a Git checkout from the repository at the Project Space level.
The Git status Updated does not work at the Project space level for all Git repos.
In GitHub, when a root folder is already untracked in a Project Space after doing the checkout, the root (1) folder is not created.
For GitLab integration, when updating the Git assets with the status Tracked Modified locally and remotely, a conflict error message displays.
For GitLab integration, while creating a Project from a Git checkout, an error message displays.
For GitLab integration, doing a Git Repository Checkout with Hard Reset results in an internal server error.
If a self-managed GitLab server does not have GitLab as part of the hostname, Status, Pull, Commit, Create Branch, and Create Tag commands fail.
Pipelines that use the default Microsoft JDBC Driver in the SQL server accounts might be impacted after the upgrade because the latest driver is backward-incompatible. For more details, refer to the FAQ for Upgrading to SQL Server JDBC Driver 12.2.0. To prevent an impact to your accounts, reconfigure the URL properties field in account settings as follows: trustServerCertificate = true
We plan to upgrade the Oracle Snap Pack to use the latest 19c JDBC JAR file (220.127.116.11 as of November 8, 2023).
Pipelines that use the default JDBC driver in the Oracle server accounts might be impacted after the driver upgrade because the latest driver is backward-incompatible. We will soon communicate how to reconfigure to prevent accounts from being impacted.
Pipelines that do not use the default driver (bundled with the Oracle Snap Pack) are not impacted.
The following new Snaps and Snap Packs are available with the August 2023 stable distribution. To access these new Snaps, enable the auto-upgrade mode or manually upgrade your Snaplex. Snaplexes are upgraded automatically 4 weeks after GA.
The Amazon S3 Snaps automatically detect the Maximum session duration value for the Cross-Account IAM role (1 through 12 hours). The Snaps round down the value to the nearest hour. So, if the Snap administrator sets the Maximum session duration at 3 hours and 45 minutes, the Snaps read it as 3 hours. The Snaps also refresh the session before the session expires. However, the automatic session refresh does not support very large file uploads or downloads that takes longer than the maximum session duration.
The Decrypt Field and Encrypt Field Snaps from the Transform Snap Pack now support CTR (Counter mode) for the AES (Advanced Encryption Standard) block cipher algorithm.
The Decrypt Field Snap now supports the decryption of various encrypted fields by providing a valid decryption key.
The Deduplicate Snap now manages memory efficiently and eliminates out-of-memory crashes using the following fields:
Minimum memory (MB)
Minimum free disk space (MB)
The pagination feature implemented for the ABFS protocol in the Hadoop Directory Browser Snap enables the Snap to return all the output documents as expected.
The Dynamics 365 for Sales Snaps includes a new Enable continue on error checkbox to allow the Snap to continue executing the remaining batch requests, even if the previous request does not execute.
The Dynamics365 for Sales Update Snap performs the update operation as expected when the Batch size is more than 1, and the Snap successfully executes when the batch size is 1.
The NetSuite Snap Pack now supports the Customer-Subsidiary and the Vendor-Subsidiary relationship Object types. Note: NetSuite's application requires the Oneworld extension for using these subsidiary relationships, and Snap integration must use WSDL 2019_1 or later versions.
The PGP Encrypt Snap now includes an Encryption key ID field, which allows you to specify the key ID for encrypting the data. If you do not specify an encryption key ID, the Snap uses the first key in the public key (master key).
The Snowflake - Bulk Load and Snowflake - Bulk Upsert Snaps are enhanced with the Flush chunk size (in bytes) field. When using internal staging, data is written to a temporary chunk file. When the file exceeds the specified size, it is copied to the Snowflake stage and deleted.
This Snap Pack is upgraded to the Microsoft JDBC Driver 12.2.0.jre11 for SQL Server. Existing pipelines that do not use the default driver (bundled with the SQL Server Snap Pack) are not impacted. If you want to use NTLM or other authentications with the latest default driver (where the DLL file is mandatory), then you must copy the DLL file to the Snaplex path. Learn more about the SQL Server driver upgrade: FAQ for Upgrading to SQL Server JDBC Driver 12.2.0.
Fixed an issue with the Send Email Snap where attachments larger than 3 MB were not getting attached to the sent email messages.
Starting from version 433patches21482, the PGP Encrypt Snap does not support encryption with an expired key. To ensure continued support for encryption, we highly recommend that you renew your PGP key before it expires or extend the expiration date of your PGP key.
Starting from version 433patches22343, the Vertica-Bulk Load Snap displays no output in the output view when the Snap receives no input.
With the 433patches21307, pagination evaluation in the HTTP Client Snap occurs before sending the output to the output view. Therefore, if snap.out.totalCount is part of the Has next or any other pagination properties, the Snap will not include the count in the output corresponding to the request that produced the output document.
If you use the PostgreSQL driver (org.postgresql.Driver) with the Redshift Snap Pack, it could result in errors if the data type provided to the Snap does not match the data type in the Redshift table schema. Either use the Redshift driver (com.amazon.redshift.jdbc42.Driver) or use the correct data type in the input document to resolve these errors.
The Send Email Snap might fail because of a 429 error response from the Graph API endpoint.
For Ultra Tasks using a continuous data flow design for polling with a Pipeline Execute Snap, the parent pipeline does not receive the document from the child pipeline, and the polling message is not generated for downstream Snaps.
The GeoJSON Parser and WKT Parser Snaps accept only application/zip or application/gml+xml; charset=utf-8 as a file input stream. However, the output content type from the File Reader Snap is binary/octet when the input file is read from SLDB.
Workaround: To read an input file (ZIP or GML) from SLDB, use the Mapper Snap to change the content type from binary/octet to one of the following:
ZIP file: Change the content type to application/zip .
GML file: Change the content type to application/gml+xml; charset=utf-8.
The Microsoft Dynamics AX Snap Pack is deprecated. Contact firstname.lastname@example.org if your existing pipelines use Snaps from the deprecated Snap Pack.
Enhanced the following Custom Snap documents by adding missing information, improving clarity in field descriptions, and implementing the latest UI changes:
Enhanced the Hyper Lib Setup and Tabcmd installation pages in the Tableau Snap Pack to enable you to perform tasks such as publishing workbooks and data sources, scheduling extract refreshes, creating users and groups, managing permissions, and interacting with Tableau Server or Tableau Online without relying on the Tableau user interface.