The SnapLogic® May 2023 Release includes the following:
Support for SOAP endpoints in Proxies
New endpoints, Google BigQuery, Amazon S3 (for CSV and JSON files), Marketo, and CSV files
Synchronize with SCD2 between Salesforce and Snowflake
For a Salesforce source, AutoSync automatically handles changes to the column precision, scale, or character limit by propagating the changes to the target.
The new Pipeline Cache feature (private beta) enables you to leverage in-memory lookup
New APIM Public APIs allow you to publish, unpublish, deprecate, or retire a set of API versions
Support for CyberArk Conjur in Secrets Manager
Introduced the GraphQL Client Snap in the API Suite Snap Pack, which enables you to query a GraphQL endpoint and retrieve data
Added the Dynamics 365 Finance and SCM Create, Update, and Delete Snaps to the Microsoft Dynamics 365 Finance and Supply Chain Management Snap Pack
Improved accessibility
New article about pipeline dependency on the control plane
Enhancements in Data Automation articles
Improved documentation of Snaps developed by the Professional Services team
New APIM video about managing your Developer Portal
This section tracks the changes made during the iterative pushes to the UAT server and the GA release. The following is the release schedule:
The SnapLogic GA release build on the production server is also deployed on the UAT server. The SnapLogic GA release build on the production server is also deployed on the UAT server. |
|
We plan to upgrade the Oracle Snap Pack to the latest JDBC JAR file 19c (19.18.0.0) in the November 2023 release.
Pipelines that use the default JDBC driver in the Oracle server accounts might be affected after the driver upgrade because the latest driver is backward-incompatible. We will soon communicate how to reconfigure to prevent accounts from being impacted.
Pipelines that do not use the default driver (bundled with the Oracle Snap Pack) are not impacted.
We plan to upgrade the Oracle Snap Pack to the latest JDBC JAR file 19c (19.18.0.0) in the November 2023 release.
Pipelines that use the default JDBC driver in the Oracle server accounts might be affected after the driver upgrade because the latest driver is backward-incompatible. We will soon communicate how to reconfigure to prevent accounts from being impacted.
Pipelines that do not use the default driver (bundled with the Oracle Snap Pack) are not impacted.
The Dynamics 365 for Sales Snaps include a new Enable continue on error checkbox to allow the Snap to continue executing the remaining batch requests, even if the previous request does not execute. (433patches21467)
The Dynamics365 for Sales Update Snap performs the update operation as expected when the Batch size is more than 1, and the Snap successfully executes when the batch size is 1. (433patches21467)
The pagination feature implemented for the ABFS protocol in the Hadoop Directory Browser Snap enables the Snap to return all the output documents as expected. (433patches21494)
The PGP Encrypt Snap now includes an Encryption key ID field, which allows you to specify the key ID for encrypting the data. If you do not specify an encryption key ID, the Snap uses the first key in the public key (master key). (433patches21482)
The following Transform Snaps include new fields to improve memory management: Aggregate, Group By Fields, Group By N, Join, Sort, Unique. (433patches21461)
The SQL Server Snap Pack is upgraded to the Microsoft JDBC Driver 12.2.0.jre11 for SQL Server. Existing pipelines that do not use the default driver (bundled with the SQL Server Snap Pack) are not impacted. If you want to use NTLM or other authentications with the latest default driver (where the DLL file is mandatory), then you must copy the DLL file to the Snaplex path. (433patches21386)
The Microsoft JDBC Driver 12.2.0.jre11 for SQL Server is currently not available with the latest distribution. However, you can still consume this patch by selecting the 433patches21386 version from the Version list in the Snap Packs page. This update is scheduled to be included in the upcoming release on July 12, 2023, and will be a part of the stable distribution with the GA release on August 9, 2023.
The Microsoft JDBC Driver 12.2.0.jre11 for SQL Server is currently not available with the latest distribution. However, you can still consume this patch by selecting the 433patches21386 version from the Version list in the Snap Packs page. This update is scheduled to be included in the upcoming release on July 12, 2023, and will be a part of the stable distribution with the GA release on August 9, 2023.
The SQL Server Snap Pack now incorporates an identifying connection property allowing the SQL Server instance to identify the source of connections for auditing purposes. (433patches21535)
The applicationName property added to all the SQL Server connections is not available with the latest distribution. However, you can still consume this patch by selecting the 433patches21535 version from the Version list in the Snap Packs page. This update is scheduled to be included in the upcoming release on July 12, 2023.
The applicationName property added to all the SQL Server connections is not available with the latest distribution. However, you can still consume this patch by selecting the 433patches21535 version from the Version list in the Snap Packs page. This update is scheduled to be included in the upcoming release on July 12, 2023.
Starting from version 433patches21482, the PGP Encrypt Snap will no longer support encryption with an expired key. To ensure continued support for encryption, we highly recommend extending the expiration of your PGP key.
Pipelines that use the default Microsoft JDBC Driver in the SQL server accounts might be impacted after the upgrade because the latest driver is backward-incompatible. To prevent an impact to your accounts, reconfigure the URL properties field in account settings as follows:trustServerCertificate = true
With the 4.33 GA release, we no longer support some algorithms for SFTP connection negotiation because we’ve updated the library used to connect to SFTP sources and to improve security. We are working on a fix to support the removed algorithms automatically. With Snap Pack version 433patches21645, you can modify the global properties. Refer to the Configuration Settings for Snaps for details about how to revert to the previous settings.
The SFTP protocol is now incompatible with some deprecated algorithms. This affects mostly the Binary Snap Pack, but might also apply to the Email Snap Pack. (433patches21482)
SFTP connections may experience connectivity issues due to the removal of some deprecated algorithms. To add these connections back, refer to Configuration Settings for Snaps for details.
Snap Pack | Date of Update | Snap Pack Version | Updates |
|---|---|---|---|
| 433patches21630 | Enhanced the performance of the Databricks - Insert Snap to improve the amount of time it takes for validation. | |
| 433patches21644 | Improved an error message in the Remote Python Script Snap to explain the reason and resolution when a Python script has errors. | |
| 433patches21744 | Fixed an issue with the Pipeline Execute Snap where a missing label produced a null pointer exception. | |
| 433patches21488 | Enhanced the LDAP Search and LDAP Lookup Snaps with a new Include string representation for binary attributes checkbox to add a string representation for binary attributes. | |
| 433patches21645 | With the 4.33 GA release, we no longer support some algorithms for SFTP connection negotiation because we’ve updated the library used to connect to SFTP sources and to improve security. With Snap Pack version | |
| 433patches21572 | The Deduplicate Snap now manages memory efficiently and eliminates out-of-memory crashes using the following fields:
| |
| 433patches21586 | The Decrypt Field Snap now supports the decryption of various encrypted fields on providing a valid decryption key. | |
| 433patches21576 | Fixed the issues with the AES Encrypt and AES Decrypt Snaps, where the Snaps previously did not include the error stack trace in the error view. The Snaps now provide detailed information in case of any issues or errors. | |
| 433patches21612 | Fixed the following issues with the Active Directory Update Entry Snap where the Snap produced incorrect results with the version main21015:
| |
| 433patches21541 | Fixed an issue with the Bulk Extract and Bulk Import Snaps where the API name was inaccessible when expressions were enabled for the Marketo Entity field and Custom Objects were selected. | |
| 433patches21349 | The Azure Active Directory Snap Pack now incorporates an identifying header allowing the Azure portal to identify the source of API requests for auditing purposes. | |
| 433patches21482 |
| |
| 433patches21386 | The SQL Server Snap Pack is upgraded to the latest version of the Microsoft JDBC Driver
The Microsoft JDBC Driver The Microsoft JDBC Driver | |
| main132 | Enhanced the COBOL Copybook Parser Snap to improve performance, data validation, and error handling. | |
| 433patches21298 | Fixed an issue with the PostgreSQL Insert Snap that inconsistently inserted some columns and missed the remaining columns(especially the Time fields), when the data was passed in the JSON format from an upstream Snap. | |
| 433patches21295 | Fixed an issue with the Marketo Bulk Extract Snap where the Snap failed to extract data from the CSV file when the data consisted of the backslash (\) character. | |
| 433patches21346 | Fixed an issue with the HTTP Client Snap where the validation of the SSL Account for a specified Keystore or Truststore value failed despite providing the correct password. | |
| 433patches21370 |
| |
| 433patches21336 | Fixed an issue with the AutoPrep Snap where dates could potentially be rendered in a currency format because currency format options were displayed for the DOB column. | |
| 433patches21367 |
| |
| 433patches21247 | Fixed an issue with the Match Snap where a null pointer exception was thrown when the second input view had fewer records than the first. | |
| 433patches21291 | Fixed an issue with the Multi File Reader Snap where it failed with the error | |
| 433patches21307 | Fixed an issue with the HTTP Client Snap that caused pagination to fail when the next Snap in the pipeline could modify the input document (for example, the Mapper Snap). | |
| 433patches21196 | Enhanced the In-Memory Lookup Snap with the following new fields to improve memory management and help reduce the possibility of out-of-memory failures:
These new fields replace the Maximum memory % field. | |
| 433patches21196 | Fixed an issue with the Pipeline Execute Snap where no input view produced a null pointer exception. | |
| 433patches21179 |
| |
| main131 | Addressed a bug in the compatibility mode of the COBOL Copybook Parser Snap that caused a working pipeline to fail. | |
| 433patches21140 |
| |
| 433patches21119 | Updated the SQL Server - Bulk Load Snap to preserve empty strings as empty strings and null as nulls. | |
| 433patches21040 | Provided a fix to ensure that the Data Validator Snap works with "unknown" data types. | |
| 433patches21070 | Fixed an issue with the Kafka Consumer Snap that caused it to skip records in a partition when a pipeline failed without successfully processing any records from that partition. This was an edge case that was only possible with a specific configuration of the Snap and a specific ordering of events during termination of the pipeline. |
Support for SOAP web services in Proxies. You can now reference a SOAP service when you create a Proxy endpoint.
Refer to the Platform section for Lifecycle Management Public APIs. You can now use Public API to publish, unpublish, deprecate, and retire APIs.
Added the Trust All Certificates checkbox in the Create proxy endpoint dialog. Similar to the setting in the REST Snaps, you can choose to allow your Proxy to trust self-signed, unsigned, or expired certificates in the endpoints.
API policy violations are now delivered in JSON. When a client executes an API that results in an authorization or authentication error, the error is returned in the JSON format.
Logging added to the Callout Authenticator API policy. You can now use the Snaplex logs to troubleshoot why the authentication failed when your Callout Authenticator policy rejects a client request.
Improved error output for the JSON Validator API policy. The API caller can easily determine which part of the JSON failed validation.
Wildcard endpoint support for dynamic base paths and error handling in Proxy endpoint mapping. When the base path is a path parameter, the Proxy uses this endpoint and routes through the mappings.
Improved error handling in the Developer Portal API specification editor. Specification violations now appear in the editor.
Added the Time Unit field in the Client Throttle API policy. You can now define a Request Limit value in requests per second.
API Manager now supports asynchronous API migration with Public APIs.
Previously selected HTTP methods are not cleared when you reopen the Create proxy endpoint dialog and select the REST to SOAP checkbox.
Workaround: Refresh the browser.
Fixed an issue where the Try it functionality did not work for an API Proxy published on the Developer Portal.
AutoSync now supports the following endpoints:
Google BigQuery as a target. Learn more.
Marketo as a source. Learn more.
CSV or JSON files stored in Amazon S3 as a source. Learn more.
CSV file upload from a local or network drive as a source. Learn more.
The ability to change load type for synchronization. The types available depend on what AutoSync supports for the source and endpoint. AutoSync supports loading from Salesforce to Snowflake using SCD2.
To change the load type after creating a data pipeline, edit it. From the Auto Synchronize tab, select the Load type. Learn more.
For a Salesforce source, AutoSync automatically handles changes to the column precision, scale, or character limit by propagating the changes to the target. Learn more.
Improved synchronization scheduling. As the last step in the wizard, choose to:
Continue without scheduling.
Pick a preset schedule and set the start time.
Create a custom schedule, which allows you to set an end date.
If you remove or rename a source table or object after using it in a data pipeline, its name will still be visible in the source configuration and you will not be able to de-select it.
For Amazon S3 data sources, you must add a forward slash if you copy the URL from the Amazon S3 console. For example, change s3://my_bucket to s3:///my_bucket.
The first row of CSV files must define the names for the column headings in the target table, with no empty or null values.
AutoSync cannot upload a CSV file to the cloud data warehouse if the file name includes single or double quotes or a special character, such as #, $, %, or &.
Google BigQuery does not support special characters for names. Data pipelines where the source file, table, or object names include special characters will fail. Whitespace, dash, and underscore in the names are not a problem.
Sometimes, AutoSync cannot clean up the Google Cloud Storage staging area after loading from Salesforce to Google BigQuery. If this occurs, you should manually remove the files to retrieve the storage.
In the IIP AutoSync Manager screen, you can create Accounts to use with AutoSync. For Google BigQuery, the Account Type dropdown allows you to select Google Service Account or Google Service Account JSON, which are not supported by AutoSync.
Sometimes, when you create or edit the synchronization schedule, you can select a start time in the past. If you do this for a data pipeline that is scheduled to run once, it will not run unless you start it manually.
Data pipelines that load the Marketo Program Manager object can time out and fail.
To use the SCD2 load type for Snowflake, you must modify Snowflake configurations created before the May 2023 release. Because AutoSync automatically sets the timezone to UTC for SCD2 operations, do the following:
For an Account created in the IIP, add a Uri property parameter with the name TIMEZONE and a value of UTC.
For credentials saved in AutoSync, delete them and create a new configuration.
On May 30, 2023 fixed an issue with SCD2 load type for Salesforce to Snowflake where the effective begin and end column values were strings. They are now timestamps
: Fixed a null pointer exception so no 5XX errors can occur if you download non-existent query details from the Pipeline Execution Statistics of an ELT (write-type) Snap.
Pipeline Cache is a subscription feature (currently available in private beta) that enables you to cache data in memory for reference in another pipeline. This information can be a dataset, and Expression Language functions can retrieve the data from this dataset. Looking up values based on database rows/columns is a costly operation if the table is queried repeatedly to use the same data as reference fields. An in-memory data store allows your pipelines to look up the data as key/value pairs in other Snaps at runtime. If you have data in an external system that you need to look up (such as an employee object or an identifier field), then you can use Pipeline Cache to optimize the mapping to match the references in this type of operation.
This feature is in private beta. Email support@snaplogic.com for an invitation to gain early access. |
Secrets Management introduces support for CyberArk Conjur. Learn more.
New Public APIs are available for APIM.
When creating a new project from a Git repository, you can also create a new branch for the new project. Learn more.
Support for HashiCorp KV Secrets Engine Version 1 is available, in addition to KV Search Engine Version 2.
Fixed an issue where Orgs could not be provisioned.
The subscription feature Secrets Management -CyberArk is now displayed correctly on the Manager > Subscriptions page.
Bearer Tokens are not included when an Org admin migrates a Triggered Task to another Org.
Workaround: Manually change the Bearer Token for the migrated task in the Triggered Task dialog.
When you delete a user or group from a Project space, their access might not be removed from the shared folder. Users or groups added to a Project space are not always correctly propagated to the respective shared folder, so shared folder permissions might not match the Project space permissions for some users or groups.
Workaround: Manually remove the user or group and make sure permissions for the remaining users and groups at the Project space and shared folder level match.
The March 2023 release added support for enabling/disabling Pipeline Validation at the Project level. This means that validation settings for a Pipeline set at the Project level could override the Org-level setting. As an Org admin, you can communicate the preferred validation settings on Projects through User Notifications.
The com.snaplogic.jsonpath.ReadVisitor.THROW_ON_BRANCH feature flag has been removed, and the platform no longer produces an error when a value is missing when using jsonpath. Previously, this would result in a null value (since the field referenced in the expression did not exist) and a lint warning. In this release, the behavior results in an empty array [], and the UI does not display a lint warning.
Added the Dynamics 365 Finance and SCM Create, Update, and Delete Snaps to the Microsoft Dynamics 365 Finance and SCM Snap Pack.
Introduced the GraphQL Client Snap in the API Suite Snap Pack, which enables you to query a GraphQL endpoint and retrieve data. This Snap features a built-in query builder to help you efficiently build queries.
Introduced the SAP Concur Snap Pack. This Snap Pack contains the following Snaps and Account:
SAP Concur Read: Retrieves the data resources from the SAP Concur endpoint.
SAP Concur Write: Creates and updates the data resources from the SAP Concur endpoint.
SAP Concur Delete: Deletes data resources from the SAP Concur endpoint.
SAP Concur OAuth2 Account: Connect SAP Concur Snaps with data sources that use the SAP Concur OAuth2 account.
The Bouncy Castle library version is upgraded from bcpg-jdk150n[1.69] to bcpg-jdk180n[1.73] in all our Snap Packs. This upgrade brings in the latest security features to enhance the performance of the SnapLogic platform.
The Generic Database Account now supports the SSH Tunneling connection. You can now encrypt the network connection between your client and the database server, ensuring a highly secure connection.
The Hive Snap Pack is Cloudera-certified for Cloudera Data Warehouse (CDW). You can use the Hive Execute Snap to work with CDW clusters through a Generic Hive Database account.
The Marketo Bulk Extract Snap works successfully in the non-lineage path in an Ultra task.
The Key passphrase field in the Private Key Account of the Binary Snap Pack now supports expressions, allowing dynamic evaluation using pipeline parameters when the expression button is enabled.
Snowflake
SnapLogic is specified as a partner tag in all requests directing to Snowflake, making it easier for Snowflake to identify the requests coming from SnapLogic.
The default JDBC JAR for the Snowflake Snap Pack is upgraded to version 3.13.28 to support the GEOMETRY data type.
Fixed an issue with the Encrypt Field Snap, where the Snap failed to support an RSA public key to encrypt a message or field. Now the Snap supports the RSA public key to encrypt the message.
Previously, in a Project, you could use Snaps from a private Snap Pack uploaded to the Project, the Project shared folder, or the global shared folder. Now, if a Snap Pack is uploaded to the Project containing different Snaps than those uploaded to a shared folder, only the Snaps uploaded to the Project are visible. If you have existing pipelines using Snap Packs that are no longer available because of this change, you must fix them by uploading the required Snaps to the Project.
As of 432patches19873 Snap Pack version, any LDAP entry attribute in the LDAP Search and LDAP Lookup Snaps containing binary data will be displayed as a byte array (Base64 encoded) instead of a string in the output.
The UI now uses accessible colors. Some icons are larger and more readable. For example, note the status icons on the summary cards in the Execution overview:
For the Execution Overview:
The Executed by column was renamed to Owner.
Invoker is a new optional column that identifies who ran a Triggered Task or manually ran an AutoSync data pipeline. It contains the user ID and their IP address. To add the Invoker column, click 
in the Search bar and click the Invoker pill:
The following screenshot shows the Owner and Invoker columns. The last row in the example has a value in the Invoker column for an AutoSync data pipeline that was run manually:
The new page Groundplexes and Pipeline Dependency on the Control Plane provides information for designing and running pipelines that reduce dependency on the control plane.
The following application configuration pages for the Exchange Online and SharePoint Online Snap Packs are enhanced with the key steps, workflow, and frequently asked questions:
Enhanced the following Snap documents in the ELT Snap Pack to improve usability and updated the content:
Detailed prerequisites for successful validation of the ELT DLP Account.
Revised and updated the existing lists of Limitations and Known Issues across the Snaps and the Accounts in the ELT Snap Pack.
Enhanced the following PS Snap documents by adding missing information, improving clarity in field descriptions, and implementing the latest UI changes.
API Management IV - Manage the SnapLogic APIM Developer Portal: To continue with the previous API Management videos, this video shows you how to configure and use a Developer Portal where all the published videos can be published for the end user.