The SnapLogic® May 2023 Release includes the following:
Support for SOAP endpoints in Proxies
New endpoints, Google BigQuery, Amazon S3 (for CSV and JSON files), Marketo, and CSV files
Synchronize with SCD2 between Salesforce and Snowflake
For a Salesforce source, AutoSync automatically handles changes to the column precision, scale, or character limit by propagating the changes to the target.
The new Pipeline Cache feature (private beta) enables you to leverage in-memory lookup
New APIM Public APIs allow you to publish, unpublish, deprecate, or retire a set of API versions
Support for CyberArk Conjur in Secrets Manager
Introduced the GraphQL Client Snap in the API Suite Snap Pack, which enables you to query a GraphQL endpoint and retrieve data
Added the Dynamics 365 Finance and SCM Create, Update, and Delete Snaps to the Microsoft Dynamics 365 Finance and Supply Chain Management Snap Pack
Improved accessibility
New article about pipeline dependency on the control plane
Enhancements in Data Automation articles
Improved Professional Snap Pack documentation
New APIM video about managing your Developer Portal
This section tracks the changes made during the iterative pushes to the UAT server and the GA release. The following is the release schedule:
The SnapLogic GA release build on the production server is also deployed on the UAT server. The SnapLogic GA release build on the production server is also deployed on the UAT server. |
|
Snap Pack | Date of Update | Snap Pack Version | Updates |
|---|---|---|---|
| 433patches21119 | Updated the SQL Server - Bulk Load Snap to preserve empty strings as empty strings and null as nulls. | |
| 433patches21040 | Provided a fix to ensure that the Data Validator Snap works with "unknown" data types. | |
| 433patches21070 | Fixed an issue with the Kafka Consumer Snap that caused it to skip records in a partition when a pipeline failed without successfully processing any records from that partition. This was an edge case that was only possible with a specific configuration of the Snap and a specific ordering of events during termination of the pipeline. |
Support for SOAP web services in Proxies. You can now reference a SOAP service when you create a Proxy endpoint.
Refer to the Platform section for Lifecycle Management Public APIs. You can now use Public API to publish, unpublish, deprecate, and retire APIs.
Added the Trust All Certificates checkbox in the Create proxy endpoint dialog. Similar to the setting in the REST Snaps, you can choose to allow your Proxy to trust self-signed, unsigned, or expired certificates in the endpoints.
API policy violations are now delivered in JSON. When a client executes an API that results in an authorization or authentication error, the error is returned in the JSON format.
Logging added to the Callout Authenticator API policy. You can now use the Snaplex logs to troubleshoot why the authentication failed when your Callout Authenticator policy rejects a client request.
Improved error output for the JSON Validator API policy. The API caller can easily determine which part of the JSON failed validation.
Wildcard endpoint support for dynamic base paths and error handling in Proxy endpoint mapping. When the base path is a path parameter, the Proxy uses this endpoint and routes through the mappings.
Improved error handling in the Developer Portal API specification editor. Specification violations now appear in the editor.
Added the Time Unit field in the Client Throttle API policy. You can now define a Request Limit value in requests per second.
API Manager now supports asynchronous API migration with Public APIs.
Previously selected HTTP methods are not cleared when you reopen the Create proxy endpoint dialog and select the REST to SOAP checkbox.
Workaround: Refresh the browser.
Fixed an issue where the Try it functionality did not work for an API Proxy published on the Developer Portal.
AutoSync now supports the following endpoints:
Google BigQuery as a target. Learn more.
Marketo as a source. Learn more.
CSV or JSON files stored in Amazon S3 as a source. Learn more.
CSV file upload from a local or network drive as a source. Learn more.
The ability to change load type for synchronization. The types available depend on what AutoSync supports for the source and endpoint. AutoSync supports loading from Salesforce to Snowflake using SCD2.
To change the load type after creating a data pipeline, edit it. From the Auto Synchronize tab, select the Load type. Learn more.
For a Salesforce source, AutoSync automatically handles changes to the column precision, scale, or character limit by propagating the changes to the target. Learn more.
Improved synchronization scheduling. As the last step in the wizard, choose to:
Continue without scheduling.
Pick a preset schedule and set the start time.
Create a custom schedule, which allows you to set an end date.
If you remove or rename a source table or object after using it in a data pipeline, its name will still be visible in the source configuration and you will not be able to de-select it.
For Amazon S3 data sources, you must add a forward slash if you copy the URL from the Amazon S3 console. For example, change s3://my_bucket to s3:///my_bucket.
The first row of CSV files must define the names for the column headings in the target table, with no empty or null values.
AutoSync cannot upload a CSV file to the cloud data warehouse if the file name includes single or double quotes or a special character, such as #, $, %, or &.
Google BigQuery does not support special characters for names. Data pipelines where the source file, table, or object names include special characters will fail. Whitespace, dash, and underscore in the names are not a problem.
Sometimes, AutoSync cannot clean up the Google Cloud Storage staging area after loading from Salesforce to Google BigQuery. If this occurs, you should manually remove the files to retrieve the storage.
In the IIP AutoSync Manager screen, you can create Accounts to use with AutoSync. For Google BigQuery, the Account Type dropdown allows you to select Google Service Account or Google Service Account JSON, which are not supported by AutoSync.
Sometimes, when you create or edit the synchronization schedule, you can select a start time in the past. If you do this for a data pipeline that is scheduled to run once, it will not run unless you start it manually.
Data pipelines that load the Marketo Program Manager object can time out and fail.
To use the SCD2 load type for Snowflake, you must modify Snowflake configurations created before the May 2023 release. Because AutoSync automatically sets the timezone to UTC for SCD2 operations, do the following:
For an Account created in the IIP, add a Uri property parameter with the name TIMEZONE and a value of UTC.
For credentials saved in AutoSync, delete them and create a new configuration.
Pipeline Cache is a subscription feature (currently available in private beta) that enables you to cache data in memory for reference in another pipeline. This information can be a dataset, and Expression Language functions can retrieve the data from this dataset. Looking up values based on database rows/columns is a costly operation if the table is queried repeatedly to use the same data as reference fields. An in-memory data store allows your pipelines to look up the data as key/value pairs in other Snaps at runtime. If you have data in an external system that you need to look up (such as an employee object or an identifier field), then you can use Pipeline Cache to optimize the mapping to match the references in this type of operation.
This feature is in private beta. Email support@snaplogic.com for an invitation to gain early access. |
Secrets Management introduces support for CyberArk Conjur. Learn more.
New Public APIs are available for APIM.
When creating a new project from a Git repository, you can also create a new branch for the new project. Learn more.
Support for HashiCorp KV Secrets Engine Version 1 is available, in addition to KV Search Engine Version 2.
Fixed an issue where Orgs could not be provisioned.
The subscription feature Secrets Management -CyberArk is now displayed correctly on the Manager > Subscriptions page.
Bearer Tokens are not included when an Org admin migrates a Triggered Task to another Org.
Workaround: Manually change the Bearer Token for the migrated task in the Triggered Task dialog.
The com.snaplogic.jsonpath.ReadVisitor.THROW_ON_BRANCH feature flag has been removed, and the platform no longer produces an error when a value is missing when using jsonpath. Previously, this would result in a null value (since the field referenced in the expression did not exist) and a lint warning. In this release, the behavior results in an empty array [], and the UI does not display a lint warning.
SnapLogic is planning to bundle the latest JDBC JAR file 19c (19.18.0.0) in the Oracle Snap Pack by the August 2023 release.
|
Added the Dynamics 365 Finance and SCM Create, Update, and Delete Snaps to the Microsoft Dynamics 365 Finance and SCM Snap Pack.
Introduced the GraphQL Client Snap in the API Suite Snap Pack, which enables you to query a GraphQL endpoint and retrieve data. This Snap features a built-in query builder to help you efficiently build queries.
Introduced the SAP Concur Snap Pack. This Snap Pack contains the following Snaps and Account:
SAP Concur Read: Retrieves the data resources from the SAP Concur endpoint.
SAP Concur Write: Creates and updates the data resources from the SAP Concur endpoint.
SAP Concur Delete: Deletes data resources from the SAP Concur endpoint.
SAP Concur OAuth2 Account: Connect SAP Concur Snaps with data sources that use the SAP Concur OAuth2 account.
The Bouncy Castle library version is upgraded from bcpg-jdk150n[1.69] to bcpg-jdk180n[1.73] in all our Snap Packs. This upgrade brings in the latest security features to enhance the performance of the SnapLogic platform.
The Generic Database Account now supports the SSH Tunneling connection. You can now encrypt the network connection between your client and the database server, ensuring a highly secure connection.
The Hive Snap Pack is Cloudera-certified for Cloudera Data Warehouse (CDW). You can use the Hive Execute Snap to work with CDW clusters through a Generic Hive Database account.
The Marketo Bulk Extract Snap works successfully in the non-lineage path in an Ultra task.
The Key passphrase field in the Private Key Account of the Binary Snap Pack now supports expressions, allowing dynamic evaluation using pipeline parameters when the expression button is enabled.
Snowflake
SnapLogic is specified as a partner tag in all requests directing to Snowflake, making it easier for Snowflake to identify the requests coming from SnapLogic.
The default JDBC JAR for the Snowflake Snap Pack is upgraded to version 3.13.28 to support the GEOMETRY data type.
Fixed an issue with the Encrypt Field Snap, where the Snap failed to support an RSA public key to encrypt a message or field. Now the Snap supports the RSA public key to encrypt the message.
Previously, in a Project, you could use Snaps from a private Snap Pack uploaded to the Project, the Project shared folder, or the global shared folder. Now, if a Snap Pack is uploaded to the Project containing different Snaps than those uploaded to a shared folder, only the Snaps uploaded to the Project are visible. If you have existing pipelines using Snap Packs that are no longer available because of this change, you need to fix them by uploading the required Snaps to the Project.
The UI now uses accessible colors. Some icons are larger and more readable. For example, note the status icons on the summary cards in the Execution overview:
For the Execution Overview:
The Executed by column was renamed to Owner.
Invoker is a new optional column that identifies who ran a Triggered Task or manually ran an AutoSync data pipeline. It contains the user ID and their IP address. To add the Invoker column, click 
in the Search bar and click the Invoker pill:
The following screenshot shows the Owner and Invoker columns. The last row in the example has a value in the Invoker column for an AutoSync data pipeline that was run manually:
The new page Groundplexes and Pipeline Dependency on the Control Plane provides information for designing and running pipelines that reduce dependency on the control plane.
The following application configuration pages for the Exchange Online and SharePoint Online Snap Packs are enhanced with the key steps, workflow, and frequently asked questions:
Enhanced the following Snap documents in the ELT Snap Pack to improve usability and updated the content:
Detailed prerequisites for successful validation of the ELT DLP Account.
Revised and updated the existing lists of Limitations and Known Issues across the Snaps and the Accounts in the ELT Snap Pack.
Enhanced the following PS Snap documents by adding missing information, improving clarity in field descriptions, and implementing the latest UI changes.
API Management IV - Manage the SnapLogic APIM Developer Portal: To continue with the previous API Management videos, this video shows you how to configure and use a Developer Portal where all the published videos can be published for the end user.