Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • New article about pipeline dependency on the control plane

  • Enhancements in Data Automation articles

  • Improved Professional Snap Pack documentationdocumentation of Snaps developed by the Professional Services team

  • New APIM video about managing your Developer Portal

...

We plan to upgrade the Oracle Snap Pack to the latest JDBC JAR file 19c (19.18.0.0) in the July November 2023 release.

  • Pipelines that use the default JDBC driver in the Oracle server accounts might be affected after the driver upgrade because the latest driver is backward-incompatible. We will soon communicate how to reconfigure to prevent accounts from being impacted.

  • Pipelines that do not use the default driver (bundled with the Oracle Snap Pack) are not impacted.

Enhancements

  • The Dynamics 365 for Sales Snaps include a new Enable continue on error checkbox to allow the Snap to continue executing the remaining batch requests, even if the previous request does not execute. (433patches21467)

  • The Dynamics365 for Sales Update Snap performs the update operation as expected when the Batch size is more than 1, and the Snap successfully executes when the batch size is 1. (433patches21467)

  • The pagination feature implemented for the ABFS protocol in the Hadoop Directory Browser Snap enables the Snap to return all the output documents as expected. (433patches21494)

  • The PGP Encrypt Snap now includes an Encryption key ID field, which allows you to specify the key ID for encrypting the data. If you do not specify an encryption key ID, the Snap uses the first key in the public key (master key). (433patches21482)

  • The following Transform Snaps include new fields to improve memory management: Aggregate, Group By Fields, Group By N, Join, Sort, Unique. (433patches21461)

  • The SQL Server Snap Pack is upgraded to the Microsoft JDBC Driver 12.2.0.jre11 for SQL Server. Existing pipelines that do not use the default driver (bundled with the SQL Server Snap Pack) are not impacted. If you want to use NTLM or other authentications with the latest default driver (where the DLL file is mandatory), then you must copy the DLL file to the Snaplex path. (433patches21386)

...

  • Starting from version 433patches21482, the PGP Encrypt Snap will no longer support encryption with an expired key. To ensure continued support for encryption, we highly recommend extending the expiration of your PGP key.

Breaking Changes

  • Pipelines that use the default Microsoft JDBC Driver in the SQL server accounts might be impacted after the upgrade because the latest driver is backward-incompatible. To prevent an impact to your accounts, reconfigure the URL properties field in account settings as follows:
    trustServerCertificate = true

  • With the 4.33 GA release, we no longer support some algorithms for SFTP connection negotiation because we’ve updated the library used to connect to SFTP sources and to improve security. We are working on a fix to

...

  • support the removed algorithms automatically. With Snap Pack version 433patches21645, you can modify the global properties. Refer to the Configuration Settings for Snaps

...

  • for details about how to revert to the previous settings.

Known Issues

  • The SFTP protocol is now incompatible with some deprecated algorithms. This affects mostly the Binary Snap Pack, but might also apply to the Email Snap Pack. (433patches21482)

  • SFTP connections may experience connectivity issues due to the removal of some deprecated algorithms. To add these connections back, refer to Configuration Settings for Snaps for details.

Dot Releases - Snaps

Snap Pack

Date of Update

Snap Pack Version

Updates

Transform

433patches21779

The Decrypt Field and Encrypt Field Snaps now support CTR (Counter mode) for the AES (Advanced Encryption Standard) block cipher algorithm.

Databricks

433patches21630

Enhanced the performance of the Databricks - Insert Snap to improve the amount of time it takes for validation.

ML Core

433patches21644

Improved an error message in the Remote Python Script Snap to explain the reason and resolution when a Python script has errors.

Flow

433patches21744

Fixed an issue with the Pipeline Execute Snap where a missing label produced a null pointer exception.

LDAP

433patches21488

Enhanced the LDAP Search and LDAP Lookup Snaps with a new Include string representation for binary attributes checkbox to add a string representation for binary attributes.

Binary

433patches21645

With the 4.33 GA release, we no longer support some algorithms for SFTP connection negotiation because we’ve updated the library used to connect to SFTP sources and to improve security. With Snap Pack version 433patches21645, you can modify the global properties. Refer to the Configuration Settings for Snaps documentation for details about how to revert to the previous settings.

ML Data Preparation

433patches21572

The Deduplicate Snap now manages memory efficiently and eliminates out-of-memory crashes using the following fields:

  • Minimum memory (MB)

  • Minimum free disk space (MB)

Transform

433patches21586

The Decrypt Field Snap now supports the decryption of various encrypted fields on providing a valid decryption key.

Binary

433patches21576

Fixed the issues with the AES Encrypt and AES Decrypt Snaps, where the Snaps previously did not include the error stack trace in the error view. The Snaps now provide detailed information in case of any issues or errors.

Active Directory

433patches21612

Fixed the following issues with the Active Directory Update Entry Snap where the Snap produced incorrect results with the version main21015:

  • When set to null, all the string attributes were updated with the literal "null" string instead of an empty string.

  • The Snap failed with javax.naming.directory.InvalidAttributeValueException error when all the number-type attributes were set to null.

Marketo

433patches21541

Fixed an issue with the Bulk Extract and Bulk Import Snaps where the API name was inaccessible when expressions were enabled for the Marketo Entity field and Custom Objects were selected.

Azure Active Directory

433patches21349

The Azure Active Directory Snap Pack now incorporates an identifying header allowing the Azure portal to identify the source of API requests for auditing purposes.

Binary

433patches21482

  • Fixed an issue that caused a String index out of range error with the SFTP protocol in the File Writer Snap when it attempted to create a top-level directory.

  • Enhanced the PGP Encrypt Snap with the Encryption key ID field, which allows you to specify the key ID for encrypting the data. If you do not specify an encryption key ID, the Snap uses the primary key in the public key (master key).

SQL Server

433patches21386

The SQL Server Snap Pack is upgraded to the latest version of the Microsoft JDBC Driver 12.2.0.jre11 for SQL Server:

  • Pipelines using the default JAR in the SQL server accounts might fail after the driver upgrade because the latest driver is backward-incompatible. To prevent your accounts from failing, reconfigure the URL properties field in account settings as follows:

    trustServerCertificate = true

  • Pipelines NOT using the default driver (bundled with the SQL Server Snap Pack) will not fail.

The Microsoft JDBC Driver 12.2.0.jre11 for SQL Server included in the SQL Server Snap Pack is currently not available with the latest distribution. However, you can still consume this patch through 433patches21386. This update is scheduled to be included in the upcoming distribution release on July 12, 2023, and will be a part of the stable distribution with the GA release on August 9, 2023.

Copybook

main132

Enhanced the COBOL Copybook Parser Snap to improve performance, data validation, and error handling.

PostgreSQL

433patches21298

Fixed an issue with the PostgreSQL Insert Snap that inconsistently inserted some columns and missed the remaining columns(especially the Time fields), when the data was passed in the JSON format from an upstream Snap.

Marketo

433patches21295

Fixed an issue with the Marketo Bulk Extract Snap where the Snap failed to extract data from the CSV file when the data consisted of the backslash (\) character.

API Suite

433patches21346

Fixed an issue with the HTTP Client Snap where the validation of the SSL Account for a specified Keystore or Truststore value failed despite providing the correct password.

Snowflake

433patches21370

  • Fixed an issue with the Snowflake Bulk Upsert Snap where the output document was missing information about the error records count and the reason for the error.

  • Fixed an issue that caused stored procedure executions to fail in the Snowflake - Multi Execute Snap.

Transform

433patches21336

Fixed an issue with the AutoPrep Snap where dates could potentially be rendered in a currency format because currency format options were displayed for the DOB column.

Salesforce

433patches21367

  • The Salesforce SOQL Snap now honors the selection of the Match Data Type checkbox when the value entered for Batch Size is greater than 50,000.

  • The Salesforce Read Snap now honors the selection of the Match Data Type checkbox if the Use PK chunking if supported checkbox is also selected.

ML Data Preparation

433patches21247

Fixed an issue with the Match Snap where a null pointer exception was thrown when the second input view had fewer records than the first.

Binary

433patches21291

Fixed an issue with the Multi File Reader Snap where it failed with the error S3 object not found when the Snap found no matching file to read and the Folder/File property value did not end with a forward slash (/).

API Suite

433patches21307

Fixed an issue with the HTTP Client Snap that caused pagination to fail when the next Snap in the pipeline could modify the input document (for example, the Mapper Snap).

Transform

433patches21196

Enhanced the In-Memory Lookup Snap with the following new fields to improve memory management and help reduce the possibility of out-of-memory failures:

  • Minimum memory (MB)

  • Out-of-memory timeout (minutes)

These new fields replace the Maximum memory % field.

Flow

433patches21196

Fixed an issue with the Pipeline Execute Snap where no input view produced a null pointer exception.

Binary

433patches21179

  • Fixed an issue with the File Delete Snap where the Snap failed with a 404 Not Found error when trying to delete files from an Amazon S3 bucket. This issue occurred only with the Identity and Access Management (IAM) role in an Amazon AWS S3 Account.

  • Fixed an issue where Binary Snaps could not handle region information for the Amazon S3 file protocol, which resulted in an error.

Copybook

main131

Addressed a bug in the compatibility mode of the COBOL Copybook Parser Snap that caused a working pipeline to fail.

API Suite

433patches21140

  • Fixed an issue affecting the HTTP Client Snap, which caused it to hang for an extended period when the user-agent contained the term java.

  • The HTTP Client Snap is enhanced with the Prevent URL encoding checkbox. This checkbox enables you to control whether the Snap should automatically encode the URL or prevent the URL encoding based on your preference.

SQL Server

433patches21119

Updated the SQL Server - Bulk Load Snap to preserve empty strings as empty strings and null as nulls.

Flow

433patches21040

Provided a fix to ensure that the Data Validator Snap works with "unknown" data types.

Kafka

433patches21070

Fixed an issue with the Kafka Consumer Snap that caused it to skip records in a partition when a pipeline failed without successfully processing any records from that partition. This was an edge case that was only possible with a specific configuration of the Snap and a specific ordering of events during termination of the pipeline.

API Management

New Features

...

...