May 2020, 4.21 Release Notes

In this article:

Important

About SnapLogic Releases

  • Per the SnapLogic Release Process, all remaining Snaplex instances across customer Orgs are upgraded to the recommended 4.21 version (master-7451 - 4.21 Patch 1) at 9 p.m. PT on June 13, 2020. If you upgraded your Snaplex to a 4.21 version prior to the auto-upgrade event, then your Snaplex version does not change. 
  • To use the new features and Snaps in the 4.21 release, ensure that your Snaplex nodes are upgraded to the recommended 4.21 patch version (master-7451 - 4.21 Patch 1).
  • This 4.21 GA release includes the patches deployed to SnapLogic after the 4.20 GA release, as listed in the 4.20 Dot Releases page. 
  • We recommend that you upgrade to the latest Chrome or Firefox browser version and clear your browser cache before logging into the SnapLogic Intelligent Integration Platform (IIP).


Java 11 Support

Starting from the 4.18 August release, SnapLogic supports Java 11 for Groundplex instances. We recommend all customers to migrate to Java 11, since Oracle no longer releases security updates for Java 8. For details, see Configuring the Java Version on Your Groundplex.

Java 11 support highlights

Groundplex:

  • The 4.21 Snaplex installer uses Java 11 by default. Java 8 is also supported as a rollback option, if required.
  • Current Groundplex installations will continue to work with Java 8 until the Snaplex is reinstalled using the new installer.
  • You are required to upgrade their Groundplex instances to Java 11 using the new installer. Support for Java 8 will cease from the August 2020 release.

Cloudplex:

  • During the current 4.21 release, all your existing Cloudplex instances hosted in the United States will be migrated to Java 11. Currently, this is being planned for Friday, June 26, 2020. For more details, see Migrating Your Cloudplex Nodes to Java 11.

New IP Whitelisting Requirements Reminder

We remind all customers of the new IP Whitelisting requirements that have been in effect since the August 2019, 4.18 release. Ensure that your network administrators review the following:

Platform 

New Features and Enhancements

  • Insights dashboardRevamps the Insights dashboard to provide data visualization of Org-level Pipeline-execution statistics for a specific date range.


  • Task monitoring: Adds a new tab for Tasks under Dashboard that does data visualization of triggered and scheduled tasks for a selected time range. The new tab displays the number of failed and completed executions for each task type along with their details.


  • Snap auto-configuration: Enhances the suggestion capabilities of Iris to include Snap auto-configuration in Snaps with suggestible properties. Iris Expression recommends the configuration of these properties based on similar Snap settings from Pipelines that you can access. 


  • Sticky Notes in SnapLogic Designer: Introduces the ability to add notes to a Pipeline in SnapLogic Designer. This enables you to collaborate with multiple Pipeline developers or add important notes for complex Pipelines. The notes are included in the Pipeline metadata and are immediately visible to other users accessing the Pipeline.

    Known Issue

    When you copy a Pipeline with Sticky Notes, the notes are also copied with the Pipeline. Additionally, when you modify or add Sticky Notes in the copied Pipeline, the notes in the original Pipeline are also updated.

  • Target recommendation: Enhances the Iris Expression recommendation capabilities to include Target path suggestions in Mapper Snap. Iris recommends Target paths based on exact matches between Expression and Target paths from other Pipelines in your Org. 

Known Issues

  • A Triggered Task on a Groundplex with an underlying Pipeline that contains child Pipelines might stall as the Data Validator Snap is executing in the parent Pipeline.
  • An Ultra Pipeline sometimes processes the same message twice even though the message was processed successfully during the first attempt. 

Updates to Expression Behavior

  • Starting in the 4.21 Release (May 2020), the escaping required in the replacement text of the ReplaceAll function is identical to that of the Replace function. Prior to this release, you had to use additional escaping in the replacement text with ReplaceAll to get the desired output.

    Previous BehaviorNew Behavior

    Example: $msg.replaceAll("-", "\\\\-")

    where $msg contains mother-in-law

    Result: mother\-in\-law

    Example: $msg.replaceAll("-", "\\-")

    where $msg contains mother-in-law

    Result: mother\-in\-law

  • Starting in the 4.21 Release (May 2020), the Date.parse() function returns the response text Not a Number when you enter a non-number or infinity. Prior to this release, doing either resulted in an error, even though the formatting of the string was correct.

    Previous BehaviorNew Behavior

    Example: $date.parse(Math.abs(1/0))

    where $date is infinity

    Result: 1970-01-01T00:00:00.000 UTC


    Example: $date.parse(“foo”, “yyyymmdd”)

    Result: Error

    Example: $date.parse(Math.abs(1/0))

    where $date is infinity

    Result: Not a number


    Example: $date.parse(“foo”, “yyyymmdd”)

    Result: Not a number


Standard-mode Snaps

New Snaps

  • Adobe Experience Platform Snap Pack: This Snap Pack enables you to integrate with the new Adobe Experience Platform and includes the following Snaps:
    • Adobe Experience Platform File Generator: Generates an Adobe Enterprise Platform (AEP)-compatible file by restructuring upstream data to match a specific schema stored in AEP.
    • Adobe Experience Platform S3 Connector: 
      • Enhances the Snap by removing the Create New Dataset checkbox. The Snap now automatically creates a new dataset if it does not exist.
      • Retrieves data from AWS S3 and writes it into AEP.  
    • Cassandra: Extends support to Cassandra version 3.11.5.
    • Adobe Experience Platform Write: Writes data from upstream Snaps into AEP.

  • SAP S/4HANA Snap Pack: This Snap Pack supports the SAP S/4 HANA on-premise environment and includes the following Snaps:
  • ML Data Preparation: Introduces the Mask Snap that enables you to hide sensitive information in your dataset before exporting the dataset for analytics or writing the dataset to a target file.

Updated Snap Packs 

  • Adobe Experience Platform Snap Pack: Enhances the Adobe Experience Platform S3 Connector Snap by removing the Create New Dataset checkbox. The Snap now automatically creates a new dataset if it does not exist. 
  • Cassandra: Extends support to Cassandra version 3.11.5.
  • Confluent Kafka: Enhances the Kafka Consumer Snap to significantly reduce the volume of commit requests for offsets when using the record acknowledgment mode (Wait after each record), and thereby, improve performance and stability.
  • ML Data Preparation: 
    • Enhances the Match Snap to add a new field, Match all, which matches one record from the first input with multiple records in the second input. Also, enhances the Comparator field in the Snap by adding one more option, Exact, which identifies and classifies a match as either an exact match or not a match at all.
    • Enhances the Deduplicate Snap to add a new field, Group ID, which includes the Group ID for each record in the output. Also, enhances the Comparator field in the Snap by adding one more option, Exact, which identifies and classifies a match as either an exact match or not a match at all.
    • Enhances the Sample Snap by adding a second output view which displays data that is not in the first output. Also, a new algorithm type, Linear Split, which enables you to split the dataset based on the pass-through percentage.
  • NetSuite 
    • Removes the obsolete Sandbox Account check box from the NetSuite Token Account.

    • Removes the now redundant Custom WSDL URL Domain field from the NetSuite Token Account to support NetSuite's transition to automatic account-specific domain identification.

  • Oracle: Updates the Oracle Thin Account and Oracle Thin Dynamic Account, enabling them to connect via Oracle Active Data Guard (ADG).

  • SAP: Enhances the SAP Execute Snap to add a new field, Output date time as Text, which enables you to output date and time as a String data type instead of a DateTime object.
  • Script Snap Pack: Adds a new Cleanup method in the ScriptHook interface associated with the Script Snap. This method enables the Script Snap to automatically execute a clean-up once the configured script completes executing.

  • ServiceNow: Adds support for ServiceNow Madrid, New York, and Orlando versions.
  • Transform: Adds support in the Mapper Snap to display schemas with complex nesting. For example, if Snaps downstream from the Mapper Snap have custom fields that contain multiple keys, then these keys automatically display in the Mapper Snap's Target Schema dropdown list when you toggle-off Expression Builder and select the field in the Mapping Table.

Breaking Change

The HTTP header field's behavior in the SOAP Execute Snap has been changed. Specifying key alone is not sufficient. Both key and value must be provided when invoking a task. 

Known Issues

  • In some Snaps, if you leave any expression-enabled mandatory fields empty, a null pointer exception error message might display instead of the usual error message on validation.
  • Cassandra

    • The Cassandra Snap Pack does not support the following data types introduced in Apache Cassandra V3.x:

      • DateRange

      • Duration

      • Geo-spatial data types such as Point, Polygon and LineString

    • Snaps in this Snap Pack display an exception when you query a map column that has a timestamp as the key.

  • Google Big Query Snap Pack: The Snaps no longer return the affectedRowCount field with a null value for DML (data manipulation language) queries. The said field no longer exists.
  • MySQL: When MySQL Execute Snaps (MySQL - Execute and MySQL - Multi Execute) are followed by MySQL Snaps, such as MySQL - Insert, MySQL - Merge, they display the following error when executed: 
    Table definition has changed, please retry transaction.

    This happens due to a known issue in the MySql Database. For more information, see MySQL Bug #65378.

  • Parquet Writer (Behavior Change): In the 4.20 Snap version (snapsmrc535 and hadoop8776) when a value for a field defined as repeated (for example, devops in the below example) is not present in the input document, the Parquet Writer Snap displays "The required field 'devops' is missing from a document" error. However, in the 4.21 release, the Parquet Writer Snap outputs a null value when there is no input for the repeated fields. If this behavior change impacts your existing Pipelines having the Parquet Writer Snap, then you might want to consider potentially redesigning your impacted Pipelines.

    repeated group devops {
    required binary lan (UTF8);
    optional binary lan2 (UTF8);
    }

Custom Snaps

During the May2020, 4.21 release cycle, we did multiple enhancements to the third-party dependencies used by the SnapLogic platform and the Snaps. Hence, we strongly recommend updating and redeploying all your custom Snap Packs to ensure compatibility with this release. For details, see Updating Custom Snap Packs in the SnapLogic Developer Documentation.


Prior 4.20 Snap Dot Releases in 4.21 GA 

 The following 4.20 Snap Dot Release updates are now a part of the current 4.21 GA release.

 Click here to expand...

Standard-mode Snaps

Snap PackUpdate DateBuildUpdates
JMS jms8824Fixes an issue in the JMS Consumer Snap due to which the Snap does not release the memory even after Pipeline termination, leading to huge memory spikes.
Workday workday8817

Fixes the Workday Write Snap that ignores the Import Synchronized check box selection while importing object data. The Snap now:

  • Waits for the current import request to complete, whether successfully or with error, before initiating the next import request.
  • Provides accurate real-time status of the request to downstream Snaps.   

Existing Pipelines using this Snap may experience longer execution times due to the synchronous behavior. However, you no longer need to use an additional Snap to capture and pass on the request status to downstream Snaps.

SOAP soap8816

Enhances the SOAP Execute Snap to allow user-specified empty fields in the SOAP request.

MySQL db/mysql8815Fixes the MySQL Stored Procedure Snap to resolve compatibility issues with the Amazon Aurora database.
Oracledb/oracle8812Enhances the Oracle Snap Pack to support connection to the Oracle ADG (Active Data Guard) URL.
Emailemail8813Fixes an issue with the Email Sender Snap that caused the Snap to fail while uploading attachments to SMB locations when the attachment password contained a forward slash "/".
SAP sap8811

Updates the SAP IDoc Write Snap with the following:

  • Fixes the stack overflow error that occurs while retrying to write the IDoc to the SAP server. 
  • Adds inbound success codes to the default value which otherwise only indicates the IDoc being created.
  • Enhances the error messages for a better user experience.
JMS jms8810Fixes the connection leak issue in the JMS Producer Snap.
SAP HANA snapsmrc535

Reverts the SAP/HANA Snap Pack to the stable version snapsmrc535.

SAP S/4HANA  saps4hana8804Introduces the SAP S/4HANA Snap Pack to support the SAP S/4 HANA Cloud and SAP S/4 HANA On-premise environments. The Snap Pack includes the following Snaps:
Oracle db/oracle8803Support for Oracle Database 19c.
Binary binary8801
  • Fixes the Decompress Snap wherein the Snap stops decompressing at 64MB without displaying any error, when decompressing using the GZIP scheme.
  • Fixes an issue in the PGP Encrypt Snap so that third-party decryption tools do not receive a .tmp file from the Snap. 
Snowflake db/snowflake8800

Certifies the Snowflake Snap Pack against JDBC Driver version 3.12.3.

Snowflake Execute and Multi-Execute Snaps may break existing Pipelines if the JDBC Driver is updated to a newer version.

With the updated JDBC driver (version 3.12.3), the Snowflake Execute and Multi-Execute Snaps' output displays a Status of "-1" instead of "0" without the Message field upon successfully executing DDL statements. If your Pipelines use these Snaps and downstream Snaps use the Status field's value from these, you must modify the downstream Snaps to proceed on a status value of -1 instead of 0.

This change in the Snap behavior follows from the change introduced in the Snowflake JDBC driver in version 3.8.1:
"Statement.getUpdateCount() and PreparedStatement.getUpdateCount() return the number of rows updated by DML statements. For all other types of statements, including queries, they return -1."

REST rest8798Fixes the REST POST Snap that fails when a single file is uploaded in multi-part form using the Single file upload Snap settings.
NetSuite netsuite8799Updates the NetSuite Create, Update, Upsert, and Async Upsert Snaps to use the Classification type instead of Class, to resolve schema generation issues. While the Class type remains available in the NetSuite Search and Async Search Snaps for backwards compatibility, use of the Classification type is recommended.
Salesforce salesforce8797Fixes the Salesforce Subscriber Snap wherein the Salesforce Subscriber and Mapper Snap combination generates Null output for Array object type.
SQL Server db/sqlserver8793Fixes an issue in the SQL Server - Insert Snap due to which the Snap failed to load values of data type INT.
SQL Server db/sqlserver8796Fixes the SQL Server - Bulk Load Snap to mask the password in the logged exception messages
Transform transform8792Resolves the NoClassDefFoundError in the Join Snap on Windows Snaplex instances.
Binary binary8791Fixes the ZipFile Write Snap wherein the Snap fails to execute if it finds different zip file paths when processing multiple input binary data.
Active Directory activedirectory8789Fixes the Active Directory Snaps to successfully read special characters in the Existing distinguished name and Distinguished name Snap settings without requiring escape characters.
Transform transform8788Resolves the NullPointerException in the Join Snap on Windows Snaplex instances.
Binary binary8785
  • Adds the User-defined object metadata and Object tags properties in the S3 File Writer Snap and the Get Object Tags property in the S3 File Reader Snap to allow inclusion of user defined metadata and object tags.
  • Fixes a regression caused by Patch binary8775 wherein the File Reader and S3 File Reader Snaps fail to read S3 files with an account which does not include GetObjectTagging permission.
MySQL  db/mysql8786Fixes the MySQL - Bulk Load Snap to use the UTF-8 character set for writing temporary files in order to successfully handle special characters, irrespective of the value specified in the Character set field in the Snap settings.  
Hadoop hadoop8776Updates the Hadoop Snap Pack to use the latest version of org.xerial.snappy:snappy-java for compression type Snappy, in order to resolve the java.lang.UnsatisfiedLinkError: org.xerial.snappy.SnappyNative.maxCompressedLength(I)I error.
JDBC jdbc8783

Fixes the Generic JDBC Execute Snap that adds the value of the previous column's field to a NULL field when SELECT queries having duplicate column names are run on the PostgreSQL database.

Binary binary8775Fixes the File Writer Snap to allow the deletion of temporary Google Storage files from the /tmp directory of the Groundplex if Pipeline execution stops before completion. 
Redshift db/redshift8774Fixes the Redshift - Execute Snap that hangs if the SQL statement field contains only a comment ("-- comment"). 
Google BigQuery  google/bigquery8773Fixes the NPE issue with stored procedures and DROP TABLE queries in the Google BigQuery Execute Snap.
ML Core mlcore8770Adds the log4j dependency to the ML Core Snaps to resolve the "Could not initialize class org.apache.log4j.LogManager" error.
ML Analytics mlanalytics8769Removes the  unused jcc-optional dependency from the ML Analytics Snap Pack.
ML Data Preparation  mldatapreparation8771Removes the  unused jcc-optional dependency from the ML Data Preparation Snap Pack.
ML Natural Language Processing mlnlp8772Removes the  unused jcc-optional dependency from the ML Natural Language Processing Snap Pack.
JMS jms8765

Fixes an issues with the JMS Consumer Snap that fails to successfully create durable subscribers when run in asynchronous mode.

SOAP soap8762

Fixes the SOAP Execute Snap to:

  • Successfully handle redirected requests, when the WSDL file specifies an HTTP URL which redirects SOAP requests to an HTTPS URL.
  • Successfully generate the schema even if the XSD does not contain the <types> element. 
Microsoft Dynamics AX  msdynamicsax8763Fixes the Microsoft Dynamics AX Execute Snap to use the correct headers list for returning the HTTP response code.
Workday Snap Pack workday8761Fixes the Workday Read Snap that fails to validate when configured in the new form UI. 
Transform transform8760
  • Fixes an issue with the CSV Parser Snap wherein the Snap fails when it is configured in the new Form UI if the Contains headers and Validate headers fields are not selected but empty rows exist in the Column names field.
  • Fixes the JSON Formatter Snap to filter the input data based on the value specified in the Content field in the Snap settings when the JSON lines field is also selected. Previously, the Snap was writing the entire input data to the output file

JSON Formatter Snap behavior might break existing Pipelines

The JSON Formatter Snap output now includes only those fields from the input file that are specified in the Content field under Settings. 

If your Pipelines use the JSON Formatter Snap with the  JSON lines field selected, they may fail to execute correctly if the Content field mentions a specific object or field but the downstream Snap is expecting the entire data. Hence, for backward compatibility, you must review the entries in the Content field based on the desired output, as follows: 

  • Enter to include all the fields in the output.
    OR
  • Enter or select the specific fields to restrict the output to. 

The behavior of the Snap when the JSON lines field is not selected is correct and remains unchanged.  

The Example: Filtering Output Data Using the JSON Formatter Snap's Content Setting further illustrates the corrected vs the old behavior.

  • Fixes an issue in the XML Generator Snap due to which the custom XML data in the Edit XML field got ignored when inbound schema and root XML element were also provided, and instead, the output was generated from upstream data. For Pipelines with a standalone XML Generator Snap, a "Premature end of file" error was displayed. 

XML Generator Snap behavior might break existing Pipelines

The XML Generator Snap now gives precedence to any custom XML data that is provided over data coming from upstream Snaps, to generate the output. 

Existing Pipelines using the XML Generator Snap may fail in the following scenarios. Use the resolution provided to update the Snap settings based on the XML data you want to pass to downstream Snaps.

Breaking Scenario

Resolution

Downstream Snaps expect XML output from upstream Snaps, but custom XML data exists in the XML Generator Snap.

  • To use custom XML data, ensure that the Inbound schema and XML root element are specified and custom XML data is entered using the Edit XML option.
  • To use data from upstream Snaps, ensure that the Inbound schema and XML root element fields are blank and no custom XML data exists in the XML editor.


In the XML Generator Snap settings, the XML root element and Inbound schema are specified but no custom XML data is provided. This will generate a validation error. 
In the XML Generator Snap settings, custom XML data is provided and Validate XML check box is selected, but XML root element and Inbound schema are not specified. This will generate a validation error.
Snowflake db/snowflake8758Re-release of fixes from db/snowflake8687 for 4.20: Fixes the Snowflake Bulk Load Snap wherein the Snap fails to load documents containing single quotes when the Load empty strings checkbox is not selected.
Confluent Kafka confluentkafka8747

Updates the Confluent Kafka SSL Account settings to make the Truststore filepath and Truststore password properties optional. 

Box box8743

Fixes an issue with the Box Snap Pack wherein the OAuth2 token auto-refresh fails for Box accounts. The fix includes a patch released by Box to restore functionality to the "https://www.box.com/api/oauth2/token" URL of the deprecated version 1 of the Box API. 

Box recommends useUnlinkrs to start using the endpoint in version 2 of the Box API: "https://api.box.com/oauth2/token". For more information, see the Box blogpost

Binary binary8741
  • Fixes the ZipFile Write Snap wherein the Snap gives a Null Pointer Exception error if no input data is received by it during execution. 
  • Fixes the Email Sender & Reader Snaps where the Snaps fail to execute due to resource exhaustion on the Snaplex.
Transform transform8738
  • Fixes the error that occurs while transforming an XML document into string by retaining only one version of Saxon library – net.sf.saxon:Saxon-HE:9.6.0-10 across the Transform Snap Pack.
  • Fixes null handling in the Aggregate Snap whereby changing the order of entries in the Aggregate fields field set results in different outputs when the input documents contains one or more null values.
  • Fixes the issue with the Excel Parser Snap whereby the Snap does not parse the date and time correctly for custom date formats.
Email email8734Fixes the Email Sender Snap wherein the Snap fails to read attachments from the SMB3 server.

SOAP

 soap8712Fixes the SOAP Execute Snap where the Snap throws an NPE error when generating a SOAP request template.
Azure Active Directory azureactivedirectory8713

Fixes Azure Active Directory Snaps where the Snaps fail after running approximately an hour when the access token times out, and the Snap fails to detect the updated token.

Binarybinary8710
  • Fixes the ZipFile Write Snap where the Snap generates files with incorrect names when the Use input view label checkbox is selected.
  • Fixes the File Writer Snap where the Snap fails intermittently when there is a network disruption.
  • Fixes the File Reader Snap where the Snap generates an output view, along with the error view, if there is an error when using the Google Service account.
Emailemail8711

Fixes the Email Sender Snap where the Snap:

  • Throws a null pointer issue when using an SMTP account.
  • Fails to read a file from SMB servers if the file name contains special characters.
DynamoDB
dynamodb8709

Fixes the DynamoDB Scan Snap and the DynamoDB Update Snap where:

  • The DynamoDB Scan Snap is unable to produce a preview output on validation. (Instead, the Snap passes the output to the downstream Snap.)
  • The DynamoDB Update Snap produces two error documents for the same error during preview output.


eXtreme (Big Data) 

New Features and Enhancements

  • Cross Account IAM Role Support: Users can now specify cross-account IAM role for an AWS account to spin up an eXtremeplex on AWS EMR without needing to share secret and access keys of their AWS accounts. 

  • Enhanced Account Encryption: Introduces Enhanced Account Encryption (EAE) support for AWS accounts. This enables you to encrypt and secure account credentials that access endpoints from SnapLogic using a private key/public key model. The data is encrypted with a public key before it leaves the browser, then decrypted with a private key on the eXtremeplex.
  • Publicly Available Datasets: Provides free limited datasets in our public AWS S3 bucket and Azure Databricks clusters. You can use these datasets to try out the eXtreme use case Pipelines. Alternatively, you can also generate your own TPC-DS benchmark datasets to use with these Pipelines. 

New Snaps

  • Spark SQL 2.X Snap Pack: Enhances the Snap Pack to support Java Database Connectivity (JDBC). This enhancement adds the following Snaps and account type:
    • JDBC InsertInserts data into a target table through a JDBC connection. 
    • JDBC SelectFetches data from a target table through a JDBC connection. 
    • JDBC Storage Account: Enables you to connect to databases that support JDBC.

Known Issue

  • eXtreme Execute Snap Pack: The Spark job continues being in the running state even after the SnapLogic Designer/Dashboard displays the Pipeline status as failed on AWS EMR. 


Prior 4.20 eXtreme Dot Releases in 4.21 GA 

The following 4.20 eXtreme (Big Data) Dot Release update is now a part of the 4.21 GA release.

 Click here to expand...

eXtreme-mode Snaps

Update DateBuildUpdate
 
N/A

Fixes an issue with eXtreme Pipelines where clicking Data Visualization in Snaps in which null values are being passed can produce internal server errors.



4.21 UAT Delta

This section tracks the changes made during the iterative pushes to the UAT server and the GA release. The schedule is:

  • UAT #1: Friday, April 24, 2020 (Release updates are published above)
  • UAT #2: Friday, May 01, 2020
  • GA: Saturday, May 09, 2020 (9 p.m. PT)

The SnapLogic GA build on the production server is also deployed on the UAT server.

UAT #2 Fixes

Platform

  • Fixes an issue where a Triggered Task invoked with a Groundplex URL fails to execute under the following two conditions:

    • The Snaplex contains a FeedMaster node.

    • The Do not start new execution if one is already active option in the Create Task dialog is selected.

  • Fixes an issue with a Scheduled Task where a Snaplex node is put into maintenance mode, and the Scheduled Task, which should be executed on a different node in the Snaplex, does not execute.

  • Fixes an issue where the expression toggle display does not render correctly for some users.

  • Fixes an issue where the Help hook for Pipeline execution statistics redirects to an incorrect documentation page.

  • Updates the Task monitoring display to make today's date default.
  • Fixes an issue where the Task Monitoring Dashboard does not display all metrics for a specified hour when seeking metrics for Tasks older than 24 hours.

Standard-mode Snaps

  • None

eXtreme-mode Snaps

  • None

4.21 Dot Releases

Platform


Date of UpdateSnaplex BuildUpdates
master-7588 - 4.21 Patch 3This patch is applicable to only select customers and is not recommended. 
 master-7513 - 4.21 Patch 2 (Restricted)This patch is applicable to only select customers and gives the option of reverting the ReplaceAll expression functionality to the 4.20 release. Contact Customer Support for more information.



master-7451 - 4.21 Patch 1 (Recommended)




  • Fixes an issue where if you invoke Triggered Tasks with only basic authentication without a bearer token, the IP address of your machine is exposed in the HTTP header.
  • Fixes an issue where the retry mechanism in an Ultra Task does not work when the input message is not acknowledged, hence removing the message from the FeedMaster queue.
  • Fixes an issue with the OAuth2 authentication across the platform where the authentication for endpoints intermittently fail because of timeouts.
  • Fixes an issue where a Pipeline does not complete an execution when invoked as a Triggered Task.
  • Improves the error messages that result from Pipelines not executing when in a queued state.
  • Fixes an issue where using HDFS ZipReader and HDFS ZipWriter Snaps as well as the Hadoop Directory Browser in Pipelines that run on Cloudplex instances result in a security exception error.

UI


Date of UpdateUpdates
Enhances Task and Snaplex error message responses.







  • Fixes an error with the eXtreme Accounts display, where you click Validate on the Create Account dialog, but the account creation fails even though the values of Access-Key ID and Secret Keys are correct.
  • Fixes an issue where the Pipeline field in the Pipeline Execute Snap with the New Form UI enabled does not accept user input.
  • Fixes an issue where refreshing Dashboard > Tasks causes the Start and End Date selectors to change color.
  • Fixes an issue where the Pie Chart in the Data Preview Visualization display for an eXtreme Pipeline does not render undefined values as null, as in Standard mode.
  • Fixes an issue with the Designer Quick Tour where clicking Get Started > Next does not upload and display a Pipeline with the Salesforce Reader Snap.
  • Fixes an issue where the downloadable CSV file from Dashboard > Task subtab also includes details for the day after the date you specify.
  • Fixes the doc links for Activity LogPipeline Execution StatisticsManagerAlerts, and Dashboard > Task displays.

Standard-mode Snaps


Snap PackUpdate DateBuildUpdates
Binary 421patches6136 Fixes the issue with Multi File Reader Snap that continues to look for a directory even when FileSystem information is unavailable.
Azure SQL, Cassandra, Hive, JDBC, MongoDB, MySQL, Oracle, PostgreSQL, Redshift, SAP HANA, Snowflake, SQL Server, Teradata, Vertica 421patches6272

Fixes the issue where Snowflake SCD2 Snap generates two output documents despite no changes to Cause-historization fields with DATE, TIME and TIMESTAMP Snowflake data types, and with Ignore unchanged rows field selected.

Binary421patches6090Fixes the 'Connection pool shut down' error in S3 File Writer Snap by keeping the AmazonS3Client class object open until all the documents are processed.
SOAP 421patches6040Requires the use of a generic envelope while executing a create request through an NTLM account in the SOAP Execute Snap to ensure the output is rendered as expected.
Azure SQL, Cassandra, Hive, JDBC, MongoDB, MySQL, Oracle, PostgreSQL, Redshift, SAP HANA, Snowflake, SQL Server, Teradata, Vertica 421patches6144Fixes the issue with DB Snaps where a connection thread waits indefinitely causing the subsequent connection requests to become unresponsive.

Fixes the issue due to connection leaks during Pipeline execution, across DB Snaps.

Fixes the exception RefCnt has gone negative across the Snaps in Snowflake Snap Pack.

Confluent Kafka 421patches6136

Enhances the Kafka Consumer Snap with an option to allow reprocessing of records that remain unacknowledged due to acknowledge timeouts. This ensures that all records from Kafka are processed.

Transform 421patches5901Enhances the JSON Generator Snap to include pass-through functionality where the Snap embeds the upstream input document under the original field of the output document along with other records.
Binary 421patches6077Fixes the UnknownHostException error in Binary Snaps due to intermittent failures in FileSystemProvider registration, by replacing ServiceLoader with direct instantiations of FileSystemProvider class objects.
Binary 421patches5994

Fixes an error in Multi File Reader and File Delete Snaps during Pipeline execution where the Snaps fail to retrieve the temporary files from the given S3 location.

Binary 

421patches5963

Enhances the Directory Browser Snap by making the File filter field optional.

SharePoint Online 421patches6011 (Latest and Stable)Introduces the productized version of SharePoint Online Snap Pack, which enables organizations to share and manage content and collaborate across an organization using the following SharePoint Online Snaps:
Binary 421patches5975

Fixes the Directory Browser Snap that fails with an Unable to create filesystem object for smb:// error, by rolling back the SMB client library to the prior working version as in snapsmrc542

Transform 421patches5848Fixes the Sort Snap that fails while performing sorting, displaying a NoClassDefFoundError. 
Azure SQL 421patches5864Adds support for UTF_8 characters with BCP (bulk copy program) command to the Azure SQL Bulk Extract Snap.
Splunk 421patches5851Fixes the Splunk Snaps that fail to route connection errors to error view, thus aborting the Snaps.
Confluent Kafka 421patches5862Fixes the Confluent Kafka Consumer Snap that aborts abruptly upon getting a CommitFailedException error. Upgraded the Kafka client library from version 2.2 to 2.5.
Transform 421patches5848Fixes the Excel Formatter Snap that fails to create an empty worksheet when you do not select the Ignore Empty Stream checkbox.
Microsoft Dynamics AX 421patches5851Fixes an issue in the Microsoft Dynamics AX Execute Snap that fails when using the Apache HTTPComponents library API if the request XML has Unicode characters.
Binary  421patches5871When configuring the SSH Auth account type, SSH passphrase is now optional, whereas it was a required field earlier. 
Anaplan 421patches5851Fixes the Anaplan Action Snap that outputs the same value for TaskID and TaskName parameters.
Coupa 421patches5851Fixes the Coupa Read Snap that is unable to read all the available records.
DynamoDB 421patches5851Removes the Snap timeout setting while waiting for the threads to complete processing.
Hive 421patches5851Fixes the Hive Execute Snap that fails with a java.lang.NullPointerException error.
REST rest8858Fixes the REST Get and REST Post Snaps that does not parse Content-Encoding and Content-Type headers for gzipped responses.
Snowflake db/snowflake8860

Adds a new field, Handle Timestamp and Date Time Data, to Snowflake Lookup Snap. This field enables you to decide whether the Snap should translate UTC time to your local time and the format of the Date Time data.

MS Dynamics 365 for Sales dynamics365forsales8856Fixes Pipeline validation error that occurs when writing data from Dynamics 365 for Sales Update Snap to output documents.
Flow flow8855Fixes the Binary Router Snap's failure in Ultra Pipelines by preventing a deep-copy (performing a copy of the original objects recursively) of binary data headers.
DynamoDB dynamodb8854

Fixes the DynamoDB Scan Snap pagination issue where the second page of output and beyond cannot be accessed during execution and validation.

Hadoop hadoop8853Updates the Parquet Writer and Parquet Reader Snaps to support the yyyy-MM-dd format for the DATE logical type.
Binary binary8849Enhances the Directory Browser Snap by adding a new advanced property Use fast file-type check for WASB/WASBS protocols for quick Snap execution. 

Binary

 

binary8843

Fixes the issue in PGP Encrypt Snap where decrypting its output file using a third-party PGP software adds a random suffix and TMP extension in the file name. This issue occurs only when using the PGP Encrypt Snap in a Windows-based Groundplex.

MySQL mysql8842
  • Fixes the SSH tunneling issue by ensuring connectivity between JCC and MySQL server

  • Updates the UI for MySQL Accounts to simplify the SSH tunneling configuration as described in Configuring MySQL Accounts

Azure SQL
Cassandra
HiveJDBC
MongoDB
MySQLOracle
PostgreSQL
Redshift
SAP HANA
Snowflake
SQL Server
TeradataVertica

MULTIPLE8841

Fixes the connection issue in Database Snaps by detecting and closing open connections after the Snap execution ends.

SQL Serverdb/sqlserver8837Fixes the SQL Server - Stored Procedure Snap to support the UUID datatype.
Salesforce salesforce8829Fixes the Salesforce Create, Salesforce Update, Salesforce Bulk Create, Salesforce Bulk Update, and Salesforce Bulk Upsert Snaps, enabling you to upload files containing duplicate values across multiple fields.

eXtreme (Big Data)

Snap PackUpdate DateBuildUpdates

eXtreme Execute 

 421patches5928
  • Adds Hierarchical Data Format v5 (HDF5) support in AWS EMR. With this enhancement, you can read HDF5 files and parse them into JSON files for further data processing. See Enabling HDF5 Support for details.
  • Adds support for Python virtual environment to the PySpark Script Snap to enable reading HDF5 files in the S3 bucket. You can specify the path for the virtual machine environment's ZIP file in this field.

eXtreme ExecuteSpark SQL 2.x 

 

421patches5851

Optimizes Spark engine execution on AWS EMR, requiring lesser compute resources.

N/A
 
master-7451 – recommended

Fixes the following issues in the DataViz feature:

  • Unicode characters do not render, displaying an internal server error.
  • Columns containing null values are not displayed in the charts.