/
August 2020, 4.22 Release Notes

August 2020, 4.22 Release Notes

In this article

About SnapLogic Releases

  • Per the SnapLogic Release Process, all remaining Snaplex instances across customer Orgs are upgraded to the recommended 4.22 version (main-8237) at 9 p.m. PT on September 12, 2020. Customer orgs who have already migrated to any of the 4.22 GA versions will not be impacted. 
  • To use the new SnapLogic features and Snaps in the 4.22 release, ensure that your Snaplex nodes are upgraded to the main-8237 4.22 Patch 4 (Recommended) version.
  • This 4.22 GA release includes the patches deployed to SnapLogic after the 4.21 GA release, as listed in the 4.21 Dot Releases page. 
  • We recommend that you upgrade to the latest Chrome or Firefox browser version and clear your browser cache before logging into the SnapLogic Intelligent Integration Platform (IIP).


Supporting Java 11 and Sunsetting Java 8

  • Groundplex—Starting from the 4.18 August, 2019 release, SnapLogic supports Java 11 for Groundplex by default. We require all customers to migrate to Java 11, since Oracle no longer releases security updates for Java 8. As previously announced, starting from this 4.22 release, we no longer support Java 8. For details, see Configuring the Java Version on Your Groundplex.
  • Cloudplex—Starting from the 4.21 release, all your existing Cloudplex instances hosted in the United States are migrated to Java 11. For details, see Migrating Your Cloudplex Nodes to Java 11.


New IP Whitelisting Requirements Reminder

We remind all customers of the new IP Whitelisting requirements that have been in effect since the August 2019, 4.18 release. Ensure that your network administrators whitelist all the given IP addresses for your Snaplex August 31, 2020. For more, see Adding the SnapLogic Platform to your Allowlist.

Platform 

New Features

  • Native API Policy Manager: Introduces SnapLogic native API Policy Manager, an in-house solution that enables you to apply user authorization and traffic shaping API Policies to your Triggered and Ultra Tasks at the folder level. Included are the following policies:
    • Generic OAuth2.0: Apply OAuth2.0 policies for the authentication method for your API consumers.
    • API Key Authenticator: Apply a unique API Key as the authentication method for your API consumers.
    • Anonymous Authentication: Allow anonymous access to your APIs by creating roles (See Authorization by Role).
    • Callout Authentication: Apply this authentication method that uses a REST service to validate tokens against API user credentials.
    • Authorization by Role: Create role-based authorization to work with your authentication policies.
    • IP Restriction: Restrict access based on client IP addresses.
    • CORS: Apply policies for request headers to allow requests from different domains to prevent browser blocking.
    • Client Throttling: Apply policies for throttling or rejecting client requests over a period of time.
    • Request Size Limit: Restrict the size of request content that exceeds the target limit.
    • Request Transformation: Apply policies for transforming a request before other API policies are applied.
    • Early/Authorize Validation: Apply generic validation to create conditions under which your APIs can be accessed before the underlying Pipeline execution is triggered.

  • API Dashboard: Coinciding with the API Policy Manager, introducing an API dashboard where you can view key metrics—such as requests per minute, error percentage, 99th percentile of latency, and calls within a specified time period.

  • Snaplex Monitoring Dashboard: Introduces the new Snaplex Dashboard Wall, enabling users to track resource metrics more easily with an enhanced design that replaces the legacy display of the Snaplex tab. Selecting a target Snaplex and the time period generates a summary report and stack area charts for key resource metrics. Included in the summary report are the following:
    • Number of nodes and cores
    • Total memory
    • System load
    • Utilization of memory and CPU resources
    • Network traffic
    • Active Pipelines per minute
    • Disk utilization

    • File descriptors

    • Active threads

  The stack area charts, which you can move around and remove/add from the Dashboard, display these utilization metrics per the specified time period.

Existing Snaplex Monitoring Dashboard

New Snaplex Monitoring Dashboard


You can also maximize the chart to populate the entire screen when the chart displays many nodes:

  • API Metrics Public API: Introduces three new public APIs where as an Org admin you can collect the concurrent and daily usage stats of your SnapLogic APIs (Pipelines executed as Tasks). These Public APIs are an extension of the Pipeline Runtimes APIs. Prior to the 4.22 release, you could only retrieve this information by logging into SnapLogic Manager as an Org admin. Now you can programmatically retrieve this information.


Enhancements

  • Improved Task Scheduling: Introduces a new Scheduled Tasks implementation for Tasks that run every five minutes or less (excluding cron Scheduled Tasks). Currently available for your developer Orgs as an opt-in feature, the following are the highlights of the new implementation:
    • Improves the accuracy of Scheduled Task executions for Tasks that run every five minutes or less.
    • You do NOT have to update your Scheduled Tasks.
    • Legacy Scheduled Tasks do not stop.

For users interested in this feature, ask your Org admin to contact SnapLogic Support.

  • Task Monitoring for Ultra Tasks: Adds support for Ultra Tasks on the Task dashboard. You can now view task monitoring information of your Ultra Tasks.

  • Universal Search: Updates the Universal Search to include results from SnapLogic Community posts. When you use the Universal search at the main toolbar at the top of SnapLogic Designer, you can enter a search term with results displaying from both Documentation and the Community

  • Snaplex Version Designation: Improves the underlying mechanism for designating Snaplex versions (Recommended, Restricted Distribution, Deprecated). Because multiple patches of a Snaplex Version may exist after the GA release, the Snaplex Version label is improving with an accurate designation being generated automatically when the version is made available.

  • Update Service Accounts via Public API: The User Management Public API is enhanced to include a new parameter called, created_by, which allows Org admins to reset service accounts whose creators' user accounts no longer exist in the Org.

  • Updated UI Dialogs: Updates both the User Settings and Enhanced Account Encryption dialogs for a more efficient design. 

Known Issues

  • If you upload a file to the File Reader Snap using the old UI, then switch to the New Form UI and preview the data, you see the File cannot be loaded error message. Reloading the browser page resolves the issue.
  • The Dashboard > Snaplex tab may display inaccurate metric values in the System Load chart and Overview section for a Windows-based Snaplex.
  • Snaplex activities, such as entering/exiting maintenance mode or restarting, may not appear in the Activity Log.
  • If your Pipeline has a MySQL Select Snap, the Pipeline Execution Statistics may not display host and port details.
  • The charts on Dashboard > Task may display inaccurate metrics for Tasks that have been deleted.
  • The underlying search mechanism behind the universal search in the SnapLogic platform does not include close matches from typos or misspellings in the list of results.

New Form UI is by Default Enabled in User Settings

In the SnapLogic 4.22 (August 2020), the New Form UI is the default setting in the Designer User Settings. The New Form UI has the following benefits:

  • Better Performance—You may notice that it takes less time to render large schemas than in the old UI. In general, the technology underlying the New Form UI reduces lag-time when switching screens.
  • Enhanced Feature and Functionality—Enables increased utilization of Iris integration assistant, resulting in features like Expression RecommendationSnap Auto-configuration, and Target Recommendation.
  • Improved User Experience—Enables the ability for dynamic forms in Snaps by introducing conditional logic. For example, in the Accounts Settings tab, the Snaplex options in Pipeline Execute Snap, and the Mapper Snap.
  • Strengthened Security—The code base for the legacy UI is outdated, and some of the referenced older third-party libraries are no longer supported. 
  • Updated Display—The New Form UI utilizes screen real estate efficiently, eliminating extraneous white space and improving alignment of components.

You can still revert to the old UI by deselecting the User Settings dialog for one more release, but we encourage you to select the New Form UI. While the functionality is the same in both UI displays, you might see these minor differences in the New Form UI:

  • All code editors (such as JSON Generator and Script Snaps) feature different grammar checking and highlighting.
  • In the Mapper Snap, the Load next Data button in the preview data fields does not appear because all the data renders at once; use the scroll bars instead.

Mapper Snap in the old UI

Mapper Snap in the new UI

  • When you resize the Mapper Snap, the Input Schema field flickers. As a work-around to this known issue, on a Mac you can navigate to System Preferences > General > Show scroll bars, then click When scrolling.
  • Because the Account tab in Snap dialogs are conditional, some buttons are not actionable because account information is being loaded.

Adding an Account in the old UI

Adding an Account in the new UI

Changes in Platform Behavior

Modified Error Messages

Starting in the SnapLogic 4.22 Release, our Public API error responses have some fields removed, and error messages are now being HTML-encoded.

Project Migration Public API Changes

The parameter async in the Project Migration Public API is renamed to async_mode to avoid conflicts where async is used elsewhere in some scenarios.


Standard-mode Snaps

Starting from the 4.22 release, we are changing the Snap build naming convention to main<build#> from snapsmrc<build#>.

Updated Snap Packs 

  • Starting from the 4.22 release, Adobe Cloud Platform (ACP) is deprecated in favor of Adobe Experience Platform (AEP) Snap Pack, which has the following updates:
    • Enhances the Adobe Experience Platform S3 Connector Snap with an option to save data to a new dataset. 
    • Adds a new Adobe Experience Platform Read Snap to read JSON or Parquet files from the Adobe Experience Platform.
  • Binary Snap Pack
    • Adds the Cross Account IAM Role feature to the existing accounts, AWS S3 Account and S3 Dynamic Account, to enable the AWS account user to access the resources of another AWS account for a specified time without sharing the login credentials.
    • Enhances the File Operation Snap by adding a new property Simple file operation that allows moving or copying a file, through FTP protocol, irrespective of the account's access permissions on source and target directories. 
  • Cassandra Snap Pack
    • Introduces a new account type Cassandra SSL/TLS Database account. Enhances encryption by ensuring data is transferred securely. 
    • Enhances the account type by enabling expressions for required fields to support dynamic account. 
  • Kafka Snap Pack (Deprecated): This Snap Pack is deprecated and is replaced with the Kafka Snap Pack.  
  • RabbitMQ Snap PackEnhances the RabbitMQ Consumer and RabbitMQ Producer Snaps to allow defining custom arguments to ensure that all declarations for the queues use the same configuration/options/arguments. 
  • REST Snap Pack:
    • Adds a new account type, REST OAuth2 SSL Account, which enables support for the OAuth2 SSL authentication for REST Snaps with endpoints that require client certificates.
    • Enhances the OAuth2 accounts by refreshing expired access tokens before connection retries.
  • SAP Snap Pack: Enhances the Snap Pack to support SNC-based authentication in SAP Accounts and the execution of BAPI call sequences in a Pipeline.
    • The SAP Accounts, SAP JCo Account and SAP Load Balanced JCo Account, can be used to extend connection security with an additional layer of Secure Network Communications (SNC) configuration by passing advanced JCo parameters.
    • Updates the SAP Execute Snap with the ability to execute a sequence of BAPI calls within the same Pipeline and session. Sequence implies that the output from a BAPI call is needed for the next BAPI call to run.
  • Script Snap PackUpgraded the Jython engine from version 2.7-b3 (a beta version from 2014) to the current version, 2.7.2 (March, 2020). See the breaking changes note for Script Snap and the deprecated Execute Script Snap below for potential impacts of this upgrade.   
  • Workday Snap Pack: Removes support for Workday WSDL Account. Pipelines and Snaps using this account type must switch to either Workday Account or Workday Dynamic Account.

Documentation Enhancements

Known Issues

  • NetSuite Snap PackWhen passing the NetSuite Token account to the Snap as an expression parameter in the Account setting, the suggestions for the Object Type field do not workThis issue does not affect existing Pipelines.
  • Salesforce Snap PackWhen passing the Saleforce.com account to the Snap as an expression parameter in the Account setting, the suggestions for the Object Type field do not work. This issue does not affect existing Pipelines.
  • Data Validator Snap: The Snap fails when using null/empty values for the Required option in the Constraints field. A possible workaround is to enter an arbitrary value, which the Snap will not use when executing the Pipelines.

Breaking changes for Pipelines using the Script Snap (or the deprecated Execute Script Snap) with the Python engine

To implement Python in the Script Snap we use Jython.

We recently upgraded the Jython engine from version 2.7-b3 (a beta version from 2014) to the current version, 2.7.2 (March, 2020). The following are the resultant issues and workarounds that we are suggesting:

  • There's an open bug in 2.7 that introduced a backwards-incompatible change in the SnapLogic platform wherein the Jython engine automatically converts BigInteger values to primitive long values. This impacts all your scripts that perform numeric manipulation of integer values from documents (SnapLogic uses the BigInteger type to represent integers in documents). Your Pipelines and Snaps with the Script Snap (or the deprecated Execute Script Snap) that use numeric manipulation scripts with integer or BigInteger data type may fail during execution. We recommend you to prospectively replace integer or BigInteger values with long values.

    Example:
    sum = a.intValue() + b.intValue()
    Here a and b are of BigInteger type that now fail as Jython 2.7.2 automatically and transparently calls longValue() on any BigInteger value it encounters. So a and b would need to use the long and not BigInteger type.

    The known fix is to rewrite the above calculation as sum = a + b by removing occurrences of .intValue() or .longValue() from your Python scripts. 

  • Before the 4.22 release (August 2020), when using the Script Snap with the Scripting language option selected as Python, requesting a key that did not exist in a dictionary (for example, my_dict['missing_key']) would return None. Starting from the 4.22 release, the same request now returns a KeyError exception. If you need to continue returning None, use the .get(key) method instead (for example, my_dict.get['missing_key']).

  • - zlib.compress():
 The zlib library compresses the JSON files retrieved from the SnapLogic APIs and backs-up Pipelines and accounts to a database. The following Python code, when trying to compress displays an ascii … ordinal not in range(128) error.
    Original code: in_doc["json"] = zlib.compress(in_doc["json"])
    Fix: in_doc["json"] = zlib.compress(in_doc["json"].encode("utf-8"))

  • {dictionary}.values().toArray()[i]:
 Prior to the 4.22 release (August 2020), to subscript a {dictionary}.values() method, you had to append the toArray() method to values(); else, you would see the Failure: ‘java.util.LinkedHashMap$LinkedValues’ object is unsubscriptable error. After the 4.22 release, toArray() returns Failure: ‘list’ object has no attribute ‘toArray’. However, the requirement for toArray() is no longer necessary for the subscript.
    Original code: sLine = data.values().toArray()[0]
    Fix: sLine = data.values()[0]


Prior 4.21 Snap Dot Releases in 4.22 GA 

The following 4.21 Snap Dot Release updates are now a part of the current 4.22 GA release.

 Click here to expand...


Snap PackUpdate DateBuildUpdates
Binary 421patches6136 Fixes the issue with Multi File Reader Snap that continues to look for a directory even when FileSystem information is unavailable.
Azure SQL, Cassandra, Hive, JDBC, MongoDB, MySQL, Oracle, PostgreSQL, Redshift, SAP HANA, Snowflake, SQL Server, Teradata, Vertica 421patches6272

Fixes the issue where Snowflake SCD2 Snap generates two output documents despite no changes to Cause-historization fields with DATE, TIME and TIMESTAMP Snowflake data types, and with Ignore unchanged rows field selected.

Binary421patches6090Fixes the 'Connection pool shut down' error in S3 File Writer Snap by keeping the AmazonS3Client class object open until all the documents are processed.
SOAP 421patches6040Requires the use of a generic envelope while executing a create request through an NTLM account in the SOAP Execute Snap to ensure the output is rendered as expected.
Azure SQL, Cassandra, Hive, JDBC, MongoDB, MySQL, Oracle, PostgreSQL, Redshift, SAP HANA, Snowflake, SQL Server, Teradata, Vertica 421patches6144Fixes the issue with DB Snaps where a connection thread waits indefinitely causing the subsequent connection requests to become unresponsive.

Fixes the issue due to connection leaks during Pipeline execution, across DB Snaps.

Fixes the exception RefCnt has gone negative across the Snaps in Snowflake Snap Pack.

Confluent Kafka 421patches6136

Enhances the Kafka Consumer Snap with an option to allow reprocessing of records that remain unacknowledged due to acknowledge timeouts. This ensures that all records from Kafka are processed.

Transform 421patches5901Enhances the JSON Generator Snap to include pass-through functionality where the Snap embeds the upstream input document under the original field of the output document along with other records.
Binary 421patches6077Fixes the UnknownHostException error in Binary Snaps due to intermittent failures in FileSystemProvider registration, by replacing ServiceLoader with direct instantiations of FileSystemProvider class objects.
Binary 421patches5994

Fixes an error in Multi File Reader and File Delete Snaps during Pipeline execution where the Snaps fail to retrieve the temporary files from the given S3 location.

Binary 

421patches5963

Enhances the Directory Browser Snap by making the File filter field optional.

SharePoint Online 421patches6011 (Latest and Stable)Introduces the productized version of SharePoint Online Snap Pack, which enables organizations to share and manage content and collaborate across an organization using the following SharePoint Online Snaps: