In this article:

Platform 

New Features and Enhancements

• Anchor
  AssetPalette AssetPalette

  Asset Palette: Introduces a new opt-in feature, providing a dynamic display of the Catalog for Snaps, Pipelines, and Pattern Pipelines. You can enable this new display through the User Settings dialog in Designer. In addition to freeing up more space on the Designer Canvas for you to build your Pipelines, the redesigned Asset Palette offers Pipeline Recommendation capabilities by using Convolutional Neural Networks that improve the quality of suggested Snap, Pipeline Pattern, and Pipeline through personalization. Iris recommends these Pipelines based on the Snaps you use. 
  
  You can take advantage of the following new capabilities when you use the opt-in Asset Palette:
  • Snap Search
  • Pipeline Search
  • Pattern Pipeline Search
  • Documentation Search in SnapLogic Designer

• Anchor
  ExpressionBuilder ExpressionBuilder

  Expression Recommendation: Enhances the suggestion capabilities of Iris to include Expression Recommendation. In Snaps that use expressions (for example, Mapper Snap), the Iris Expression Catalog enables you to receive expression recommendations based on the input schema, and expressions in the Pipelines from projects where you have read access.

Image Modified

• Enhancing the UI icons for the Expression Toggle to better distinguish between On and Off modes: 

Expression Toggle Off–Enter a value as a string.
Image Modified

Expression Toggle On–Enter a value as an expression.
Image Modified

When you click , you can enter an expression in the adjacent field by clicking () to display a drop-down list of available expressions.

• Additions to the SnapLogic Expression Language:
  • toLocaleString: When formatting a date, you can now pass a locale language-tag in the options object.
  • Array Functions: The following array functions are now available for Uint8 arrays.
    • Uint8Array.Of: Creates a new Uint8 array from a variable number of arguments.
    • Uint8Array.subarray(): Returns a new subarray from an Uint8 array, where the arguments are the indexes begin and end offsets.
    • Uint8Array.indexOf(): Returns the first index at which a given element is present.
    • Uint8Array.lastindex.Of(): Returns the last index at which a given element is present.
    • concat: Returns a new array that concatenates the initial array with the arrays or values provided as arguments.
      
      
• Node Alerts: You can now view Snaplex node alerts on the Dashboard, where other information about your Pipeline executions is displayed. This change centralizes the location of monitoring information, enhancing the usability of this reporting functionality. Accordingly, these alerts no longer appear in the Activity Log in Manager.
  
  
• New Pipeline Parameter: You can now capture the start and end times of Tasks with task.start and task.end, which you can pass as an expression in a Mapper Snap. This capability is particularly useful for Scheduled Tasks, where you view the last execution window.
  
  
• Size Limitation on Pipeline Parameters: In this release (February 2020, 4.20), an additional limit is added which caps the total size of the Pipeline parameter values for one Pipeline execution (including parameters that are not captured) to 8MB. If the Pipeline parameter values exceed 8MB in size, then the Pipeline does not execute.

Known Issues

When you create or update a Task, the dialog displays a checkbox labeled Anomaly Notifications in preparation to enable a future functionality. Selecting the checkbox does not actually trigger any notifications.

Standard-mode Snaps

New Features and Enhancements

• If you have built custom Snaps, then we strongly recommend you to recompile your custom Snaps using our latest SDK, so that your Snaps can take advantage of the latest performance optimizations and security updates. 

New Snaps

• Flow Snap Pack: Introducing the Gate Snap, which relies on all upstream Snaps successfully executing before you send their consolidated contents as a single document to the downstream Snaps.
  
  
• Salesforce Snap Pack: Introducing the Salesforce Bulk Query Snap, which enables you to perform bulk SOQL queries on Salesforce.
  
  

• Anchor
  SuccessFactors SuccessFactors

  Introduces the SuccessFactors Snap Pack, which enables you to work with the human capital management (HCM) system, covering everything from payroll to employee engagement. This Snap Pack includes the following Snaps:
  
  • SuccessFactors Create: Creates data objects in the SuccessFactors Data Center server.
  • SuccessFactors Delete: Deletes data objects from the SuccessFactors Data Center server.
  • SuccessFactors Read: Retrieves data objects from your SuccessFactors instance.
  • SuccessFactors Update: Updates data objects from the SuccessFactors Data Center server to a selected item.
  • SuccessFactors Upsert: Updates or inserts data objects from the SuccessFactors Data Center server to a selected item.

Updated Snap Packs

• Active Directory Snap Pack
  • View all elements of array fields in Search and List Users Snaps: Fixes the Search Snap and the List Users Snap wherein the Snaps displayed only the first element of an array field. Now, the Snaps display all the elements of an array field.
    
    
• Flow Snap Pack: The Pipeline Execute Snap now provides the following capabilities:
  • Specify the Snaplex on which to run child Pipelines. You can now select the Snaplex on which to run the child Pipelines through the Execute On and Snaplex Path fields. The default option is executing the child Pipeline on the same Snaplex node as the Parent, but you can also specify a different Snaplex through the drop-down list, which includes the option to manually enter a Snaplex name.
    

  • Anchor
    PipelineExecuteRetry PipelineExecuteRetry

    Configure retry logic for child Pipeline executions. You can now configure retry logic for child Pipeline executions though the following fields: Number of Retries, Retry Interval, and Timeout fields. 
    
    
• REST Snap Pack
  • Retain original file names when posting to Google Drive: The new Multipart related option in the Upload body type property in the REST Post Snap enables you to post files to the Google Drive API and retain the original names.
  • Obtain access tokens from REST endpoints automatically: The REST Account now has a new Grant Type option called Client Credentials. This enables the Snap to obtain an access token to the client ID and client secret through the token endpoint URL.
    
    
• ServiceNow Snap Pack
  • Query both autogenerated and custom fields in ServiceNow: The ServiceNow Query Snap enhances its scope to include both autogenerated and custom fields.
    
    

• Workday Prism Analytics Snap Pack
  • Specify bucket completion timeout values: The Workday Prism Analytics Bulk Load Snap now has a new field, Bucket Completion Timeout, which enables you to specify the maximum number of minutes for which you want the Snap to wait for the Workday Prism bucket to complete the bulk upload.
    
  • Work with Workday Prism API v1 and v2: The Workday Prism Analytics Bulk Load Snap and the Workday Prism Analytics Account now support both v1 and v2 versions of the Workday Prism API.

Deprecated Snaps

• LinkedIn Snap Pack: In this release, we retire the LinkedIn Snap Pack from the SnapLogic Snaps Catalog. Pipelines having the LinkedIn Snaps will no longer work.

Known Issues

• Excel Parser Snap: Does not read password-protected Excel files and fails with the error: "Failure: Failed to parse input data."
• PostgreSQL-Execute Snap: Does not support $$ as an escape character for enclosing string input, as $ is a reserved character for variable substitution in SnapLogic.
• Cassandra-Execute Snap: The Cassandra-Execute Snap using Apache Cassandra V3 does not support $$ as an escape character for enclosing string input, as $ is a reserved character for variable substitution in SnapLogic.

• Generic JDBC-Execute Snap: If a SELECT query in the SQL statement field in the Snap Settings contains duplicate column names, then the query result displays the column name twice, with the latter mention prefixed with the table name. If the column name occurs more than twice, the second entry is still prefixed with the table name but it displays the value of the last mention of the column in the query. The example below illustrates this behavior. 

  Expand
  title Example: Incorrect results for SELECT queries with duplicate column names
  Example Consider the following query, where the column name "name" is repeated three times:  select id, name, city, location, id as name, city as name, location as name from address Expected output: The  Generic JDBC - Execute Snap's output should reflect the correct query result containing all the seven fields, as shown in the image below: Actual output: The Snap's output displays the repeat occurrence of the column "name" only once, prefixed with the table name "address" as shown in the image below. Further, it holds the value of the table column "location", which was mentioned last in the SELECT query.

  To avoid this issue, we recommend that you give unique column names in the query.

• SOAP Execute Snap: After upgrading your Snaplex to 4.20 GA version, Pipelines with SOAP Execute Snap that uses the Apache CXF library might hang without executing. The Snap using HTTPComponents library executes successfully.
  
  
• All Database Write (Insert/Update/Upsert) Snaps: When you set Auto commit to True in a database Snap Settings and Number of retries to more than 0 the retry functionality inserts duplicate records if the Snap tries reconnecting to the database. To avoid this error, we recommend you to do one of the following depending upon your use case:
  • Set Auto commit to false, or 
  • Set Number of retries to 0 to avoid duplicate records passing downstream while executing a Pipeline, or
  • Use a primary key in your database that does not allow duplicate records to be inserted into the database, or

  • Use an Upsert Snap instead of Insert Snap that overwrites any duplicate records. 

• NetSuite SOAP Snap Pack: The following NetSuite Snaps currently do not support custom records: NetSuite Get, NetSuite Get List, NetSuite Async Get, and NetSuite Delete.
  
  
• Reltio Snap Pack: You cannot roll back the productized Reltio Snap Pack to a prior version due to an upgrade to the platform-provided AWS SDK version that it leverages.
  
  
• Transform Snap Pack: If you are using OpenJDK 11 in the Snaplex on which you want to execute Excel Formatter Snaps, you must install Fontconfig on the Snaplex. Otherwise, the Snap displays an error.
  
  
• Hadoop Snap Pack:
  
  • After upgrading your Snaplex to 4.20 GA version, Pipelines with HDFS Reader Snap that use Kerberos authentication might remain in the start state.
  • ORC Writer/Reader Snaps fail on S3 when using the 4.20 Snaplex with the previous Snap Pack version (hadoop8270 and snapsmrc528) displaying this error – Unable to read input stream, Reason: For input string: "100M" error 
  • ORC Reader/Writer and Parquet Reader/Writer Snaps fail when executing a Pipeline on S3 with this error – org.apache.hadoop.fs.StorageStatistics


• Diff Snap (Standard mode): When using the new UI form to create new or editing existing Pipelines with Diff Snap, Output view name values go missing in Snap Settings after switching to Views or Info tab. The workaround is to switch back to the old UI form (disable New UI Form in User Settings) and then create the new Pipelines or edit existing Pipelines.
  
  
• Workday Read Snap:
  • Using the old UI form, the Snap does not find existing Workday accounts.
  • Using the new UI form, the Snap crashes when validating a Pipeline displaying a null pointer exception error.
    
    
• A Null Pointer Exception error displays when updating Snaps that contain empty property values in the new UI form. The existing Pipelines are not impacted unless a change is made on the Snap and saved. A workaround is to switch to the old UI form (disable New UI Form in User Settings). 
  

eXtreme-mode Snaps (Big Data) 

New Features and Enhancements

• Anchor
  eXtremeDataPreview eXtremeDataPreview

  Data Preview Enhancement: Displays Snap output data as pie, bar, line, and scatter charts. You can see the entire dataset while navigating through your data records in the paginated tabular view. Apart from offering better controls for navigating through your previewed data per Snap, the enhanced Data Preview form provides multiple options for plotting the various categories of data. 
  
  
  You can also select the number of records (up to 2000) that you want to preview through the settings menu in SnapLogic Designer. Go to SnapLogic Designer > User Settings > Preview doc count. The default is 50 records. 
  
  
  

• Anchor
  eXtremeAzureAccount eXtremeAzureAccount

  eXtreme Execute Snap Pack: Introducing a new account type, Azure Databricks Account /wiki/spaces/AP/pages/891289679. This enhancement makes account configuration mandatory for the PySpark Script and JAR Submit Snaps.

  Note
  title Breaking Change
  Spark 2.x PySpark Script and JAR Submit Snaps now require an account to be configured for the Pipelines to successfully execute. Hence, all existing and new eXtreme Pipelines having either of these Snaps will need either an AWS or Azure account configured.

  
  

• Spark SQL 2.x Snap Pack: Introducing a new account type, Amazon Web Services (AWS) Account, to support object encryption using AWS Key Management Service (KMS). This enhancement makes account configuration mandatory for the Spark 2.x File Writer and File Reader Snaps. 

  Note
  title Breaking Change
  Spark 2.x File Writer and File Reader Snaps now require an account to be configured for the Pipelines to successfully execute. Hence, all existing and new eXtreme Pipelines having either of these Snaps will need either an AWS or Azure account configured.

  
  

New Snaps

• Anchor
  eXtremeSample eXtremeSample

  Spark SQL 2.x Snap Pack: Introducing the Sample Snap, which enables you to generate a sample dataset from the main dataset. You can use the sample dataset to test Pipelines, thereby saving resources while designing Pipelines.

Updated Snap Packs

• Delta Lake Snap Pack: Renames the Table Action field in the Delta Lake Insert Snap to Table Insert Policy.
• eXtreme Execute Snap Pack: Enhances the PySpark Script Snap to display the Pipeline Execution Statistics after a Pipeline with the Snap executes successfully.
• eXtreme Snowflake Snap Pack: Adds a new field, Table Insert Policy, in the Snowflake Insert Snap to handle duplicate records in the insert statement.

Deprecated Features

• We have ceased support for Hadooplex and Spark 1.6 Snaps with this release. 

Known Issues

• eXtreme Pipelines configured with Azure Databricks Accounts having Snowflake Select and Snowflake Insert Snaps need at least one additional eXtreme Snap in between them for the Pipelines to successfully execute.  
• Data visualization for eXtreme Pipelines on AWS EMR gives an Internal Server Error in DataViz in the following conditions:
  • when you have null and nested structure in preview data, or
  • when previewing complex objects (such as pie charts).
• eXtreme Pipelines on Azure Databricks with Jar Submit Snap fail with the job canceled as the SparkContext shuts down.
  
  

Prior 4.19 eXtreme Dot Releases in 4.20 GA 

The following 4.19 eXtreme (Big Data) Dot Release update is now a part of the 4.20 GA release.

4.20 UAT Delta

4.20 Dot Releases