Winter 2016 (4.4.0) Launch Updates

In this Page

Live on production February 13, 2016.

Before You Begin

You may need to clear your browser cache before you log into the latest SnapLogic Elastic Integration Platform.

See the appropriate documentation for your browser:

You may see an Upgrade Notice dialog informing you that you will be logged out immediately to complete the update process. Because the SnapLogic Elastic Integration Platform consists of multiple applications, you will likely see this message multiple times.

Accounts and Platform Updates

Some accounts may have a fixed time for refresh tokens, such as Google accounts must be refreshed every hour. If that refresh needs to occur when the platform is down for an update, the refresh does not occur. To prevent these accounts from failing after a new platform deployment, it is recommended that you refresh your accounts before the designated down time.

Enhancements

General

  • You are now informed if you are using an unsupported browser version.
  • A maintenance page will display for downtime during future releases.
  • The size of the buffer between Snaps has been reduced from 1024 slots to 32 to reduce excess memory usage while streaming. It can be changed back by setting the jcc.disruptor_ring_buffer_size in the global.properties

Note: This functionality was rolled back with 4.4 Patch 2 (mrc232).

Designer

  • You can now rotate Snaps from the right-click menu. You have the ability to rotate the entire Snap or just the head (input) or tail (output) side. 
  • User Settings: The option to enable/disable sound effect when placing Snaps on the canvas has been added to the user settings.
  • Improvements were made to the validation process to make execution faster.
  • You can now open the pipeline stats dialog during a validation run to get more details on how it is progressing.
  • Pipeline Queueing: An execution in the Queued state will attempt to be restarted when resources become available on the Snaplex.

Manager

  • Settings: Pipeline validation can now be disabled at the org level.
  • Updates made to the Snaplex dialog in Manager for Pipeline Queuing.
  • Runtime Archive was added under Projects to move runtimes to a zip file to improve the performance. Archiving will be started when the platform has shown stability after the release.
  • Behavior Change: You can no longer set Tasks to start in the past.

Dashboard

  • Additional information is now available in the runtime history to inform you who stopped the pipeline.

  • Org admins can restart a Snaplex node from the Snaplex Health Wall.
  • More information is now available for Snaplex nodes (JCCs) under Additional Information.  
  • Snaplex nodes check their configuration on initialization and raise an alert if any issues are found. If alerts are present, a warning icon is displayed at the Snaplex level and node level within Dashboard for the org admin. Alerts for that node display below it.

Big Data

  • Support for Spark Pipelines
    • Spark Support added to the following Snaps: Aggregate, Copy, CSV Formatter, CSV Parser, Filter, HDFS Reader, HDFS Writer, Mapper, Router, Sequence Formatter, Sequence Parser, Sort, Union, Unique.
  • NEW! Spark Script Snap
  • NEW! Parquet Reader and Writer Snaps
  • NEW! ORC Reader and Writer Snaps
  • Behavior change: HDFS Writer in SnapReduce mode now requires the File property to be blank.

For information on Snap updates, see the Winter 2016 section of Snap Updates, 2016.

Migration Impacts

None

Known Issues

  • A pipeline with a Parquet Writer Snap followed by a Parquet Reader Snap will fail.
  • Nested pipelines that contain a ForEach or Task execute can fail with an authorization error. Resolved with mrc229.
  • Long running pipelines my not accurately reflect the status if the Status dialog is left open during the run. Workaround: Close the dialog and reopen to refresh it.
  • A SnapReduce pipeline does not read all Sequence or RC files in a directory if some of them are empty. Workaround: Delete empty files.
  • Spark Script Snap: YARN mode cannot use environment variables.
  • Sequence File Formatter does not support expressions. 
  • HDFS Reader and Writer do not support WASB/S protocols in Spark mode.
  • In certain circumstances, pre-existing pipelines may not show detailed runtime history.
  • Runtime Archive: Archiving will be started when the platform has shown stability after the release.
  • The reduced ring buffer change in the platform caused the Multi Join Snap to hang. Workaround: Select Unmatched data to error view to complete the pipeline successfully.