Enhancements

General

• Snap thread wait time added to Run Info stats.
• Expression Language
  • New Object Methods added: get, merge
• Public API for Groundplex installers and configurations.

• Pipeline Execution Statistics
  • Stacktraces of a Snap thread are now displayed in the Pipeline Execute Statistics page when the Snap thread fails or is stopped.
    
  • Warnings may be attached to Snaps if a potential problem was detected and should be addressed. For example, pipelines that use the deprecated version of the Join Snap will display a warning.
    
    Pipeline executions that have warnings will show up in the Dashboard with a warning icon next to their status to make them easy to identify.
    
  • Updated with additional network statistics.
    
    
  • Updated with an Extra Details tab to provide additional information about the pipeline run.
    • Ultra pipelines display the timestamp and other metadata of the documents currently being processed by an ultra pipeline in the Extra Details tab.

• Pipeline Snapshot shows what the pipeline looked like at execution time.
• Integration Assistant is now enabled by default for new users.

Manager

• Project Level Migration. If you have access to more that one org in SnapLogic, you can move projects between orgs.
• The Event Log page has been split into two separate pages: Activity Log and Alerts.
• Pipeline Execution Timeout. Users can now configure the maximum duration that a pipeline is allowed to run by entering Timeout (in minutes) through task configuration. The execution stops executing after the specified timeout duration is reached. The minimum duration the pipeline is allowed timeout is 1 minute. The field takes only integer input. Timeout of 0 or leaving the field empty is equivalent to NO timeout on pipeline execution hence running the pipeline till it reaches its final state. 
• Alerts now lists when the Daily API Usage and Concurrent API Usage exceed 75% of the limit.
• Settings: Alerts/Activity Log Notifications section added for creating org level notifications.
• Allow/disallow UI access to accounts.
• Snap Packs can now be uploaded to projects. The lower level Snap takes precedence over the Shared level, but an account will use the version at it's same level.
• Asset Search
  • The page now loads without data presented. A search must be performed for data to display.
  • Enhanced with Snap Statistics.

Dashboard

• You can now restart a Snaplex from within the Dashboard
• Additional information now includes a Secure URI option for Groundplex. This option opens a local dashboard that would be available during maintenance mode or other disconnections and provide information on your ultra pipelines in progress.
• Added the See executing pipelines option to the Node Actions menu in Snaplex Health. It opens the Pipeline tab in Dashboard, filtered to the selected Snaplex node.
• See also, Pipeline Execution Statistics above under General.
• Pipeline Execution Runtime Logs for Spark

This section provides the Snap updates for the Summer 2017 (4.10) release. This release also includes the patches made to Snaps in the 4.9.0 release, as listed in the Summer 2017 - 4.10 Release Notes page. For information on Snap Pack build versions, see Summer 2017 - 4.10 Release Notes (updated at product release).

New Snap Packs

• Azure SQL Snap Pack: The Snap Pack includes Polybase Bulk Load, Azure SQL Bulk Load, Azure SQL Bulk Extract, Azure SQL Update, Azure SQL Execute, Azure SQL Table List and Azure SQL Stored Procedure.
• JWT Snap Pack: This Snap Pack contains JWT Generate and JWT Validate. These Snaps allow pipelines to issue, and validate against restricted scope tokens.

Updated Snap Packs

Confluent Kafka

• Confluent v3.2 support for Kafka Snap Pack.

Data Snaps

• A new Snap property Auto commit is added in all database Execute and Select Snaps (except the Cassandra, Hive and Snowflake, Select and Execute Snaps). This enables users to override the status of Auto commit in the account settings. This property has three options: 'True', 'False' and 'Use account setting’. The default value is 'Use account setting' in Execute Snap. In Select Snap, the default value is 'False’ in order to preserve backwards compatibility.

  Auto commit may be enabled for certain use cases if PostgreSQL JDBC driver is used in either Redshift, PostgreSQL or generic JDBC Snap. But the JDBC driver may cause out of memory issues when Select statements are executed. In those cases, Auto commit in Snap property should be set to ‘False’ and the Fetch size in the “Account setting" can be increased for optimal performance.

DynamoDB

• DynamoDB Scan, Bulk Write, Delete Table Item and Update Snaps support the Set types (Binary, List and Map).

Email 

• Configuring Email Accounts Updated with Attachment password property (encrypted) to support attachment function for the Email Sender and Reader Snaps.
• Email Reader Snap updated with the Attachment folder property to store the attached files on a path.
• Email Sender Snap extended support to SLDB, SFTP and SMB file protocols for the attachments.

Google Sheet

• Updated the Google Google Sheets Writer Snap with Row count property to resolve the ' Data writing error'. 

Hadoop

• Support for S3 accounts with IAM Roles added to Parquet Reader and /wiki/spaces/SD/pages/1438227
• HDFS Reader/Writer with Kerberos support on Groundplex (including user impersonation).
• /wiki/spaces/SD/pages/1438227 partitioning support in Standard mode.

Microsoft Dynamics CRM

• Added the new Snap, CRM Upsert.
• Performance enhancements for Microsoft DEPRECATED | Dynamics CRM Create.

RabbitMQ

• RabbitMQ Consumer Snap updated for the Message count property updated to a default value of -1. 
• Added support to the Configuring RabbitMQ Accounts with the Use TLSv1.2,Trust all certificates, Key Store, TrustStore, Key/Truststore password and Key alias properties to support the SSL/TLS for secured communications.
• RabbitMQ Producer Snap updated with Retry request interval and Maximum request attempts fields to support the retry logic in case of a network failure.

SAP

• Added SAP account to connect to a message server, SAP Load Balanced JCo Account.

Snowflake

• Updated the Snowflake Bulk Load Snap with Load empty strings property for the empty string values in the input documents to be loaded as empty strings to the string-type fields.
• Updated the Snowflake Bulk Load Snap with Table Columns to support the order of the entries on the staged files that contain a subset of the columns in the Snowflake table.

Teradata

• Teradata Select Snap added.

Transform

• CSV Parser Snap updated with Ignore empty stream property to support passing the empty data.
• XML Formatter Snap updated with Max schema levels property to support the outbound schema XSD containing import statements.
• Addressed spelling error in messages across the Snap Pack.

This section tracks the changes made during the iterative pushes to the UAT server and between the last push to UAT and the GA release. The expected schedule is:

• UAT Push #1, July 22, 2017
• UAT Push #2, July 29, 2017
• UAT Push #3, August 5, 2017
  
• GA Push, August 12, 2017

UAT #2 Delta

Snap Updates

• Fixed issues with Filter not working as expected for amazon s3 directory
• Join Snap: Fixed issue with Input not completely consumed during execution when Unmatched data to error view is enabled
• Redshift Execute: Fixed issue with closing database cursor connection
• Snowflake Table list: Fixed issue with the Snap throwing throwing Null pointer Exception while fetching the list of tables
• Azure SQL Bulk Extract: Fixed issue with invoking BCP command in Windows Snaplex
• Database Execute: Improved the error message when table is not found
• Oracle Execute: Enhance error message to advise the user to remove the trailing semicolons in the query statement if necessary
• Oracle Bulk Load: Fixed issue with records being inserted into the table even though Snaps fails with SQLLDR command error
• Removing Snap level auto commit property from Database Select Snaps: Hive, MySQL, Oracle, SAP HANA, Snowflake, SQL Server, Vertica, Teradata
• SAP HANA Insert: Fixed issue with inserting null into database when a string value is provided for NUMERIC data type
• Workday Write: Propagate error message routed to error view
• Rabbit MQ Producer: Added retry logic to initialize connection
• Google Worksheet Writer: Improved Performance of Snap execution to write data

Platform Updates

• Fixed issue seen with project export with invalid Snap Pack
• Ultra pipeline restart after task name change
• Performance improvement for fetching account list
• Disabled multi-line string warning in expressions and fix caching of warning message
• Fix JCC startup issues seen when org admins group is improperly configured
• Changed local_admins group to be additive with admins group for local dashboard
• Updated email delivery blacklist
• Fixed issues with insights API
• Fixed issue causing JCC restarts by the monitor process
• Fix race condition causing errors in child pipeline started using PipeExec Snap

UI Updates

• Minor style updates
• Resolved Project Migration UI showing non-visible Project Spaces
• Updated MOTD notification message
• Resolved Dashboard not showing JCC details
• Resolved issue with auto-refresh closing JCC additional info dialog
• Error updating first last name for newly created basic user
• Resolved Project Migration UI showing Life Cycle Management phases
• Updated logo and background to new corporate graphics
• Resolved script error closing pipeline tabs

Big Data Updates

• Fix the behavior of error views in HDFS reader and HDFS writer Snaps for Groundplex and Hadooplex.
• Resolved issue related to setting permissions for created files in HDFS Writer Snap.

UAT #3 Delta

Snap Updates

• Addressed an issue in Azure SQL Bulk Load with the Azure SQL account where account validation succeeded but failed in Snap execution.
• Removed Active Directory Account support from Azure SQL Bulk Extract.
• Addressed an issue with theSnowflake Execute Snap throwing an error for few DDL statements.
• Addressed an issue in SQL Server Bulk Load with data not getting loaded into the target table.
• Addressed an issue with RabbitMQ Producer to make sure we wait for the queue name to confirm
  
• Addressed an issue with Database execute snaps, which was failing when snap level auto-commit was false but the account level was true.
  
• Enable visibility of the Snap level auto commit field in listed database Select Snaps
  

Platform Updates

• Fix error handling for invalid project path in Metadata create snap 
• Fix error in insights API fetch
• Fix email notification check for API throttling
• Fix issue with shared file copy with project migration.
  

UI Updates

Minor style updates:

• In Firefox fixed styling with Designer/Manager/Dashboard header tabs
• Expand Icon is replaced by checkbox for dropdowns across Asset search and Notification project select tree.

Designer defect fix

• Dialog is not really closed when user pressed esc button to close Snap infobox. It re-appeared when a new Snap was dragged in to the canvas.

Big Data Updates

• Fixed issue where Parquet reader was validating with invalid keys 
• Remove Azure account from Parquet reader and writer
• Fixed issue with Parquet reader and writer failing with default s3 endpoint.
• Fixed issues with HDFS writer behavior of writing to error and output views, on selecting different file actions.
• Improve error message in HDFS writer when input directory is invalid.
• Fixed ADL append support in HDFS writer.
• Fixed HDFS reader not being able to read files which were written with explicit permission being set.

GA Delta

UI Updates

• Fixed issue where the expand icon was replaced by a checkbox icon for drop-downs across Asset search and User email notification
• Fixed issue where Workbench was not creating the pipeline in the current org.

Snap Updates

• Addressed an issue with Hive and Teradata Execute Snap with managing database connection
• Addressed an issue with Cassandra Select Snap to handle timestamp when using simple date format
• Addressed an issue with handling of DDL and DML statements in Snowflake Execute Snap with newer JDBC drivers

Big Data Updates

• Fixed issue in Hadooplex where multiple containers were spawned during auto-update
• Fixed NPE on providing empty directory in HDFS Reader Snap.
• Fixed error messages on HDFS Reader Snap.
• Fixed routing error to error view in HDFS Reader and Writer Snaps.