May 2018, 4.13 Release Notes
- Ankur Parekh
- Kalpana Malladi
- Mohammed Iqbal
In this Page
Before you begin
You may need to clear your browser cache before you log into the latest SnapLogic Intelligent Integration Platform (IIP). See the applicable documentation for your browser:
You may see an Upgrade Notice dialog informing that you will be logged out immediately to complete the update process. Because the SnapLogic platform consists of multiple applications, you will likely see this message a few times.
We recommend that you upgrade to the latest browser version.
Accounts and Platform updates
Some accounts may have a fixed time for refresh tokens, such as Google accounts that must be refreshed every hour. If that refresh needs to occur when the platform is down for an update, the refresh does not occur. To prevent these accounts from failing after a new platform deployment, we recommend you to refresh your accounts before the designated down time. If the platform is down for longer than an hour, you will need to refresh those accounts once the platform is back online.
Platform
Enhancements
Designer and Manager
- Next Snap suggestions in Integration Assistant now include personalized recommendations based on your username, org, and current project. SnapLogic takes these items into consideration to provide the best possible Snap recommendations based on your current situation. For example, users working on a project to migrate from Oracle to ServiceNow would see different Next Snap suggestions than users working on a project to migrate from Oracle to Salesforce.
For additional security, you can configure a Groundplex Whitelist to control where nodes in a Groundplex can originate, in order to be authorized to connect with the SnapLogic control plane. This enables you to make sure only nodes inside your network safety zones can connect.
The Lifecycle Management feature and related references are removed from the SnapLogic user interface and user documentation.
- When you migrate accounts from an org with standard account encryption to an org with standard or enhanced account encryption, the source org must recognize the destination org as a "trusted" org. Previously, you had to work with SnapLogic Support to establish trust between orgs. Now, org admins can establish trust between orgs by configuring Trusted Orgs in Organization Settings.
General
Starting from 4.13, customers enjoying Platinum support have an option to upgrade the org Snaplex (Groundplex or Cloudplex) and Snap Packs within 35 days after GA. This gives you an opportunity to ensure that the new Snaplex and Snap Packs are stable and working as expected, before rolling out across your orgs in the company. Read more about the new upgrade process.
On 9th June, 2018, all customer Snaplex nodes are upgraded to 4.13 per the new SnapLogic Release Process. All subscribed Snap Packs are updated to
snapsmrc486 (default)4.13 stable version. In case of any questions, contact SnapLogic Support.- Groundplex nodes can now be containerized, allowing customers to optimize their IT infrastructure for costs and performance.
SnapLogic now supports API management integration with Apigee Edge, a popular platform for managing API services. You can export Triggered Tasks, and Ultra Tasks in your SnapLogic environment and manage them as APIs in the Apigee Management Console. API services provided by Apigee Edge include:
- Specification Publishing
- API Proxy/Protocol Translation
- Throttling/Traffic Shaping
- Market Place
- Developer Portal
- Analytics
As part of this solution, we have augmented defining the pipeline parameters that conform with the Open API specifications. This enables SnapLogic to deliver a complete and seamless API management solution that customers can benefit from.
- SnapLogic extends support to Swagger. You can now download the specification of the SnapLogic Tasks as REST APIs in .yaml file format and view the APIs in Swagger. Generate the OpenAPI (Swagger) description of the SnapLogic Task APIs in a project with a new option in Manager.
- Enhancements to the Snap schema propagation, to make it easier to build pipelines which do data transformations.
- Increased the default compressed size limit from 1GB to 2GB for Groundplex and Cloudplex, if the JVM heap size is detected to be more than 10GB. This allows customers to improve utilization of Snaplex nodes.
- Added high availability (HA) support for Ultra tasks, wherein you can deploy tasks on multiple Snaplexes. The fail over functionality will trigger in any aliased locations, if any of the Snaplexes where the task is running goes down.
- Enhancements to support updated SAML (Security Assertion Markup Language) specifications for Single Sign On.
- Enhancements to the SnapLogic expression language:
- Enhancements to expressions that are dynamically validated and an output preview displays with the result of each sub-expression as well as the final result. View a sub-expression's result by clicking or moving the cursor to the sub-expression. Refer to Expression Language Usage.
- Performance improvements in the expression evaluation engine to allow for faster processing of complex expressions.
- Added error recovery functionality for Ultra pipelines, making it possible to use error pipelines to recover and retry the original request via two new methods –
original.load() and original.id
Pipeline Asset Fix Resulting in Functional Change
- In the 4.12 release, a bug was introduced wherein if parent and child pipelines are running on the same node, but belonged to separate project folders, then files required by the child pipelines were accessed from the parent pipeline's project. This was seen when using relative paths to access assets like files and expression libraries. While this is fixed in 4.13, it is likely to impact a small set of pipelines that are designed as stated.
- We recommend you to run any impacted Pipelines in your test orgs, after ensuring that the dependent assets are present in the child pipelines' project folder, before upgrading your Groundplex and Cloudplex instances in your production orgs. Storing the required files in both the project folders also avoids the issue. Pipelines using absolute paths are not affected.
- Contact SnapLogic Support if you have any follow-up questions on this functional change as a result of this fix.
Snaps
This section provides the Snap updates for the Spring 2018 (4.13) release. This release also includes the patches made to Snaps in the 4.12 release, as listed in the 4.12 Dot Releases page.
- Pipelines that are already started before a Snap update will continue to run with the older version of the Snaps. Only pipelines started after the new Snaps are deployed will run with the new Snaps.
- For Ultra pipelines, the currently running instances will continue to run with the older Snap packs. Editing and saving the task instance in Manager will cause a rolling restart of the ultra pipeline instances and the new Snap Packs will get picked up.
- Customers using SQL Server with Windows authentication and SAP will need to restart their Groundplex and Cloudplex instances.
New Snap Packs
Adobe Cloud Platform: This Snap Pack reads and writes data sets in Adobe Cloud Platform. This Snap Pack includes the following Snaps:
- Adobe Cloud Platform Read
- Adobe Cloud Platform Write
SAP Analytics Cloud Snap Pack: This Snap Pack enables you to connect to the SAP Analytics Cloud instance and perform CRUD operations in it. This Snap Pack includes the following Snaps:
- SAP Analytics Cloud Read
- SAP Analytics Cloud Create
- SAP Analytics Cloud Upsert
- SAP Analytics Cloud Delete
New Snap Packs are displayed in the Designer only after all the JCCs in an org are upgraded to the latest 4.13 version.
Updated Snap Packs
Binary:
- Added AWS Key Management Service (KMS) encryption support to AWS S3 Account and S3 Dynamic accounts in the Binary Snap Pack.
- Added KMS support to these Binary Snaps: Directory Browser, File Operation, File Poller, File Reader, File Writer, Multi File Reader, ZipFile Read, ZipFile Write, S3 File Reader, and S3 File Writer.
Hadoop:
- Added KMS encryption support to AWS S3 account in the Hadoop Snap Pack.
- Enhanced the Parquet Reader, Parquet Writer, HDFS Reader, and HDFS Writer Snaps to support WASB and ADLS file protocols.
- Added the AWS S3 account support to the Parquet Reader and /wiki/spaces/SD/pages/1438227 Snaps.
- Added the second input view to the Parquet Writer Snap. When enabled, the Snap accepts the table schema from the second input view.
RabbitMQ:
- Added a new Snap, the RabbitMQ Acknowledge Snap to perform an acknowledge, reject or a recover action on the received message from the RabbitMQ server.
- Added a new property “Message Acknowledge Mode” in the RabbitMQ Consumer Snap to specify the mode of message acknowledgement -- Auto Acknowledge or Pipeline Control (if the RabbitMQ Acknowledge Snap is used).
Redshift: Added the KMS encryption support to the Redshift Snaps -- Redshift Unload, Redshift Bulk Load, Redshift Bulk Upsert, and Redshift S3 Upsert.
Reltio:
- Added a new property, S3 File to the Reltio Bulk Export Snap to support data to be exported to the specified S3 location.
- Added new properties Batch size, and Thread limit to the Reltio Delete (URI) and Reltio Delete (Object) Snaps to enhance the performance.
Salesforce:
- Fixed the Salesforce Read Snap that fails to download the PK-Chunking CSV file due to network failure or being out of disk space.
- Fixed the SFDC Account (standard) that fails to validate.
- Fixed the Salesforce Lookup Snap that does not match a data type with the bulk API. For instance, boolean data type returns as string.
- Fixed the Salesforce SOQL Snap that does not preview data in pipelines.
- Fixed the PK-Chunking files that do not delete from the "tmp" directory.
ServiceNow: Added Jakarta support to the ServiceNow Snap Pack.
Tableau: Enhanced Tableau Snap Pack to support Tableau version 10.5. This includes support for REST API calls related to Refresh tasks in the /wiki/spaces/AP/pages/1438855.
Big Data
- Fixed a memory leak issue in Hadooplex, caused by Spark pipeline executions.
In view of the new upgrade process mentioned in the Platform section above, we recommend that you bring down your current Hadooplex and start again with the new bits, before using the latest SparkSQL Snap Pack version.
Ensure that there are no private Snap Packs associated with your Org.
Known Issues
Snaps:
- Expression parameters from upstream Snaps do not get parsed in HDFS Writer, Parquet Writer, and ORC Writer Snaps. Nonetheless, the pipeline parameters do parse as expected.
- Using unicode characters in file path or file name result in the files not being read in HDFS Reader, HDFS Writer, Parquet Reader, Parquet Writer, and Binary Snaps.
- File and Directory suggestion does not work in case of a Groundplex using name service in HDFS Reader, HDFS Writer, Parquet Reader, and Parquet Writer Snaps.
UAT Delta
This section tracks the changes made during the iterative pushes to the UAT server and the GA release:
- UAT Push #1, April 20, 2018 (Release updates are published above)
- UAT Push #2, April 27, 2018
- UAT Push #3, May 04, 2018
- GA Push, May 12, 2018
UAT #2 Delta (April 27, 2018)
Platform Updates
- Fixed an issue where a child pipeline execution on the current node looks for dependencies in the parent pipeline's project instead of the child pipeline's project.
- Fixed an issue with task update API that causes the task to enable after an update.
- Added details to the pipeline failure notification email.
- Fixed an error condition handling when creating API proxy in Apigee.
- Fixed an issue when passing encrypted properties using the Metadata Snaps.
- Fixed an issue that causes CSV formatting to fail with unicode delimiter character.
Snap Updates
- Adobe Cloud Platform: Dataset suggestion now displays all datasets and not just the first 20.
- Box: Fixed an issue wherein the Box Read Snap enters an infinite loop situation for connection errors.
- Confluent Kafka: Fixed an issue with the Confluent Kafka Consumer Snap that did not respect the acknowledgement timeout window on a per message basis.
- Hadoop: Fixed an issue with the HDFS and Parquet Snaps that presume a default port value of 8020, preventing directory and file suggestions for nameservice-enabled nodes.
- JDBC: Updated the Hikari connection pool management library version. This has resulted in a breaking change for JDBC3 drivers. This newer version requires databases to be able to support a validation query of "SELECT 1;" or have the driver itself implement a custom validation method. Some older JDBC3 drivers may not have such an implementation, meaning those drivers configured in a JDBC Account may no longer validate if the "Database name" setting has the value of "Auto-detect." For example, DB2-based JDBC Accounts must now explicitly choose "DB2" from the "Database name" setting. JDBC4 drivers are not affected by this change.
- Oracle: Fixed a regression in the Oracle Bulk Load Snap due to null values resulting in records being rejected.
- RabbitMQ: Fixed an issue with the RabbitMQ Consumer Snap that did not output the "deliveryTag" and "redelivered" fields for use when re-consuming an unacknowledged/rejected message.
UI Update
- Fixed an issue wherein pipelines did not print.
UAT #3 Delta (May 04, 2018)
Platform Updates
- Fixed an issue that causes parameterized accounts to fail with an older version of Snaplexes.
- Fixed issues with encryption of account properties having null values.
- Fixed the timeout and load balancer settings for Apigee proxy creation.
- Increased the neighbor health check timeout value.
Snap Pack Updates
- Binary and Hadoop: In a previous UAT deployment, all AWS KMS-related options were available in an "Encryption type" selection setting for S3-related accounts. For greater backwards-compatibility, the "Server-side encryption" setting is re-introduced and removed from the "Encryption type" selection setting.
- Box: Fixed an issue where an additional error document incorrectly generates when an invalid file path or folder name is provided.
- Database Snaps (Oracle, PostgreSQL, MySQL, Redshift, SAP HANA, Vertica): Addressed an issue when "Create table is not present" is enabled and the "Auto-commit" setting is not enabled.
- Hadoop:
- Fixed an issue with the HDFS Writer Snap where an extra error document is generated while using an unsupported file operation.
- Fixed an issue when an HDFS Reader, HDFS Writer, Parquet Reader, or Parquet Writer Snap is used on a kerberized Hadooplex without configuring an Account (meaning the Snap should use the Hadooplex's configuration for authentication/authorization).
- PostgreSQL: The Postgres Execute Snap now permits expressing functions that include binding to input document fields (specifically, if the parameters contain the $ character)
- RabbitMQ:
- Fixed an issue when all messages are set as Unacknowledged when the RabbitMQ Consumer Snap is set to PIPELINE_CONTROL (for manual acknowledgement) and Asynchronous consumption with an explicit message count.
- Prevented the RabbitMQ Consumer Snap from hanging when a message with a malformed or invalid deliveryTag is provided to the RabbitMQ Acknowledge Snap. Instead, an error document generates now.
- Improved the error messages.
- Redshift: In a previous UAT deployment, all AWS KMS-related options were available in an "Encryption type" selection setting. For greater backwards-compatibility, the "Server-side encryption" setting is re-introduced and removed from the "Encryption type" selection setting.
- Reltio: Fixed an issue in the Delete (Object) Snap that prevents successful deletion of Relations, Categories, or Entities.
- Tableau: Corrected a behavior where API requests are malformed when the payload is empty. Resulting in actions like "run_extract_refresh_task" failing, as the required placeholder information is not included.
- Vertica: Fixed a race condition in the Bulk Upsert Snap that would result in a NullPointerException.
Big Data Update
- Fixed an issue where the Spark pipeline execution fails when running on HDP 2.6.3 with kerberos and user impersonation enabled.
UI Update
- Fixed an issue wherein if a user injects a script directly into a Snap's Expression builder it throws an error.
GA Delta (May 12, 2018)
Snap Pack Updates
- Box:
- Improved connection failure management to prevent pipelines from hanging.
- A new property, Enable staging, is added to the Box Write Snap to stage and write a temp file locally before sending the write request to Box for long-running pipelines.
- Hadoop:
- Improved error messages when an authentication fails on Hadooplex.
- Improved error messages when the Parquet Reader and Parquet Writer Snaps attempt to read from an unsupported S3 region or endpoint.
4.13 Dot Releases
SnapLogic dot releases help optimize and continuously improve the platform. This dot release section documents all customer-impacting updates across the SnapLogic platform.
Platform and UI
| Release # | Date | Updates | Build | Actions |
|---|---|---|---|---|
| 4.13.2 |
| master-3379 | If the Snaplex is upgraded to the latest version of JRE 1.8, then use this version. The current Snaplex Linux installers use version 1.8.0_162. | |
| 4.13.1 | Fixed an issue that caused uneven distribution of child pipeline executions across nodes in a Snaplex when using the Pipeline Execute Snap. This issue is observed when a child pipeline is using nested pipelines. | master-3268 | NA |
For information on updating Snaplexes, refer to How to Update a Snaplex War File.
Snaps
| Snap Pack | Update Date | Build | Updates | |
|---|---|---|---|---|
| Salesforce | salesforce5629 | Fixed the Salesforce SOQL Snap that fails to execute if the parameter values of a SOQL query expression contain the WHERE clause. | ||
| Confluent Kafka | confluentkafka5614 | Fixed the Confluent Kafka Snap Pack that does not delete the temp files after executing pipelines. | ||
| SQL Server | db/sqlserver5522 | Implemented a connection retry logic in the SQL Server account to resolve the connection loss issue in a customer's org. | ||
| MongoDB | mongo5537 | Fixed the Update, Delete, and Find Snaps to populate input view schema for a given table, similar to the Insert Snap. | ||
| Binary | binary5372 | Updated the File Operation Snap for S3 to support files larger than 5GB by using an API for multipart upload copy. | ||
| Transform | transform5411 | Fixed an issue in the CSV Formatter Snap where the output showed extra values that were not provided in the input. | ||
email5368 |
| |||
| Binary | binary5315 | Added a new property "Enable staging" to enable File Reader Snap perform as expected even in a slow network connection to AWS S3. | ||
| Confluent Kafka | confluentkafka5330 | Added support in Confluent Kafka Producer Snap the ability to write data to multiple partitions, in a round robin fashion. | ||
| Redshift | db/redshift5303 | Added a new property "Validate input data" in the Redshift Bulk Load Snap to help users troubleshoot the input data schema. | ||
| Hadoop | hadoop5318 |
| ||
| RabbitMQ | rabbitmq5296 | Fixed the RabbitMQ Acknowledge Snap that returns a NullPointerException error when no DeliveryTag is sent during pipeline execution. | ||
| Workday | workday5247 | Fixed the Workday Read Snap that took an extended time to read from the Workday application. | ||
| Google Spreadsheet | spreadsheet5293 | Fixed the Google Worksheet Writer Snap that incorrectly displays numeric values as a string in the spreadsheet when writing in batch mode. The spreadsheet now preserves the exact data type of the input value. | ||
| SOAP | soap5214 | Fixed a bug in the SOAP Execute Snap wherein the end tag is removed incorrectly when generating the envelope. | ||
| Hive | db/hive5269 | Fixed the Hive Execute Snap that incorrectly stores account passwords in plain text in the log file. | ||
| Generic JDBC | db/jdbc5231 | Added binding information for DB2 database connection on the Generic JDBC Snap Pack to ensure that pipelines correctly validate and execute. | ||
| Generic JDBC | db/jdbc5229 | Added test connection query for SQLMX database in the JDBC Accounts to check if the database connection is valid. | ||
| Oracle | db/oracle5192 |
| ||
| Reltio | reltiosnapconnector5200 |
| ||
For information on updating the Snap Packs, refer to How to upgrade to a new Snap Pack version.
Have feedback? Email documentation@snaplogic.com | Ask a question in the SnapLogic Community
© 2017-2024 SnapLogic, Inc.