August 2021, 4.26 Release Notes

In this article

Key New Features in SnapLogic

Platform:

  • Zero Downtime during quarterly release updates.
  • Snaplex-based Scheduler will be the default scheduling mechanism for all Scheduled Tasks after the auto-upgrade on September 18.
  • Ability to encrypt SnapLogic user emails.

API Management: 

  • Ability to manage lifecycle of APIs.
  • Redesigned API Catalog and API documentation for the Developer Portal.
  • Support for creating APIs from existing SnapLogic Assets.

New Snaps:

  • Shopify Snap Pack.
  • Tableau Snaps—Hyper Formatter, Hyper Parser, and Hyper Write.
  • Zuora Snaps—Zuora OpenAPI and Zuora REST Read.

ELT for Cloud Data Platforms:

  • Support for Databricks Lakehouse Platform as a target database.
  • Ability to run multiple SQL queries using ELT Execute Snap.

New! Introducing SnapLogic Flows


4.26 UAT Delta

 Click here to expand...

This section tracks the changes made during the iterative pushes to the UAT server and the GA release. The following is the release schedule:

  • UAT 1: Friday, July 30, 2021 – Snaplex version: 4.26_rc-10116 - 4.26 GA (Recommended)

  • UAT 2: Friday, August 06, 2021 

  • GA: Saturday, August 14, 2021 (9 p.m. PT)  

The SnapLogic GA release build on the production server is also deployed on the UAT server.

Important

  • Per the SnapLogic Release Process, all remaining Snaplex instances across Orgs are auto-upgraded to the main-10245 - Patch 1 version at 9 p.m. PT on September 18, 2021. Orgs migrated to the 4.26 GA version prior to the auto-upgrade are not impacted. 

    The Snaplex upgrade process also upgrades the Java version to the latest version. After the upgrade, the Java version of your Cloudplex nodes may not be the same as your FeedMaster nodes. However, this does not impact the operations of your SnapLogic instance. We will be fixing this issue in a subsequent release.

  • To use the new SnapLogic features and Snaps in the 4.26 release, ensure that your Snaplex nodes are upgraded to the main-10245 - Patch 1 (Recommended) version.
  • This 4.26 release includes the patches deployed to SnapLogic after the 4.25 release, as listed in the 4.25 Dot Releases section. 

4.26 Dot Releases

Platform

Date of UpdateSnaplex BuildUpdates
 main-10631 - Patch 3
  • Fixed an issue that caused Pipelines to be stuck in the NoUpdate state.
  • Fixed an issue where some validated Pipelines would not show preview data.
  • Fixed an issue where some schedule tasks would fail to execute.
  • Fixed an issue that prevented the Snowflake Snap from loading JAR files.
main-10451 - Patch 2
  • Fixed an issue where the FeedMaster node would not enter maintenance mode. 
  • Fixed an issue where the Load Balancer node would return an error instead of stopping further requests when in maintenance mode.
N/A

Control Plane update:

  • Fixed an issue where system administrators would not receive password reset emails if email encryption was enabled. 
  • Fixed an issue where users were unable to see Tasks under the Task Column of the Dashboard
  • Fixed an issue where opening Ultra Task Details would return an Invalid ID error.
  • Fixed an issue where Ultra Tasks would continuously restart.
  • Fixed an issue that prevented users from updating a Snaplex with the Manage Snaplex Versions setting if the Snaplex had Slack users or channels added.
  • Fixed an issue that prevented users from creating APIs in projects that had Tasks with empty Snaplex instances configured. 
  • Added a read-only view for non-admin users to the Developer Portal.
 main-10245 - Patch 1 (Recommended)
  • Fixed an issue where Scheduled Tasks would execute twice after switching between control plane scheduling and Snaplex scheduling.
  • Fixed an issue where a Snaplex would wait an hour before starting to execute scheduled jobs after switching from control plane level scheduling to Snaplex level scheduling.

UI

Date of UpdateSnaplex BuildUpdates
 N/A
  • Fixed an issue where the Join Snap would fail after its input view was updated.
  • Fixed an issue where the default value of newly-added properties would not appear in existing instances of the Snaplex.
  • Fixed an issue where the Router Snap returned an error if a view was renamed multiple times.
  • Fixed an issue where some Pipeline Execute Snaps were missing the Reuse executions to process documents checkbox.
  • Fixed an issue that prevented the conditional fields of the Team Operations Snap and SQS Account from rendering.

Standard

Snap PackDate of UpdateSnaplex BuildUpdates
REST426patches12749

Fixed an issue with the REST Post Snap, where the Snap delayed the fetching of documents, when the batch size is greater than or equal to one.

SQL Server426patches12535

Fixed an issue in the SQL Server Bulk Load snap where boolean data type failed to load into BIT field.

Binary
Hadoop
426patches12288

Fixed memory leak issue when using the HDFS protocol in Hadoop and Binary Snaps.

Transform

426patches11922Fixed an issue with the Join Snap, where it exhausted the memory while buffering millions of objects.
Binary426patches11922Fixed an issue with the File Writer Snap, where the Snap failed to open a channel while accessing through a SFTP protocol.
Salesforce426patches12054Enhanced the Salesforce Subscriber Snap with improved resilience to network failures and fixed an issue with duplicate documents.
REST426patches11746

Enhanced the REST Post Snap to display all cookies (securedauth cookies and httponly cookies) from the CookieStore along with headers when you select the Show all headers checkbox.

Transform426patches11780Fixed an issue in the XML Formatter Snap, where the Map input to first repeating element in XSD checkbox is selected, while no XSD is specified for mapping the input.
RabbitMQ426patches11770

Enhanced the RabbitMQ Account with a new checkbox Cluster Support that enables multiple cluster node support for RabbitMQ Consumer and RabbitMQ Producer Snaps. You can select this checkbox to enable high availability of clusters by adding multiple hosts (Host name and Port) for each cluster node.

Marketo 426patches11555Fixed an issue with the Marketo Lead Read Snap, wherein the Snap generated the same next page token anytime during the day.
Email 426patches 11591

Fixed the following issues with the Email Archive Snap where the Snap:

  • Failed with an error User is authenticated but not connected, as the Snap did not re-authenticate with the server after the connection expired due to the long durations in the execution of Ultra Pipelines. 
  • Caused multiple copies of the same source message in the target folder when the Snap was executed multiple times in preview.
Transform 426patches11725Fixed an issue with the Join Snap wherein the upstream document flow of the right view is blocked by the left view, which hung the Join Snap.
DynamoDB426patches11593Fixed an issue with the DynamoDB Update Snap, where the data type conversion for input keys caused an error.
Binary 426patches11529
  • Fixed an issue with the Decompress Snap, where the Snap prematurely uncompressed multi-stream BZIP2 files.
  • Enhanced the PGP Encrypt Snap with improved memory usage and also fixed an issue with this Snap, where the Snap failed to encrypt large size files due to the lack of memory.
Workday 426patches11525Enhanced the Workday Read Snap with the new field Pool Size that controls the maximum number of threads in the pool. This field is available only when you select the Multi-threaded checkbox.
Snowflake 426patches11469Fixed an issue with Snowflake Insert and Snowflake Bulk Load Snaps where the schema names or database names containing an underscore (_) caused the Pipelines to time out.
Active Directory 426patches11280Fixed an issue with the Active Directory Add Member Snap that failed when using forward slash "/" in the Distinguished name field.
SharePoint Online main89Enhanced all Snaps in the SharePoint Online Snap Pack to support dynamic site and document library values using input document variables.

Data Automation

Snap PackDate of UpdateSnap Pack Build NameUpdates
ELT 426patches12534
  • Fixed an issue with ELT Transform Snap where it may display incorrect schema only in the previews (during Pipeline validation). This occurs especially when the incoming SQL statement (defined in the SQL Statement Editor of the upstream Snap) contains one or more of the WHERE, GROUP BY, HAVING , ORDER BY, LIMIT, LIMIT followed by OFFSET, and SAMPLE clauses. Here are a few Pipeline scenarios where this issue might surface:

ELT 426patches12021
  • Fixed an issue where the ELT Load Snap connecting to a Databricks Lakehouse Platform (DLP) instance failed to perform the load operation. Ensure that you provide a valid DBFS Folder path in the Snap's account settings as the Snap requires this folder path.
ELT 426patches11646
  • Enhanced the ELT Database Account to support token-based authentication (Source Location Session Credentials) to S3 locations for Snowflake and Redshift target databases.
  • Enhanced the ELT Aggregate Snap with the following changes:
    • Revised the field labels from:
      • GROUP BY Fields List field set > Output Field to GROUP BY Field.
      • ORDER-By Fields to ORDER-BY Fields (Aggregate Concatenation Functions Only).
    • Removed the Suggestion option for Field Name field under General Aggregate Functions List field.
    • Made the Alias Name fields in the Aggregate Concatenation Functions List and the Percentile Distribution Functions List field sets mandatory.
  • If your target database is a Databricks Lakehouse Platform (DLP) instance, then the ELT Load Snap supports loading data from source CSV files that contain only comma as the separator between values.
ELT 426patches11323
  • Enhanced the ELT Database Account to allow parameterization of field values using Pipeline Parameters. You can define and use these parameters in expression-enabled fields to pass values during runtime.

ELT 426patches11262
  • Fixed the following Known Issues recorded in the 4.26 GA version:
    • For a Snowflake target instance, the ELT Insert Select Snap does not suggest column names to select for the Insert Column field in the Insert Expression List.
    • The Snaps—ELT Merge Into, ELT Select, ELT Join, and ELT Filter—do not prevent the risk of SQL injection when your target database is Databricks Lakehouse Platform (DLP).
    • Intermittent null-pointer exceptions in the ELT Load Snap on Databricks Lakehouse Platform (DLP).

    • The ELT Insert Select Snap attempts to create the target table even when it exists in the Snowflake database.
    • When loading data from a JSON file into a target Databricks Lakehouse Platform (DLP) instance using an ELT Load Snap, if you choose the Drop and Create Table option as the Load Action and specify an additional column (that is not available in the JSON file) for the new table, it results in one more column null added to the new target table.
    • When you use the SQL editor in the ELT Select Snap configuration to define your SQL query, the Pipeline validation fails due to a syntax error in the following scenarios. However, the Pipeline execution works as expected. The only workaround is to drop the LIMIT clause and the optional OFFSET clause from the SQL query during Pipeline validation.
      • The query contains a LIMIT clause on a Snowflake, Redshift or Databricks Lakehouse Platform target instance: The SQL query created during Pipeline validation includes an additional LIMIT clause, for example: SELECT * FROM "STORE_DATA"."ORDERS" LIMIT 10 LIMIT 990

      • The query contains an OFFSET clause (supported in case of Snowflake and Redshift): The SQL query created during Pipeline validation looks like SELECT * FROM "STORE_DATA"."ORDERS" LIMIT 10 OFFSET 4 LIMIT 990

Platform 

Snaplex-based Scheduler

During the 4.26 release cycle's auto-upgrade date of September 18, the Snaplex-based Scheduler will be enabled by default through the Control Plane. The new scheduling mechanism improves the timeliness and reliability of Scheduled Task executions. You do not need to update your Snaplex instances to enable the new scheduler, unless you have set the feature flag to false for your Org, in which case you will have to switch it to true in order for the Snaplex-based Scheduler to be enabled for your Org. Contact support@snaplogic.com for more information. 

Zero downtime of the SnapLogic Platform during the release windowStarting with this release, we are implementing zero downtime for the SnapLogic Platform during quarterly release updates. Thus, you will be able to access the SnapLogic UI and all your Pipelines will run as schedule during the release window. For details, see the SnapLogic release process article.

Email Encryption: Adds the ability to upload a public key in Manager to be used for emails. You can now encrypt your emails using this public key. Email Encryption is a subscription feature.

New APIM Features

Lifecycle Management for APIs: Added the capability to manage the lifecycle of your SnapLogic APIs. You can now publish APIs created from API specifications to the new developer portal in the Portal Manager console. You can also unpublish, deprecate, and retire an API version, providing full API lifecycle management features.

Portal Manager: Added console for managing the developer portal for your Org. You can now customize the new Developer Portal with your own branding and URL suffix. The Portal Manager also enables you to view API status on the API Catalog.

API Developer Portal: Introduced a new page where API consumers can explore and view APIs created in your Org. The API Catalog provides a space where APIs can be exposed to users outside of the SnapLogic ecosystem. You control the consumer's ability to view and call APIs by adding API policies in the API Version Details page.

Create APIs from Assets in Manager: Added options in the API Manager console to create APIs directly from Projects and the Assets. You can now build your APIs by developing Pipelines and Tasks, and add other assets like Accounts and Files to the Project to be uploaded as an API in the API Manager. You can also create an Empty API asset as a placeholder.

OAuth 2.0 Credentials API Policy: Added a new OAuth2 flow to authenticate users using client credentials in the API Policy.

Enhancements to Existing Features

Batching Support: Added batching support to the Pipeline Execute Snap. In contrast to the Reuse mode, the batching field enables users to specify the number of documents to be processed to completion through the child Pipelines before processing the next document in the batch. Accordingly, batch mode does not support reuse mode.

/wiki/spaces/DRWIP/pages/2295988225 Added Windows 2019 support for your Groundplex instances.

SnapLogic will sunset support for Windows Server 2012 on . Hence, ensure that you upgrade your Groundplex instances to Windows 2016 or 2019. 

/wiki/spaces/DRWIP/pages/2294218801 Snaplex tab widgets now display one-minute data intervals for time ranges up to seven days.

OAuth Accounts: Optimized the OAuth account refresh operations by moving them to the customer Groundplex instances. Additionally, the Box OAuth 2.0 account type was also added to Snaps that do not restart Ultra Task instances because Box has the capability of reloading the new account token.

Known Issues

  • Few Snaplex instances with an unusually high number of tasks may take slightly longer to start scheduling tasks when a node is starting.
  • For Scheduled Tasks scheduled every 5–10 minutes, the next execution time is not being updated even though the Task is executed.
  • In the Version Details page of an API, clicking Edit Details from the API > Versions page displays the Server field for selecting another Snaplex, but the field is disabled if the specified Snaplex is in a down state.

Behavior Change

  • Users are no longer unsubscribed from a Snap Pack if the Snap Pack's name is changed.
  • Your Groundplex thread/heap dump files are now stored in a folder as configured in the java.io.tmpdir property, which is usually /tmp on Linux.


Standard-mode Snaps

Behavior Fix in the CSV Formatter Snap

In 4.26, we fixed the Ignore empty stream checkbox functionality in the Snap to actually produce an empty binary stream output in case there is no input document when the checkbox is NOT enabled. Previously, there was no output even when the checkbox was NOT enabled.

If your existing Pipelines (prior to 4.26) use the Ignore empty stream functionality, then you might need to revisit your Pipeline design and either enable or disable the said checkbox depending upon the expected behavior.

New Snaps

Binary Copy: This Snap that belongs to the Flow Snap Pack enables you to copy a binary stream to the Snap’s output views. Use this Snap if you want to send the same information to multiple endpoints.

Copybook Snap Pack: This Snap Pack enables you to work with Cobol Copybooks. This Snap Pack has the following Snaps:

  • Cobol Copybook Data Generation: Generates sample records based on Cobol Copybooks.
  • Cobol Copybook Parser: Parses Cobol Copybook and data.

Shopify: This Snap Pack allows businesses to set up an online store and sell their products online. You can use this Snap Pack to create orders, products, customers, and run automated workflows. The Shopify Snap Pack has the following Snaps:


Tableau

  • The Snap Pack consisting of the Tableau 8 version Snaps (Tableau 8 Write and TDE 8 Formatter) is deprecated due to no customer usage. See the Deprecated Snaps section for more information.
  • Starting from the 4.26 release, the existing Tableau 9 & 10 Snap Pack is renamed to Tableau Snap Pack.

Added the following Tableau Snaps to support hyper extract files for Tableau 10.25 and later versions. Hyper is Tableau's in-memory data engine that is optimized for fast data ingestion and analytical query processing on large or complex data sets.

  • Hyper Formatter: Converts the incoming documents to Tableau hyper extract format and sends the output (extract) to the binary output view. 
  • Hyper Parser: Parses Tableau hyper extract files and converts them to a document format.
  • Hyper Write: Creates a hyper file in a temporary location (local disk) for incoming documents and publishes it to Tableau Server/Online without packaging it into a data source.


Zuora: Added the following Snaps and account types to connect these Snaps with the Zuora REST API. 

Updated Snap Packs

Amazon SQS

  • Enhanced the Amazon SQS Account with a new checkbox Enable Large Message Support that enables Amazon SQS client to send and receive messages up to 2GB through Amazon S3. 
  • Enhanced the AWS S3 and Amazon SQS accounts to add the Snap’s information to the User Agent header for Cross IAM account requests.

AnaplanAdded the following fields to the Anaplan Write Snap:

  • Override server settings
  • Separator
  • Text Delimiter

Azure Active Directory: Enhanced the Azure Active Directory Create Entry and Azure Active Directory Update Entry Snaps to support Pipeline parameters and upstream values for the Attribute name setting under Attributes and the Snap Pack to support proxy authentication. 

Binary

  • Enhanced the Directory Browser Snap to process files with the owner field for the SFTP protocol. Upon validation, the output of the Snap fetches the owner’s name, which was previously displayed as unknown.
  • Fixed an issue with the File Writer Snap where the Snap fails to connect to the SFTP endpoint using the AWS Transfer Family.
  • Fixed an issue where Pipelines fail to connect the SFTP Server with the Binary Snap Pack after upgrading to the SnapLogic 4.25 release version (main9554).
  • Removed the Project ID field from the Google Service Account, since the JSON Key already contains information about the Project ID. 

Box:

  • Modified default values of the following fields across the Snap Pack to trigger automatic retries by default, in case of connection failures or timeouts:
    • Number of Retries - Changed from 0 to 3
    • Retry Interval (seconds) -  Changed from 1 to 5

Google SheetsEnhanced the Worksheet Writer Snap to populate the target schema preview with headers and associated data types (when the data is written to an existing worksheet with a valid header) in the upstream Snap.

JDBC:

  • Fixed an issue with Generic JDBC - Select Snap where if there are URL properties in the account, the unPooledConnection does not use the properties when creating a connection. This causes the auto-detect functionality to use the wrong database name.
  • Fixed an issue with MySQL and MariaDB in Generic JDBC - Update Snap when a table is incorrectly reported as not being found.

Kafka:

  • Enhanced the Kafka Producer Snap to support the Avro serialization format with the new target data type Avro under Key and Value Serializer.
  • Enhanced the Kafka SSL Account with new fields (Registry Username or Key and Registry Password or Secret) for Schema Registry authentication. The two existing SASL properties (SASL Username and SASL Password) are revised to SASL Username or Key and SASL Password or Secret, respectively.

  • Enhanced the Kafka Producer and Kafka Consumer Snaps to create a default client ID string that identifies the specific Snap instance.
  • Improved the handling of interrupted/aborted Kafka Snaps to ensure a proper clean-up of metrics.

  • Optimized the Kafka Producer Snap to initialize the Kafka API only if there is at least one input document.

  • Fixed an issue of account passwords being included in the log messages output of Kafka Snaps. The account passwords are now hidden in the logs for both Kafka Consumer and Kafka Producer Snaps.

  • Apache Kafka client library is upgraded from version 2.6.0 to 2.8.0.

  • Confluent Kafka client libraries are upgraded from version 5.2.1 to 6.2.0.

MySQL:

  • Enhanced the MySQL - Lookup Snap suggestions to load the Output Fields and Lookup Column settings.
  • Fixed an issue with the MySQL - Select Snap where the second output view of the Snap was showing empty column values and improved error messages in the Snap where the Snap fails with a null pointer exception error when the given account information is invalid.

OpenAPIEnhanced the OpenAPI Snap with the following two fields:

  • HTTP Headers
  • Query Parameters

PostgreSQL: Enhanced the performance of PostgreSQL - Bulk Load Snap significantly. We expect the Snap to execute up to three times faster than the previous version for enterprise workloads.

REST

  • Added Send Client Data as Basic Auth Header checkbox to the Create Account settings of the OAuth2 and OAuth2 SSL accounts to include and pass the client credentials information in the header when connecting to the OAuth endpoint.
  • Added a new account type REST AWS SigV4 that adds the ability to call any AWS API.

Salesforce: Enhanced the Salesforce Read Snap to enable you to add an optional second output view to display the schema of a target object as the output document. 

ServiceNowEnhanced the ServiceNow QueryServiceNow InsertServiceNow Update, and ServiceNow Delete Snaps with a retry mechanism that includes the following fields:

  • Number Of Retries
  • Retry Interval (seconds)

Snowflake:

  • Enhanced the Snowflake S3 Database and Snowflake S3 Dynamic accounts with S3 AWS Token field that allows you to connect to private and protected Amazon S3 buckets.
  • Added a new account type Snowflake Google Storage Database to connect to Google Cloud Storage to load data.
  • Added support for all existing Snowflake Snap accounts to connect to a Snowflake instance hosted on the Google Cloud Platform

  • Enhanced Snowflake - Lookup and Snowflake SCD2 Snaps with the Input Date Format field to select from the following two options:
    • Continue to execute the snap with the given input Date format
    • Auto Convert the format to Snowflake default format

SOAPEnhanced the SOAP Execute Snap with a new checkbox Escape special characters to escape XML special characters in variable values when inserting values into the Velocity template.

SQL Server: Fixed an issue with the SQL Server - Bulk Load Snap where the Snap fails when the login password contains a colon or a less than (<) symbol.

Transform:

  • Enhanced the CSV Parser Snap with a new checkbox Preserve Surrounding Spaces that enable the Snap to preserve the surrounding spaces for the values that are non-quoted.
  • Enhanced the JSON Splitter Snap with a new field Show Null Values for Include Paths that enables the Snap to show key-value entries of the null values for the objects added to the Include Paths field in the output document.
  • Enhanced the Join Snap with a new field Available Memory Threshold (%) to enable the Snap to keep all the Right input view documents with the same join-path values in memory until the join operation is done for the specific join-path values. 
  • Replaced Strict XSD output field in the XML Formatter Snap with Map input to first repeating element in XSD. If selected, the Snap ignores the root element from the XSD file.  

Improved the error messages in Teradata Snap Pack, Oracle Snap Pack, MySQL - Select in MySQL Snap Pack, Channel Operations Snap in Slack Snap Pack, Create Event Snap in Exchange Online and Teams - Create Team in the Teams Snap Pack where the Snaps fail with a null pointer exception error when the given account information is invalid. 

Enhanced the JIRACoupa, SOAP Snap Packs and Workday Prism Analytics Bulk Load Snap in Workday Prism Analytics Snap Pack to support HTTP proxy authentication.

Updated the AWS SDK from version 1.11.688 to 1.11.1010 in the DynamoDB Snap Pack Redshift Snap Pack, and added a custom SnapLogic User Agent header value.

Renamed Snap Packs

Revised the names of the following Snap Packs. This does not affect your subscription to these Snap Packs.

Old Snap Pack NameNew Snap Pack Name
Confluent Kafka Kafka 
Google Spreadsheet Google Sheets 
Tableau 9&10 

Tableau 

The Snap Pack consisting of the Tableau 8 version Snaps is deprecated starting from the 4.26 GA. See the Deprecated Snaps section for more information.

Deprecated Snaps

The Snap Pack consisting of the Tableau 8 version Snaps (Tableau 8 Write and TDE 8 Formatter) is deprecated due to no customer usage. Contact support@snaplogic.com, if your existing Pipelines use Snaps from the deprecated Snap Pack.

Known Issues

None.

Prior 4.25 Snap Dot Releases that are Merged into 4.26 GA

 Click here to expand...

Snap PackDate of UpdateSnap Pack Build NameUpdates
Box425patches10840

Modified default values for the following fields in Box Read and Box Write Snaps:

  • Number of Retries - Changed from 3 to 0.
  • Retry Interval (seconds) -  Changed from 5 to 1.

For existing Pipelines which have the Enable Staging (deprecated field) checkbox selected, this Snap retries the connection once even if the Number of Retries is zero.

DynamoDB 
Flow
JIRA 
Microsoft OneDrive
Teams
SAP
 425patches10571
  • Updated the AWS SDK from version 1.11.688 to 1.11.1010 in the DynamoDB Snap Pack and added a custom SnapLogic User Agent header value.
  • Enhanced the performance of Binary Router Snap through an efficient IO utilization.
  • Enhanced the JIRA Snap Pack to support HTTP proxy authentication.
  • Updated the error output view of the OneDrive - Upload File Snap from a Binary view to a Document view.
  • Improved the error messages in the Teams - Create Team Snap where the Snap fails with a null pointer exception error when the given account information is invalid.
  • Renamed the RFC Listener Snap to SAP RFC Listener Snap. 
Amazon SQS
Binary 
Coupa 
Exchange Online
ML Data Preparation
Slack 
SOAP
Tableau
Workday Prism Analytics 
 425patches10994
  • Enhanced the AWS S3 and Amazon SQS accounts to add a Snap’s information to the User Agent header for Cross IAM account requests.
  • Removed duplicate strings in a request’s User Agent header sent by Amazon SQS or Binary S3 Snaps.
  • In the Binary Snap Pack:
    • Enhanced the Directory Browser Snap to process files with the owner field for the SFTP protocol. Upon validation, the output of the Snap fetches the owner’s name, which was previously displayed as unknown.
    • Fixed an issue with the File Writer Snap where the Snap fails to connect to the SFTP endpoint using the AWS Transfer Family.
    • Fixed an issue where Pipelines fail to connect the SFTP Server with the Binary Snap Pack after upgrading to the SnapLogic 4.25 release version (main9554).
  • Enhanced the Coupa Snap Pack to support HTTP proxy authentication.
  • Improved the Exchange Online Snap Pack to handle authentication token errors and acquire new tokens and improved the error messages in the Create Event Snap where the Snap fails with a null pointer exception error when the given account information is invalid.
  • Fixed an issue with the Deduplicate Snap where the Snap breaks when running on a locale that does not format decimals with Period (.) character. 
  • Improved the error messages in the Channel Operations Snap where the Snap fails with a null pointer exception error when the given account information is invalid.
  • Enhanced the SOAP Snap Pack to support HTTP proxy authentication.
  • Fixed an issue with Tableau REST Snap where the Snap was not able to filter or route the output data shows either duplicate records or incorrect results.
  • Enhanced the Workday Prism Analytics Bulk Load Snap to support HTTP proxy authentication.
JDBC 
MySQL
Oracle
Redshift
Teradata
 425patches11008
  • In the JDBC Snap Pack:
    • Fixed an issue with the Generic JDBC - Select Snap where if there are URL properties in the account, the unPooledConnection does not use the properties when creating a connection. This causes the auto-detect functionality to use the wrong database name.
    • Fixed an issue with MySQL and MariaDB in the Generic JDBC - Update Snap when a table is incorrectly reported as not being found.
  • In the MySQL Snap Pack:
    • Enhanced the MySQL - Lookup Snap suggestions to load the Output Fields and Lookup Column settings.
    • Fixed an issue with the MySQL - Select Snap where the second output view of the Snap was showing empty column values and improved error messages in the Snap where the Snap fails with a null pointer exception error when the given account information is invalid.
  • In the Oracle Snap Pack:
    • Improved the support for the CLOB data type in the Oracle - Insert Snap. 
    • Improved the error messages for all the Snaps in the Oracle Snap Pack where the Snaps fail with a null pointer exception error when the given account information is invalid. 
    • Updated documentation links for Oracle Thin Account and Oracle Thin Dynamic Account in the Snap settings. 
  • Updated the AWS SDK from version 1.11.688 to 1.11.1010 in the Redshift Snap Pack and added a custom SnapLogic User Agent header value.
  • Improved the error messages for all the Snaps in the Teradata Snap Pack where the Snaps fail with a null pointer exception error when the given account information is invalid. 
Microsoft Dynamics CRM  425patches10880Improved the Dynamics 365 for Sales Upsert Snap by caching the parameter lookup values.
Azure Active Directory 425patches10868Enhanced the Azure Active Directory Create Entry and Azure Active Directory Update Entry Snaps to support Pipeline parameters and upstream values for the Attribute name setting under Attributes and the Snap Pack to support proxy authentication. 
SQL Server 425patches10797

Fixed an issue with the SQL Server - Bulk Load Snap where the Snap fails when the login password contains a colon or a less than (<) symbol.

REST

425patches10419

Fixed a memory leak issue in the REST Snap Pack.
Anaplan 425patches10360

Added the following fields to the Anaplan Write Snap:

  • Override server settings
  • Separator
  • Text Delimiter
Salesforce 425patches10182Fixed an issue with the Salesforce Read Snap where the retry fields do not work as expected when a Pipeline has more than ten Snaps and improved the error messages in the Snap where it fails with a null pointer exception error when the account reference provided is invalid.
REST 425patches9929Enhanced the REST Snap Pack to support Proxy Authentication.
Transform 425patches10663

Fixed an issue in the CSV Formatter Snap where even if the Ignore empty stream checkbox is not enabled, the Snap did not produce an empty binary stream output in case there is no input document.

Reltio 425patches10410Fixed an issue with the Reltio Create Snap where the Snap processes a lesser number of documents while loading data into Reltio.
Transform 425patches10152

Enhanced the CSV Parser Snap with a new checkbox Preserve Surrounding Spaces that enables you to preserve the surrounding spaces for the values that are non-quoted.

Replaced the Strict XSD output field with Map input to first repeating element in XSD field in the XML Formatter Snap. If selected, the Snap ignores the root element from the XSD file.

SOAP 425patches10152

Enhanced the SOAP Execute Snap with a new checkbox Escape special characters that enable the snap to escape XML special characters in XML template variable values when this Snap inserts these values into the Apache Velocity template.

Snowflake 425patches10190Enhanced the Snowflake S3 Database and Snowflake S3 Dynamic accounts with a new field S3 AWS Token that allows you to connect to private and protected Amazon S3 buckets.
Confluent Kafka 425patches10543
  • Improved the handling of interrupted/aborted Kafka Snaps to ensure proper clean-up of metrics.

  • Optimized the Kafka Producer Snap to initialize the Kafka API only if there is at least one input document.

  • Fixed an issue of account passwords being included in the log messages output of Kafka Snaps. The password is now hidden in the logs for both Kafka Consumer and Kafka Producer Snaps.

  • Enhanced the Kafka Producer Snap to support Avro serialization format with a new target data type Avro under Key and Value Serializer.
  • Enhanced the Kafka SSL Account with new fields (Registry Username or Key and Registry Password or Secret) for Schema Registry authentication. The two existing SASL properties (SASL Username and SASL Password) have been revised to SASL Username or Key and SASL Password or Secret, respectively.

  • The Kafka Producer and Consumer Snaps create a default client ID string to identify the specific Snap instance.
  • Apache Kafka client library is upgraded from version 2.6.0 to 2.8.0.

  • Confluent Kafka client libraries are upgraded from version 5.2.1 to 6.2.0.

OpenAPI 425patches10417

Enhanced the OpenAPI Snap with the following two fields:

  • HTTP Headers
  • Query Parameters
Salesforce 425patches9609Enhanced the Salesforce Read Snap to allow you to add an optional second output view that exposes the schema of the target object as the output document. The output view of the Snap has a minimum of one output and a maximum of two outputs. 
Microsoft Dynamics CRM 425patches9813

Fixed an issue with Dynamics CRM Search Snap where the Snap returns duplicate records when reading results from multiple pages.

Binary  425patches10217
  • Fixed the following issues in the PGP Decrypt Snap caused due to the Snap patch 425patches9581:
    • The Snap writes an empty document to the output view when there is an error in the Snap. It will now decrypt the document and write it to the output view only if it succeeds.
    • The Snap fails to decrypt any non-signed encrypted file more than 500 bytes. It will now decrypt both non-signed and signed encrypted files. 
  • Removed the Project ID field from the Google Service Account in the Binary Snap Pack, since the JSON Key already contains information about the Project ID. 
MS Dynamics 365 for Sales 425patches10303

Fixed an error in the Dynamics 365 for Sales Create Snap where the Snap does not send the original documents to the error view in the output when the batch size property is set to greater than 1.

ServiceNow 425patches10014

Enhanced the ServiceNow Query Snap with the retry mechanism that includes the following settings:

  • Number of retries: Specifies the number of attempts the Snap should make to perform the selected operation in case of a connection failure or a timeout.
  • Retry interval (seconds)Specifies the minimum number of seconds the Snap must wait before each retry attempt.
Box  425patches10072Modified default values of the following fields across the Box Snap Pack to trigger automatic retries by default, in case of connection failures or timeouts:
  • Number of retries – Changed from 0 to 3
  • Retry interval (seconds) – Changed from 1 to 5
Hadoop 425patches9975

Fixed a dependency issue in Hadoop Parquet Reader Snap while reading data from the AWS S3 folder. The issue is caused due to conflicting definitions for some of the AWS class dependencies in the classpath.

PostgreSQL 425patches9879

Enhanced the performance of PostgreSQL - Bulk Load Snap significantly. We expect the Snap to execute up to 3 times faster than the previous version for enterprise workloads.

Transform 425patches9815Fixed a ClassCastException error in the Avro Parser Snap and handling of the map, fixed, enum, and bytes data types in the Avro Formatter and Avro Parser Snaps.
Binary 425patches9581

Fixed an issue with the PGP Decrypt Snap where the Snap writes an empty document to the output view when there is an error in the Snap. It will now decrypt the document and write it to the output view only if it succeeds.

As of  we have discovered an issue with this Binary patch affecting the PGP Decrypt Snap. In case your Pipelines use the PGP Decrypt Snap, we recommend you to NOT use this patch. Else, your Pipelines will likely fail. We're working on a fix and will be deploying a Snap patch soon.

Transform 425patches9749Enhanced the XML Parser Snap to recognize input headers when defining the inbound schema.
Amazon SQS 425patches9596Added the Security token field to the Amazon SQS account.
REST 425patches9684
  • Fixed the REST Post Snap that fails to correctly set the Content-Length header when working with multi-file uploads.
  • Added Send Client Data as Basic Auth Header check box to the Create Account settings of the OAuth2 and OAuth2 SSL accounts. This check box enables you to include and pass the client credentials information in the header when connecting to the OAuth endpoint.
  • Enhanced the REST Snap Pack with the new account REST AWS Sig V4 Account that adds the ability to call any AWS API.
Binary 425patches9564Enhanced the SFTP session in Binary Snaps to support the SFTP servers that close the SFTP sessions slowly.
Transform 425patches9638

Reverted the Join Snap to the 4.24 release behavior. This is in response to an issue encountered in the Join Snap in the 4.25 release version (main9554) that can result in incorrect outputs from all Join Types. 425patches9638 is the default version for both stable and latest Transform Snap Pack versions for orgs that are on the 4.25 release version. No action is required by customers to receive this update and no impact is anticipated. For more information, contact support@snaplogic.com.

Data Automation

SnapLogic's data automation solution speeds up the identification and integration of new data sources, and the migration of data from legacy systems. The solution can automatically detect duplicate, erroneous, or missing data, and identify structures and formats that do not match the data model. Data automation can accelerate the loading and transformation of your data into the data warehouse, speeding up the data-to-decisions process.

New Features in ELT Snap Pack

Starting with the 4.26 release, you can use the Snaps in the ELT Snap Pack to perform ELT operations on Databricks Lakehouse Platform (DLP). This is in addition to the existing list of supported target databases—Snowflake, Redshift, and Azure Synapse.

Supported JDBC JAR Versions

ELT Snaps automatically use a corresponding JDBC JAR file to connect to your target database and to perform the load and transform operations

DatabaseCertified JDBC JAR File
Azure Synapsemssql-jdbc-9.2.1.jre8.jar
Databricks Lakehouse Platform (DLP)SimbaSparkJDBC42-2.6.17.1021.jar
Redshiftredshift-jdbc42-2.0.0.2.jar
Snowflakesnowflake-jdbc-3.13.1.jar

Using Alternate JDBC JAR File Versions

Though we recommend you to use the above JAR file versions, you can choose to use a different version, based on your environment. 

Updated ELT Snaps

  • Enhanced the ELT Snap preview to support the following Snowflake data types: array, object, variant, and timestamp.

    • The Snaps convert the values to hexadecimal (HEX) equivalents—the default setting for the session parameter BINARY_OUTPUT_FORMAT in Snowflake. See Session Parameters for Binary Values for more information.

    • If this setting is different from hexadecimal (such as base64) in the Snowflake table, the Snaps still convert the values to hexadecimal equivalents for rendering them in the Snap preview.

  • Enhanced all ELT Snaps to display the Get preview data checkbox below the Snap's Label field.
  • The ELT Database account is now mandatory for all Snaps in the ELT Snap Pack.

    Breaking change

    Starting with the 4.26 release, all Snaps in the ELT Snap Pack (except the ELT Copy Snap) require an account to connect to the respective target database. Your existing Pipelines that do not use an account may fail. We recommend you to associate an ELT Database Account to each of the ELT Snaps (except ELT Copy Snap) for your Pipelines.

  • Enhanced the ELT Aggregate Snap to support Linear Regression functions on Redshift and Azure Synapse. The Snap also supports these functions on Databricks Lakehouse Platform.
  • Enhanced the ELT Execute Snap to enable running multiple DML, DDL, and DCL SQL statements from the same Snap instance.
  • Enhanced the ELT Join Snap to:
    • Support LEFT ANTI JOIN and LEFT SEMI JOIN types on all supported databases.
    • Display or hide the Resultant Column Names Prefix Type field based on the target database selected in the Snap's account.
  • Enhanced the ELT Load and ELT SCD2 Snaps to provide a list of suggested data types, while adding columns to or creating a table.

Known Issues

  • Selecting the Pipeline Parameters from the drop-down list enabled for using expressions or the expression editor to define fields within your ELT Database Account does not currently respond as it should. Hence, we recommend that you manually key-in the parameter names with the preceding underscore ( _ ). For example, _dbfsPwd, _sfUsername.
  • ELT Transform Snap displays incorrect data types (string instead of the actual data type) for column names populated in its Input schema section.
  • ELT Pipelines created prior to 4.24 GA release using one or more of the ELT Insert Select, ELT Merge Into, ELT Load, and ELT Execute Snaps may fail to show expected preview data due to a common change made across the Snap Pack for the current release (4.26 GA). In such a scenario, replace the Snap in your Pipeline with the same Snap from the Asset Palette and configure the Snap's Settings again.

  • In case of Databricks Lakehouse Platform, all ELT Snaps' preview data (during validation) contains a value with precision higher than that of the actual floating point value (float data type) stored in the Delta. For example, 24.123404659344 instead of 24.1234. However, the Snap reflects the exact values during Pipeline executions.

  • In any of the supported target databases, this Snap does not appropriately identify nor render column references beginning with an _ (underscore) inside SQL queries/statements that use the following constructs and contexts (the Snap works as expected in all other scenarios):
    • WHERE clause (ELT Filter Snap)
    • WHEN clause
    • ON condition (ELT Join, ELT Merge Into Snaps)
    • HAVING clause
    • QUALIFY clause
    • Insert expressions (column names and values in ELT Insert Select, ELT Load, and ELT Merge Into Snaps)
    • Update expressions list (column names and values in ELT Merge Into Snap)
    • Secondary AND condition
    • Inside SQL query editor (ELT Select and ELT Execute Snaps)

Workaround

As a workaround while using these SQL query constructs, you can:

Precede this Snap with an ELT Transform Snap to re-map the '_' column references to suitable column names (that do not begin with an _ ) and reference the new column names in the next Snap, as needed.

  • In case of Databricks Lakehouse Platform where CSV files do not have a header (column names), a simple query like SELECT * FROM CSV.`/mnt/csv1.csv` returns default names such as _c0, _c1, _c2 for the columns which this Snap cannot interpret. To avoid this scenario, you can:
    • Write the data in the CSV file to a DLP table beforehand, as in: CREATE TABLE csvdatatable (a1 int, b1 int,…) USING CSV `/mnt/csv1.csv` where a1, b1, and so on are the new column names.
    • Then, read the data from this new table (with column names a1, b1, and so on) using a simple SELECT statement.
  • Fixed with 426Patches11262 on When loading data from a JSON file into a target Databricks Lakehouse Platform (DLP) instance using an ELT Load Snap, if you choose the Drop and Create Table option as the Load Action and specify an additional column (that is not available in the JSON file) for the new table, it results in one more column null added to the new target table. In case of CSV files, this Snap behaves the same way as mentioned above, but only when the CSV file uses a comma separator.
  • Fixed with 426Patches11262 on : For a Snowflake target instance, the ELT Insert Select Snap does not suggest column names to select for the Insert Column field in the Insert Expression List.
  • Due to an issue with the COPY_INTO mechanism provided by Azure Synapse, the ELT Load Snap does not display an error message (the Pipeline appears to run successfully) even when the source file path does not resolve to a valid file. We recommend that you verify whether the source file location you provide in the Snap’s settings is valid and accessible, before running your Pipeline.
  • ELT Pipelines targeting a Databricks Lakehouse Platform (DLP) instance might fail due to a very long or complex SQL query that they build.
    • As a workaround, you can set an advanced (URL) property useNativeQuery to 1 in your ELT Database Account configuration as shown below:
  • Fixed with 426Patches11262 on : The Snaps—ELT Merge Into, ELT Select, ELT Join, and ELT Filter—do not prevent the risk of SQL injection when your target database is Databricks Lakehouse Platform (DLP).
  • After the fix with 426Patches11262 on , the ELT Load Snap on Databricks Lakehouse Platform (DLP) may cause intermittent null-pointer exceptions only when you specify an incorrect source file path in any of the DBFS Folder Path and the File List > File fields.

  • Fixed with 426Patches11262 on : In case you are writing into a Snowflake target table, the ELT Insert Select Snap attempts to create the target table even when it exists in the database.
  • Fixed with 426Patches11262 on : When you use the SQL editor in the ELT Select Snap configuration to define your SQL query, the Pipeline validation fails due to a syntax error in the following scenarios. However, the Pipeline execution works as expected. The only workaround is to drop the LIMIT clause and the optional OFFSET clause from the SQL query during Pipeline validation.
    • The query contains a LIMIT clause on a Snowflake, Redshift or Databricks Lakehouse Platform target instance: The SQL query created during Pipeline validation includes an additional LIMIT clause, for example: SELECT * FROM "STORE_DATA"."ORDERS" LIMIT 10 LIMIT 990

    • The query contains an OFFSET clause (supported in case of Snowflake and Redshift): The SQL query created during Pipeline validation looks like SELECT * FROM "STORE_DATA"."ORDERS" LIMIT 10 OFFSET 4 LIMIT 990

Prior 4.25 ELT Dot Releases that are Merged into 4.26 GA

 Click here to expand...

Snap PackDate of UpdateSnap Pack Build NameUpdates
ELT 425patches10017
  • Updated the ELT SCD2 Snap to replace End date of historical row option in the Meaning field of Target Table SCD2 Fields field set with End Date of Current Row.

    Breaking change

    This may cause the existing Pipelines to fail as the End date of historical row option no longer exists.

    You need to make the following update in the ELT SCD2 Snap's settings across your Pipelines after upgrading your Snap Pack to this patch version:

    • Select End Date of Current Row from the Meaning drop-down list in the second entry (highlighted in the image).
  • Fixed the issue with the ELT Insert Select Snap containing an open output preview that fails to retrieve output preview data in case of Redshift and Azure Synapse databases, though the Pipeline runs work as expected.
  • Fixed an issue where the ELT Execute Snap does not error out (Snap turns Green) even when running an SQL query to drop a non-existent table from a Snowflake or Azure Synapse database.

  • [Update on ]: Enhanced the ELT Snap previews to support the following data types: array, object, variant, and timestamp.

    • The Snaps convert the values to hexadecimal (HEX) equivalents—the default setting for the session parameter BINARY_OUTPUT_FORMAT in Snowflake. See Session Parameters for Binary Values for more information.

    • If this setting is different from hexadecimal (such as base64) in the Snowflake table, the Snaps still convert the values to hexadecimal equivalents for rendering them in the Snap previews.

ELT
 
425patches9725
  • Enhanced the ELT Snap preview to display the exact binary and varbinary values from the Snowflake database during Pipeline validation, by converting the values to hexadecimal equivalents—the default setting in SnowflakeIf the setting is different from hexadecimal in the Snowflake table, then the Snaps still convert the values to hexadecimal for rendering the Snap preview.
  • Enhanced the ELT Transform Snap to display the appropriate data type (binary or varbinary) for the column names populated in the output schema.
  • Enhanced the ELT Window Functions Snap to address potential issues due to an incorrect definition for MINUS function in the case of Redshift and Azure Synapse databases.

Introducing SnapLogic Flows

SnapLogic Flows is a breakthrough new user interface that empowers business users to self-build new application integrations and data automations to support the specific needs of their respective functions and departments. Flows removes coding barriers and makes it easy for business users to succeed in developing and integrating applications such as Salesforce, Marketo, Google Sheets, Slack, to get the data and insights they need to make faster business decisions.

Flows also enables IT to step away from the core development of these solutions but gives them the ability to add requirements and guardrails for non-technical developers, enabling IT to centrally maintain visibility, control access, and oversee what is developed before it is pushed to production. 

To get started, register for Flows.