In this article

Key New Features in SnapLogic

Platform:

• New Public API endpoints to facilitate integration with your CI/CD tools:

  • Create new Project Spaces and Projects to run your tests.

  • Manage Permissions in specific Orgs as you promote Pipelines and Assets.

  • List Git status of Assets.

API Management: 

• Expand your API ecosystem to manage, secure, and control third-party and public APIs through the API Management solution.

• Secure external APIs with policies for security, traffic management, and more.

Developer Portal:

• Improved application development time by testing and sampling custom APIs within the Developer Portal using the “Try it out” feature in the API documentation page.

Snaps and Accounts:

• Added Azure Service Bus Snap Pack.

• Added Exact Online Snap Pack.

• Added Active Directory Remove Member Snap.

• Added SharePoint Online - Get Sites Snap.

• Added OAuth2 Account support for ServiceNow Snap Pack.

• Added OAuth2 Account support for SuccessFactors Snap Pack.

• Added OAuth2 Account support for Snowflake Snap Pack.Snowflake Snap Pack.

ELT for Cloud Data Platforms:

• Subquery Pushdown Optimization.

• Added new ELT Snaps: ELT Coalesce, ELT Conditional Function, ELT Math Function, and ELT SQL Case Expression.

• Support for loading data from S3 and Redshift source locations to BigQuery target tables.

SnapLogic Flows:

• Added Webhook support for Salesforce and HubSpot.

• Added two new endpoints: HubSpot and Jira (source).

• Added Auto Data Type Conversion.

4.28 UAT Delta

4.28 Dot Releases

SnapLogic Studio

Snaps

Platform 

New Features

• New and enhanced Public APIs for improved integration with CI/CD tools

  • The following new public APIs are now available:

    • POST /assetapi/acl/{project_path} – Grants a user or group access to the specified project.

    • GET /assetapi/acl/{project_path} – Retrieves the privileges of the current user to access the specified project.

    • DELETE /assetapi/acl/{project_path} – Revokes all project access from a user or group.

    • POST /project/{projpath} – Creates a new project or project space.

    • PUT /project/{projpath} – Renames an existing project or project space.

    • GET /repo-status/{projpath} – Determines the status of a project’s GitHub repository.

    • GET /migrate/{status_token} – Determines the status of a migration.

  • The following public APIs were modified:

    • GET /activities/{org} has a new optional query parameter: filter_snaplogic_admin_update.

Enhancements

• Updated the Snaplex installer to use JRE 11.0.12+7 instead of 11.0.10+9. We have committed to upgrade the version twice a year as part of security compliance. New Groundplex installations will use this version, and Cloudplex installations will get upgraded as part of the version upgrade. Existing Groundplex installations will continue on the current version until the RPM/DEB is manually reinstalled.

Fixed Issues

• Fixed the issue in ServiceNow Snaps where Number of Retries was showing empty or blank values.

• Fixed 404 errors when selecting 8 Hours or 24 Hours in the Dashboard under the Health tab.

• Fixed the missing default value for Skip lines in the CSV Parser Snap settings dialog.

Known Issues

• When creating new Projects using the Public API (POST /project/{projpath}), the checkout Public API (POST /project/checkout/{project_path}) is currently unable to create the initial link between the Project and the repository, and the checkout will fail. As a workaround, check out using the UI. Target fix version: 4.28 patch in March.

• When committing files to GitHub from SnapLogic, some subdirectories might be inadvertently deleted from the repository, including GitHub Actions workflows in the .github directory. Target fix version: 4.28 patch in March.

API Management

New Features

• Proxies. SnapLogic's new Proxies feature allows you to seamlessly integrate with your custom APIs and third-party APIs by providing API administration, monitoring, and security management within the API Manager.

  

• Naming restrictions for APIM assets. The following restrictions now apply to the names for API Manager assets (APIs, API versions, Proxies, Proxy Endpoints):

  • Must be unique.

  • Must begin with alphanumeric characters A-Z, a-z, or 0-9.

  • Cannot contain the reserved keyword shared.

  • Cannot contain |, <, >, [, ], {, }, #, ?, /, and \.

  • Must be no more than 256 characters. If the name includes double-byte characters, the maximum length is shorter.

• Deleting API version assets. You can now delete multiple assets associated with an API version at the same time. To do so:

1. Go to the API Manager and select the API.

2. In the Versions tab, select the version.

3. In the Assets tab, select the assets to delete.

4. Click the trash icon ( ).

Developer Portal

New Features

• Try It Out. You can now test your APIs in the Developer Portal using the Try it out button. It works with all APIs with only two requirements:

  • The API must include a CORS Restriction policy.

  • The API must be published.

    

• Subscribe button in the API description. You can now subscribe to a custom API while viewing its description page in the Developer Portal.

  

Enhancements

• Account and Subscription Management Based on User Role. Only Org Administrators can manage (create, approve, deny, or delete) Developer Portal user accounts and API subscriptions. System Administrators and regular users can only view these users and API subscriptions.

Snaps

New Snaps

Azure Service Bus: A cloud messaging service that connects applications, devices, and services running in Azure Cloud to any other applications or services. This Snap Pack offers the following Snaps:

• ASB Acknowledge: Acknowledges messages from a queue in ASB.

• ASB Producer: Sends messages to a queue or topic in ASB.

• ASB Consumer: Reads messages from a queue or topic in ASB.

Exact Online: Allows companies to speed up their business process and automation for payroll, expenses, and other accounting workflows. This Snap Pack offers the following Snaps:

• Exact Online Read: Reads and retrieves data objects from the Exact Online instance.

• Exact Online Create: Creates data objects in the Exact Online instance.

• Exact Online Update: Updates a data object in the Exact Online instance.

• Exact Online Delete: Deletes a data object in the Exact Online instance.

Active Directory: Added the Active Directory Remove Member Snap that enables you to remove an existing member from a specified group in Active Directory.

SharePoint Online: Added the SharePoint Online - Get Sites Snap that enables you to get the list of sites specific to the logged-in user from the SharePoint Online site.

Snap Enhancements

• Binary

  • Enhanced the File Operation Snap to support Azure Data Lake Storage (ADLS) Gen2 protocol for moving and copying files in the Azure Blob File System (ABFS).

  • Fixed an issue with the File Poller Snap to enhance the performance of the polling operations that allow the Snap to poll the files without stopping.

  • Enhanced the S3 File Writer Snap with the Suggest fully-qualified file names checkbox that allows you to use fully-qualified file names in the File name Suggestions list for gov cloud instances.

  • Enhanced the File Writer, S3 File Writer, and ZipFile Write Snaps with the AWS Canned ACL (Access Control List) permission mode that enables you to use Canned ACLs when creating the S3 Object. This feature allows you to grant full access of the object to the S3 bucket owner.
    

• Database Delete Snaps

  • Updated the label for Delete Condition to Delete Condition (Truncates Table if empty) in the following Snaps:

    • Snowflake Delete

    • SAP HANA Delete

    • MySQL Delete

    • Redshift Delete

    • Oracle Delete

    • PostgreSQL Delete

    • SQL Server Delete

    • Vertica Delete
      

• Flow: Enhanced the Filter Snap with the Null-safe access checkbox. Select this checkbox to consider the valid values in the filter expression and remove the missing or empty data instead of treating them as null.
  

• JDBC

  • Enhanced the following Snaps to support DML (Data Manipulation Language) operations (
CREATE, ALTER, INSERT, and SELECT) when using AWS Athena database.

    • Generic JDBC - Execute

    • Generic JDBC - Insert

    • Generic JDBC - Select
      

• Marketo Snap Pack: Enhanced the Bulk Lead Extract and Bulk Lead Upsert Snaps with retry mechanism fields to set a time limit to complete the processing of operations:

  • Timeout (seconds)

  • Number of Retries

  • Retry Interval (seconds)
    

• ML Data Preparation: Enhanced the Type Converter Snap with the Fail safe upon execution checkbox. Select this checkbox to enable the Snap to convert data with valid data types, while ignoring invalid data types.
  

• Oracle

  • Enhanced the Oracle - Update Snap to accept values in the Number of Retries and Retry Interval (seconds) fields from Pipeline parameters only. 

  • Enhanced the Oracle Bulk Load Snap with the Column Length Overrides field set to support CLOB (Character Large Object) and NCLOB (National Character Large Object) data types that may require a large buffer size, allowing users to load large amounts of data into a single column store of data type CLOB or NCLOB.
    

• REST

  • Enhanced the REST Post, REST Put, and REST Patch Snaps with a new field Multipart Type to support multipart (files and text) upload.

  • Renamed the following fields:

    • Upload File(s) field set to Form Upload

    • Upload-file key to Multipart Key

    • Upload file to Multipart Value
      

• SAP

  • Upgraded the Java Native Library used by the SAP Java Connector (JCo) to version 3.1.

• Salesforce

  • Removed Null Setting with Bulk API property in the following because the Salesforce.com server software has been changed such that these Snaps can insert and update null values regardless the Null Setting with Bulk API setting is true or false.

    • Salesforce Update

    • Salesforce Upsert

    • Salesforce Create

• ServiceNow

  • Added a new account type ServiceNow OAuth2 that enables OAuth2 authentication for all the ServiceNow Snaps.
    

• SuccessFactors

  • Added a new account type SuccessFactors OAuth2 that enables OAuth2 authentication for all the SuccessFactors Snaps.
    

• Snowflake

  • Enhanced the Snowflake S3 Database Account with the S3 Storage Integration field to support authentication of Amazon S3 bucket hosting as the external stage. When you provide a value in this field, the Snap enables the COPY command to avoid using AWS credentials for authentication.

  • Added the following account types that enable OAuth2 authentication for all the Snowflake Snaps.

    • Snowflake Azure OAuth Account

    • Snowflake Google Storage OAuth2 Account

    • Snowflake S3 OAuth2 Account
      

• Workday

  • Enhanced the Workday Read Snap with the following:

    • Added the Pass-through on no lookup match checkbox that allows the input document to pass through to the output view when there are no records matching an input document.

    • Parameterization of the Page Number and Page Size fields using Pipeline parameters. You can define and use the parameters in these fields using the
      Expression Enabler  icon to pass values during runtime.

Renamed Snap

Renamed the SuccessFactors Snap Pack to SAP SuccessFactors. This does not affect your subscription to this Snap Pack.

Known Issues

• Expressions used in the Mapper Snap, downstream of any Snowflake Snap, that evaluate to very large values such as EXP(900) are displayed in the input/output previews as Infinity . However, you can see the exact values in the validation previews; hence, ignore this error.

• Google BigQuery does not support very large exponential values—larger than EXP(700). So, while displaying values of such high exponential order in the validation preview, the Google BigQuery Execute Snap routes to the error view, and displays the
  "Data conversion failed for field f0_(FLOAT) of value Infinity." error.
  The Snap also displays empty output in the preview, which is not expected.

• Generic JDBC - Select Snap

  • The metadata output in the second output preview is not displayed in table format when your target database is AWS Athena.

  • The suggestions list is not populated for Table name field when your target database is AWS Athena.
    

• File Operations Snap

  • This Snap does not support using ABFS protocol with a Windows-based Snaplex.

  • When you use special characters, such as ðø©¢¾A²½µ®÷¶þ~ for Source and Target directory and filenames, the Snap fails and results in the following error as the special characters are not supported.
    Error: Illegal character in fragment at index 71: abfs://bigdataqa@bigdataqassl.dfs.core.windows.net/simplechar/owner!@#$^&()_¢äâêîôûñç¡¿ÉÙËǨ°¸ðø©¢¾A²½µ®§÷¶þ~.json,

  • The format: abfs(s)://filesystem@accountname.endpoint/<path> for file path for Source and Target fields does not work as this URL syntax is not supported. The file protocol must begin with abfs(s):///, else the container, account name, and endpoint are interpolated into the URL at runtime, which results in the following error. 
    Error: Unsupported protocol or URL syntax error in abfs(s)://filesystem@accountname.endpoint/<path>,
    Workaround: Use the supported file protocol and correct URL syntax.
    

• Marketo

  • The Bulk Lead Upsert and Bulk Lead Extract Snaps do not display valid error messages when you provide large, empty, null, string, or other invalid values (timeout value that is less, for instance, 10) in the Timeout (seconds) and Number of Retries fields. These Snaps work successfully and do not display any error message when you pass large input data through the Pipeline parameters. However, when you pass large input static data in these fields the Snap displays an error.

  • The Bulk Lead Upsert Snap fails upon parsing data greater than 100000 records.
    

• Snowflake

  • Snowflake OAuth2 Account has a maximum validity of the Refresh tokens as 90 days (7776000 seconds). The Snowflake platform does not allow you to get a new refresh token when refreshing the access token for the Snowflake External OAuth.

    Solution: To get a new token after every 90 days you must re authorize your Snowflake account for the token to be valid for the next 90 days. We recommend you to set the oauth_refresh_token_validity to 7776000 seconds as shown below when creating the Security Integration in Snowflake.

    alter integration <integration name> set oauth_refresh_token_validity = 7776000;

Prior 4.27 Snap Dot Releases that are Merged into 4.28 GA

ELT for Cloud Data Platforms

SnapLogic's ELT data solution for cloud data platforms accelerates the identification and integration of new data sources and the migration of data from legacy systems. The solution can automatically detect duplicate, erroneous, or missing data, and identify structures and formats that do not match the data model. Data automation can accelerate the loading and transformation of your data into the data warehouse, speeding up the data-to-decisions process.

New Features in ELT Snap Pack

• Subquery Pushdown Optimization: SnapLogic now optimizes SQL queries before they are passed to the CDW to ensure the queries are performant and cost-efficient in the respective CDW. An SQL subquery means a query inside a query. Pushdown optimization refers to rewriting these incremental (nested) SQL queries produced in your ELT Pipeline to form a more optimal/performant version.

  • Enhanced all Snaps in the ELT Snap Pack with the ability to push down optimized SQL subqueries along the ELT Pipelines. See Subquery Pushdown Optimization (SPDO) for details.

New ELT Snaps

• Introduced the following new ELT Snaps:

  • ELT Case Expression: Snap to return the action to perform on an event based on a list of events and respective expected actions.

  • ELT Coalesce: Snap to return the first non-NULL value from a list of arguments.

  • ELT Conditional Functions: Snap to perform unary and binary conditional operations on data.

  • ELT Math Functions: Snap to perform mathematical—arithmetic, logarithmic, trigonometric, exponent, root, rounding, and truncation—operations on data.

ELT Snap Enhancements

• Enhanced all the expression-enabled fields in ELT Snaps to display suggestions from the Input Schema (emanating from the upstream Snaps) in addition to the existing standard SQL expressions and functions list.

• Enhanced the ELT Aggregate Snap to support:

  • HAVING clause within GROUP BY clause, when the WHERE clause cannot be used.

  • GROUP BY ROLLUP.

  • New aggregate functions for DLP: ANY, SOME, KURTOSIS, and STDDEV.

• Enhanced the ELT Load Snap to support loading data into BigQuery targets from S3 buckets and Redshift CDW. These load operations use the BigQuery Data Transfer Service (DTS) client libraries and are carried out in asynchronous mode.

  • Enhanced the ELT Database Account to support these operations.

• Enhanced the ELT Load and ELT Insert Select Snaps with a new fieldset Table Options List to support defining the Table Options for creating a new table in your target CDW.

• Enhanced the ELT Select Snap to support Common Table Expression (CTE)-based SQL queries that contain a WITH clause inside the SQL Query Editor field, when your target CDW is Azure Synapse.

Known Issues

• When using the ELT SCD2 Snap to load your source data into a Redshift target instance, the Snap fails if you use constant values such as ‘2022-01-01’ or SQL expressions such as GETDATE() and current_time() that evaluate to constant values instead of the target table column names in the Target Table Sort Fields.

• [This issue is fixed in 428patches15290] When any of the functions in the ELT Math Function Snap encounters an infinity value—while converting the data type of double values to big decimal—the Snap fails during validation with the error Character I is neither a decimal digit number, decimal point, nor "e" notation exponential mark.

Flows

New Features

• Added Webhook support for the Salesforce and HubSpot endpoints. Webhooks enable users to receive notifications (events) when they happen. You can configure events within the Salesforce or HubSpot endpoints and a notification will be sent to SnapLogic when the event occurs.

  

• Added two object types for the Marketo endpoint: Get Lead by List Name and Get Campaign by Name.

• Added two new endpoints:

  • HubSpot: The HubSpot endpoint connects to the CRM platform for marketing, sales, content management, and customer service.

  • Jira (source): The Jira source endpoint connects to Jira’s agile project management software.

• You can now import/export a Flow from one project space to another.

Auto Data Type Conversion

• You can now see data types in the filter dropdown when selecting input data in the Filter Source tab of the Data Wrangler.

• Flows will now automatically suggest data types based on introspection of the preview data. Users can override the data type as needed.

  

Enhancements

• The Slack endpoint now has an input box for the Users target type. You can use the dropdown to choose from a list of users.

  

• Now when Read from a pipeline is chosen as a source, only Pipelines that are compatible with Flows can be selected.

Fixed Issues

• Fixed an issue where users were unable to map object fields in the Map Destination tab of the Data Wrangler.

• Fixed an issue where the in-app documentation for the Slack endpoint was not displaying.

• Fixed an issue where users were unable to select the Column field in the Filter condition tab of the Data Wrangler.