Versions Compared

Version Old Version 80 New Version 81
Changes made by

Anand Vedam

John Brinckwirth

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents
minLevel1
maxLevel1
include
outlinefalse
indent
exclude
typeflat
separatorbrackets
class
printablefalse

...

The APIM dashboard displays more data for Ultra requests. The dashboard displays the number of requests waiting in the queue for each minute, every FeedMaster node, and the total number of requests across all FeedMaster nodes. You can now view the average wait time, average processing time, and average total time for each Ultra Task.

...

The API dashboard also introduces a new Queue Size widget that displays the number of requests inline to be processed per time segment.

...

...

Known Issues

In the Response Mapper and Response Transformer policies, APIM functions display in the expressions dropdown menu but do not actually work.

AutoSync

Features

  • AutoSync now supports the following endpoints:

    • Google Sheets as a source

    • Snowflake as a source

    • Google BigQuery as a source

  • Detection of changes to source data types: Now AutoSync detects changes to source column or field data types and updates the destination table with a compatible type. Supported for Amazon Redshift, Google BigQuery, and Snowflake destinations. Learn more.

...

  • You can now append data to a table created from a CSV file by splitting the file. This is useful, for example, when the data exceeds the 100 MB size limit or when a system periodically generates CSV files and you want them to load to the same table. Learn more.

  • New experience for sharing data pipelines. Learn more.

  • The Snowflake destination endpoint now uses internal Snowflake staging, which removes the need to have and configure a separate staging environment.

  • The Amazon Redshift endpoint now supports using an S3 bucket for staging that is in a different AWS region. If the S3 bucket is in the same region as the Redshift instance, you can leave the region unspecified.

Known Issues

  • When a data pipeline with Snowflake as a destination encounters an error, it stops loading. It should instead log the error record and continue.

...

Panel
panelIconIdatlassian-warning
panelIcon:warning:
bgColor#FFEBE6

Breaking Change

With the new support for LDAP authentication to HashiCorp Vaults, the Snaplex node configuration file secrets-config.json requires a new key, auth_method, with a value approle or ldap.

Action needed:

You must update each of your existing Snaplex node configuration files to include this key-value pair: "auth_method": "approle", as prescribed in the Step 1a-Approle, for the approle-based authentication to succeed.

Known Issues

  • In GitHub, you cannot create a branch doing a Git checkout from the repository at the Project Space level.

  • The Git status Updated does not work at the Project space level for all Git repos.

  • In GitHub, when a root folder is already untracked in a Project Space after doing the checkout, the root (1) folder is not created.

  • For GitLab integration, when updating the Git assets with the status Tracked Modified locally and remotely, a conflict error message displays.

  • For GitLab integration, while creating a Project from a Git checkout, an error message displays.

  • For GitLab integration, doing a Git Repository Checkout with Hard Reset results in an internal server error.

  • If a self-managed GitLab server does not have GitLab as part of the hostname, Status, Pull, Commit, Create Branch, and Create Tag commands fail.

...

  • Starting from version 433patches21482, the PGP Encrypt Snap does not support encryption with an expired key. To ensure continued support for encryption, we highly recommend that you renew your PGP key before it expires or extend the expiration date of your PGP key.

  • Starting from version 433patches22343, the Vertica-Bulk Load Snap displays no output in the output view when the Snap receives no input.

Known Issues

  • For the Binary and Email Snap Packs, SFTP connections might experience connectivity issues because of the removal of some deprecated algorithms. To add these connections back, refer to Configuration Settings for Snaps for details.

  • If you use the PostgreSQL driver (org.postgresql.Driver) with the Redshift Snap Pack, it could result in errors if the data type provided to the Snap does not match the data type in the Redshift table schema. Either use the Redshift driver (com.amazon.redshift.jdbc42.Driver) or use the correct data type in the input document to resolve these errors

  • The Send Email Snap might fail because of a 429 error response from the Graph API endpoint.

  • For Ultra Tasks using a continuous data flow design for polling with a Pipeline Execute Snap, the parent pipeline does not receive the document from the child pipeline, and the polling message is not generated for downstream Snaps.

  • The GeoJSON Parser and WKT Parser Snaps accept only application/zip or application/gml+xml; charset=utf-8 as a file input stream.
    However, the output content type from the File Reader Snap is binary/octet when the input file is read from SLDB.

    Workaround: To read an input file (ZIP or GML) from SLDB, use the Mapper Snap to change the content type from binary/octet to one of the following:

    • ZIP file: Change the content type to application/zip .

    • GML file: Change the content type to application/gml+xml; charset=utf-8.

...