Fixed the thread leak issue in Google BigQuery Snaps.

Fixed an issue with the BigQuery Upsert (Streaming) Snap where an HTTP 404 error occurred when retrieving job status for multi-regional datasets in BigQuery. This occurred because the location was not explicitly specified in the request. The job request now includes the location information, ensuring successful job polling regardless of the region (for example, the US or EU).

• The Google BigQuery Bulk Load (Cloud Storage) Snap now displays a lint warning when the bulk load process completes successfully but the temporary file deletion fails.

• Enhanced the following Google BigQuery Snaps with implicit retry functionality to improve the reliability of CRUD operations by retrying after errors:

  • BigQuery Write

  • BigQuery Execute

  • BigQuery Bulk Load (Cloud Storage)

439patches29499

• The BigQuery Bulk Load (Cloud Storage) Snap now displays a lint warning when the bulk load process completes successfully but the temporary file deletion fails.

• Enhanced the following Google BigQuery Snaps with implicit retry functionality to improve the reliability of CRUD operations in BigQuery and handle all retriable BigQuery errors:

  • BigQuery Write

  • BigQuery Table Delete

  • BigQuery Table Data List

  • BigQuery Table Create

  • BigQuery Execute

  • BigQuery Dataset List

  • BigQuery Dataset Delete

  • BigQuery Dataset Create

  • BigQuery Table List

  • BigQuery Bulk Load (Cloud Storage)

Fixed an issue with the BigQuery Table Data List Snap that displayed a null pointer exception when the table source schema contained a nested Array list schema.

August 2024

main27765

21 Aug 2024 

Stable

Updated and certified against the current SnapLogic Platform release.

August 2023

main22460

Stable

Updated and certified against the current SnapLogic Platform release.

Introduced the Google BigQuery Upsert (Streaming) Snap, which enables you to perform bulk update/insert operations into a BigQuery table from existing tables or any input data stream.

432patches20298

• Disabled retries on truncated tables for the the BigQuery Write and BigQuery Bulk Load (Streaming) Snaps.

• Added better error messages for error conditions in the BigQuery Table Data List Snap.

• Fixed an issue with the BigQuery Table Create Snap that caused an error to display when fields were separated with a comma and a space.

• Fixed an issue with the sorting of the Partitioning time dropdown in the BigQuery Table Create Snap.

Fixed an issue with the Google BigQuery Bulk Load (Streaming) Snap that caused the Table not found message to display even when Create table if not present was selected.

• Introduced the following new Snaps:

  • Google BigQuery Table Create Snap—enables you to create tables that support clustering and partitioning.

  • Google BigQuery Table Data List Snap—enables you to read table data from a BigQuery dataset and lists the contents of the table in rows in the output.

• The Location field in the BigQuery Execute Snap lists all the locations in the suggestions.

• Introduced the following Snaps and accounts to manage datasets in Google BigQuery (GBQ):

  • BigQuery Dataset Create: Creates datasets in the GBQ.

  • BigQuery Dataset Delete: Removes datasets from the GBQ.

  • BigQuery Dataset List: Lists datasets from the GBQ.

  • BigQuery Table Delete: Removes tables from the GBQ.

  • BigQuery Table List: Lists tables that are read from the GBQ dataset.

  • Google Service Account JSON: Accesses the resources on GBQ using a JSON Key.

Upgraded with the latest SnapLogic Platform release.

• Fixed an issue with Google BigQuery Execute Snap, where the Snap displayed 404 Job not found error when calling a procedure.

• Fixed an issue with the Google BigQuery Bulk Load (Cloud Storage) Snap where the Snap failed, because the access token expired when it had to wait longer to execute. With this fix, the Snap is reloaded to get refreshed access token.

• Fixed an issue with the Google BigQuery Execute Snap, where the Snap displayed an error when the input data contained a table having the record type column and its value was null.

• Fixed an issue with the Google BigQuery Write Snap, when the input data contained complex data type columns (such as nested fields) and Create table if not present checkbox was selected.

Upgraded Google BigQuery driver to 1.119.0 version to support time partition intervals by MONTH and YEAR.

Fixed an issue with the Google BigQuery Bulk Load (Cloud Storage) Snap, where the Snap failed with an exception for big query tables. The CreateDisposition is now set conditionally on the basis of the setting in the Create table if not present checkbox.

4.27

main12833

13 Nov 2021

Stable

Enhanced the Google BigQuery Bulk Load (Cloud Storage) Snap with the following batching and retry properties to process input records:

• Batching: Processes the input records in batches.

• Batch Size: The number of records batched per request.

• Batch Timeout (milliseconds): Time in milliseconds to elapse following which the batch, if not empty, will be processed even though it might be lesser than the given batch size.

• Enhanced the Google BigQuery Execute Snap and Google BigQuery accounts to enable you to choose the SQL dialect to use in the Query field. See Interpreting the SQL Query Dialect for more information.
• Fixed the precision loss in Google BigQuery Execute Snap output that strips millisecond values while retrieving TIMESTAMP values from Google BigQuery tables.

Fixed the NPE issue with stored procedures and DROP TABLE queries in the Google BigQuery Execute Snap.

Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers.

Added the Snap Execution field to all Standard-mode Snaps. In some Snaps, this field replaces the existing Execute during preview check box.

snapsmrc486

snapsmrc480

Added a new property Schema auto detect in the Google BigQuery Bulk Load (Cloud Storage) Snap to support CSV and JSON files where one or more columns in the source file may not contain any values. 

• Added new Snap: Google BigQuery Bulk Load (Cloud Storage)
• Added new Snap: Google BigQuery Bulk Load (Streaming).
• Updated Google Big Query Write Snap with a new Create table if not present property.

Addressed an issue when authenticating with Dynamic OAuth accounts.

snapsmrc414

Supports refreshing OAuth access tokens during long-running pipeline executions. Fixed an issue with writing small batch sizes and when querying empty dataset tables.

Reload OAuth account from Platform when the access token expires during pipeline execution.

snapsmrc398

snapsmrc382

snapsmrc344

• Resolved the following issues with the Google BigQuery Execute Snap:
  • throwing binary data in stacktrace and two error messages.
  • improve error handling for suggestions
  • improve error handling on bad queries
  • suggestion bubble missing for Destination table ID
• Resolved an issue with Auto refresh token not working in Google BigQuery account.