$customHeader
Skip to end of banner
Go to start of banner

Google BigQuery OAuth2

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 17 Current »

In this article

Overview

Use this SnapLogic Account type to connect Snaps to Google BigQuery using OAuth2.

Prerequisites

Valid account setting values, such as hostname or server path, as applicable. These are received from the endpoint that you access using the account.

Limitations and Known Issues

None.

Account Settings

  • Asterisk ( * ): Indicates a mandatory field.

  • Suggestion icon ( (blue star) ): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon ( (blue star) ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon ( (blue star) ): Indicates that you can add fields in the fieldset.

  • Remove icon ( (blue star) ): Indicates that you can remove fields from the fieldset.

Field Name

Field Type

Description

Label*

Default Value:  None
ExampleBigQuery OAuth2 Account

String/Expression

Specify a unique label for the account instance.

Access token*

Default Value:  None

String

Specify the access token for the application that is auto-generated on authorization. You receive an access token when the account is setup for the endpoint. During setup, the OAuth2 flow is executed and the access token that gets generated is stored in this property. If the access token is refreshed, then the optional refresh token is stored. Get a new access token, if:

  • the access token has expired and no refresh token is defined, or

  • both the access token and the refresh token have expired.

Refresh token

Default Value:  None

String

The refresh token for the application that gets auto-generated on authorization. You receive a refresh token when the account is setup for the endpoint. During setup, the OAuth2 flow is executed and the refresh token that is generated is stored in this property. If the refresh token is already stored, then the access token can be refreshed automatically before it expires.

Access token expiration

Default Value:  None
Example: 16942198634

String

The access token expiration value that gets auto-generated on authorization. The retrieval of the access token expiration is done when setting up the account for the endpoint. There, the OAuth2 flow will be executed and the resulting access token expiration will be stored in this property.

Refers to the access token expiration value for the application. You get the access token expiration when the account is setup for the endpoint. During setup, the OAuth2 flow is executed and the access token expiration that is generated is stored in this property.

OAuth2 Endpoint*

Default valuehttps://accounts.google.com/o/oauth2/auth

String/Expression

Authorization endpoint to authorize the application.

OAuth2 Token*

Default Valuehttps://accounts.google.com/o/oauth2/token

String/Expression

Token endpoint to get the access token.

Auth endpoint config

Provides custom properties for the OAuth2 auth endpoint.

Access Type

Default value: offline

String/Expression

The access type for the token. If you are offline, then it will persist in the account.

Approval Prompt

Default value: force

String/Expression

The approval type for the token.

Default Standard SQL

Default value: Deselected

Checkbox

Select this checkbox to default the dialect in the Query field of the Snaps using this account to Standard SQL.

To override this default setting, use the #legacySQL tag within the Query field of the Snap to treat it as a Legacy SQL query.

Application scope

Default Valuehttps://www.googleapis.com/auth/dfareporting

Examplehttps://www.googleapis.com/auth/cloud-platform

String/Expression

The scope for the applications execution. As an example, the Snap could define that only a read of data by the application should be allowed.

Auto-refresh token

Default value: Deselected

Checkbox

If the auto-refresh token is enabled, the token refreshes automatically. Otherwise, the token might expire and will not be automatically refreshed.

Authorize

Button

When you click Authorize, the OAuth2 authorization flow begins; the account is saved before authorizing.  SnapLogic fetches the Access token, Refresh token, and Access token expiration details and populates these fields with the generated values.

Refresh

Button

Refresh access token. Available after the account has been saved.

Snap Pack History

 Click to view/expand
Release Snap Pack VersionDateType  Updates
May 2024main26341 StableUpdated and certified against the current SnapLogic Platform release.
February 2024main25112 StableUpdated and certified against the current SnapLogic Platform release.
November 2023main23721 StableUpdated and certified against the current SnapLogic Platform release.

August 2023

main22460

 


Stable

Updated and certified against the current SnapLogic Platform release.

May 2023433patches22057  Latest

Introduced the Google BigQuery Upsert (Streaming) Snap, which enables you to perform bulk update/insert operations into a BigQuery table from existing tables or any input data stream.

May 2023433patches21955 LatestFixed an issue with the GBQ-Google Service Account that caused an input stream to remain open.
May 2023main21015 StableUpdated and certified to be compatible with the August SnapLogic Platform release.
February 2023432patches20962 LatestFixed an intermittent null pointer exception that occurred in the BigQuery Write Snap.
February 2023

432patches20298

 Latest
February 2023432patches19840 Latest

Fixed an issue with the Google BigQuery Bulk Load (Streaming) Snap that caused the Table not found message to display even when Create table if not present was selected.

February 2023main19844 Stable
November 2022431patches19301 LatestThe Google BigQuery Bulk Load (Streaming) Snap works as expected, with no active Timer threads remaining when the Pipeline execution fails. 
November 2022main18944 StableUpgraded with the latest SnapLogic Platform release.
August 2022main17386 Stable
4.29main15993 Stable

Upgraded with the latest SnapLogic Platform release.

4.28 Patch428patches15459 Latest
  • Fixed an issue with Google BigQuery Execute Snap, where the Snap displayed 404 Job not found error when calling a procedure.
  • Fixed an issue with the Google BigQuery Bulk Load (Cloud Storage) Snap where the Snap failed, because the access token expired when it had to wait longer to execute. With this fix, the Snap is reloaded to get refreshed access token.

4.28 Patch428patches14743 Latest
  • Fixed an issue with the Google BigQuery Execute Snap, where the Snap displayed an error when the input data contained a table having the record type column and its value was null.
  • Fixed an issue with the Google BigQuery Write Snap, when the input data contained complex data type columns (such as nested fields) and Create table if not present checkbox was selected.

4.28main14627 StableUpgraded with the latest SnapLogic Platform release.
4.27 Patch427patches13752 Latest

Upgraded Google BigQuery driver to 1.119.0 version to support time partition intervals by MONTH and YEAR.

4.27 Patch427patches13615 LatestFixed the table truncate 404 error with the Google BigQuery Load (Streaming) Snap by supporting the retry functionality. The Snap now waits in case of an error and retries before loading the data.
4.27 Patch427patches12691 Latest

Fixed an issue with the Google BigQuery Bulk Load (Cloud Storage) Snap, where the Snap failed with an exception for big query tables. The CreateDisposition is now set conditionally on the basis of the setting in the Create table if not present checkbox.

4.27

main12833

Stable

Enhanced the Google BigQuery Bulk Load (Cloud Storage) Snap with the following batching and retry properties to process input records:

  • Batching: Processes the input records in batches.

  • Batch Size: The number of records batched per request.

  • Batch Timeout (milliseconds): Time in milliseconds to elapse following which the batch, if not empty, will be processed even though it might be lesser than the given batch size.

4.26main11181 StableUpgraded with the latest SnapLogic Platform release.
4.25main9554
 
StableUpgraded with the latest SnapLogic Platform release.
4.24main8556
StableUpgraded with the latest SnapLogic Platform release.
4.23main7430
 
Stable
4.22main6403
 
StableUpgraded with the latest SnapLogic Platform release.
4.21snapsmrc542

 

StableUpgraded with the latest SnapLogic Platform release.
4.20 Patch google/bigquery8773 Latest

Fixed the NPE issue with stored procedures and DROP TABLE queries in the Google BigQuery Execute Snap.

4.20snapsmrc535
 
StableUpgraded with the latest SnapLogic Platform release.
4.19snaprsmrc528
 
StableUpgraded with the latest SnapLogic Platform release.
4.18snapsmrc523
 
StableUpgraded with the latest SnapLogic Platform release.
4.17ALL7402
 
Latest

Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers.

4.17snapsmrc515
 
Stable

Added the Snap Execution field to all Standard-mode Snaps. In some Snaps, this field replaces the existing Execute during preview check box.

4.16snapsmrc508
 
StableUpgraded with the latest SnapLogic Platform release.
4.15snapsmrc500
 
StableUpgraded with the latest SnapLogic Platform release.
4.14snapsmrc490
 
StableUpgraded with the latest SnapLogic Platform release.
4.13

snapsmrc486

 
StableUpgraded with the latest SnapLogic Platform release.
4.12

snapsmrc480

 
Stable

Added a new property Schema auto detect in the Google BigQuery Bulk Load (Cloud Storage) Snap to support CSV and JSON files where one or more columns in the source file may not contain any values. 

4.11snapsmrc465
 
Stable
  • Added new Snap: Google BigQuery Bulk Load (Cloud Storage)
  • Added new Snap: Google BigQuery Bulk Load (Streaming).
  • Updated Google Big Query Write Snap with a new Create table if not present property.
4.10 Patch google/bigquery4046 Latest

Addressed an issue when authenticating with Dynamic OAuth accounts.

4.10

snapsmrc414

 
StableUpgraded with the latest SnapLogic Platform release.
4.9snapsmrc405
 
StableUpgraded with the latest SnapLogic Platform release.
4.8 Patchbigquery2952 Latest

Supports refreshing OAuth access tokens during long-running pipeline executions. Fixed an issue with writing small batch sizes and when querying empty dataset tables.

4.8.0 Patchbigquery2813 Latest

Reload OAuth account from Platform when the access token expires during pipeline execution.

4.8

snapsmrc398

 
StableUpgraded with the latest SnapLogic Platform release.
4.7

snapsmrc382

 
StableUpgraded with the latest SnapLogic Platform release.
4.6snapsmrc362
 
StableUpgraded with the latest SnapLogic Platform release.
4.5.1

snapsmrc344

 
StableUpgraded with the latest SnapLogic Platform release.
4.4.1NA StableUpgraded with the latest SnapLogic Platform release.
4.4NA StableUpgraded with the latest SnapLogic Platform release.
4.3.2NA Stable
  • Resolved the following issues with the Google BigQuery Execute Snap:
    • throwing binary data in stacktrace and two error messages.
    • improve error handling for suggestions
    • improve error handling on bad queries
    • suggestion bubble missing for Destination table ID
  • Resolved an issue with Auto refresh token not working in Google BigQuery account.


Related Content

  • No labels