Use this SnapLogic Account type to connect Snaps to Google BigQuery using a Google service account.
Prerequisites
Existing Google Service Account with P12 file containing the private key. Learn how to import P12 Fileinto the Google Service Account.
JDK with Keytool utility to set up the Keystore.
Limitations and Known Issues
None.
Account Settings
Asterisk ( * ): Indicates a mandatory field.
Suggestion icon ( ): Indicates a list that is dynamically populated based on the configuration.
Expression icon ( ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.
Add icon ( ): Indicates that you can add fields in the field set.
Remove icon ( ): Indicates that you can remove fields from the field set.
Field Name
Field Type
Description
Label*
Default Value: None Example: Service Account
String/Expression
Specify a unique label for the account.
Properties for Google BigQuery
Account ID*
Default Value: None Example: 108275755
Integer/Expression
Specify the ID of the service account available in the Google web console.
Private Key ID*
Default Value: N/A Example: 4fb718fd54de73f0
String/Expression
Specify the ID of the private key available in the Google web console.
To perform the above steps, open a Command Prompt window. The steps along with the requisite syntax are described in detail below:
Step 1 - Create New Keystore
Execute the following command at the Command Prompt. keytool -genkey -alias <some alias> -keyalg RSA -keysize 2048 -keystore <keystore name>
Example: keytool -genkey -alias demoalias1 -keyalg RSA -keysize 2048 -keystore demokeystore1
Specify the follow-up questions for First and Last name, Organizational unit name, Organization name, City/Locality, State/Province, and Two-letter country code.
Click Yes in the Details confirmation dialog box.
Enter the key password for the provided alias.
Press Enter.
The same key password will be used in the subsequent steps.
Step 2 - Empty the Keystore
Execute the following command at the Command Prompt: keytool -delete -alias <alias name> -keystore <keystore name>
Example:
keytool -delete -alias demoalias1 -keystore demokeystore1
Enter the password as given in Step 1.
Step 3 - Verify if Keystore is Empty
Execute the following command at the Command Prompt: keytool -v -list -keystore <keystre name>
Example:
keytool -v -list -keystore demokeystore1
Enter password as given in Step 1. The response should be "Your keystore contains 0 entries".
Step 4 - Import the P12 file
Execute the following command at the Command Prompt: keytool -v -importkeystore -srckeystore <p12file.p12> -srcstoretype PKCS12 -destkeystore <keystore name> -deststoretype JKS -destkeypass <password for private key (should be same as keystore passphrase)>
Example:
keytool -v -importkeystore -srckeystore sample_p12_file.p12 -srcstoretype PKCS12 -destkeystore demokeystore1 -deststoretype JKS -destkeypass demoPassword1
Enter the destination Keystore password given in Step 1.
Enter the source Keystore password that was obtained while generating P12 file.
The response should be "Import command completed: 1 entries successfully imported, 0 entries failed or canceled".
Step 5 - Check Keystore to be Non-empty
Execute the following command at the Command Prompt:
The response will be blank if the change is successful.
Step 7 - Verify New Alias
Repeat Step 5 and ensure the alias name is what is given in Step 6. The Keystore "demokeystore1" is ready to be used in the snaps with the passphrase given in Step 1.
Snap Pack History
Click to view/expand
Release
Snap Pack Version
Date
Type
Updates
August 2024
438patches28058
Latest
Fixed an issue with the BigQuery Table Data List Snap that displayed a null pointer exception when the table source schema contained a nested Array list schema.
August 2024
main27765
Stable
Updated and certified against the current Snaplogic Platform release.
May 2024
main26341
Stable
Updated and certified against the current SnapLogic Platform release.
February 2024
main25112
Stable
Updated and certified against the current SnapLogic Platform release.
November 2023
main23721
Stable
Updated and certified against the current SnapLogic Platform release.
August 2023
main22460
Stable
Updated and certified against the current SnapLogic Platform release.
May 2023
433patches22057
Latest
Introduced the Google BigQuery Upsert (Streaming) Snap, which enables you to perform bulk update/insert operations into a BigQuery table from existing tables or any input data stream.
Fixed an issue with the BigQuery Table Create Snap that caused an error to display when fields were separated with a comma and a space.
Fixed an issue with the sorting of the Partitioning time dropdown in the BigQuery Table Create Snap.
February 2023
432patches19840
Latest
Fixed an issue with the Google BigQuery Bulk Load (Streaming) Snap that caused the Table not found message to display even when Create table if not present was selected.
Upgraded with the latest SnapLogic Platform release.
4.28 Patch
428patches15459
Latest
Fixed an issue with Google BigQuery Execute Snap, where the Snap displayed 404 Job not found error when calling a procedure.
Fixed an issue with the Google BigQuery Bulk Load (Cloud Storage) Snap where the Snap failed, because the access token expired when it had to wait longer to execute. With this fix, the Snap is reloaded to get refreshed access token.
4.28 Patch
428patches14743
Latest
Fixed an issue with the Google BigQuery Execute Snap, where the Snap displayed an error when the input data contained a table having the record type column and its value was null.
Fixed an issue with the Google BigQuery Write Snap, when the input data contained complex data type columns (such as nested fields) and Create table if not present checkbox was selected.
4.28
main14627
Stable
Upgraded with the latest SnapLogic Platform release.
4.27 Patch
427patches13752
Latest
Upgraded Google BigQuery driver to 1.119.0 version to support time partition intervals by MONTH and YEAR.
4.27 Patch
427patches13615
Latest
Fixed the table truncate 404 error with the Google BigQuery Load (Streaming) Snap by supporting the retry functionality. The Snap now waits in case of an error and retries before loading the data.
4.27 Patch
427patches12691
Latest
Fixed an issue with the Google BigQuery Bulk Load (Cloud Storage) Snap, where the Snap failed with an exception for big query tables. The CreateDisposition is now set conditionally on the basis of the setting in the Create table if not present checkbox.
Batch Size: The number of records batched per request.
Batch Timeout (milliseconds): Time in milliseconds to elapse following which the batch, if not empty, will be processed even though it might be lesser than the given batch size.
4.26
main11181
Stable
Upgraded with the latest SnapLogic Platform release.
4.25
main9554
Stable
Upgraded with the latest SnapLogic Platform release.
4.24
main8556
Stable
Upgraded with the latest SnapLogic Platform release.
Fixed the precision loss inGoogle BigQuery ExecuteSnap outputthat strips millisecond valueswhile retrieving TIMESTAMP values from Google BigQuery tables.
4.22
main6403
Stable
Upgraded with the latest SnapLogic Platform release.
4.21
snapsmrc542
Stable
Upgraded with the latest SnapLogic Platform release.
4.20 Patch
google/bigquery8773
Latest
Fixed the NPE issue with stored procedures and DROP TABLE queries in the Google BigQuery Execute Snap.
4.20
snapsmrc535
Stable
Upgraded with the latest SnapLogic Platform release.
4.19
snaprsmrc528
Stable
Upgraded with the latest SnapLogic Platform release.
4.18
snapsmrc523
Stable
Upgraded with the latest SnapLogic Platform release.
4.17
ALL7402
Latest
Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers.
4.17
snapsmrc515
Stable
Added the Snap Execution field to all Standard-mode Snaps. In some Snaps, this field replaces the existing Execute during preview check box.
4.16
snapsmrc508
Stable
Upgraded with the latest SnapLogic Platform release.
4.15
snapsmrc500
Stable
Upgraded with the latest SnapLogic Platform release.
4.14
snapsmrc490
Stable
Upgraded with the latest SnapLogic Platform release.
4.13
snapsmrc486
Stable
Upgraded with the latest SnapLogic Platform release.
4.12
snapsmrc480
Stable
Added a new propertySchema auto detectin theGoogle BigQuery Bulk Load (Cloud Storage) Snap to support CSV and JSON files where one or more columns in the source file may not contain any values.
4.11
snapsmrc465
Stable
Added new Snap: Google BigQuery Bulk Load (Cloud Storage)
Added new Snap: Google BigQuery Bulk Load (Streaming).
Updated Google Big Query Write Snap with a new Create table if not presentproperty.
4.10 Patch
google/bigquery4046
Latest
Addressed an issue when authenticating with Dynamic OAuth accounts.
4.10
snapsmrc414
Stable
Upgraded with the latest SnapLogic Platform release.
4.9
snapsmrc405
Stable
Upgraded with the latest SnapLogic Platform release.
4.8 Patch
bigquery2952
Latest
Supports refreshing OAuth access tokens during long-running pipeline executions. Fixed an issue with writing small batch sizes and when querying empty dataset tables.
4.8.0 Patch
bigquery2813
Latest
Reload OAuth account from Platform when the access token expires during pipeline execution.
4.8
snapsmrc398
Stable
Upgraded with the latest SnapLogic Platform release.
4.7
snapsmrc382
Stable
Upgraded with the latest SnapLogic Platform release.
4.6
snapsmrc362
Stable
Upgraded with the latest SnapLogic Platform release.
4.5.1
snapsmrc344
Stable
Upgraded with the latest SnapLogic Platform release.
4.4.1
NA
Stable
Upgraded with the latest SnapLogic Platform release.
4.4
NA
Stable
Upgraded with the latest SnapLogic Platform release.
4.3.2
NA
Stable
Resolved the following issues with the Google BigQuery Execute Snap:
throwing binary data in stacktrace and two error messages.
improve error handling for suggestions
improve error handling on bad queries
suggestion bubble missing for Destination table ID
Resolved an issue with Auto refresh token not working in Google BigQuery account.