In this article

1 Overview
- 1.1 Snap Type
- 1.2 Prerequisites
- 1.3 Support for Ultra Pipelines
- 1.4 Limitations and Known Issues
2 Snap Views
3 Snap Settings
4 Examples

Overview

You can use this Snap to create Google BigQuery tables that support clustering and partitioning.

Snap Type

The Google BigQuery Table Create Snap is a Write-type Snap that creates BigQuery tables.

Prerequisites

A valid Google BigQuery account with the required permissions.

Support for Ultra Pipelines

Works in Ultra Pipelines.

Limitations and Known Issues

The rounding_mode option for BIGNUMERIC and NUMERIC data type fields is not supported. These data type fields fields can be configured on the Google Cloud console.
This Snap considers only the following JSON keys in the schema during table creation (any other keys given are ignored):
- name
- type
- mode
- description
- fields

In the following example, the Snap creates the table with the with name, type, and mode keys but ignores defaultValueExpression. There is no error displayed for unsupported keys.

[{
"name": "sales",
"type": "FLOAT",
"mode": "NULLABLE",
"defaultValueExpression": "2.55"
}]

Snap Views

Type	Format	Number of Views	Examples of Upstream and Downstream Snaps	Description

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input

Document

Min: 0
Max: 1

Mapper
JSON Generator

The Project ID and the Document ID.

Output

Document

Min: 1
Max: 1

JSON Parser
File Writer

The list of Table IDs along with their Project IDs, Dataset IDs, and Table type.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

Stop Pipeline Execution: Stops the current pipeline execution if the Snap encounters an error.
Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.
Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

Asterisk ( * ): Indicates a mandatory field.
Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.
Expression icon ( ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.
Add icon ( ): Indicates that you can add fields in the field set.
Remove icon ( ): Indicates that you can remove fields from the field set.

Field Name		Field Type	Field Dependency	Description

Field Name		Field Type	Field Dependency	Description
Label* Default Value: BigQuery Table Create Example: Collegenames Table Create		String/Expression	None	Specify a unique name for the Snap. Modify this to be more specific, especially if you have more than one Snap of the same type in your pipeline.
Project ID Default Value: N/A Example: test-project-12345		String/Expression/Suggestion	None	Specify the project ID in which the dataset resides.
Dataset ID Default Value: N/A Example: dataset-12345		String/Expression/Suggestion	None	Specify the dataset ID of the destination.
Table ID Default Value: N/A Example: table-12345		String/Expression/Suggestion	None	Specify the table ID of the table you are creating. This is a unique ID that you must provide (it is not automatically created or assigned). Learn more about creating valid BigQuery table names.
Table Schema (JSON)		String	None	Enter the JSON schema for the table.
Detailed Information Default value: Deselected		Checkbox	None	Select this checkbox to enable the Snap to access additional fields for displaying them in the output.
Default table expiration (in days)		String/Expression	Appears when you select the Detailed Information* checkbox.*	New tables created in this dataset will be automatically deleted in the number of days specified.
Partitioning	Use this field set to define partitioning requirements.
	Enable partitioning Default Value: Deselected	Checkbox	Appears when you select the Partitioning* dropdown.*	Select to configure partitioning.
	Require partitioning filter Default Value: Deselected	Checkbox	Appears when you select the Enable Partitioning* checkbox.*	Select to require users to include a WHERE clause that specifies the partitions to query each time they query the table. Select the partitioning type (time or range) and the partitioning time (time interval after which a new partition is created).
Clustering	Use this field set to define clustering requirements.
Clustering	Enable clustering Default Value: Deselected	Checkbox	Appears when you select the Clustering* dropdown.*	Select to configure clustering.
Snap Execution Default Value: Validate & Execute Example: Disabled		Dropdown list	None	Select one of the following three modes in which the Snap executes: Validate & Execute: Performs limited execution of the Snap, and generates a data preview during pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during pipeline runtime. Execute only: Performs full execution of the Snap during pipeline execution without generating preview data. Disabled: Disables the Snap and all Snaps that are downstream from it.

Examples

Creating a BigQuery Table

This example demonstrates how to create a table from a BigQuery dataset.

Configure the Snap with the Project ID, Dataset ID, Table ID, and Table schema in JSON format.

On validation, the Snap lists information about the newly created table in the output.

Creating a BigQuery Table After Filtering the Dataset

This example pipeline demonstrates how to read tables from a BigQuery dataset and create a new table from the filtered the dataset.

First, configure the BigQuery Table List Snap with the Project ID and Dataset ID of the dataset from which the table data is to be listed.

On validation, the Snap lists the tables in the dataset.

Next, configure the Filter Snap with an expression that filters the data to limit it to just the Inventory table: $table_id=="Inventory"

On validation, the Snap lists information about the filtered data in the output.

Configure the Table Create Snap with the Project ID, Dataset ID, Table ID, and Table schema in JSON format.