BigQuery Dataset Create

In this article

Overview

You can use this Snap to create BigQuery Datasets. Datasets are top-level containers that you can use to organize and control access to your tables and views.

Snap Type

BigQuery Dataset Create Snap is a WRITE-type Snap that creates BigQuery Dataset.

Prerequisites

  • A valid Google BigQuery Account with the required permissions.

  • Access/Valid permissions to the project to create a dataset

Support for Ultra Pipelines

Limitations

  • You can set the geographic location only at the time of creating a dataset. After you create the dataset, the location becomes immutable.

  • Dataset names must be unique for each project.

Known Issues

None.

Snap Views

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input 

Document

  • Min: 0

  • Max: 1

  • Mapper

  • Any BigQuery Snap

The Snap can have at the most one input document that provides the value for the project ID or Dataset ID

Output

Document

  • Min: 1

  • Max: 1

  • File Writer

  • Mapper

The Snap has a minimum of one output that provides information on whether the dataset is created or not. You can add an input and error views to view the result.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

  • Stop Pipeline Execution: Stops the current pipeline execution if the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

  • Asterisk (*): Indicates a mandatory field.

  • Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon (): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon (): Indicates that you can add fields in the fieldset.

  • Remove icon (): Indicates that you can remove fields from the fieldset.

Field Name

Field Type

Field Dependency

Description

Field Name

Field Type

Field Dependency

Description

Label*

Default ValueBigQuery Dataset Create
ExampleBigQuery_Dataset_Create

String

None

Specify a unique name for the Dataset that is to be created.

Dataset names must be unique for each project.

Project ID*

Default Value: None
Example: case16370

String/Expression

None

Specify the Project ID in which the dataset is to be created.

Dataset ID*

Default Value: None
Example: dscreate5

String/Expression

None

Specify the Project ID in which the dataset is to be created.

Detailed Information

Default ValueDeselected
Example: Selected

Checkbox

None

Select this checkbox to allow the Snap to display additional fields which can be included while creating the dataset.

Location

Default ValueNone
Example: Europe - Finland (europe-north1)

String/Expression

Displays when Detailed Information is selected.

Specify the geographic location where you want to create the dataset.

You can set the geographic location only at the time of creating a dataset. After you create the dataset, the location becomes immutable.

Default Table Expiration (in Days)

Default ValueNone
Example: 10

String/Expression

Displays when Detailed Information is selected.

Specify the number of days after which any new table created in this dataset should be automatically deleted. You can use this field only if you want to create a dataset to store temporary data that is not to be preserved for longer.
If you want to preserve the tables for longer, then leave this field blank.

Snap Execution

Default Value
Example: Validate & Execute

Dropdown list

N/A

Select one of the following three modes in which the Snap executes:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Examples

Creating a Dataset and then Deleting the Same

This example Pipeline demonstrates how to create a Dataset by using the Dataset Create Snap at the specified location.

First, configure the Dataset Create Snap to create the dataset as specified below.

On validation, the output displays that the Dataset has been created with the timestamp.

Download this Pipeline

Downloads


Related Links