In this article
Table of Contents | ||||||
---|---|---|---|---|---|---|
|
Overview
You can use this Snap to create BigQuery Datasets. Datasets are top-level containers that you can use to organize and control access to your tables and views.
...
Snap Type
BigQuery Dataset Create Snap is a WRITE-type Snap that creates BigQuery Dataset.
Prerequisites
A valid Google BigQuery Account with the required permissions.
Access/Valid permissions to the project to create a dataset
Support for Ultra Pipelines
Works in Ultra Pipelines.
Limitations
<<Notes to the reviewer - Including the generic limitations of BigQuery datasets. Kindly confirm which one to keep.>>
You can set the geographic location only at the time of creating a dataset. After you create the dataset, the location becomes immutable.
Dataset names must be unique for each project.
A dataset is contained within a specific project. A table or view must belong to a dataset, so you need to create at least one dataset before loading data into BigQuery.
All tables that are referenced in a query must be stored in datasets in the same location. (This can be used for Dataset List - your view)
When you copy a table, the datasets that contain the source table and destination table must reside in the same location.
Known Issues
None.
Snap Views
Type | Format | Number of Views | Examples of Upstream and Downstream Snaps | Description |
---|---|---|---|---|
Input | Document |
|
| The Snap can have at the most one input document that provides the value for the project ID or Dataset ID |
Output | Document |
|
| The Snap has a minimum of one output that provides information on whether the dataset is created or not. You can add an input and error views to view the result. |
Error | Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:
Learn more about Error handling in Pipelines. |
Snap Settings
Info |
---|
|
Field Name | Field Type | Field Dependency | Description | ||
---|---|---|---|---|---|
Label* Default Value: BigQuery Dataset Create | String | None | Specify a unique name for the Dataset that is to be created.
| ||
Project ID* Default Value: None | String/Expression | None | Specify the Project ID in which the dataset is to be created. | ||
Dataset ID* Default Value: None | String/Expression | None | Specify the Project ID in which the dataset is to be created. | ||
Detailed Information Default Value: Deselected | Checkbox | None | Select this checkbox to allow the Snap to display additional fields which can be included while creating the dataset. | ||
Location Default Value: None | String/Expression | Displays when Detailed Information is selected. | Specify the geographic location where you want to create the dataset.
|
Default Table Expiration (in Days) Default Value: None | String/Expression | Displays when Detailed Information is selected. | Specify the number of days after which any new table created in this dataset should be automatically deleted. You can use this field only if you want to create a dataset to store temporary data that is not to be preserved for longer. |
Snap Execution Default Value: | Dropdown list | N/A | Select one of the following three modes in which the Snap executes:
|
Examples
Creating a Dataset and then Deleting the Same
This example Pipeline demonstrates how to create a Dataset by using the Dataset Create Snap at a the specified location and then deleting it by using the Dataset Delete Snap.
...
First, configure the Dataset Create Snap to create the dataset as specified below.
...
On validation, the Dataset shows the following output displays that the Dataset has been created with the timestamp.
...
The Union Snap is included to view the output in a proper format.
...
The BigQuery Dataset Delete Snap deletes the dataset that you just created.
...
The output of the BigQuery Dataset Delete Snap displays that the delete is successful.
...
Downloads
Info |
---|
|
Attachments | ||||
---|---|---|---|---|
|
Multiexcerpt include macro | ||||||||
---|---|---|---|---|---|---|---|---|
|
...