...
This Snap enables you to perform bulk update or insert (upsert) operations into a BigQuery table from existing tables or any input data stream of data.
The upsert operation updates existing rows if the specified value exists in the target table , and inserts a new row if the specified value does not exist in the target table.
...
Field Name | Field Type | Description |
|---|---|---|
Label Default Value: BigQuery Bulk Upsert (Streaming) | String | Specify the name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline. |
Project ID Default Value: N/A | String/Expression/Suggestion | Specify the project ID in which the dataset resides. |
Dataset ID
Default Value: N/A Example: dataset-12345 | String/Expression/Suggestion | Specify the dataset ID of the destination. |
Table ID
Default Value: N/A | String/Expression/Suggestion | Specify the table ID of the table you are creating. |
Batch size Default value: 1000 | String | The number of records batched per request. If the input has 10,000 records and the batch size is set to 100, the total number of requests would be 100.Default value: 1000 |
Batch timeout (milliseconds) Default value: 2000 | String | Time in milliseconds after which the batch will be processed (even though it might be less than the specified batch size).Default value: 2000 Batch timeout value must be set with care. When this limit is reached, the batch will be flushed whether or not all the records in the batch were loaded. |
Batch retry count Default value: 0 | String | The number of times the server should try to load a failed batch.Default value: 0 |
Batch retry delay (milliseconds) Default value: 500 | String | The time delay between each retry.Default value: 500 |
Snap Execution
Default Value: Validate & Execute | Dropdown list | Select one of the three modes in which the Snap executes. Available options are:
|
...