Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In this article

...

You can use this Snap to insert data into Snowflake using the Snowpipe Streaming , a feature that allows for API, which enables the continuous ingestion of data into Snowflake tables as and when it is available.

...

Field Name

Field Type

Description

Label*

Default ValueSnowflake – Snowpipe Streaming
ExampleStream Data

String

The Specify the name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

Schema name

Default Value: N/A
Example: “PUBLIC”

String/Expression/Suggestion

Specify the database schema name.

Table Name*

Default Value: N/A
Example: “PUBLIC”.”SNOWPIPESTREAMING”

String/Expression/Suggestion

Specify the table name on which the insert operation has to be executed.

Create table if not present

Default ValueDeselected

Checkbox

Select this checkbox if you want the table to be automatically created if it does not already exist.

Max client lag*

Default Value1000
Example: 1500

Integer/Expression/Suggestion

Specify the client data flush interval in milliseconds. You can set Adjust this value as high as based on the maximum latency your target latency allows system can handle (60,000 ms).

Snap execution

Default Value
Example: Validate & Execute

Dropdown list

Select one of the following three modes in which the Snap executes:

  • Validate & Execute: Performs limited execution of the Snap and generates a data preview during pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during pipeline runtime.

  • Execute only: Performs full execution of the Snap during pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Troubleshooting

Provide a private key in the account configuration.

Error

Reason

Resolution

Private key not found

The private key is required for Snowflake Database accounts configuration.

Schema name not found

The schema name is required for Snowpipe Streaming.

Provide a schema name in the Snap configuration.

...