Required. Enter the name of the database schema. In case the schema name is not defined, then the suggestion for the schema name retrieves all schema names in the specified database when you click the suggest button.

Specify the hash distribution column name for the target table in Azure Synapse, if you choose the Load Action as Drop and Create table.

This field set enables you to define the table options for creating the target table before performing the load operation. You must specify each table option in a separate row. Click to add a new row.

• Table Option

Click the Suggest icon (Image Modified) and choose a table option that you want to use, from the suggested values. Specify the required values for the selected option.

The table options you can use are:

• PARTITION BY
• CLUSTER BY
• OPTIONS
• COMMENT

This field set enables you to configure the schema of the target table. You can use this field set to add/drop columns in the target table if you select the Drop and Create table or Alter table options in the Load Action field. 

This field set consists of the following fields:

• Column
• Data Type
• Modifier

ID

FNAME

Activates when you select the Drop and Create table or Alter table options in the Load Action field.

Enter the data type of the values in the specified column. 

INT

VARCHAR

Activates when you select the Alter table option in the Load Action field.

Select whether you want to add/drop the specified column.

Available options are:

• Add
• Drop

Applicable only when you are loading data from a Redshift source to a BigQuery target.

Specify the schema name of the source table in Redshift.

Applicable only when you are loading data from a Redshift source to a BigQuery target.

Specify the name of the Redshift source table (from the schema selected above).

Specify the fully-qualified path to the folder in the Cloud storage in this field, if you want to override the default path defined in the Snap's Account settings. For example:

• AWS S3 path: s3://<bucket name>/<folder>
• DBFS folder: /<dbfs folder name>
• Azure: azure://<accountname>.blob.core.windows.net/<containername>/<azurefolder>
• Google Cloud Storage: gcs://<bucketname>/<folder>/

This field set enables you to specify the staged files to load. You must specify each file in a separate row. Click + to add a new row.

This field set consists of the following field:

• File 

Snowflake: '.*key.*[.]csv'

Redshift: './key.*csv'

This field set enables you to define the file format options for the load operation. You must specify each file format option in a separate row. Click + to add a new row.

• Format Option

See Snowflake Data Loading Commands, Redshift Data Format Parameters, or COPY INTO (Transact-SQL) for Azure Synapse Analytics for the list of of options that you can use based on the target database.

Click the Suggest icon (Image Modified) and choose a file format option that you want to use during the load operation, from the suggested values. Specify the required value for the selected option.

This field set enables you to define the COPY options for the load operation. You must specify each COPY option in a separate row. Click + to add a new row.

This field set comprises the following field:

• Copy Option

See COPY Options for Snowflake, COPY Parameter Reference for Redshift, or COPY INTO (Transact-SQL) for Azure Synapse Analytics.

Activates when you select the RETURN_n_ROWS option in the Validation Mode field.

Enter the number of records to retrieve from the target table to validate whether the load operation was successful.

Select this checkbox to include the column position from the source table in case of Snowflake or Azure Snapse.

In case of other target CDWs (Redshift, DLP and BigQuery), you can leave this checkbox unchecked.

This field set enables you to define the mapping between the columns in the target table and the corresponding columns in the source file. You must define each mapping in a separate row. Click to add a new row.

The available options are:

• Target Table Column Name
• Source File Column Name

Depending on your target database, use the respective File Format Option in the ELT Load Snap.

• For Snowflake: SKIP_HEADER = 1
• For Azure Synapse: FIRSTROW = 2
• For Databricks Lakehouse Platform: 'header' = 'true'

Invalid DBFS folder name

(Target CDW: Databricks Lakehouse Platform)

Database cannot be blank.

(when seeking the suggested list for Schema Name field)

Suggestions in the Schema Name and Target Table Name fields do not work when you have not specified a valid value for the Database Name field in this Snap.

Specify the target Database Name in this Snap to view and choose from a suggested list in the Schema Name and Target Table Name fields respectively.

Unknown Source / Not found: <path>

(Target CDW: BigQuery)

BigQuery has a unique way of resolving wildcard patterns like *.* (that you provide in the File or File Name Pattern fields of this Snap).

Make sure that the file name pattern you provide conforms to the rules defined in Batch loading data > Load Wildcards section of BigQuery documentation.

Since the Redshift Copy Into SQL does not natively support File Pattern option unlike the other CDWs, SnapLogic supports this externally through Java Regular Expressions (RegEx).

The RegEx *.* returns matching results from the root location instead of the folder location specified before *.* in the File Name Pattern field.

Keyword RANGE is not acceptable as a column name.

(CDW: Databricks Lakehouse Platform)

Keyword ORDER is not acceptable as a column name in the JSON file.

(Target CDW: BigQuery)

[Simba][SparkJDBCDriver](500051) ERROR processing query/statement. Error Code: 0

Cannot create table ('<schema name>`.`<table name>`'). The associated location (`…<table name>`) is not empty but it's not a Delta table

(Target CDW: Databricks Lakehouse Platform)

The specified location contains one of the following:

• A non-Delta table (such as CSV, ORC, JSON, PARQUET)

• A corrupted table

• A Delta table with a different table schema

So, the Snap/Pipeline cannot overwrite this table with the target table as needed.

Ensure that you take appropriate action (mentioned below) on the existing table before running your Pipeline again (to create another Delta table at this location).

Move or drop the existing table from the schema manually using one of the following commands:

Access the DBFS through a terminal and run:

• dbfs mv dbfs:/<current_table_path> dbfs:/<new_table_path> to move the table or

• dbfs rm -r dbfs:/<table_path> to drop the table.

OR

Use a Python notebook and run:

• dbutils.fs.mv(from: String, to: String, recurse: boolean = false): boolean to move the table/file/directory or

• dbutils.fs.rm(dir: String, recurse: boolean = false): boolean to drop the table/file/directory.