On this Page
| Table of Contents | ||||
|---|---|---|---|---|
|
Overview
The Catalog Delete Snap enables you to delete the partitions within a specific Table Asset in the SnapLogic Data Catalog.
Input and Output
- Expected input: A document containing the metadata to be deleted from the Data Catalog.
- Expected output: A document stream containing status messages on the result of the delete operation.
- Expected upstream Snaps: Required. A Snap that offers document streaming data. Examples: Parquet Writer and Mapper.
- Expected downstream Snaps: Any Snap that accepts binary streaming data document in its input view. Examples: Mapper and File Writer.
Prerequisites
- Write access to the SnapLogic Data Catalog.
Configuring Accounts
Accounts are not used with this Snap.
Configuring Views
Input | This Snap has exactly one document input view. |
|---|---|
| Output | This Snap has exactly one document output view. |
| Error | This Snap has one document error view. |
Troubleshooting
- None.
Known Issues
None.
Modes- Ultra pipelines: Does not work in Ultra pipelines.Spark mode: Supports Spark mode.
Snap Settings
| Label | Required. The name for the Snap. Modify this to be more specific, especially if there are more than one of the same Snap in the pipeline. | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Table Name | Enter the name of the target Table Asset in the Data Catalog. You can either enter this information manually or you can select the table from the suggestible drop-down. Example: /<Org>/<Project_Space>/<Project>/<Table_Name> Default value: None | ||||||||||||
| Data Location | Enter the location of the source data from which the Table Asset in the Data Catalog originates. Example: parquetesting1.parquet Default value: None | ||||||||||||
| Partition Keys | Add the partition keys which you want to delete. You list the partition keys by specifying Key Column and Key Value combinations to identify the precise row and column from which to create a partition. | ||||||||||||
| Key Name | Specify the key you want to use to locate a specific partition in a metadata table. Example: Default value: None | ||||||||||||
| Operator | Specify the operator to use to locate the table partition. Example: != Default value: = | ||||||||||||
| Key Value | Specify the value to use to locate the table partition. Example: 10 Default value: None | ||||||||||||
|
|
Example
Pipeline Description
The following Pipeline demonstrates how to overwrite an existing partition by using the three Data Catalog Service Snaps: Catalog Query, Catalog Delete, and Catalog Insert. Because you cannot edit the Table Assets in Manager, you can use this Pipeline as a way for overwriting partitions in the Data Catalog.
The Catalog Query Snap queries for a partition, age, in the Asset named table1.
The Catalog Delete Snap deletes the partition, PK_1, indicated by the Key Name and Key Value.
To replace this metadata, the JSON Generator Snap is used to generate a document with the data for name, age, and height.
The JSON Snap passes the document data to the Parquet Writer, which writes the JSON document to a S3 bucket.
The Catalog Insert Snap populates table1 with the new data.
You can navigate to Manager > Projects > Assets to view the metadata inserted in table1.
In Manager, click table1, and click Show Table Schema.
Download
| Multiexcerpt include macro | ||||
|---|---|---|---|---|
|
| Attachments | ||||
|---|---|---|---|---|
|
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|