Overview
The Catalog Delete Snap enables you to delete the partitions within a specific Table Asset in the SnapLogic Data Catalog.
Input and Output
- Expected input: A document containing the metadata to be deleted from the Data Catalog.
- Expected output: A document stream containing status messages on the result of the delete operation.
- Expected upstream Snaps: Required. A Snap that offers document streaming data. Examples: Parquet Writer and Mapper.
- Expected downstream Snaps: Any Snap that accepts binary streaming data in its input view. Examples: Mapper and File Writer.
Prerequisites
Write access to the SnapLogic Data Catalog.
Configuring Accounts
Accounts are not used with this Snap.
Configuring Views
Input | This Snap has exactly one document input view. |
|---|---|
| Output | This Snap has exactly one document output view. |
| Error | This Snap has one document error view. |
Troubleshooting
None.
Limitations and Known Issues
None.
Modes
- Ultra pipelines: Does not work in Ultra pipelines.
- Spark mode: Supports Spark mode.
Snap Settings
| Label | Required. The name for the Snap. Modify this to be more specific, especially if there are more than one of the same Snap in the pipeline. | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Table Name | Enter the name of the target Table Asset in the Data Catalog. You can either enter this information manually or you can select the table from the suggestible drop-down. | ||||||||||||
| Data Location | Enter the location of the source data from which the Table Asset in the Data Catalog originates. | ||||||||||||
| Partition Keys | Add the partition keys which you want to delete. You list the partition keys by specifying Key Column and Key Value combinations to identify the precise row and column from which to create a partition. Default value: None | ||||||||||||
| Key Name | Specify the key you want to use to locate a specific partition in a metadata table. Example: Default value: None | ||||||||||||
| Operator | Specify the operator to use to locate the table partition. Example: != Default value: = | ||||||||||||
| Key Value | Specify the value to use to locate the table partition. Example: 10 Default value: None | ||||||||||||
|
|
Example
Pipeline
The following Pipeline demonstrates how to overwrite an existing partition by the functions of using the three Data Catalog Service Snaps: Catalog Query, Catalog Delete, and Catalog Insert. Because you cannot edit the Data Catalog Table Assets in Manager, you can use this Pipeline as a pattern way for overwriting partitions in the Data Catalog.
The Catalog Query Snap queries for a partition, age, in the Asset named table1.
The Catalog Delete Snap deletes the partition, PK_1, indicated by the Key Name and Key Value.
To replace this metadata, a the JSON Generator Snap is used to generate a document with the metadata data for name, age, and height.
The data is passed through JSON Snap passes the document data to the Parquet Writer, which writes the JSON document to an a S3 bucket.
The Catalog Insert Snap populates the table table1 with the new metadatadata.
You can navigate to Manager > Projects > Assets to view the metadata inserted in the Data Catalog from the resulting operation when you open table1 in Manager > Projects > Assets: table1.
In Manager, click table1, and and click Show Table Schema.
Download
| Attachments | ||||
|---|---|---|---|---|
|
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|