Catalog Delete
On this Page
Overview
The Catalog Delete Snap enables you to delete the partitions within a specific Table Asset in the SnapLogic Data Catalog.
Input and Output
- Expected input: A document containing metadata to be deleted from the Data Catalog.
- Expected output: A document containing status messages on the result of the delete operation.
- Expected upstream Snaps: Required. A Snap that offers document. Examples: Parquet Writer and Mapper.
- Expected downstream Snaps: Any Snap that accepts document in its input view. Examples: Mapper and File Writer.
Prerequisites
- Write access to the SnapLogic Data Catalog.
Configuring Accounts
Accounts are not used with this Snap.
Configuring Views
Input | This Snap has exactly one document input view. |
|---|---|
| Output | This Snap has exactly one document output view. |
| Error | This Snap has one document error view. |
Troubleshooting
- None.
Known Issues
- Does not work in Ultra pipelines.
Snap Settings
| Label | Required. The name for the Snap. Modify this to be more specific, especially if there are more than one of the same Snap in the pipeline. |
|---|---|
| Table Name | Enter the name of the target Table Asset in the Data Catalog. You can either enter this information manually or you can select the table from the suggestible drop-down. Example: /<Org>/<Project_Space>/<Project>/<Table_Name> Default value: None |
| Data Location | Enter the location of the source data from which the Table Asset in the Data Catalog originates. Example: parquetesting1.parquet Default value: None |
| Partition Keys | Add the partition keys which you want to delete. You list the partition keys by specifying Key Column and Key Value combinations to identify the precise row and column from which to create a partition. |
| Key Name | Specify the key you want to use to locate a specific partition in a metadata table. Example: Default value: None |
| Operator | Specify the operator to use to locate the table partition. Example: != Default value: = |
| Key Value | Specify the value to use to locate the table partition. Example: 10 Default value: None |
Snap execution | Select one of the three modes in which the Snap executes. Available options are:
|
Example
Pipeline Description
The following Pipeline demonstrates how to overwrite an existing partition by using the three Data Catalog Service Snaps: Catalog Query, Catalog Delete, and Catalog Insert. Because you cannot edit the Table Assets in Manager, you can use this Pipeline as a way for overwriting partitions in the Data Catalog.
The Catalog Query Snap queries for a partition, age, in the Asset named table1.
The Catalog Delete Snap deletes the partition, PK_1, indicated by the Key Name and Key Value.
To replace this metadata, the JSON Generator Snap is used to generate a document with the data for name, age, and height.
The JSON Snap passes the document data to the Parquet Writer, which writes the JSON document to a S3 bucket.
The Catalog Insert Snap populates table1 with the new data.
You can navigate to Manager > Projects > Assets to view the metadata inserted in table1.
In Manager, click table1, and click Show Table Schema.
Download
Important steps to successfully reuse Pipelines
- Download and import the Pipeline into SnapLogic.
- Configure Snap accounts as applicable.
- Provide Pipeline parameters as applicable.
Snap Pack History
Have feedback? Email documentation@snaplogic.com | Ask a question in the SnapLogic Community
© 2017-2024 SnapLogic, Inc.