Catalog Delete

On this Page

Overview

The Catalog Delete Snap enables you to delete the partitions within a specific Table Asset in the SnapLogic Data Catalog.

Input and Output

  • Expected input: A document containing metadata to be deleted from the Data Catalog.
  • Expected output: A document containing status messages on the result of the delete operation.
  • Expected upstream SnapsRequired. A Snap that offers document. Examples: Parquet Writer and Mapper.
  • Expected downstream Snaps: Any Snap that accepts document  in its input view. Examples: Mapper and File Writer.

Prerequisites

  • Write access to the SnapLogic Data Catalog.

Configuring Accounts

Accounts are not used with this Snap.

Configuring Views

Input

This Snap has exactly one document input view.
OutputThis Snap has exactly one document output view.
ErrorThis Snap has one document error view.

Troubleshooting

  • None.

Known Issues

  • Does not work in Ultra pipelines.

Snap Settings


LabelRequired. The name for the Snap. Modify this to be more specific, especially if there are more than one of the same Snap in the pipeline.
Table Name

Enter the name of the target Table Asset in the Data Catalog. You can either enter this information manually or you can select the table from the suggestible drop-down.

Example: /<Org>/<Project_Space>/<Project>/<Table_Name>

Default value: None

Data Location

Enter the location of the source data from which the Table Asset in the Data Catalog originates.

Example: parquetesting1.parquet

Default value: None

Partition Keys

Add the partition keys which you want to delete. You list the partition keys by specifying Key Column and Key Value combinations to identify the precise row and column from which to create a partition.

Key Name

Specify the key you want to use to locate a specific partition in a metadata table.

Example: airline_code

Default value: None

Operator

Specify the operator to use to locate the table partition.

Example: !=

Default value: =

Key Value

Specify the value to use to locate the table partition.

Example: 10

Default value: None

Snap Execution

Select one of the three modes in which the Snap executes. Available options are:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.
  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.
  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Example


Pipeline Description

The following Pipeline demonstrates how to overwrite an existing partition by using the three Data Catalog Service Snaps: Catalog Query, Catalog Delete, and Catalog Insert. Because you cannot edit the Table Assets in Manager, you can use this Pipeline as a way for overwriting partitions in the Data Catalog.  

The Catalog Query Snap queries for a partition, age, in the Asset named table1.

The Catalog Delete Snap deletes the partition, PK_1indicated by the Key Name and Key Value. 

To replace this metadata, the JSON Generator Snap is used to generate a document with the data for name, age, and height.

The JSON Snap passes the document data to the Parquet Writer, which writes the JSON document to a S3 bucket.

The Catalog Insert Snap populates table1 with the new data.

You can navigate to Manager > Projects > Assets to view the metadata inserted in table1.

In Manager, click table1, and click Show Table Schema.



Download

Important steps to successfully reuse Pipelines

  1. Download and import the Pipeline into SnapLogic.
  2. Configure Snap accounts as applicable.
  3. Provide Pipeline parameters as applicable.


  File Modified

File DataCatalog_Read_Delete_Insert_Basic_UseCase.slp

Feb 08, 2019 by John Brinckwirth

Snap Pack History

 Click to view/expand
Release Snap Pack VersionDateTypeUpdates
4.29

main15993

 StableUpgraded with the latest SnapLogic Platform release.
4.28main14627 StableUpgraded with the latest SnapLogic Platform release.

4.27

main12833

 

Stable

Upgraded with the latest SnapLogic Platform release.
4.26main11181 StableUpgraded with the latest SnapLogic Platform release.
4.25main9554
 
StableUpgraded with the latest SnapLogic Platform release.
4.24main8556
StableUpgraded with the latest SnapLogic Platform release.
4.23main7430
 
StableUpgraded with the latest SnapLogic Platform release.
4.22main6403
 
StableUpgraded with the latest SnapLogic Platform release.
4.21snapsmrc542-StableUpgraded with the latest SnapLogic Platform release.
4.20snaprsmrc528-StableUpgraded with the latest SnapLogic Platform release.
4.19snaprsmrc528-StableUpgraded with the latest SnapLogic Platform release.
4.18snapsmrc523-StableUpgraded with the latest SnapLogic Platform release.
4.17 PatchALL7402-Latest

Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers.

4.16

snapsmrc508-Stable

New Snap. Added the Catalog Delete Snap to remove tables and table partitions from the Data Catalog. 

4.15snapsmrc500-Stable

New Snap Pack: Introducing the Data Catalog Snap Pack with the following Snaps: Catalog Insert and Catalog Query.