Catalog Query
On this Page
Overview
The Catalog Query Snap enables you to read the metadata associated with a specific table in the SnapLogic data catalog.
Expected Input and Output
- Expected Input: A document stream containing information related to the metadata table that must be queried.
- Expected Output: A document stream containing the contents of the the metadata table that must be queried.
- Expected Upstream Snaps: Any Snap that offers document streaming data. Examples: Catalog Insert, Mapper.
- Expected Downstream Snaps: Required. Any Snap that accepts document streaming data in its input view. Examples: JSON Formatter, Catalog Insert.
Prerequisites
Read access to the SnapLogic metadata catalog.
Configuring Accounts
Accounts are not used with this Snap.
Configuring Views
Input | This Snap has at most one document input view. |
---|---|
Output | Required. This Snap has exactly one document output view. |
Error | This Snap has at most one document error view. |
Troubleshooting
None.
Known Issues
- Does not work in Ultra pipelines.
Snap Settings
Label | Required. The name for the Snap. Modify this to be more specific, especially if there are more than one of the same Snap in the pipeline. |
---|---|
Table Name | The location and name of the table that you want to query. Structure: /<Org>/<Project>/<Table_Name> Example: /TestOrg/shared/Docs_Test_Table Default value: None. |
Partition Keys | The partition keys in the target table from which you want to extract metadata |
Key Name | The key you want to use to query a specific partition in a metadata table. Example: airline_code Default value: None |
Operator | The operator that you want to use in the condition required to locate the table partition. Example: != Default value: = |
Key Value | The value that you want to use in the condition required to locate the table partition. Example: 10 Default value: None |
Snap Execution | Select one of the following three modes in which the Snap executes:
Default Value: Execute only |
Examples
Inserting and Querying Custom Metadata from the Flight Metadata Table
The Pipeline in this zipped example, MetadataCatalog_Insert_Read_Example.zip, demonstrates how you can:
- Use the Catalog Insert Snap to update metadata tables.
- Use the Catalog Query Snap to read the updated metadata information.
In this example:
- We import a file containing the metadata.
- We create a parquet file using the data in the imported file
- We insert metadata that meets specific requirements into a partition in the target table.
- We read the newly-inserted metadata using the Catalog Query Snap.
Downloads
Important steps to successfully reuse Pipelines
- Download and import the pipeline into the SnapLogic application.
- Configure Snap accounts as applicable.
- Provide pipeline parameters as applicable.