Filter

In this article

Overview

The Filter Snap filters a document stream based on a Filter expression. If the filter expression evaluates to true, the document is passed through. If the filter expression evaluates to false, the document is dropped in the stream. Expressions that do not evaluate to a boolean value will cause the pipeline to fail. Learn more about Expression functions.

  • The Router and Filter Snaps are similar, but the Filter Snap handles only a single stream. The Router Snap is designed to handle multiple output views.

  • You do not require accounts to use this Snap.

Snap Type

The Filter Snap is a Flow type Snap.

Support for Ultra Pipelines

Works in Ultra Pipelines. 

When using the Filter Snap in an Ultra Task pipeline, the pipeline returns an empty response if the Snap filters the incoming document.


Snap Views

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input

Document 

Min: 1

Max: 1

JSON Generator

The data in the input document to filter.

Output

Document 

Min: 1

Max: 1

  • Mapper

  • Pipeline Execute

The filtered document.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the pipeline by choosing one of the following options from the When errors occur list under the Views tab:

  • Stop pipeline Execution: Stops the current pipeline execution if the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

  • Asterisk ( * ): Indicates a mandatory field.

  • Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon ( ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon ( ): Indicates that you can add fields to the field set.

  • Remove icon ( ): Indicates that you can remove fields from the field set.

  • Upload icon ( ): Indicates that you can upload files.

Field 

Field Type

Description

Field 

Field Type

Description

Label*


Default Value: Filter
Example: Filter

String

The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

Null-safe access


Default Value: Deselected

Checkbox

Select this checkbox to consider the missing data or empty value as null when accessing the source path.

If you deselect this checkbox, the Snap fails and displays error in case there is missing data or empty value when accessing the source path.

Filter Expression*

Default Value: None
Examples:

$first_name = "Joe"
$people[0].age > 15
$Customers.contains(_customer)

startsWith($description, "Snap") && $quantity < 100 

String/Expression

Enter the expression to filter the documents. If the expression evaluates to true, the document is passed through. Else, the document is not passed through.
If the expression does not evaluate to a boolean, then the pipeline fails.


Snap Execution



Default Value: Validate & Execute
Example: Execute Only

Dropdown list

Select one of the three modes in which the Snap executes. Available options are:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Examples

Filter missing data or empty values using the Null-safe access option

This example pipeline demonstrates how to filter data with missing data or empty values using the Null-safe access checkbox.

  1. Configure the JSON Generator Snap with the following data. Each record has location and salary except ID102, which does not have the location.

  1. Configure the Filter expression as $location=="Hyderabad" || $salary>=20000 which means the Snap should filter and display the records in the output that has the location as Hyderabad OR the salary at least INR 20000. Select the Null-safe access checkbox—the Snap then finds any missing data while accessing the source path ( such as record#102, which does not have a location); the Snap considers the field null, a valid filter criteria.

  1. Validate the pipeline, the Filter Snap returns the output as follows:

Downloads

  File Modified

File Example_Filter_Enabling_Null_Safe_Option.slp

Feb 08, 2022 by Subhajit Sengupta