Sort

In this article

Overview

You can use this Snap to sort a document stream using its values in the order in which they show up in the sort path list.

Snap Type

The Sort Snap is a Transform-type Snap that sorts the input document streams in the memory buffer.

Prerequisites

None.

Support for Ultra Pipelines 

Does not work in Ultra Pipelines

Limitations and Known Issues

None.

Snap Views

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input 

Document

  • Min: 1

  • Max: 1

  • Mapper

  • Union

The Snap reads input documents and sorts them in a memory buffer.

Output

Document

  • Min: 1

  • Max: 1

  • Mapper

  • Join

This Snap sorts a document stream which show up in the sort path list.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

  • Stop Pipeline Execution: Stops the current pipeline execution when the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

  • Asterisk (*): Indicates a mandatory field.

  • Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon ( ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon (): Indicates that you can add fields in the fieldset.

  • Remove icon (): Indicates that you can remove fields from the fieldset.

 

Field Name

Field Type

Description

Label*

Default value: Sort
Example: Sort input streams

String

The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

Sort Paths*

Use this field set to provide the list of paths to sort on.

Sort path*

Default Value: None
Example: $person.first_name

String/Suggestion

Paths of values in the document to sort on. For example, if the sort paths are $first_name and $last_name, then the value at first_name is sorted first.

This field supports only scalar values such as string, number, date and so on.

Sort order*

Default value: global
Example: ascending

Dropdown list

Order of sorting. The available options are:

  • global

  • ascending

  • descending

If you select the option global, the Sort order (Global) property becomes functional for the corresponding Sort path.

Sort order (Global)

Default value: ascending
Example: descending

Dropdown list

Sort the specified values in ascending or descending order.

Null greater

Default value: Not Selected
Example: Selected

Checkbox

Select this checkbox to allow the Snap to consider null value greater than the non-null values. When you select this checkbox, null values are placed last when sorted in the ascending order.

  • When you select this checkbox, and the Sort order is:

    • ascending, null values appear at the end of the list.

    • descending, null values appear at the beginning of the list.

  • When you deselect this checkbox and the Sort order is:

    • ascending, null values appear at the beginning of the list.

    • descending, null values appear at the end of the list.

Null-safe access

Default value: Selected (True)  
Example: Deselected

Checkbox

Select this checkbox if you consider the values of non-existent or incorrect sort paths as null. If unselected, the Snap validates each input document to throw errors for non-existent or incorrect sort paths. Clearing this property will result in a lower performance.

Maximum memory

Default Value: 10
Example: 20

Integer

Specify the maximum memory to compute the size of the internal memory buffer for the external merge sort. The value you choose must be in MB or % , which you can specify in the Maximum memory unit. If its unit is %, the value of this property is the percentage of maximum used memory as compared to the maximum system memory.

Maximum memory unit

Default Value: %

Example: MB

Dropdown

Select the appropriate unit for the Maximum memory property.

  • %

  • MB

Minimum memory (MB)

 

Default value: 500
Example750

String/Expression

If the available memory is less than this property value while processing input documents, the Snap stops to fetch the next input document until more memory is available. This feature is disabled if this property value is 0.

Minimum free disc space (MB)

 

Default value: 500
Example750

String/Expression

If the free disc space is less than this property value, the Snap stops processing input documents until more free disc space is available. This feature is disabled if this property value is 0.

Out-of-resource timeout (minutes)

 

Default value: 30
Example20

String/Expression

If the Snap pauses longer than this property value while waiting for more memory available, it throws an exception to prevent the system from running out of memory or disk space.

 

Snap Execution

Dropdown list

Select one of the three modes in which the Snap executes. Available options are:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Troubleshooting:

Error

Reason

Resolution

Error

Reason

Resolution

Failed to sort data at the input document.

Insufficient free disk space to stage sort data into temporary files.

Increase the free disk space.

Input document does not contain sort path.

If Null-safe access is false, all input documents should contain keys for sort paths.

Address the reported issue.

Example


Sort fields in ascending and descending order

The example pipeline demonstrates how to sort the data by age and last name from a JSON file in ascending and descending order.

  1. Configure the JSON Generator Snap with multiple records.

     

2. Configure the Sort Snap to sort the Age in descending order and the Last Name as global. Further, set the Sort order (Global) as ascending.

3. Validate the Snap. The Sort Snap displays the following output, with Age in descending order and, further, the Last Name in ascending order.

Download the Pipeline.

Downloads