Parquet Formatter

In this article

Overview

You can use this Snap to read both the input document data and write the data in the binary (Parquet) format to the output.

parquet-formatter-overview.png

Snap Type

The Parquet Formatter Snap is a Format-type Snap.

Prerequisites

None.

Support for Ultra Pipelines

Does not work in Ultra Pipelines

Limitations and Known Issues

None.

Snap Views

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input 

Document

 

  • Min: 1

  • Max: 2

  • Mapper

  • Copy

Requires document data as input.

You can override the schema setting by inserting an object like this into the second input view.

Output

Binary

 

  • Min: 1

  • Max: 1

  • Parquet Writer

  • Parquet Parser

Writes the document data in the binary (Pprquet) format to the output.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the pipeline by choosing one of the following options from the When errors occur list under the Views tab:

  • Stop Pipeline Execution: Stops the current pipeline execution if the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

  • Asterisk ( * ): Indicates a mandatory field.

  • Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon ( ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon ( ): Indicates that you can add fields in the field set.

  • Remove icon ( ): Indicates that you can remove fields from the field set.

  • Upload icon ( ): Indicates that you can upload files.

Field Name

Field Type

Description

Field Name

Field Type

Description

Label*

 

Default ValueParquet Formatter
ExampleTransform Parquet Formatter

String

Specify the name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

 

Edit Schema

 

 

 

 

 

 

 

Button

Specify a valid Parquet schema that describes the data.  

The following is an example of a schema using all the primitive and some examples of logical types:

message document { # Primitive Types optional int64 32_num; optional int64 64_num; optional boolean truth; optional binary message;