Input 

Document

• Min: 1

• Max: 2

• Mapper

• Copy

Requires document data as input.

Output

Binary

• Min: 1

• Max: 1

• Parquet Writer

• Parquet Parser

Writes the document data in the binary (parquet) format to the output.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

• Stop Pipeline Execution: Stops the current Pipeline execution if the Snap encounters an error.

• Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

• Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Label*

Default Value: Parquet Formatter
Example: Hadoop Parquet Formatter

String

Specify the name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your Pipeline.

Edit Schema

Button

Specify a valid Parquet schema that describes the data.  

The following is an example of a schema using all the primitive and some examples of logical types:

message document {
  # Primitive Types
  optional int64 32_num;
  optional int64 64_num;
  optional boolean truth;
  optional binary message;
  optional float pi;
  optional double e;
  optional int96 96_num;
  optional fixed_len_byte_array (1) one_byte;
  # Logical Types
  optional binary snowman (UTF8);
  optional int32 8_num (INT_8);
  optional int32 16_num (INT_16);
  optional int32 u8_num (UINT_8);
  optional int32 u16_num (UINT_16);
  optional int32 u32_num (UINT_32);
  optional int64 u64_num (UINT_64);
  optional int32 dec_num (DECIMAL(5,2));
  optional int32 jan7 (DATE);
  optional int32 noon (TIME_MILLIS);
  optional int64 jan7_epoch (TIMESTAMP_MILLIS);
  optional binary embedded (JSON);
}

Compression

Default Value: NONE

Example: SNAPPY

Dropdown list

Select the type of compression to use when writing the file. The following are the available options:

• NONE

• SNAPPY

• GZIP

Decimal rounding mode

Default Value: Half up

Example: Up

Dowpdown list

Select the required rounding method for decimal values when they exceed the required number of decimal places. The available options are:

• Half up

• Half down

• Half even

• Up

• Down

• Ceiling

• Floor

• Truncate

Snap execution

Default Value: Validate & Execute
Example: Execute only

Dropdown list

Select one of the following three modes in which the Snap executes:

• Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

• Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

• Disabled: Disables the Snap and all Snaps that are downstream from it.

Account validation failed.

The Pipeline ended before the batch could complete execution due to a connection error.

Verify that the Refresh token field is configured to handle the inputs properly. If you are not sure when the input data is available, configure this field as zero to keep the connection always open.