JSON Formatter

 In this article

Overview

You can use this Snap to read JSON document data from its input view, format and write it as JSON binary data to its output view. This is a Transform Snap type. A schema file can also be attached to the Snap that allows it to share the schema with upstream Snaps like the Mapper Snap.  

Prerequisites

None.

Support for Ultra Pipelines

Works in Ultra Tasks if Format each document is selected.

Limitations and Known Issues

None.

Snap Views

TypeFormatNumber of views Examples of upstream/downstream SnapsDescription
InputDocument
Min: 1
Max: 1
  • JSON Generator


JSON document data to be formatted
OutputBinary
Min: 1
Max: 1
  • Group by Fields
  • Mapper
 JSON binary data stream.

Error

Binary

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab. The available options are:

  • Stop Pipeline Execution: Stops the current pipeline execution when the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the rest of the records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

  • Asterisk (*): Indicates a mandatory field.

  • Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon (): Indicates whether the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon ( ): Indicates that you can add fields in the field set.

  • Remove icon (): Indicates that you can remove fields from the field set.


FieldField TypeDescription

Label*


Default Value
JSON Formatter
Example
:JSON Formatter

StringSpecify the name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your Pipeline.

Schema


Default Value: None
Example: contact.schema

String/Expression

Upload the JSON schema files, especially when the formatted JSON has to follow a certain schema. This is particularly useful when used with the Mapper Snap, the schema from the JSON Formatter Snap will be back-propagated to the Mapper Snap. This is a suggestible property, click on the browse button to select file from your project folder.

The pipeline should be validated once when the schema file has been uploaded in order for the schema to be visible in the upstream Snap.

Binary header properties



Use this field set to add properties to the binary document's header. The properties in the header of a binary document can be accessed in expression properties of downstream Snaps.  For example, a 'content-location' property added to the header in this Snap can be referenced in the file name property of a File Writer snap with the expression: $['content-location'].  When this Snap is configured to output a single binary document, the headers are computed from the first input document.  When the "Format each document" option is enabled, the headers are computed separately for each binary output.

Expression

Default Value: None

String/Expression

Specify the function to be used to transform the data such as combine, concat or flatten.


Target Path


Default Value: None
Example: $person.firstname to write the field 'firstname' of the 'person' object.

String/Suggestion

Specify the target JSON path where the value from the expression will be written.


Content


Default Value: $

String/Suggestion

Specify the input object or field whose data must be written to the output stream. For example, $group$payload or $emp_salary. Use $ to write the entire data.

See Filtering Output Data Using the JSON Formatter Snap's Content Setting further in this page for filtering the input data based on the Content field.


Ignore empty stream

Default Value: Deselected


Checkbox

Select this checkbox to ignore empty stream when no document is received at the input view throughout the Pipeline execution. 
Deselect this checkbox to enable the Snap to write an empty array to the output stream.


Format each document


Default Value:Deselected 


Checkbox

Select this checkbox to enable the Snap to create one binary output stream for every document on its input view.
Deselect this checkbox to write one binary output stream for all documents on its input view which are enclosed by a JSON array element [ .. ]. See the example: 
Using JSON Formatter Snap to render outputs from Group By Snaps, for a simple but less-frequent scenario.

For Data Groups in Input View

Select this checkbox along with JSON lines checkbox and set the Content field to $group to output one document per group with member in the group as a separate JSON line.

JSON lines


Default Value
Deselected 


Checkbox

Select this checkbox to enable the Snap to write all documents to a single output file, with each document appearing as a single line followed by a new line. This field cannot be used with Pretty-print check boxSee the example: Using JSON Formatter Snap to render outputs from Group By Snaps, for a simple but less-frequent scenario.

For Data Groups in Input View

Select this check box along with Format each document check box and set the Content field to $group to output one document per group with each member in the group as a separate JSON line.

  

Pretty-print


Default Value
Deselected 


Checkbox

Select this checkbox to format the output to make it more readable/printable.

 

Derive Schema from a Sample Size of*


Default Value
:

Dropdown list

Select the size of the number of initial input documents to be used when deriving the schema to be added to the binary output header. The sizing that you choose depends on the uniformity of the source schema. For example, if the schema is uniform across the documents, then choose Small. If the schema differs from document to document, then choose a larger size (Medium or Large) to attain a more accurate sampling of your schema. The performance impact on the Pipeline execution/validation is greater, the larger the sample size is. If you select None, then the schema is not sampled.

Support Type Extensions


Default Value
Deselected 

Checkbox

Select this checkbox to enable the Snap to format/parse the Snaplogic-specific syntax indicating objects of the special types, such as byte arrays and date objects in JSON. For more information, see the Input Schema Types example.

Snap Execution


Default Value: Validate & Execute
Example: Execute only

Dropdown list

Select one of the three modes in which the Snap executes. Available options are:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Examples


Render outputs from Group By Snaps

This example demonstrates how to render groups data from upstream Snaps such as Group By Fields Snap.

In a scenario where the JSON Formatter Snap needs to format data groups, you can enhance the Snap's output to format each group as a separate document, where each line is a member of the group.

Configure the JSON Generator Snap to pass the raw JSON content to be formatted.

JSON Generator SnapOutput (JSON content to be formatted)

The Group By Fields Snap is set to group the incoming data by $key with the Target field as $group.

Group By Fields SnapOutput (Raw JSON rendered into groups)

Notice that the incoming data is grouped by key in this output. 

JSON Formatter SnapOutput (one document per group and one line per group member)

When the JSON Formatter Snap received data groups, we can select the fields JSON lines and Format each document checkboxes in unison and the Snap formats the content as multiple documents – one per group (key1, key 2 and key3 in this example) with one line for each element in these groups.

Download this Pipeline.

Filter Output Data Using the JSON Formatter Snap's Content Setting

This example demonstrates how, effective 4.20 Patch transform8670 and onwards, you can use the JSON Formatter Snap to filter the input JSON data. 

Download the Sample Pipeline.

  1. Using the sample Pipeline shown above, generate a JSON document using the JSON Generator Snap.
  2. In the Mapper Snap, map the input fields: interface_id, failed_pipeline, target_url, and payload to the required output fields.
  3. In in the JSON Formatter Snap, in the Content field under Settings select $payload as the object to be included in the output.
  4. Select the JSON lines check box to ensure each record appears as a separate line in the output file.
  5. Provide the name of the file to write to in the File Writer Snap and save it.
  6. Validate and execute the Pipeline.

Current behaviorOld behavior

With Patch transform8670 onwards, the JSON Formatter Snap correctly acknowledges the Content field, irrespective of the JSON lines check box selection. So, in this example, the output would contain only the specified object’s data ($payload):

Prior to Patch transform8670, the JSON Formatter Snap would ignore the value in the Content field and write the entire input data to the output file if the JSON lines check box was selected, as shown in the image below:

Pass the Execution Status Code through Binary Header Properties of the JSON Formatter Snap

In this pipeline, the pipeline execution status is routed to the error view, which is redirected to reflect on the browser using the Binary Header Properties of the  JSON Formatter Snap. This allows us to view the   status on the browser same as on the output view of the triggered pipeline. 

This workaround of using the Binary Header properties is mainly useful when the user wants to view the exact execution status of the triggered task which may or may not be consistent with the status code otherwise reflected on the browser.


 

The JSON Formatter Snap passes the actual status code to the browser via the Binary Header Properties:

The status code (405) as retrieved from the pipeline is mapped to the response:

The JSON Formatter Snap passes the actual status code to the browser via the Binary Header Properties:

The status code (405) as reflected on the browser (Postman):

 

Create a File with Header Value in a Specific Location

In this pipeline, the JSON Formatter Snap, creates a file with a header value on a specified location. The metadata is passed to the File Writer Snap using the Binary Header and  the Content properties on the JSON Formatter Snap.

The JSON Generator Snap passes the values for the file and the records to be created.

The JSON Formatter Snap passes the metadata to the File Writer Snap.


The File Writer Snap creates the file "demo-" + $['content-location'] + ".json" on a specified content location.

The successful execution of the pipeline displays the below output preview:

Note the file headers created on the sldb location, and the files names as demo-abc.json and so on.

Schema Suggest to Upstream Snap

In the example pipeline, Basic Use Case_JSON Formatter Schema Property, the Mapper Snap is used to write employee data into a file. The input data has to cater to a specified JSON schema, to enable this the Schema property is configured with a schema file. this allows the user to see the schema of the target file in the Mapper Snap. 

The JSON Formatter Snap is configured in the following manner, the schema file contact.schema is uploaded in the Schema property:


The schema defined in the contact.schema file is shown below:


Below is the Mapper Snap's configuration, note the Target Schema and the Output preview data sections where the schema of the target file and the mapped data are shown:

When executed the input record from the Mapper will be mapped to the specified path based on the target schema and written into the target file.

Exported pipeline is available in the Downloads section below.

/wiki/spaces/AP/pages/1439181

Downloads

Important steps to successfully reuse Pipelines

  1. Download and import the pipeline into the SnapLogic application.
  2. Configure Snap accounts as applicable.
  3. Provide pipeline parameters as applicable.

  File Modified