JSON Formatter
In this article
Overview
You can use this Snap to read JSON document data from its input view, format and write it as JSON binary data to its output view. This is a Transform Snap type. A schema file can also be attached to the Snap that allows it to share the schema with upstream Snaps like the Mapper Snap.
Prerequisites
None.
Support for Ultra Pipelines
Works in Ultra Tasks if Format each document is selected.
Limitations and Known Issues
None.
Snap Views
Type | Format | Number of views | Examples of upstream/downstream Snaps | Description |
---|---|---|---|---|
Input | Document | Min: 1 Max: 1 |
| JSON document data to be formatted |
Output | Binary | Min: 1 Max: 1 |
| JSON binary data stream. |
Error | Binary | Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab. The available options are:
Learn more about Error handling in Pipelines. |
Snap Settings
Asterisk (*): Indicates a mandatory field.
Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.
Expression icon (): Indicates whether the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.
Add icon ( ): Indicates that you can add fields in the field set.
Remove icon (): Indicates that you can remove fields from the field set.
Field | Field Type | Description | ||
---|---|---|---|---|
Label*
| String | Specify the name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your Pipeline. | ||
Schema Default Value: None | String/Expression | Upload the JSON schema files, especially when the formatted JSON has to follow a certain schema. This is particularly useful when used with the Mapper Snap, the schema from the JSON Formatter Snap will be back-propagated to the Mapper Snap. This is a suggestible property, click on the browse button to select file from your project folder. The pipeline should be validated once when the schema file has been uploaded in order for the schema to be visible in the upstream Snap. | ||
Binary header properties | Use this field set to add properties to the binary document's header. The properties in the header of a binary document can be accessed in expression properties of downstream Snaps. For example, a 'content-location' property added to the header in this Snap can be referenced in the file name property of a File Writer snap with the expression: $['content-location']. When this Snap is configured to output a single binary document, the headers are computed from the first input document. When the "Format each document" option is enabled, the headers are computed separately for each binary output. | |||
Expression Default Value: None | String/Expression | Specify the function to be used to transform the data such as combine, concat or flatten. | ||
Target Path
| String/Suggestion | Specify the target JSON path where the value from the expression will be written. | ||
Content
| String/Suggestion | Specify the input object or field whose data must be written to the output stream. For example, $group, $payload or $emp_salary. Use $ to write the entire data. See Filtering Output Data Using the JSON Formatter Snap's Content Setting further in this page for filtering the input data based on the Content field. | ||
Ignore empty stream Default Value: Deselected | Checkbox | Select this checkbox to ignore empty stream when no document is received at the input view throughout the Pipeline execution. | ||
Format each document Default Value:Deselected | Checkbox | Select this checkbox to enable the Snap to create one binary output stream for every document on its input view. For Data Groups in Input View Select this checkbox along with JSON lines checkbox and set the Content field to $group to output one document per group with member in the group as a separate JSON line. | ||
JSON lines
| Checkbox | Select this checkbox to enable the Snap to write all documents to a single output file, with each document appearing as a single line followed by a new line. This field cannot be used with Pretty-print check box. See the example: Using JSON Formatter Snap to render outputs from Group By Snaps, for a simple but less-frequent scenario. For Data Groups in Input View Select this check box along with Format each document check box and set the Content field to $group to output one document per group with each member in the group as a separate JSON line.
| ||
Pretty-print
| Checkbox | Select this checkbox to format the output to make it more readable/printable.
| ||
Derive Schema from a Sample Size of*
| Dropdown list | Select the size of the number of initial input documents to be used when deriving the schema to be added to the binary output header. The sizing that you choose depends on the uniformity of the source schema. For example, if the schema is uniform across the documents, then choose Small. If the schema differs from document to document, then choose a larger size (Medium or Large) to attain a more accurate sampling of your schema. The performance impact on the Pipeline execution/validation is greater, the larger the sample size is. If you select None, then the schema is not sampled. | ||
Support Type Extensions
| Checkbox | Select this checkbox to enable the Snap to format/parse the Snaplogic-specific syntax indicating objects of the special types, such as byte arrays and date objects in JSON. For more information, see the Input Schema Types example. | ||
Snap Execution
| Dropdown list | Select one of the three modes in which the Snap executes. Available options are:
|
Examples
Render outputs from Group By Snaps
This example demonstrates how to render groups data from upstream Snaps such as Group By Fields Snap.
In a scenario where the JSON Formatter Snap needs to format data groups, you can enhance the Snap's output to format each group as a separate document, where each line is a member of the group.
Configure the JSON Generator Snap to pass the raw JSON content to be formatted.
JSON Generator Snap | Output (JSON content to be formatted) |
---|---|
The Group By Fields Snap is set to group the incoming data by $key with the Target field as $group.
Group By Fields Snap | Output (Raw JSON rendered into groups) |
---|---|
Notice that the incoming data is grouped by key in this output.
JSON Formatter Snap | Output (one document per group and one line per group member) |
---|---|
When the JSON Formatter Snap received data groups, we can select the fields JSON lines and Format each document checkboxes in unison and the Snap formats the content as multiple documents – one per group (key1, key 2 and key3 in this example) with one line for each element in these groups.
Filter Output Data Using the JSON Formatter Snap's Content Setting
This example demonstrates how, effective 4.20 Patch transform8670 and onwards, you can use the JSON Formatter Snap to filter the input JSON data.
Download the Sample Pipeline.
- Using the sample Pipeline shown above, generate a JSON document using the JSON Generator Snap.
- In the Mapper Snap, map the input fields:
interface_id, failed_pipeline, target_url,
andpayload
to the required output fields. - In in the JSON Formatter Snap, in the Content field under Settings select
$
payload as the object to be included in the output. - Select the JSON lines check box to ensure each record appears as a separate line in the output file.
- Provide the name of the file to write to in the File Writer Snap and save it.
- Validate and execute the Pipeline.
Current behavior | Old behavior |
---|---|
With | Prior to |
Pass the Execution Status Code through Binary Header Properties of the JSON Formatter Snap
In this pipeline, the pipeline execution status is routed to the error view, which is redirected to reflect on the browser using the Binary Header Properties of the JSON Formatter Snap. This allows us to view the status on the browser same as on the output view of the triggered pipeline.
This workaround of using the Binary Header properties is mainly useful when the user wants to view the exact execution status of the triggered task which may or may not be consistent with the status code otherwise reflected on the browser.
The JSON Formatter Snap passes the actual status code to the browser via the Binary Header Properties:
The status code (405) as retrieved from the pipeline is mapped to the response:
The JSON Formatter Snap passes the actual status code to the browser via the Binary Header Properties:
The status code (405) as reflected on the browser (Postman):
Create a File with Header Value in a Specific Location
In this pipeline, the JSON Formatter Snap, creates a file with a header value on a specified location. The metadata is passed to the File Writer Snap using the Binary Header and the Content properties on the JSON Formatter Snap.
The JSON Generator Snap passes the values for the file and the records to be created.
The JSON Formatter Snap passes the metadata to the File Writer Snap.
The File Writer Snap creates the file "demo-" + $['content-location'] + ".json" on a specified content location.
The successful execution of the pipeline displays the below output preview:
Note the file headers created on the sldb location, and the files names as demo-abc.json and so on.
Schema Suggest to Upstream Snap
In the example pipeline, Basic Use Case_JSON Formatter Schema Property, the Mapper Snap is used to write employee data into a file. The input data has to cater to a specified JSON schema, to enable this the Schema property is configured with a schema file. this allows the user to see the schema of the target file in the Mapper Snap.
The JSON Formatter Snap is configured in the following manner, the schema file contact.schema is uploaded in the Schema property:
The schema defined in the contact.schema file is shown below:
Below is the Mapper Snap's configuration, note the Target Schema and the Output preview data sections where the schema of the target file and the mapped data are shown:
When executed the input record from the Mapper will be mapped to the specified path based on the target schema and written into the target file.
Exported pipeline is available in the Downloads section below.
Downloads
Important steps to successfully reuse Pipelines
- Download and import the pipeline into the SnapLogic application.
- Configure Snap accounts as applicable.
- Provide pipeline parameters as applicable.