This Snap has exactly one binary input view, where it gets the JSON binary data to be parsed.
Output
This Snap has exactly one document output view, where it provides the JSON document data stream.
Error
This Snap has at most one document error view and produces zero or more documents in the view.
If the Snap fails during the operation, an error document is sent to the error view containing the fields error, reason, original, resolution, and stack trace:
{
error: "Cannot parse JSON data"
reason: "Unable to create json parser for the given input stream, Unexpected character ('F' (code 70)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')"
original: {cache-control:no-store, content-type:text/csv; charset=utf-8, connection:keep-alive, last-modified:{, ...}, etag:W/6c206b5eb3f9f13535158ad7cd09c951, content-length:236, content-dis...}
resolution: "Please check for correct input JSON data."
stacktrace: "com.Snaplogic.api.ExecutionException: ...
}
Settings
Label
Required. The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.
Filter path
The Filter path property specifies the JSON path to the filter to be applied when parsing the given JSON data. The Snap extracts the parts of data that applies to the filter condition. For more information regarding JSON path, refer to the site http://goessner.net/articles/JsonPath/ Example: $.items.item.[?] $.items.item[?(@.price > 10)] Default value: [None]
Non-standard JSON
Required. The Non-standard JSON property specifies whether or not the Component should allow non-standard features in the given JSON data. Example: Selected Default value: Selected
Process array
Required. The Process array property specifies whether or not the Component should take a root JSON array and writes each element of the array as a JSON document.
Example: Selected
Do not select this property if the input data is not in JSON lines format. If it has no newline character and its data size exceeds available memory, it may cause an out of memory error.
Default value: Selected
JSON lines
This property specifies the Snap's error handling when the input JSON data is newline-delimited JSON (JSON lines) and contains invalid JSON data:
If selected, the Snap writes an error and continues to parse the next JSON lines. The documents in a line after the invalid data are not parsed if the line contains more than one document. Lower performance is expected since the data is parsed one line at a time. If not selected, the Snap writes an error and stops parsing. The Snap can parse JSON lines if each line has only one document.
Do not select this property if the input data is not in JSON lines format. If it has no newline character and its data size exceeds available memory, it may cause an out of memory error.
Example: Selected
Default value: Not selected
Support Type Extensions
Select this checkbox to enable the Snap to format/parse the Snaplogic-specific syntax indicating objects of the special types,such as byte arrays and date objects in JSON. For more information, see the Input Schema Types example.
Snap Execution
Select one of the three modes in which the Snap executes. Available options are:
Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.
Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.
Disabled: Disables the Snap and all Snaps that are downstream from it.
Default Value: Execute only Example: Validate & Execute
Input as a non UTF-8 compliant
JSON Parser Snap supports reading the input file in UTF-8 format only. However, when the input data from the upstream is a non UTF-8 format (such as ISO-8859-13, IBM01144, windows-1258 and so on), use the Transcoder Snap to decode the format back to the UTF-8 compliant and then parse the data.
Examples
JSON Parser with JSON lines selected
JSON Parser with JSON Lines Selected
In the following pipeline, the JSON Parser Snap parses the valid lines from the upstream File Reader Snap and routes the invalid lines to the error view.
The File Reader passes four lines of which one line carries an invalid data with a missing delimiter:
The JSON Parser Snap parses the data from the upstream. Note the output preview displays the valid three lines only:
The invalid line is routed to the error view:
Reading an input file other than the UTF-8 format
Reading an Input File Other Than the UTF-8 Format
Reading an input file other than the UTF-8 format:
The following pipeline reads the input file in ISO-8859-15 format to generate an output file in UTF-8 format.
The Transcoder Snap converts the incoming ISO-8859-15 format document into the UTF-8 compliant format.
The JSON Parser Snap parses the transcoded upstream document and displays the below output preview:
Reading an upstream JSON file and Parsing the data
Reading an Upstream JSON File and Parsing the Data
If you have a JSON file that looks like this:
When it is read by a File Reader Snap, the output looks like this:
If you add a JSON Parser Snap after the File Read, the output then looks like this:
Click to view/expand
Release
Snap Pack Version
Date
Type
Updates
August 2024
438patches28073
Latest
Fixed an issue with the JSON Generator and XML Generator Snaps that caused unexpected output displaying '__at__' and '__h__' instead of '@' and '-' respectively because the Snap could not update them to their original values after the Velocity library upgrade.
August 2024
438patches27959
Latest
Fixed an issue with the Sort where the Snap could not sort files larger than 52 MB. This fix applies to Join Snap also.
August 2024
main27765
Stable
Upgraded the org.json.json library from v20090211 to v20240303, which is fully backward compatible.
May 2024
437patches26643
Latest
Fixed an issue with the Sort Snap that displayed an error when estimating the size of the input document provided by the upstream S3 Browser Snap.
Fixed an issue with the Parquet Formatter Snap that was unable to route errors to the error view.
May 2024
437patches26453
Latest
Added expression support to the Skip lines field in the CSV Parser Snap to enable passing pipeline parameters and upstream values.
Fixed an issue with the XML Parser Snap that caused an error when using the Splitter option in the Snap settings.
May 2024
main26341
Stable
Added Parquet Parser and Parquet Formatter Snaps to the Transform Snap Pack:
Parquet Parser: Reads the binary Parquet data and writes document data to the output.
Parquet Formatter: Reads the document data and writes it to the output in binary Parquet format.
Enhanced the JSON Splitter Snap to capture metadata and lineage information from the input document.
February 2024
436patches25564
Latest
Fixed an issue with theJSON FormatterSnap that generated incorrect schema.
February 2024
436patches25292
Latest
Fixed an out-of-memory error issue with the Aggregate Snap. This Snap no longer performs the presort for the input documents.
If the input documents areunsorted and GROUP-BY fields are used, you must use the Sort Snap upstream of the Aggregate Snap to presort the input document stream and set the Sorted stream field Ascending or Descending to prevent the out-of-memory error. However, if the total size of input documents is expected to be relatively small compared to the available memory, then Sort Snap is not required upstream.
Updated and certified against the current SnapLogic Platform release.
November 2023
435patches24802
Latest
Fixed an issue with the Excel Parser Snap that caused a null pointer exception when the input data was an Excel file that did not contain a StylesTable.
November 2023
435patches24481
Latest
Fixed an issue with the Aggregate Snap where the Snap was unable to produce the desired number of output documents when the input was unsorted and the GROUP-BY fields field set was used.
November 2023
435patches24094
Latest
Fixed a deserialization issue for a unique function in the Aggregate Snap.
November 2023
main23721
Stable
Updated and certified against the current SnapLogic Platform release.
August 2023
434patches23076
Latest
Fixed an issue with the Binary to Document Snap where an empty input document with Ignore Empty Stream selected caused the Snap to stop executing.
August 2023
434patches23034
Latest
Fixed an issue with the Transform Snap Pack that caused an error when the input file was a binary JSON file that contained a string value of more than 20,000,000 characters.
Fixed a memory issue with the Aggregate Snap that occurred when using GROUP-BYfields.
August 2023
434patches22705
Latest
Fixed an issue with the JSON Splitter Snap that caused the pipeline to terminate with excessive memory usage on the Snaplex node after the 4.33 GA upgrade. The Snap now consumes less memory.
August 2023
main22460
Stable
Updated and certified against the current SnapLogic Platform release.
May 2023
433patches22431
Latest
Fixed an issue with the Excel Multi Sheet Formatter Snap that caused it to produce binary output data when there was no input document and Ignore empty stream was selected.
Introduced the following new Snaps:
GeoJSON Parser: Parses geospatial data from binary data input and outputs the contents as a GeoJSON document downstream.
WKT Parser: Parses geospatial data from binary data input and outputs the contents as a WKT (Well Known Text) document downstream.
May 2023
433patches21779
Latest
The Decrypt Field and Encrypt Field Snaps now support CTR (Counter mode) for the AES (Advanced Encryption Standard) block cipher algorithm.
May 2023
433patches21586
Latest
The Decrypt Field Snap now supports the decryption of various encrypted fields on providing a valid decryption key.
Fixed an issue with the AutoPrep Snap where dates could potentially be rendered in a currency format because currency format options were displayed for the DOB column.
May 2023
433patches21196
Latest
Enhanced the In-Memory Lookup Snap with the following new fields to improve memory management and help reduce the possibility of out-of-memory failures:
Minimum memory (MB)
Out-of-memory timeout (minutes)
These new fields replace the Maximum memory % field.
May 2023
main21015
Stable
Upgraded with the latest SnapLogic Platform release.
February 2023
432patches20535
Latest
Fixed an issue with the Encrypt Field Snap, where the Snap failed to support an RSA public key to encrypt a message or field. Now the Snap supports the RSA public key to encrypt the message.
The Pipeline Execution Statistics of the Join Snap now has a status message that displays the parameters - Free disk space, Available memory, and Average document size.
The internal sort buffer size is reduced to a minimum of 10MB when the available memory in the node becomes lower than 500MB to avoid the out-of-memory crash.
The internal sort buffer size is restored to its original size when the available memory becomes larger than 2GB.
We have improved the readability of the error message for the out of disk space on node error. The updated error message now provides clearer information and guidance for users, as shown below: Reason: Insufficient free disk space available to stage sort data into temporary files. Resolution: Increase the amount of free disk space and try again.
February 2023
432patches20250
Latest
Fixed an issue with the JSON Splitter Snap that was causing errors when using multiple repeated dots in the JSON Path.
The Sort Snap includes the following improvements:
The Maximum memory % field is revised to Maximum memory.
The Maximum memory unit (new dropdown list) enables you to choose a unit, percentage (%), or MB for better memory control.
February 2023
432patches20151
Stable/Latest
Fixed an issue that occurred with the JSON Splitter Snap when used in an Ultra pipeline. The request was acknowledged before it was processed by the downstream Snaps, which caused a 400 Bad Request response.
February 2023
432patches20062
Stable/Latest
Fixed the behavior of the JSON Splitter Snap for some use cases where its behavior was not backward compatible with the 4.31 GA version. These cases involved certain uses of either the Include scalar parents feature or the Include Paths feature.
February 2023
432patches19974
Stable/Latest
Fixed the "Json Splitter expects a list" error by restoring the JSON Splitter Snap's previous behavior of handling the case where the document element referenced by the JSON Path to Split field is an object instead of a list or array.
Review your pipelines where this error occurred to check your assumptions about the input to the JSON Splitter and whether the value referenced by the JSON Path to Split field will always be a list. If the input is provided by an XML-based or SOAP-based Snap like the Workday or NetSuite Snaps, a result set or child collection that’s an array when there's more than one result or child will be an object when there's only one result or child. In these cases, we recommend using a Mapper Snap and the sl.ensureArray() function to ensure that the value being split by the JSON Splitter is always an array (even for the single element cases).
February 2023
432patches19918
Stable/Latest
Fixed an issue with the CSV Formatter Snap where the Unicode character delimiters using [0-9a-f] did not work.
Fixed an issue with the JSON Splitter Snap that was generating null values for empty input data.
February 2023
main19844
Stable
Upgraded with the latest SnapLogic Platform release.
The Transform Join Snap now doesn’t fail with the Null Pointer Exception when you configure the Sorted streams field with Ascending.
November 2022
431patches19359
Latest
The JSON Splitter Snap includes memory improvements and a new Exclude List from Output Documents checkbox. This checkbox enables you to prevent the list that is split from getting included in output documents, and this also improves memory usage.
The Mapper Snap now has a Sorted checkbox in the Input Schema and Target Schema panels, which allows you to sort the input and target schemas. When unchecked, the Snap unsorts the input and the target schema.
October 2022
430patches18800
Latest
The Sort and Join Snaps now have improved memory management, allowing used memory to be released when the Snap stops processing.
The AutoPrep feature now includes the following new transformation options that enable you to:
Change the field data type from Data type menu.
Format dates and date Strings.
Rename a field.
Mask sensitive data using an MD5, SHA-1, SHA-256, or SHA-512 algorithm.
The data in the Preview data pane format is easier to read and the buttons have been changed to improve usability.
The CSV Parser Snap now parses data with empty values in the columns when using a multi-character delimiter.
September 2022
430patches18119
Latest
The Transcoder Snap used in a low-latency feed Ultra Pipeline now acknowledges the requests correctly.
September 2022
430patches17802
Latest
The Avro Parser Snap now displays the decimal number correctly in the output view if the column’s logical type is defined as a decimal.
September 2022
430patches17737
Stable/Latest
AutoPrep now enables you to handle empty or null values.
September 2022
430patches17643
Latest
The CSV Parser and CSV Formatter Snaps now support either \ or \\ for a single backslash delimiter which were failing earlier.
September 2022
430patches17589
Latest
The CSV Formatter Snap does not hang when running in specific situations involving multibyte characters in a long field. If you notice the CSV Formatter Snap is hung, we recommend that you update to the 430patches17589 version and restart your Snaplex.
August 2022
main17386
Stable
New Snap Application: The Auto Prep Snap provides a data preparation application where you can flatten structured data, include and exclude data fields, and change data types before forwarding the data for further processing.