- Created by Aparna Tayi, last modified by Anand Vedam on Jul 10, 2020
You are viewing an old version of this page. View the current version.
Compare with Current View Page History
« Previous Version 15 Next »
In this article
Overview
Use this Snap to generate a JSON document for the next Snap in the Pipeline.
- This Snap is often used at the beginning of a pipeline to create a file.
- This Snap does not need any Accounts.
Prerequisites
None.
Support for Ultra Pipelines
- Works in Ultra Pipelines.
See Snap Support for Ultra Pipelines.
Limitations and Known Issues
- The maximum size for a JSON file generated using this Snap is 16 MB.
Snap Input and Output
Input/Output | Type of View | Number of Views | Examples of Upstream and Downstream Snaps | Description |
---|---|---|---|---|
Input | Document |
|
| This Snap can have an upstream Snap that can pass a JSON document in output view. |
Output | Document |
|
| This Snap generates a document that contains the expected result along with the original JSON document (when Pass through is enabled). |
Snap Settings
Parameter Name | Data Type | Description | Default Value | Example |
---|---|---|---|---|
Label | String | Specify a name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your Pipeline. | N/A | JSON Generator |
Edit JSON | Apache Velocity template | Optional. Launches the JSON editor which will allow defining JSON. The editor will load an existing JSON document if it was defined previously. The editor lets you save JSON in SnapLogic's file system or import existing JSON documents from it. Note that this text is treated as an Apache Velocity template, so you can substitute values from input documents or the pipeline parameters. The Apache Velocity template can be used to pass dynamic values from upstream Snaps. For more information, see | Empty JSON file | File with JSON content |
Process Array | Check box | Optional. The Process array property specifies whether or not the Component should take a root JSON array and write each element of the array as a JSON document. | Selected | Selected |
Pass through | Check box | Optional. Use this property to pass the data in the input document through to the output document and merge it under the key 'original'. See example for more details. | Deselected | Selected |
Snap Execution | String | Indicates how the Snap must be executed. Available options are:
| Validate & Execute | Execute only |
Troubleshooting
Error | Reason | Resolution |
---|---|---|
Unexpected character ('{' (code 123)) | The array entries in the JSON file do not follow the syntax. A comma ',' is expected before the beginning of every second array—'{'. | Verify the array at the row and column mentioned beside the error message using the Edit JSON feature and add a comma appropriately. |
Editing JSON
Clicking Edit JSON will launch the JSON Editor. The JSON will be stored in the Snap (after clicking OK). The JSON can be exported to SnapLogic's file system using the export action or be imported from there using the import action.
The JSON can be customized to meet specific needs, such as having:
- Static content that produces one output document OR
- A list of static documents within an array definition which can then be transformed into multiple documents upstream by using a JSON Splitter Snap OR
- Variables in the JSON, which can be substituted with document values from the input view of the Snap.
Escaping Special Characters
Additionally, some special characters can be escaped using the following notations.
Type | To pass this value | Specify |
---|---|---|
Single characters | $ | #[$]# or \$ |
JSON String | ## | #[##]# |
For information on how to use pipeline parameters and field values within this Snap, see the "Velocity Templates in Generator Snaps" section of Parameters and Fields.
Examples
Using the Pass Through Functionality
Consider the following Pipeline that combines two simple datasets.
Snap Configurations
JSON Object (JSON Generator Snap with Pass through disabled)
This JSON Generator Snap, JSON Object, is used as an input by the downstream JSON Generator Snap, Combined JSON.
Snap Configuration (JSON Object) | |
---|---|
![]() | |
Input JSON | Output JSON |
![]() | ![]() |
Combined JSON (JSON Generator Snap with Pass through enabled)
- The Pass through check box is selected.
- In addition to this, the upstream parameter, $details.username, is included in this Snap's JSON script to capture information based on the username field in the upstream document.
Snap Configuration (Combined JSON) | |
---|---|
Input JSON | Output JSON |
![]() | ![]() |
The output JSON contains:
- Data from the input document.
- Data captured from the upstream parameter.
Downloads
Important Steps to Successfully Reuse Pipelines
- Download and import the Pipeline into SnapLogic.
- Configure Snap accounts as applicable.
- Provide Pipeline parameters as applicable.
File | Modified |
---|

Fixed an issue with CSV Formatter and CSV Generator Snaps that displayed an incorrect error when one or more header values in the CSV file were missing. This error is now routed to the error view. Fixed an issue with the CSV Parser Snap, where the Snap stopped indefinitely while processing certain inputs. Fixed an issue with the CSV Formatter Snap, where the escape character selected was not used. Updated the CSV library used in the CSV Generator, CSV Parser, and CSV Formatter Snaps. This library update enforces values for each column in a parsed CSV header. Enhanced the XML Formatter Snap with the Output Character Set field. You can select the character set you want for your binary output. For example, UTF-8 or ISO860. Fixed an issue with the XML Generator Snap, where the Snap failed with an invalid UTF-8 sequence error when running on the Windows Snaplex. Fixed an issue in the Group by Fields Snap that caused the Snap to abort with an error when executed with zero input documents. Added dynamic memory management to the Group By Fields Snap, which can activate with the following settings: Memory Sensitivity: The Snap's response to the memory changes. Min Part Size: The minimum part size that you want the Snap to split larger groups into multiple parts. Added the following checkboxes in the Excel Multi Sheet Formatter Snap: Convert numeric string to number: Sets the behavior of the Snap when a value in the input document is a numeric string. This checkbox supports the conversion of numeric values to string values. Translate date and time types: Sets the behavior of the Snap when a value in the input document is a date and datetime type. This checkbox supports translating date and time types to Excel date numeric type with date format style. Fixed an issue in the XML Formatter Snap, where the Map input to first repeating element in XSD checkbox is selected, while no XSD is specified for mapping the input. Fixed an issue in the CSV Formatter Snap, where even if the Ignore empty stream checkbox is not enabled, the Snap did not produce an empty binary stream output in case there is no input document. Replaced the Strict XSD output field with Map input to first repeating element in XSD field in the XML Formatter Snap. If selected, the Snap ignores the root element from the XSD file. Fixed a Enhanced XML Parser Snap to recognize input headers when defining inbound schema. Reverts the Join Snap to the 4.24 release behavior. This is in response to an issue encountered in the Join Snap in the 4.25 release version (main9554), which can result in incorrect outputs from all Join Types. 425patches9638 is the default version for both stable and latest Transform Snap Pack versions for orgs that are on the 4.25 release version. No action is required by customers to receive this update and no impact is anticipated. Enhanced the Group By N Snap with the following settings: Fixed the timestamp issue in the JSON Formatter Snap that changed the time zone offset to include colon by default after upgrading to 4.24. Fixes an issue in the JSON Splitter Snap by logging an error when the matcher does not find a pattern. Fixes an issue in the In-Memory Lookup Snap to correctly handle the Join path in the format like $['join path']. Fixes an issue in the XML Formatter Snap when it fails to convert input JSON data, with JSON property having a special character as its prefix, to XML format by sorting the elements. Fixes an issue with the JSON Splitter Snap's behavior in Ultra Pipelines that prevents processed requests to be acknowledged and removed from the FeedMaster queue, resulting in retries of requests that are already processed successfully. Enhances the JSON Formatter Snap to render groups output from upstream (Group by) Snaps with one document per group and a new line per group element. You can now select the Format each document and JSON lines check boxes simultaneously. Fixes the JSON Splitter Snap data corruption issue by copying the data in JSON Splitter Snap before sending it to other downstream Snaps. Fixes the XML Generator Snap issue reflecting empty tags and extra space by removing the extra space in the XML output. Enhances the JSON Generator Snap to include pass-through functionality where the Snap embeds the upstream input document under the Adds support in the Mapper Snap to display schemas with complex nesting. For example, if Snaps downstream from the Resolves the Resolves the NullPointerException in the Join Snap on Windows Snaplex instances. Fixes the JSON Formatter Snap to filter the input data based on the value specified in the Content field in the Snap settings when the JSON lines field is also selected. Previously, the Snap was writing the entire input data to the output file. The JSON Formatter Snap output now includes only those fields from the input file that are specified in the Content field under Settings. If your Pipelines use the JSON Formatter Snap with the JSON lines field selected, they may fail to execute correctly if the Content field mentions a specific object or field but the downstream Snap is expecting the entire data. Hence, for backward compatibility, you must review the entries in the Content field based on the desired output, as follows: The behavior of the Snap when the JSON lines field is not selected is correct and remains unchanged. The Example: Filtering Output Data Using the JSON Formatter Snap's Content Setting further illustrates the corrected vs the old behavior. XML Generator Snap behavior might break existing Pipelines The XML Generator Snap now gives precedence to any custom XML data that is provided over data coming from upstream Snaps, to generate the output. Existing Pipelines using the XML Generator Snap may fail in the following scenarios. Use the resolution provided to update the Snap settings based on the XML data you want to pass to downstream Snaps. Breaking Scenario Downstream Snaps expect XML output from upstream Snaps, but custom XML data exists in the XML Generator Snap. Fixed an issue with the Excel Parser Snap wherein the Snap incorrectly outputs Unformatted General Number format. The output of the AVG function in the Aggregate Snap now rounds up all numeric values that have more than 16 digits. Fixed an issue with the Excel Multi Sheet Formatter Snap wherein the Snap fails to create sheets in the expected order. Added a field, Round dates, to the Excel Parser Snap which enables you to round numeric excel data values to the closest second. Fixed an issue with the Sort and Join Snaps wherein the platform removes all temp files at the end of Pipeline execution. Fixed an issue with the XML Parser Snap wherein a class cast exception occurs when the Snap is configured with a Splitter and Namespace Context. Added a new field, Ignore empty stream, to the Avro Formatter Snap that writes an empty array in the output in case of no input documents. Added a new field, Format as canonical XML, to the XSLT Snap that enables canonical XML formatting. Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers. Fixed an issue with the XSLT Snap failure by enhancing the Saxon version. Fixed an issue with the XML Parser Snap wherein XSD with annotations were incorrectly interpreted. Fixed an issue wherein the XML Generator Snap was unable to escape some of the special characters. Fixed an issue with the Excel Parser Snap wherein headers were not parsed properly and header columns were also maintained in an incorrect order. Fixed an issue with an object type in the Fixed Width Formatter Snap. Fixed an issue with the Fixed Width Parser Snap wherein Turkish characters caused incorrect parsing of data on Windows plex. Fixed an issue with the XML Parser Snap wherein the Snap failed to get data types from XSD. Fixed an issue with the XML Parser wherein XML validation was failing if the XSD contained Added a new property, Binary header properties, to the Document to binary Snap that enables you to specify the properties to add to the binary document's header. Fixed an issue wherein the XML Parser Snap was not maintaining the datatype mentioned in the XSD file. Fixed an issue with the Avro Formatter Snap wherein the Snap was failing for a complex JSON input. Fixed an issue with S3 file reads getting aborted intermittently because of incomplete consumption of input stream. Fixed the JSON Parser Snap that causes the File Reader Snap to fail to read S3 file intermittently with an Fixed an issue in the CSV Formatter Snap where the output showed extra values that were not provided in the input. Fixed an issue with the Excel Parser Snap that drops columns when the value is null at the end of the row. Fixed an issue in the Excel Formatter Snap, wherein opening an output stream prematurely causes the Box Write Snap to fail. Excel Formatter now awaits the first input document, before opening an output stream. Fixed an issue that caused the Join Snap to go out of memory. Improved performance for both Aggregate and Join Snaps. Added Select All and Deselect All buttons to the Mapper table Enforced UTF-8 character encoding for the Fixed Width Formatter Snap. Enhanced enum labels on the Binary to Document and the Document to Binary Snaps for the Encode or Decode property with DOCUMENT and NONE options. Fixed a sorting issue with the Join and Sort Snaps where the end of the file was not detected correctly. Added support on the Kryo serialization for UUID and other types to the Sort, Join and In-Memory Lookup Snaps. Addressed an issue with the Excel Parser Snap that failed with out of memory when using large input Data (eg. 191 MB). Conditional Snap: fixed an issue with the "Null-safe access" Snap Setting not being respected for return values. Join Snap - All input documents from all input views should be consumed before the end of Snap execution. Made all four output views in Diff snap as mandatory. Made all four output views in Diff snap as mandatory. CSV Parser Snap - A new Snap property Ignore empty data with true default is added Addressed an issue with the transform2989 build. Addressed an issue with Excel Parser not displaying the most recent cached value for vlookups containing missing external references. Introduced Encrypt Field and Decrypt Field Snaps. [CSVParser] Fixed an issue where an empty Quote Character config field was defaulting to the unicode quote character U+0000 (null). This caused issues if the input CSV had U+0000 characters in it. Addressed an issue with CSV Parser causing a spike in CPU usage. Addressed an issue with Excel Formatter dropping the first record when Ignore empty stream is selected. Addressed an issue with Excel Formatter altering decimal numbers to text. Resolved an issue with validation of pipelines taking more time than executing a pipeline when a large amount of data is used. Resolved an issue with XML Parser failing with error: 'Maximum attribute size (524288) exceeded'. Resolved an issue with JSON Generator failing with an "Invalid UTF-8 middle byte 0x70" error on Windows. Resolved an issue where Excel Parser did not reliably set header row when "contains headers" is checked Resolved an issue in Sort Snap where buffer size should not be fixed for optimal performance Resolved a performance issue in Excel Multi Sheet Formatter. December 20, 2014 Aggregate now provides 2 new functions: concat and unique_concat CSV Parser with empty skip lines field leads to null pointer exception during validation CSV Formatter does not parse headers with space XML Formatter: Snap fails converting document into XML JSON Splitter: The JSON Splitter Snap splits a list of values into separate documents. Snap Pack History
Release Snap Pack Version Date Type Updates 4.29 main15993 Stable 4.28 patch 428patches14370 Latest 4.28 main14627 Stable Upgraded with the latest SnapLogic Platform release. 4.27 Patch 427patches12966 Latest Enhanced the Avro Formatter Snap to display meaningful error message while processing invalid and null values from the input. 4.27 main12989 Stable and Latest 4.27 main12833 Stable 4.26 Patch 426patches12086 Latest Fixed an issue with the Join Snap, where it exhausted the memory while buffering millions of objects. 4.26 Patch 426patches11780 Latest 4.26 Patch 426patches11725 Latest Fixed an issue with the Join Snap where the upstream document flow of the right view is blocked by the left view, which hung the Join Snap. 4.26 main11181 Stable 4.25 Patch 425patches10663 Latest 4.25 Patch 425patches10152 Latest 4.25 Patch 425patches9815 Latest ClassCastException
error in the Avro Parser Snap and handling of the map, fixed, enum, and bytes data types in the Avro Formatter and Avro Parser Snaps.4.25 Patch 425patches9749 Latest 4.25 Patch 425patches9638 - Latest 4.25 main9554 Stable 4.24 Patch 424patches8938 Latest 4.24 main8556 Stable Upgraded with the latest SnapLogic Platform release. 4.23 Patch 423patches7958 Latest 4.23 Patch 423patches7898 Latest 4.23 Patch 423patches7792 Latest 4.23 Patch 423patches7753 Latest 4.23 main7430 Stable 4.22 Patch 422patches6395 Latest 4.22 Patch 422patches6505 Latest 4.22 main6403 Latest Upgraded with the latest SnapLogic Platform release. 4.21 Patch 421patches5901 Latest original
field of the output document along with other records.4.21 Patch 421patches5848 Latest NoClassDefFoundError
. 4.21 snapsmrc542 - Stable 4.20 Patch transform8792 - Latest NoClassDefFoundError
in the Join Snap on Windows Snaplex instances.4.20 Patch transform8788 - Latest 4.20 Patch transform8760 - Latest
ORPremature end of file"
error was displayed. Resolution In the XML Generator Snap settings, the XML root element and Inbound schema are specified but no custom XML data is provided. This will generate a validation error. In the XML Generator Snap settings, custom XML data is provided and Validate XML checkbox is selected, but XML root element and Inbound schema are not specified. This will generate a validation error. 4.20 Patch transform8738 - Latest 4.20 snapsmrc535 - Stable Upgraded with the latest SnapLogic Platform release. 4.19 Patch transform8280 - Latest 4.19 snaprsmrc528 - Stable 4.18 Patch transform8199 - Latest 4.18 Patch transform7994 - Latest 4.18 Patch transform7780 - Latest 4.18 Patch transform7741 - Latest 4.18 Patch transform7711 - Latest 4.18 snapsmrc523 - Stable 4.17 Patch Transform7431 - Latest 4.17 Patch Transform7417 - Latest 4.17 Patch ALL7402 - Latest 4.17 snapsmrc515 - Stable 4.16 Patch transform7093 - Latest 4.16 Patch transform6962 - Latest 4.16 Patch transform6869 - Latest 4.16 snapsmrc508 - Stable Upgraded with the latest SnapLogic Platform release. 4.15 Patch transform6736 - Latest 4.15 Patch transform6680 - Latest 4.15 Patch transform6402 - Latest 4.15 Patch transform6386 - Latest 4.15 Patch transform6321 - Latest 4.15 Patch transform6265 - Latest xsd:include
.4.15 snapsmrc500 - Stable 4.14 Patch transform6098 - Latest 4.14 Patch transform6080 - Latest 4.14 Patch MULTIPLE5732 - Latest 4.14 Patch transform5684 - Latest AbortException
error.4.14 snapsmrc490 - Stable Upgraded with the latest SnapLogic Platform release. 4.13 Patch transform5411 - Latest 4.13 snapsmrc486 - Stable Upgraded with the latest SnapLogic Platform release. 4.12 Patch transform5005 - Latest 4.12 Patch transform4913 - Latest 4.12 Patch transform4747 - Latest 4.12 snapsmrc480 - Stable 4.11 Patch transform4558 - Latest 4.11 Patch transform4343 - Latest 4.11 Patch transform4361 - Latest 4.11 Patch transform4281 - Latest 4.11 snapsmrc465 - - 4.10 Patch transform4058 - - 4.10 Patch transform3956 - - 4.10 snapsmrc414 - - 4.9.0 Patch transform3343 - - 4.9.0 Patch transform3281 - - 4.9.0 Patch transform3264 - - 4.9.0 Patch transform3220 - - 4.9.0 Patch transform3019 - - 4.9.0 Patch transform2989 - - 4.9.0 - - 4.8.0 Patch transform2956 - - 4.8.0 Patch transform2848 - - 4.8.0 Patch transform2768 - - 4.8.0 Patch transform2736 - - 4.8.0 - - 4.7.0 Patch transform2549 - - 4.7.0 Patch transform2344 - - 4.7.0 Patch transform2335 - - 4.7.0 Patch transform2206 - - 4.7.0 - - 4.6.0 Patch transform2018 4.6.0 Patch transform1905 4.6.0 Patch transform1901 4.6.0 Patch transform1871 4.6.0 4.5.1 4.5.0 4.4.1 4.4 4.3.2 4.3.1 4.3.0 4.2.2 August 7, 2015 (2015.25/4.2.1) June 27, 2015 (2015.22) June 6, 2015 (2015.20) May 2, 2015 March 2015 January 2015 November 2014 October 18, 2014 Fall 2014 August 2014 July/Summer 2014 June 30, 2014 May 2014 April 2014 March 2014 January 2014 December 2013 November 2013 August 2013 July 2013 Initial release
See Also
- No labels