This Snap has exactly one binary input view, where it gets the binary data to be parsed for fixed length columns.
Output
This Snap has exactly one document output view, where it provides a document data stream. Each document contains the extracted field values. All fields in the document data are of string-type. The Mapper (Data) Snap can be used to transform the data type from string-type to required type.
Error
This Snap has at most one document error view and produces zero or more documents in the view. Note: An error view is highly recommended to help track down errors in field configuration.
Settings
Label
Required. The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.
Skip lines
Number of lines to skip at beginning of the data
Example: 2
Default value: 0
Line separator
The character used to separate lines in the input. Leave empty to separate lines using new line character or specify the character used for separating lines.
Default value: \n
Field configuration
Details to be filled for each field that is required from the input.
Column names
Required. Column names to be used as headers for the extracted values.
Example:
First Name
Last Name
Default value: [None]
Start position
Required. Starting position of each column to be used while extracting field values.
Example: 1
Default value: [None]
End position
Required. Ending position of each column to be used while extracting field values. Example: 25 Default value: [None]
Trim column data
Required. If removal of leading and trailing spaces is required on the extracted data Default value: False
Ignore Lines
This is a table property allows user to ignore lines in the input document satisfying the provided condition
Function
This is an LOV property having functions to be applied on the data line to be ignored.
Values:
startsWith: To ignore a line starting with specific value
endsWith: To ignore a line ending with specific value
contains: To ignore a line containing specific value
regex: To ignore a line with data in provided regular expression format
Default value: startsWith
Value
The value to be used for the function. If this value is empty then that property is ignored.
Format: String
Snap Execution
Select one of the three modes in which the Snap executes. Available options are:
Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.
Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.
Disabled: Disables the Snap and all Snaps that are downstream from it.
Default Value: Execute only Example: Validate & Execute
Examples
Fixing a Pipeline Containing Incorrect Field Configurations
In this sample pipeline, information is brought in through a Constant Snap, then sent to the Fixed Width Parser.
In the Constant Snap, supply the following information in the Content field:
81888888888800002ST06/20/2014JOHN SMITH 05-24-2012A
82777777777700003ST06/20/2014MARY SMITH 05-24-2012A
81888888888800002ST06/20/2014JOHN SMITH 05-24-2012A
82777777777700003ST06/20/2014MARY SMITH 05-24-2012A
81888888888800002ST06/20/2014JOHN SMITH 05-24-2012A
82777777777700003ST06/20/2014MARY SMITH 05-24-2012A
81888888888800002ST06/20/2014JOHN SMITH 05-24-2012A
82777777777700003ST06/20/2014MARY SMITH 05-24-2012A
81888888888800002ST06/20/2014JOHN SMITH 05-24-2012A
82777777777700003ST06/20/2014MARY SMITH 05-24-2012A
FIXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX00000010
For reference, most of the rows are 70 characters long. Note that the last row of data intentionally does not follow the same format.
In the Fixed Width Parser add the following Field configurations, containing incorrect field positions to demonstrate error handling.
Column Names
Start position
End Position
Trim Column Data
COLA
1
12
COLB
13
20
DATECOL
21
30
NAME
31
60
Selected
DATE2
61
70
A
71
72
When you save the pipeline, you will see the error: Failure: The input data format is not supported.
To help determine the error, set the error view on the Fixed Width Parser to route error data to error view. Now when you save the pipeline, data is written to the error view. If you look at the schema preview, you'll see the following.
Note that:
DATECOL contains the first character of NAME
NAME contains the first character of DATE2
DATE2 contains the character for A, which does not exist as a column.
The last row of data only fills the first three columns (expected because its data format).
Now, update the Field configurations as follows:
Set DATECOL end at 29.
Set NAME to start at 30 and end at 59.
Set DATE2 to start at 60 and end at 69.
Set A to start and end at 70.
Now when you save the pipeline, data preview is available at both the output view and error view.
The output view contains the first 10 rows of data correctly formatted.
The error view now conatins only the last row of data that does not match the Field configuration settings.
Using the regex Function
To use regex to ignore lines with the digits "17" in the 16 & 17th positions in the lines, select regex in the function, and apply a value of:
\w{15}17.* (including the starting \ and ending *)
To Ignore all those except those with 17 in that position, we use:
^\w{15}17.*
To ignore all which had 17 or 99:
\w{15}(17|99).*
Click to view/expand
Release
Snap Pack Version
Date
Type
Updates
August 2024
438patches28073
Latest
Fixed an issue with the JSON Generator and XML Generator Snaps that caused unexpected output displaying '__at__' and '__h__' instead of '@' and '-' respectively because the Snap could not update them to their original values after the Velocity library upgrade.
August 2024
438patches27959
Latest
Fixed an issue with the Sort where the Snap could not sort files larger than 52 MB. This fix applies to Join Snap also.
August 2024
main27765
Stable
Upgraded the org.json.json library from v20090211 to v20240303, which is fully backward compatible.
May 2024
437patches26643
Latest
Fixed an issue with the Sort Snap that displayed an error when estimating the size of the input document provided by the upstream S3 Browser Snap.
Fixed an issue with the Parquet Formatter Snap that was unable to route errors to the error view.
May 2024
437patches26453
Latest
Added expression support to the Skip lines field in the CSV Parser Snap to enable passing pipeline parameters and upstream values.
Fixed an issue with the XML Parser Snap that caused an error when using the Splitter option in the Snap settings.
May 2024
main26341
Stable
Added Parquet Parser and Parquet Formatter Snaps to the Transform Snap Pack:
Parquet Parser: Reads the binary Parquet data and writes document data to the output.
Parquet Formatter: Reads the document data and writes it to the output in binary Parquet format.
Enhanced the JSON Splitter Snap to capture metadata and lineage information from the input document.
February 2024
436patches25564
Latest
Fixed an issue with theJSON FormatterSnap that generated incorrect schema.
February 2024
436patches25292
Latest
Fixed an out-of-memory error issue with the Aggregate Snap. This Snap no longer performs the presort for the input documents.
If the input documents areunsorted and GROUP-BY fields are used, you must use the Sort Snap upstream of the Aggregate Snap to presort the input document stream and set the Sorted stream field Ascending or Descending to prevent the out-of-memory error. However, if the total size of input documents is expected to be relatively small compared to the available memory, then Sort Snap is not required upstream.
Updated and certified against the current SnapLogic Platform release.
November 2023
435patches24802
Latest
Fixed an issue with the Excel Parser Snap that caused a null pointer exception when the input data was an Excel file that did not contain a StylesTable.
November 2023
435patches24481
Latest
Fixed an issue with the Aggregate Snap where the Snap was unable to produce the desired number of output documents when the input was unsorted and the GROUP-BY fields field set was used.
November 2023
435patches24094
Latest
Fixed a deserialization issue for a unique function in the Aggregate Snap.
November 2023
main23721
Stable
Updated and certified against the current SnapLogic Platform release.
August 2023
434patches23076
Latest
Fixed an issue with the Binary to Document Snap where an empty input document with Ignore Empty Stream selected caused the Snap to stop executing.
August 2023
434patches23034
Latest
Fixed an issue with the Transform Snap Pack that caused an error when the input file was a binary JSON file that contained a string value of more than 20,000,000 characters.
Fixed a memory issue with the Aggregate Snap that occurred when using GROUP-BYfields.
August 2023
434patches22705
Latest
Fixed an issue with the JSON Splitter Snap that caused the pipeline to terminate with excessive memory usage on the Snaplex node after the 4.33 GA upgrade. The Snap now consumes less memory.
August 2023
main22460
Stable
Updated and certified against the current SnapLogic Platform release.
May 2023
433patches22431
Latest
Fixed an issue with the Excel Multi Sheet Formatter Snap that caused it to produce binary output data when there was no input document and Ignore empty stream was selected.
Introduced the following new Snaps:
GeoJSON Parser: Parses geospatial data from binary data input and outputs the contents as a GeoJSON document downstream.
WKT Parser: Parses geospatial data from binary data input and outputs the contents as a WKT (Well Known Text) document downstream.
May 2023
433patches21779
Latest
The Decrypt Field and Encrypt Field Snaps now support CTR (Counter mode) for the AES (Advanced Encryption Standard) block cipher algorithm.
May 2023
433patches21586
Latest
The Decrypt Field Snap now supports the decryption of various encrypted fields on providing a valid decryption key.
Fixed an issue with the AutoPrep Snap where dates could potentially be rendered in a currency format because currency format options were displayed for the DOB column.
May 2023
433patches21196
Latest
Enhanced the In-Memory Lookup Snap with the following new fields to improve memory management and help reduce the possibility of out-of-memory failures:
Minimum memory (MB)
Out-of-memory timeout (minutes)
These new fields replace the Maximum memory % field.
May 2023
main21015
Stable
Upgraded with the latest SnapLogic Platform release.
February 2023
432patches20535
Latest
Fixed an issue with the Encrypt Field Snap, where the Snap failed to support an RSA public key to encrypt a message or field. Now the Snap supports the RSA public key to encrypt the message.
The Pipeline Execution Statistics of the Join Snap now has a status message that displays the parameters - Free disk space, Available memory, and Average document size.
The internal sort buffer size is reduced to a minimum of 10MB when the available memory in the node becomes lower than 500MB to avoid the out-of-memory crash.
The internal sort buffer size is restored to its original size when the available memory becomes larger than 2GB.
We have improved the readability of the error message for the out of disk space on node error. The updated error message now provides clearer information and guidance for users, as shown below: Reason: Insufficient free disk space available to stage sort data into temporary files. Resolution: Increase the amount of free disk space and try again.
February 2023
432patches20250
Latest
Fixed an issue with the JSON Splitter Snap that was causing errors when using multiple repeated dots in the JSON Path.
The Sort Snap includes the following improvements:
The Maximum memory % field is revised to Maximum memory.
The Maximum memory unit (new dropdown list) enables you to choose a unit, percentage (%), or MB for better memory control.
February 2023
432patches20151
Stable/Latest
Fixed an issue that occurred with the JSON Splitter Snap when used in an Ultra pipeline. The request was acknowledged before it was processed by the downstream Snaps, which caused a 400 Bad Request response.
February 2023
432patches20062
Stable/Latest
Fixed the behavior of the JSON Splitter Snap for some use cases where its behavior was not backward compatible with the 4.31 GA version. These cases involved certain uses of either the Include scalar parents feature or the Include Paths feature.
February 2023
432patches19974
Stable/Latest
Fixed the "Json Splitter expects a list" error by restoring the JSON Splitter Snap's previous behavior of handling the case where the document element referenced by the JSON Path to Split field is an object instead of a list or array.
Review your pipelines where this error occurred to check your assumptions about the input to the JSON Splitter and whether the value referenced by the JSON Path to Split field will always be a list. If the input is provided by an XML-based or SOAP-based Snap like the Workday or NetSuite Snaps, a result set or child collection that’s an array when there's more than one result or child will be an object when there's only one result or child. In these cases, we recommend using a Mapper Snap and the sl.ensureArray() function to ensure that the value being split by the JSON Splitter is always an array (even for the single element cases).
February 2023
432patches19918
Stable/Latest
Fixed an issue with the CSV Formatter Snap where the Unicode character delimiters using [0-9a-f] did not work.
Fixed an issue with the JSON Splitter Snap that was generating null values for empty input data.
February 2023
main19844
Stable
Upgraded with the latest SnapLogic Platform release.
The Transform Join Snap now doesn’t fail with the Null Pointer Exception when you configure the Sorted streams field with Ascending.
November 2022
431patches19359
Latest
The JSON Splitter Snap includes memory improvements and a new Exclude List from Output Documents checkbox. This checkbox enables you to prevent the list that is split from getting included in output documents, and this also improves memory usage.
The Mapper Snap now has a Sorted checkbox in the Input Schema and Target Schema panels, which allows you to sort the input and target schemas. When unchecked, the Snap unsorts the input and the target schema.
October 2022
430patches18800
Latest
The Sort and Join Snaps now have improved memory management, allowing used memory to be released when the Snap stops processing.
The AutoPrep feature now includes the following new transformation options that enable you to:
Change the field data type from Data type menu.
Format dates and date Strings.
Rename a field.
Mask sensitive data using an MD5, SHA-1, SHA-256, or SHA-512 algorithm.
The data in the Preview data pane format is easier to read and the buttons have been changed to improve usability.
The CSV Parser Snap now parses data with empty values in the columns when using a multi-character delimiter.
September 2022
430patches18119
Latest
The Transcoder Snap used in a low-latency feed Ultra Pipeline now acknowledges the requests correctly.
September 2022
430patches17802
Latest
The Avro Parser Snap now displays the decimal number correctly in the output view if the column’s logical type is defined as a decimal.
September 2022
430patches17737
Stable/Latest
AutoPrep now enables you to handle empty or null values.
September 2022
430patches17643
Latest
The CSV Parser and CSV Formatter Snaps now support either \ or \\ for a single backslash delimiter which were failing earlier.
September 2022
430patches17589
Latest
The CSV Formatter Snap does not hang when running in specific situations involving multibyte characters in a long field. If you notice the CSV Formatter Snap is hung, we recommend that you update to the 430patches17589 version and restart your Snaplex.
August 2022
main17386
Stable
New Snap Application: The Auto Prep Snap provides a data preparation application where you can flatten structured data, include and exclude data fields, and change data types before forwarding the data for further processing.
The Hide whitespace option in the CSV Generator and JSON Generator Snaps allows you to hide the rendering of whitespace as symbols (dot or underscore) in the output that you may have in the CSV or JSON input documents.
The Render whitespace checkbox in the Mapper Snap enables or disables the rendering of whitespace in the input document. When a value in the Expression field has blank spaces (leading, trailing, or spaces in the middle of a string), the spaces are rendered as symbols (dot “.” or underscore “_”) in the output on selecting this checkbox.
The Excel Parser Snap includes the Custom Locale dropdown list that allows selecting a user-defined locale for formatting numbers..
The Selected fields inthe Pivot Snap allow you to define fields to be unpivoted so that the remaining fields are automatically pivoted.
The XML Generator Snap includes examples on how to escape single (') and double quotes (“) when used with elements or attributes.
4.29 Patch
429patches16990
Latest
Fixed an issue with the Aggregate Snap where the Snap failed to validate (after first successful validation) while using a field that may contain a date for MIN and MAX functions. The Snap now supports DATE-type fields.
Enhanced the Pivot Snap with the Treat selected fields as static checkbox that enables the Snap to treat the selected fields as static to preserve the structure of the selected fields while all other fields are pivoted.
4.29 Patch
429patches16923
Latest
Enhanced the CSV Formatter and CSV Parser Snaps to support multiple characters or strings as delimiters.
Fixed an issue with the Join Snap where the Snap displayed an incorrect error if the Left path or Right path fields were expression-enabled or if you have specified properties other than the field name and irrespective of whether the Sorted streams field is Unsorted or not. For example, "$first + '2' . Now, the Snap runs properly if the Left path or Right path was expression enabled and you have specified properties other than the field name and the Sorted streams field is Sorted or Unsorted. The Snap now displays a proper error that is more informative, in case there is a problem while executing this Snap.
Fixed an issue with CSV Parser Snap where the Snap failed when more than six characters are used as delimiters. Now, the Snap executes properly when you use more than six characters as delimiters.
4.29 Patch
429patches16521
Latest
Removed the default value for the Root element field in the XML Formatter Snap.
Fixed an issue with the Transcoder Snap where the Input character-set field was not displaying the suggestions properly.
4.29 Patch
429patches16026
Latest
Enhanced the Excel Parser Snap with the Custom Locale dropdown list that allows you to select a user-defined locale to format numbers as per the selected locale.
4.29
main15993
Stable
Fixed an issue with CSV Formatter and CSV Generator Snaps that displayed an incorrect error when one or more header values in the CSV file were missing. This error is now routed to the error view.
Fixed an issue with the CSV Parser Snap, where the Snap stopped indefinitely while processing certain inputs.
Fixed an issue with the CSV Formatter Snap, where the escape character selected was not used.
Updated the CSV library used in the CSV Generator, CSV Parser, and CSV Formatter Snaps. This library update enforces values for each column in a parsed CSV header.
Enhanced the XML FormatterSnap with theOutput Character Setfield. You can select the character set you want for your binary output. For example, UTF-8 or ISO860.
4.28 patch
428patches14370
Latest
Fixed an issue with the XML Generator Snap, where the Snap failed with an invalid UTF-8 sequence error when running on the Windows Snaplex.
4.28
main14627
Stable
Upgraded with the latest SnapLogic Platform release.
4.27 Patch
427patches12966
Latest
Enhanced the Avro Formatter Snap to display meaningful error message while processing invalid and null values from the input.
4.27
main12989
Stable and Latest
Fixed an issue in the Group by Fields Snap that caused the Snap to abort with an error when executed with zero input documents.
4.27
main12833
Stable
Added dynamic memory management to theGroup By FieldsSnap, which can activate with the following settings:
Memory Sensitivity: The Snap's response to the memory changes.
Min Part Size: The minimum part size that you want the Snap to split larger groups into multiple parts.
Enhanced the JSON Formatter, JSON Generator, and JSON Parser Snaps with the Support Type Extensionscheckbox. On selecting this checkbox, the Snap supports SnapLogic-specific syntax for representing non-standard data types such as byte arrays and date objects in JSON.
Convert numeric string to number: Sets the behavior of the Snap when a value in the input document is a numeric string. This checkbox supports the conversion of numeric values to string values.
Translate date and time types: Sets the behavior of the Snap when a value in the input document is a date and datetime type. This checkbox supports translating date and time types to Excel date numeric type with date format style.
4.26 Patch
426patches12086
Latest
Fixed an issue with theJoinSnap, where it exhausted the memory while buffering millions of objects.
4.26 Patch
426patches11780
Latest
Fixed an issue in the XML Formatter Snap, where theMap input to first repeating element in XSD checkbox is selected, while no XSD is specified for mapping the input.
4.26 Patch
426patches11725
Latest
Fixed an issue with the JoinSnap where the upstream document flow of the right view is blocked by the left view, which hung the Join Snap.
4.26
main11181
Stable
Enhanced the JSON Splitter Snap with a new field Show Null Values for Include Paths that enables the Snap to show key-value entries of the null values for the objects added to the Include Paths field in the output document.
Enhanced the Join Snap with a new field Available Memory Threshold (%) that enables the Snap to keep all the Right-input view documents with the same join-path values in memory until the join operation is done for the specific join-path values.
4.25 Patch
425patches10663
Latest
Fixed an issue in the CSV Formatter Snap,where even if theIgnore empty streamcheckbox is not enabled, the Snap did not produce an empty binary stream output in case there is no input document.
4.25 Patch
425patches10152
Latest
Replaced the Strict XSD output field with Map input to first repeating element in XSD field in the XML Formatter Snap. If selected, the Snap ignores the root element from the XSD file.
Enhanced the CSV Parser Snap with a new checkbox Preserve Surrounding Spaces that enables you to preserve the surrounding spaces for the values that are non-quoted.
4.25 Patch
425patches9815
Latest
Fixed a ClassCastExceptionerror in the Avro Parser Snap and handling of themap, fixed, enum, and bytes data types in theAvro Formatter and Avro Parser Snaps.
4.25 Patch
425patches9749
Latest
Enhanced XML Parser Snap to recognize input headers when defining inbound schema.
4.25 Patch
425patches9638
-
Latest
Reverts the Join Snap to the 4.24 release behavior. This is in response to an issue encountered in the Join Snap in the 4.25 release version (main9554), which can result in incorrect outputs from all Join Types. 425patches9638 is the default version for both stable and latest Transform Snap Pack versions for orgs that are on the 4.25 release version. No action is required by customers to receive this update and no impact is anticipated.
4.25
main9554
Stable
Enhanced the Group By N Snap with the following settings:
Memory Sensitivity: The Snap'sresponse to the memory changes.
Group Size:The maximum number of input documents to be grouped into a single output document.
Min Group Size: The minimum number of input documents to be grouped into a single output document.
4.24 Patch
424patches8938
Latest
Fixed the timestamp issue in the JSON Formatter Snap that changed the time zone offset to include colon by default after upgrading to 4.24.
Fixed the null pointer exception at runtime in the Fixed Width Parser Snap by setting the Trim column data field to false for empty columns.
Enhances the Group By N Snap to process the records efficiently by adding the FlushTimeout field that enables the Snap to flush a partial group of records if the time specified in this field passes with no new input.
4.24
main8556
Stable
Upgraded with the latest SnapLogic Platform release.
4.23 Patch
423patches7958
Latest
Fixes an issue in theJSON Splitter Snap by logging an error when the matcher does not find a pattern.
4.23 Patch
423patches7898
Latest
Fixes an issue in the In-Memory Lookup Snap to correctly handle the Join path in the format like $['join path'].
Fixes an issue in the XSLT Snap, wherein null binary header values are now converted to blank strings when injecting them as parameters in the stylesheet.
4.23 Patch
423patches7792
Latest
Fixes an issue in the XML Formatter Snap when it fails to convert input JSON data, with JSON property having a special character as its prefix, to XML format by sorting the elements.
4.23 Patch
423patches7753
Latest
Fixes an issue with the JSON Splitter Snap's behavior in Ultra Pipelines that prevents processed requests to be acknowledged and removed from the FeedMaster queue, resulting in retries of requests that are already processed successfully.
4.23
main7430
Stable
Enhances the JSON FormatterSnap to render groups output from upstream (Group by) Snaps with one document per group and a new line per group element.You can now select the Format each documentandJSON lines check boxes simultaneously.
4.22 Patch
422patches6395
Latest
Fixes the JSON Splitter Snap data corruption issue by copying the data in JSON Splitter Snap before sending it to other downstream Snaps.
4.22 Patch
422patches6505
Latest
Fixes the XML Generator Snap issue reflecting empty tags and extra space by removing the extra space in the XML output.
4.22
main6403
Latest
Upgraded with the latest SnapLogic Platform release.
4.21 Patch
421patches5901
Latest
Enhances the JSON Generator Snap to include pass-through functionality where the Snap embeds the upstream input document under the original field of the output document along with other records.
4.21 Patch
421patches5848
Latest
Fixes the Sort Snap that fails while performing sorting, displaying a NoClassDefFoundError.
Fixes the Excel Formatter Snap that fails to create an empty worksheet when you do not select the Ignore Empty Stream checkbox.
4.21
snapsmrc542
-
Stable
Adds support in the Mapper Snap to display schemas with complex nesting. For example, if Snaps downstream from the
4.20 Patch
transform8792
-
Latest
Resolves the NoClassDefFoundError in the Join Snap on Windows Snaplex instances.
4.20 Patch
transform8788
-
Latest
Resolves the NullPointerException in the Join Snap on Windows Snaplex instances.
4.20 Patch
transform8760
-
Latest
Fixes an issue with the CSV Parser Snap wherein the Snap fails when it is configured in the new Form UI if the Contains headers and Validate headers fields are not selected but empty rows exist in the Column names field.
Fixes the JSON Formatter Snap to filter the input data based on the value specified in the Content field in the Snap settings when the JSON lines field is also selected. Previously, the Snap was writing the entire input data to the output file.
The JSON Formatter Snap output now includes only those fields from the input file that are specified in the Content field under Settings.
If your Pipelines use the JSON Formatter Snap with the JSON lines field selected, they may fail to execute correctlyif the Content fie