COBOL Copybook Parser
In this article
- 1 Overview
- 2 Snap Views
- 3 Snap Settings
- 4 Troubleshooting
- 5 Additional Information
- 6 Examples
- 6.1 Processing a Single COBOL Structure using EBCDIC Binary File input from a Variable Blocked File
- 6.2 Processing an ASCII Binary Input File with Fixed Length Records and Trailer Record
- 6.3 Processing an ASCII Input Binary File with Fixed Length Records
- 6.4 Processing ASCII Binary Input File with Data Mapped by a COBOL Structure with Array
- 6.5 Reading a Binary file From the Enterprise Ecosystem
- 7 Downloads
- 8 Snap Pack History
Overview
You can use the Copybook Snap for consuming one or more COBOL Copybooks and a data stream to produce a JSON formatted message containing the structured payload.
Prerequisites
None.
Support for Ultra Pipelines
Does not support Ultra Pipelines.
Limitations and Known Issues
None.
Snap Views
Type | Format | Number of Views | Examples of Upstream and Downstream Snaps | Description |
---|---|---|---|---|
Input | Document |
|
| The input to this Snap must be a flat binary file containing the data that must be mapped by the COBOL structure(s) provided as part of the configuration. |
Output | Document |
|
| The output is structured results in JSON format based on the data provided. |
Error | Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter while running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:
Learn more about Error handling in Pipelines. |
Snap Settings
Asterisk (*): Indicates a mandatory field.
Suggestion icon ( ): Indicates a list that is dynamically populated based on the configuration.
Expression icon (): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.
Add icon ( ): Indicates that you can add fields in the field set.
Remove icon (): Indicates that you can remove fields from the field set.
Field Name | Field Type | Description | |
---|---|---|---|
Label*
Default Value: Cobol Copybook Parser | String | Specify a unique name for the Snap. You can modify the default name especially if you have more than one of the same Snaps in your pipeline.
| |
Copybooks* | Use this field set to define the Copybooks to use. | ||
Copybook File*
Default Value: N/A | String/Expression | Specify the Copybook file which describes the data. Alternatively, click the upload icon to browse and upload the desired copybook file.
| |
Copybook Split*
Default Value: SPLIT_NONE | String/Expression | Select the option to split the copybook for processing. The available options are:
| |
Organization*
Default Value: IO_DEFAULT | String/Expression | Specify the type of file organisation for the data payload. Alternatively click the Suggestion icon to fetch a list of file organisation types and select the desired organisation.
| |
Dialect*
Default Value: FMT_MAINFRAME | String/Expression | Specify the COBOL dialect. The available options are:
The selected dialect determines how the parser deals with binary information in the input file. | |
Encoding*
Default Value: UTF8 | String/Expression | Specify the format for encoding the input data. Any supported Java encoding is available for use. Some common code pages to be used:
| |
Enclose character values in quote
Default Value: Selected | Checkbox | Select this checkbox to wrap all character fields in single quotes. Many applications need to differentiate between character and numeric fields. | |
Compatibility
| Checkbox | Select this checkbox to maintain backward compatibility with the original version of the Snap. However, we recommend that you update your pipelines to handle the new format immediately, as this checkbox will be removed in a future release. The key distinctions introduced with the Compatibility checkbox are:
| |
Truncate picX field spaces*
Default Value: BOTH | String/Expression | Specify or select the setting that determines the blanks that are stripped off from a character field. The available options are:
| |
Selection Criteria Default Value: N/A The following selection criteria would match: REC-TYPE = “P” or “T” to the DETAIL record. REC-IND = “X1” and REC-TYPE = “T” to the TRAILER record. <sc:selection
xmlns:sc="http://www.snaplogic.com/namespaces/cobol/selectionCriteria">
<recordSelection>
<name>DETAIL</name>
<fieldSelection>
<field name="REC-TYPE">P</field>
<fieldSelection>
<fieldSelection>
<field name="REC-TYPE">T</field>
</fieldSelection>
</recordSelection>
<recordSelection>
<name>TRAILER</name>
<fieldSelection>
<field name="REC-IND">X1</field>
<field name="REC-TYPE">T</field>
</fieldSelection>
</recordSelection>
</sc:selection> | Button | The provision of XML selection criteria offers complex selection criteria. This is provided using an XML structure as described below. Selection criteria must be provided in the Snap to match a data record to the copybook structure. The selection criteria is an XML-based configuration as follows: <sc:selection
xmlns:sc="http://www.snaplogic.com/namespaces/cobol/selectionCriteria">
<recordSelection>
<name>STRUCTURE_NAME</name>
<fieldSelection>
<field name="FIELD_NAME">VALUE</field>
</fieldSelection>
</recordSelection>
</sc:selection> Where:
It is possible to have AND along with OR conditions:
| |
Header Structure (for first record)
Default Value: N/A | String | Specify a Header record which is the first record in the file.
| |
Middle Structure Name (all records in between)
Default Value: N/A | String | Specify one or more Middle records which contain the main data.
| |
Trailer Structure Name(for last record)
Default Value: N/A | String | Specify a Footer record which is the last record in the file.
| |
Configure error processing | Configure the action the Snap must take when an error occurs. | ||
Set maximum acceptable number of errors | String | Set the maximum number of acceptable errors that can occur before the Snap terminates. Zero (0) indicates no limit on errors that can be issued. | |
Configure field error processing | Use this field set to configure the Snap's action when an error ocucurs for each field. | ||
Field Name | String/Expression/Suggestion | Specify or select the full name (this must include structure for uniqueness) of the COBOL field being configured. | |
Error processing configuration* | Expression/Dropdown list | Select or set the action the Snap must take when an error occurs when processing:
| |
Default value for field | String/Expression | This is the value that will be output for a field when the DefaultValue is configured for the field. | |
Debugging configuration | Configure the debugging properties. | ||
Name of only structure to output | String/Expression/Suggestion | Specify or select the name of the only structure for which data is to be sent to the output. Data for any other structure will be ignored and is not sent to the output. | |
Ranges of records to be output | Specify one or more ranges of records to be output. | ||
Low record for range | String | Specify the first record in the range to be output. | |
High record for range | String | Specify the last record in the range to be output. | |
Snap Execution
Default Value: Validate & Execute | Dropdown list list | Select one of the three modes in which the Snap executes:
|
Troubleshooting
Error | Reason | Resolution |
---|---|---|
| At least one copybook must be configured for the Copybook Parser Snap. | Configure one or more Copybooks for the Snap. |
| A Header, Middle or Trailer structure has been specified along with the selection criteria for records. | Change the configuration to specify either the Selection criteria XML _OR_ one of more of the Header, Middle or Trailer structure configuration parameters. |
| The file name provided for a Copybook cannot be found while trying to read the copybook file or files configured. | Review the configuration for the copy book files and ensure that they all exist and are accessible. |
| The protocol <protocol> provided for a copy book is not supported | Ensure the file names provided for each copybook use a supported protocol. |
| At least one copybook must be configured for the Copybook Parser Snap. | Configure one or more copybooks for the Snap. |
| The Snap encountered difficulties while processing the provided copybooks using the available configurations, resulting in unsuccessful processing attempts. | Review the message provided in “<error message>” and correct the error highlighted there. |
| The structure name <name> has been provided as a configuration parameter but the structure does not exist in the copybooks provided. | Ensure that you provide the accurate copybooks or specify the correct structure name referenced to identify a structure included in the provided copybooks. |
| The field was not found in the structure . | Correct the field name to reference the correct name in the named structure or the correct structure where the field name is defined. |
| A system error occurred combining multiple copybooks for processing. | Determine the problem based on the “<error message>” provided and correct the error. |
| Multiple debug ranges have been provided and one range overlaps with another range. | Identify the range based on the ranges in the message and correct the overlap. |
| The lower range is higher than the higher range. | Correct the range specification so that the low range is less than or equal to the high range. |
| An attempt to parse the XML provided in the selection criteria failed due to the error provided as “<error message>”. | Determine the problem with the XML based on the <error message> and correct the provided XML. |
| A field error configuration setting is invalid. | Correct the setting for the field configuration based on the suggestions provided in the configuration. |
| An I/O error occurred while processing the record number. | Search for system or other errors during the runtime and correct the issue. |
| The Snap processing has terminated because the number of errors encountered while processing the data is more than the maximum errors configured.
| Correct the errors that have occurred or increase the error threshold. You can increase the number of expected errors allowed by adjusting the maximum error configuration. By doing so, the Snap can complete its processing even if additional errors are encountered. |
Snap fails to process any data. | The File Organization is incorrect.
| The data must be manually inspected along with the COBOL structures to determine:
|
Snap processes some data but fails mid-way through the process. | This can be caused by any of the reasons where no data is processed so check these first. Invalid packed decimal data has been encountered which causes an exception processing an individual data record. |
|
Character data is invalid or contains single invalid data. | The encoding selected does not correctly reflect the encoding used for the input character data. | If the character data is totally corrupted, it suggests that you have selected an ASCII related encoding for EBCDIC input or vice versa. You can view a character field in the data to establish if the data is EBCDIC or ASCII encoded.
|
Unable to determine structure for field names. | The COBOL structures that you have provided cannot support field configuration. | Contact SnapLogic Customer Support for more details. |
Additional Information
The input to this Snap must be a flat binary file containing the data that must be mapped by the COBOL structure(s) provided as part of the configuration. This file will consist of data that have records with the following formats. For the records to be processed correctly, you must configure the correct record type processing for the records to be processed accurately:
Fixed Length records: Records with a consistent fixed length, the total length of the COBOL structure or structures configured for the Snap.
Records delimited by a carriage return/line feed (CR/LF) or just an LF: Records separated by CR/LF characters within the data. The data length between these CR/LFs may be fixed or variable and must be correctly represented by the length of the COBOL structure or structures configured for the Snap.
Variable blocked records: Variable length records created on IBM z/OS environments with an RDW before each record. This RDW includes the record length, including the length of the following data followed by the Record-Data.
Alternate variable blocked records: Records consist of a byte swapper record length excluding the length of the following data followed by the record data.
To ensure accurate processing of the data, you must configure the following types of data correctly:
Character data: This data should have a code page you must correctly specify in your configuration.
Binary data: These are binary numeric values with a specific endianness (the order in which the bits are transmitted over a communication channel) depending on the platform from which they have come.
Big Endian generally comes from older mainframe platforms (z Series, i Series, Solaris, etc.) and is where the high-order byte of the binary value is the most significant byte.
Little Endian generally comes from newer platforms (x86, x86-64, and so on) and is where the low-order byte of the binary value is the most significant byte.
Examples
Processing a Single COBOL Structure using EBCDIC Binary File input from a Variable Blocked File
This example demonstrates using a single record format described by a single COBOL Structure being processed from a binary file to a JSON formatted output for postprocessing by a downstream snap.
First, we configure the File Reader Snap to accept the data for processing. Provide the filename and leave the other settings with default values.
Next, we configure the Copybook COBOL Parser Snap with the Copybook structure file and the settings required to parse the input data.
Snap configuration settings:
The further options mean:
|
Validate the pipeline. You can preview the JSON formatted output:
Processing an ASCII Binary Input File with Fixed Length Records and Trailer Record
The following example demonstrates a standard file structure where the input file consists of a header as the first record, one or more detail records in the middle, with the last record being a trailer record.
First, we configure the File Reader Snap to accept the data for processing. In this case, we need to provide the filename and leave the other setting values defaulted.
Next, we must provide the Copybook COBOL Parser Snap with the Copybook structure file and settings required to parse the input data. To process this, the Snap must be configured to identify the structure to use for the first record (the header structure), the structure to use for the middle/detail records (the Middle Structure) and the structure to be used for the last record (the trailer structure).
Snap configuration settings:
The further options mean:
|
The Pipeline is now ready to process the binary file and produce a JSON-formatted output. Following pipeline validation, we can preview the JSON formatted output:
Processing an ASCII Input Binary File with Fixed Length Records
This example demonstrates another common file structure where different COBOL structures describe different record types in the same file. The Snap must be configured using an XML selection structure to understand what record structure to use when a specific field (normally at the start of the record) contains different information to indicate the record type.
First, we configure the File Reader Snap to accept the data for processing. Provide the filename and leave the other settings with default values.
Next, we must provide the Copybook COBOL Parser Snap with the Copybook structure file and settings required to parse the input data:
Snap configuration settings:
The further options mean:
|
To process this, the Snap must be provided with the required selection criteria to differentiate between record types. To provide this, click the ‘Selection Criteria’ button in the settings and provide the XML selection criteria. This can be entered directly or provided in an XML file using the import option:
This is interpreted based on the ‘<recordSelection>’ XML node:
COBOL Structure HEADER will be used to interpret records found with the HEADER-RECORD-TYPE field containing the value ‘9998’.
COBOL Structure AH00-ACCOUNT-HISTORY-BASE will be used to interpret records found with the RECORD-TYPE field containing the value ‘200A’.
COBOL Structure AH01-ACT-CYCLE-HIST-INFO will be used to interpret records found with the RECORD-TYPE field containing the value ‘200B’.
COBOL Structure AH31-BALANCE-HISTORY-INFO will be used to interpret records found with the RECORD-TYPE field containing the value ‘200C’.
COBOL Structure AH32-TERMS-BALANCE-INFO will be used to interpret records found with the RECORD-TYPE field containing the value ‘200D’.
COBOL Structure AH35-TLP-HIS will be used to interpret records found with the RECORD-TYPE field containing the value ‘200E’.
COBOL Structure AH02-ACT-HIST-DETAIL-INFO will be used to interpret records found with the RECORD-TYPE field containing the value ‘200F’.
COBOL Structure TRAILER will be used to interpret records found with the TRAILER-RECORD-TYPE field containing the value ‘9999.
The Pipeline is now ready to process the binary file and produce a JSON-formatted output. Following pipeline validation, we can preview the JSON formatted output:
Processing ASCII Binary Input File with Data Mapped by a COBOL Structure with Array
This example pipeline demonstrates how to process ASCII binary data with the data mapped by a COBOL structure with an array. Often COBOL Structures have an array that can have a maximum number of entities. To save space, this was often restricted in the data to a specific number of array entries using the DEPENDSON COBOL
structure.
First, we configure the File Reader Snap to read the data for processing. Provide the filename and leave the other settings with default values.
Next, we configure the Parser Snap with the Copybook structure file and settings required to parse the input data.
The above configuration options specify the following:
The further options mean:
|
Validate the Pipeline. You can view the JSON formatted output in the preview.
Reading a Binary file From the Enterprise Ecosystem
A standard implementation of a Copybook Snap Pipeline will look like this:
The following example Pipeline demonstrates how to read a binary file from the enterprise ecosystem with a sample definition.
Configure the File Reader Snap as follows:
Configure the COBOL Copybook Snap as follows. This Snap uses the output from the upstream Snap:
The resultant JSON output can then be used from the Copybook Snap and passed into the downstream Snaps to achieve the desired result with the data.
Downloads
Snap Pack History
Related Content
Have feedback? Email documentation@snaplogic.com | Ask a question in the SnapLogic Community
© 2017-2024 SnapLogic, Inc.