Categorical to Numeric

On this Page

Overview

This is a Transform type Snap that converts categorical fields into numeric fields. There are two available encoding options: Integer Encoding and One Hot Encoding.

Input and Output

Expected input

  • First input view: A document containing categorical fields.
  • Second input view: A document containing data statistics computed by the Profile Snap.

Expected output: A document containing numeric field.

Expected upstream Snaps

  • First input view: A Snap that has a document output view. For example, Mapper, CSV Generator, or Categorical to Numeric.
  • Second input view: A sequence of File Reader and JSON Parser Snaps. These Snaps read the data statistics computed by the Profile Snap in another pipeline. It is required to select Value distribution in the Profile Snap and set Top value limit according to the number of unique values; or set to 0, which means unlimited.

Expected downstream Snaps: A Snap that has a document input view. For example, Mapper, JSON Formatter, or Type Converter.

Prerequisites

None.

Configuring Accounts

Accounts are not used with this Snap.

Configuring Views

Input

This Snap has exactly two document input views, the Data input view and the Profile input view.

OutputThis Snap has exactly one document output view.
ErrorThis Snap has at most one document error view.

Troubleshooting

None 

Limitations and Known Issues

None 

Modes

Snap Settings


LabelRequired. The name for the Snap. Modify this to be more specific, especially if there are more than one of the same Snap in the pipeline.
Policy

The preferences for fields and encoding methods. For each policy, select the input field with categorical values, the encoding method, and the result field.

Field

Required. The field that must be transformed. This is a suggestible property that lists all available fields in the input documents.

Default value: None.

Rule

Required. The type of transformation to be performed on the selected field. Two options are available:

  • Integer Encoding: Each unique value is mapped to an integer starting from 1.
  • One Hot Encoding: Each unique value results in a field. The value is 1 if it matches the field name; otherwise, it is 0.

Default value: Integer Encoding

Result field

Required. The result field that is used in the output map. If the variable in the Result field property is the same as the one in the Field property, the values are overwritten. If the Result field does not exist in the original input document, a new field is added.

Default value: None.

Snap Execution

Select one of the three modes in which the Snap executes. Available options are:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.
  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.
  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Examples


Assigning Team based on Position

This pipeline demonstrates how you can use the Categorical to Numeric Snap to assign team number for employees (numeric) based on their position (categorical).

Download this pipeline.

 Understanding the Pipeline

In this example, the CSV Generator Snap contains employee data with categorical fields. The Copy Snap duplicates the data flow and feeds it into the Categorical to Numeric and Profile Snaps. The Profile Snap computes data statistics and sends them to the Categorical to Numeric Snap. The Categorical to Numeric Snap then converts the position values (categorical) into team numbers (numeric), based on the selected encoding methods.

The CSV Generator Snap contains employee data as shown below.


The Categorical to Numeric Snap contains two policies that 
assign employees to groups based on their position. Integer Encoding maps each position into an integer ($team_numer). One Hot Encoding creates a field for each position ($team_instructional_designer and $team_technical_writer).


The Profile Snap computes the value distribution of the input data, which is required by Integer Encoding and One Hot Encoding.


As you can see, the value distribution of $position contains "technical_writer" and "instructional_designer". Integer Encoding assigns values 1 and 2 to these, while One Hot Encoding creates a field for each of them.


The preview output of Categorical to Numeric Snap is shown below. This Snap adds 3 fields to the output.

  • $team_number: This is the output of the first policy, where you use Integer Encoding to convert values in the $position field to numeric values.
  • $team_instructional_designer and $team_technical_writer: These two fields are created by the second Policy. The first field indicates the "instructional_designer" position, and the second field indicates the "technical_writer" position.

Additional Example

The following use case demonstrates a real-world scenario for using this Snap:


Downloads

Important steps to successfully reuse Pipelines

  1. Download and import the Pipeline into SnapLogic.
  2. Configure Snap accounts as applicable.
  3. Provide Pipeline parameters as applicable.

  File Modified

File Categorical_to_Numeric.slp

Nov 09, 2018 by Rakesh Chaudhary

Snap Pack History

 Click to view/expand

4.29 (main15993)

  •  Upgraded with the latest SnapLogic Platform release.

4.28 (main14627)

  • Enhanced the Type Converter Snap with the Fail safe upon execution checkbox. Select this checkbox to enable the Snap to convert data with valid data types, while ignoring invalid data types.

4.27 (427patches13730)

  • Enhanced the Type Converter Snap with the Fail safe upon execution checkbox. Select this checkbox to enable the Snap to ignore invalid data types and convert data with valid data types.

4.27 (427patches13948)

4.27 (main12833)

  • No updates made.

4.26 (main11181)

  • No updates made.

4.25 (425patches10994)

  • Fixed an issue when the Deduplicate Snap where the Snap breaks when running on a locale that does not format decimals with Period (.) character. 

4.25 (main9554)

  • No updates made.

4.24 (main8556)

  • No updates made.

4.23 (main7430)

  • No updates made.

4.22 (main6403)

  • No updates made.

4.21 (snapsmrc542)

  • Introduces the Mask Snap that enables you to hide sensitive information in your dataset before exporting the dataset for analytics or writing the dataset to a target file.
  • Enhances the Match Snap to add a new field, Match all, which matches one record from the first input with multiple records in the second input. Also, enhances the Comparator field in the Snap by adding one more option, Exact, which identifies and classifies a match as either an exact match or not a match at all.
  • Enhances the Deduplicate Snap to add a new field, Group ID, which includes the Group ID for each record in the output. Also, enhances the Comparator field in the Snap by adding one more option, Exact, which identifies and classifies a match as either an exact match or not a match at all.
  • Enhances the Sample Snap by adding a second output view which displays data that is not in the first output. Also, a new algorithm type, Linear Split, which enables you to split the dataset based on the pass-through percentage.

4.20 Patch mldatapreparation8771

  • Removes the unused jcc-optional dependency from the ML Data Preparation Snap Pack.

4.20 (snapsmrc535)

  • No updates made.

4.19 (snapsmrc528)

  • New Snap: Introducing the Deduplicate Snap. Use this Snap to remove duplicate records from input documents. When you use multiple matching criteria to deduplicate your data, it is evaluated using each criterion separately, and then aggregated to give the final result.

4.18 (snapsmrc523)

  • No updates made.

4.17 Patch ALL7402

  • Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers.

4.17 (snapsmrc515)

  • New Snap: Introducing the Feature Synthesis Snap, which automatically creates features out of multiple datasets that share a one-to-one or one-to-many relationship with each other.
  • New Snap: Introducing the Match Snap, which enables you to automatically identify matched records across datasets that do not have a common key field.
  • Added the Snap Execution field to all Standard-mode Snaps. In some Snaps, this field replaces the existing Execute during preview check box.

4.16 (snapsmrc508)

  • Added a new Snap, Principal Component Analysis, which enables you to perform principal component analysis (PCA) on numeric fields (columns) to reduce dimensions of the dataset.

4.15 (snapsmrc500)

  • New Snap Pack. Perform preparatory operations on datasets such as data type transformation, data cleanup, sampling, shuffling, and scaling. Snaps in this Snap Pack are: 
    • Categorical to Numeric
    • Clean Missing Values
    • Date Time Extractor
    • Numeric to Categorical
    • Sample
    • Scale
    • Shuffle
    • Type Converter