Feature Synthesis

On this Page

Overview

The Feature Synthesis Snap generates new features for a base or primary dataset by joining it with other datasets (reference datasets) linked together by common identifiers. Features are measurements of data points in a dataset. Using this Snap enables you to view statistical data of all related datasets in the base dataset itself. Categorical and numeric data will get different set of features. Some of the features generated are:

  • Mean (Numeric data)
  • Min (Numeric data)
  • Max (Numeric data)
  • Mode (Categorical data)
  • Unique (Categorical data)
  • Count (Generated at the base dataset level)


Input and Output

Expected input

  • First input: The base dataset.
  • Subsequent input(s): The reference dataset(s).

Expected output: The base dataset containing all the features generated based on the reference datasets. 

Expected upstream Snaps: Snaps that offer a document output. For example, MySQL - Select, or PostgreSQL - Select.

Expected downstream Snaps: A Snap that accepts documents. For example, Mapper, JSON Formatter, or AutoML.

Prerequisites

The base dataset must have one-to-many or one-to-one relationship with the reference dataset; and the reference datasets must have one-to-one or one-to-many relationship with each other.

Configuring Accounts

Accounts are not used with this Snap.

Configuring Views

Input

This Snap has at least two document input views.
OutputThis Snap has exactly one document output view.
ErrorThis Snap has at most one document error view.

Troubleshooting

None.

Limitations and Known Issues

None.

Modes

Snap Settings

LabelRequired. The name for the Snap. You can modify this to be specific, especially if you have more than one of the same Snap in your pipeline.
PolicySpecify the base and reference datasets and fields.
Base view

The input view to which the base dataset is connected.

Default value: input0 (name of the input view)

Base path

The field in the base dataset that is to be used as the base field. This has to be the common identifier for joining with the reference dataset.

Example: $customer_id

Default value: [None]

Ref view

View where the reference dataset is the input.

Default value: input0 (name of the input view)

Ref path

The field in the reference dataset that is to be used. This field must have the same values as in the base field.

Example: $customer_id

Default value: [None]

  Snap Execution

Specifies the execution type:

  • Validate & Execute: Performs limited execution of the Snap (up to 50 records) during Pipeline validation; performs full execution of the Snap (unlimited records) during Pipeline execution.
  • Execute only: Performs full execution of the Snap during Pipeline execution; does not execute the Snap during Pipeline validation.
  • Disabled: Disables the Snap and, by extension, its downstream Snaps.

Default value: Validate & Execute

Examples

Getting Customer Insights Based on Transaction Data

This example shows how the Feature Synthesis Snap is used to generate features based on transaction dataset and added to customer dataset.

Download this Pipeline.

 Understanding the Pipeline

The base dataset in this example is a collection of customer records. It has the following fields:

  • $customer_id
  • $firstname
  • $lastname
  • $create_time

The reference dataset is a collection of transactions made by the customers listed in the base dataset. It has the following fields:

  • $transaction_id
  • $customer_id
  • $source
  • $num_item
  • $total 

Both datasets are provided by the CSV Generator Snaps titled Customers and Transactions. These are passed through a Type Converter Snaps so that all data types are mapped correctly. This is required to enable the Feature Synthesis Snap to generate features accurately. 

A preview of the customer and transaction datasets that are output by the CSV Generator Snaps is as shown below:

 

The field $customer_id is common between both datasets. The Feature Synthesis Snap will use this field to join datasets and is configured as shown below:

 

The customer dataset is connected to the first input view (titled customer); so this view becomes the Base view. Similarly, the transaction dataset is connected to the second input view (titled transaction); so that becomes the Ref view.

Upon successful execution, the Feature Synthesis Snap generates features and adds them to the base dataset as shown below: 

The same output is shown in a JSON format to let you see the full list of features:

Download this Pipeline.

Downloads

  File Modified

File Feature_Synthesis.slp

Apr 19, 2019 by Mohammed Iqbal
 

Snap Pack History

 Click to view/expand

4.27 (main12833)

  • No updates made.

4.26 (main11181)

  • No updates made.

4.25 (425patches10994)

  • Fixed an issue when the Deduplicate Snap where the Snap breaks when running on a locale that does not format decimals with Period (.) character. 

4.25 (main9554)

  • No updates made.

4.24 (main8556)

  • No updates made.

4.23 (main7430)

  • No updates made.

4.22 (main6403)

  • No updates made.

4.21 (snapsmrc542)

  • Introduces the Mask Snap that enables you to hide sensitive information in your dataset before exporting the dataset for analytics or writing the dataset to a target file.
  • Enhances the Match Snap to add a new field, Match all, which matches one record from the first input with multiple records in the second input. Also, enhances the Comparator field in the Snap by adding one more option, Exact, which identifies and classifies a match as either an exact match or not a match at all.
  • Enhances the Deduplicate Snap to add a new field, Group ID, which includes the Group ID for each record in the output. Also, enhances the Comparator field in the Snap by adding one more option, Exact, which identifies and classifies a match as either an exact match or not a match at all.
  • Enhances the Sample Snap by adding a second output view which displays data that is not in the first output. Also, a new algorithm type, Linear Split, which enables you to split the dataset based on the pass-through percentage.

4.20 Patch mldatapreparation8771

  • Removes the unused jcc-optional dependency from the ML Data Preparation Snap Pack.

4.20 (snapsmrc535)

  • No updates made.

4.19 (snapsmrc528)

  • New Snap: Introducing the Deduplicate Snap. Use this Snap to remove duplicate records from input documents. When you use multiple matching criteria to deduplicate your data, it is evaluated using each criterion separately, and then aggregated to give the final result.

4.18 (snapsmrc523)

  • No updates made.

4.17 Patch ALL7402

  • Pushed automatic rebuild of the latest version of each Snap Pack to SnapLogic UAT and Elastic servers.

4.17 (snapsmrc515)

  • New Snap: Introducing the Feature Synthesis Snap, which automatically creates features out of multiple datasets that share a one-to-one or one-to-many relationship with each other.
  • New Snap: Introducing the Match Snap, which enables you to automatically identify matched records across datasets that do not have a common key field.
  • Added the Snap Execution field to all Standard-mode Snaps. In some Snaps, this field replaces the existing Execute during preview check box.

4.16 (snapsmrc508)

  • Added a new Snap, Principal Component Analysis, which enables you to perform principal component analysis (PCA) on numeric fields (columns) to reduce dimensions of the dataset.

4.15 (snapsmrc500)

  • New Snap Pack. Perform preparatory operations on datasets such as data type transformation, data cleanup, sampling, shuffling, and scaling. Snaps in this Snap Pack are: 
    • Categorical to Numeric
    • Clean Missing Values
    • Date Time Extractor
    • Numeric to Categorical
    • Sample
    • Scale
    • Shuffle
    • Type Converter