Bag of Words
On this Page
Overview
This Snap vectorizes sentences into a set of numeric fields. It takes two inputs:
- An array of tokenized words drawn from a set of input sentences. If your sentences are not already tokenized, use the Tokenizer Snap.
A document containing the most common words in the training set. You can generate this document using the Common Words Snap.
It then processes the data received and outputs a count frequency of the most common words in the text in each sentence.
For example: in the first output, say we have an array containing all the tokens in each sentence in a dataset; and the frequency of the 100 most common words in the second input. The Bag of Words Snap enables you to list out the number of times each of the 100 most common words appear in each input sentence.
Expected Input and Output
Expected input:
- First input view: An array of tokenized words drawn from a set of input sentences. If your sentences are not already tokenized, use the Tokenizer Snap.
- Second input view: A document containing the most common words in a given dataset, together with their frequency. You can generate this document using the Common Words Snap.
Expected output: A document containing the count frequency of the most common words in the text field.
Expected upstream Snaps:
- First input view: Any Snap that offers documents. For example: Tokenizer or Mapper.
- Second input view: Any Snap that offers documents. For example: Common Words, or a combination of File Reader and JSON Parser Snaps.
Expected downstream Snaps: Any Snap that accepts documents. For example: Cross Validator, Trainer, or a combination of JSON Formatter and File Writer Snaps.
Prerequisites
None.
Configuring Accounts
Accounts are not used with this Snap.
Configuring Views
Input | This Snap has exactly two document input views. |
---|---|
Output | This Snap has exactly one document output view. |
Error | This Snap has at most one document error view. |
Troubleshooting
None.
Limitations and Known Issues
None.
Modes
- Ultra Pipelines: Works in Ultra Pipelines.
Snap Settings
Label | Required. The name for the Snap. Modify this to be more specific, especially if there are more than one of the same Snap in the pipeline. |
---|---|
Token array field | Required. The field containing the array of tokens on which you want to apply the Bag of Words technique. This suggestible property is a JSONPath. Default value: None |
Snap Execution | Select one of the three modes in which the Snap executes. Available options are:
Default value: Validate & Execute Example: Execute only |
Examples
Identifying the Occurrence Frequency of the Top-100 Common Words in a Tokenized Input Dataset
In this example, we demonstrate how you can use the Bag of Words Snap to capture the frequency with which members of a set of common words appear in a dataset of tokenized sentences.
Download this pipeline.
Downloads
Important steps to successfully reuse Pipelines
- Download and import the pipeline into the SnapLogic application.
- Configure Snap accounts as applicable.
- Provide pipeline parameters as applicable.
Snap Pack History
Have feedback? Email documentation@snaplogic.com | Ask a question in the SnapLogic Community
© 2017-2024 SnapLogic, Inc.