On this Page
Table of Contents | ||||
---|---|---|---|---|
|
Snap type: | Transform | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Description: | This Snap eliminates duplicate documents in a document stream, such as duplicate rows in a CSV file. To understand the functionality of the Unique Snap, it is important to understand that SnapLogic is streaming data, processing "one document at a time" in each Snap.
If the data being passed was in the following order: The result would be exactly the same as the input, as the duplicate rows are not adjacent in the flow and therefore, are not identified.
| |||||||||||||
Prerequisites: | [None] | |||||||||||||
Support and limitations: | Works Does not work in Ultra Pipelines. | |||||||||||||
Account: | Accounts are not used with this Snap. | |||||||||||||
Views: |
| |||||||||||||
Settings | ||||||||||||||
Label | Required. The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline. | |||||||||||||
|
|
Multiexcerpt include macro name Temporary Files page Join
Example
A simple pipeline for this Snap would include:
File Reader + CSV Parser + Unique + CSV Formatter + File Writer
Insert excerpt | ||||||
---|---|---|---|---|---|---|
|