On this Page
Snap type: | Transform | |||||||
---|---|---|---|---|---|---|---|---|
Description: | This Snap eliminates duplicate documents in a document stream, such as duplicate rows in a CSV file. To understand the functionality of the Unique Snap, it is important to understand that SnapLogic is streaming data, processing "one document at a time" in each Snap.
If the data being passed was in the following order: The result would be exactly the same as the input, as the duplicate rows are not adjacent in the flow and therefore, are not identified.
| |||||||
Prerequisites: | [None] | |||||||
Support and limitations: | Works in Ultra Pipelines. | |||||||
Account: | Accounts are not used with this Snap. | |||||||
Views: |
| |||||||
Settings | ||||||||
Label | Required. The name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline. | |||||||
A simple pipeline for this Snap would include:
File Reader + CSV Parser + Unique + CSV Formatter + File Writer