...
Using the Marketo Snap Pack, you can automate the deduplication of records by merging the lead source data with that contain the same email ID. The example pipelines below demonstrate how this can be done. The deduplication of the records can be done using a Smart List, Static List, and createdAt and updatedAt filters.
...
This method involves building the following three pipelines:three pipelines, a main pipeline, and two child pipelines, capable of fetching all the data from the day the data was created in the Marketo database.
Main pipeline
The main or first pipeline takes the start date and the end date of the period for which the deduplication must be done. The first pipeline executes the child pipeline (2nd pipeline) to segregate the number of days into batches of 31 days (Marketo has a limitation to fetch records for 31 days using Bulk Extract Snap).
...
The second pipeline executes another child pipeline (3rd pipeline) that fetches the Marketo records for the specified date range. This pipeline identifies the duplicate records by email ID and merges the Person Source field data to form a unique record for an email Id.
...