MongoDB - Atlas Vector Search

In this article

Overview

The MongoDB - Atlas Vector Search Snap enables you to perform advanced vector-based queries efficiently. This Snap is useful for Similarity searches, Approximate Nearest Neighbor (ANN) queries, and Range queries on vector data stored in MongoDB Atlas. Learn more about how to run vector search queries.

mongodb-atlas-vector-search-overview.png

Snap Type

The MongoDB - Atlas Vector Search Snap is a Read-type Snap.

Prerequisites

MongoDB Atlas account in Mongo database.

Support for Ultra Pipelines

Works in Ultra Pipelines

Limitations and Known Issues

None.

Snap Views

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input 

Document

 

  • Min: 1

  • Max: 1

  • Join

  • JSON Generator

Requires MongoDB aggregate command to perform vector search.

Output

Document

 

  • Min: 1

  • Max: 1

  • Mapper

  • Copy

Retrieved query results from the collection as specified in the settings.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the Pipeline by choosing one of the following options from the When errors occur list under the Views tab:

  • Stop Pipeline Execution: Stops the current Pipeline execution if the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

  • Asterisk ( * ): Indicates a mandatory field.

  • Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon ( ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon ( ): Indicates that you can add fields in the field set.

  • Remove icon ( ): Indicates that you can remove fields from the field set.

  • Upload icon ( ): Indicates that you can upload files.

Field Name

Field Type

Description

Field Name

Field Type

Description

Label*

 

Default ValueMongoDB - Atlas Vector Search
ExampleAtlas Vector Search

String

Specify the name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

 

Database name

 

Default Value: N/A
Example: chunking_strategy

String/Expression/Suggestion

Specify the database where the query is executed. If you do not specify, the database configured in the account is used.

Collection name*

 

Default Value: N/A
Example: chunk_metadata

String/Expression/Suggestion

Specify or select a MongoDB collection name to find indexes for vector search.

Search index*

 

Default Value: N/A
Example: vector_index

String/Expression

Specify the name of the search index to query.

Vector field*

 

Default Value: N/A
Example: embedding

String/Expression/Suggestion

Specify the name of the vector field that you want to search.

Number of candidates*

 

Maximum value: 10000
Minimum value: 0

Default Value100
Example: 50

Integer/Expression

Specify the number of candidates for vector search.

The Number of candidates value must be higher than the Limit value to increase the accuracy of the results.

Limit*

 

Maximum value: 10000
Minimum value: 0

Default Value4
Example: 10

Integer/Expression

Specify the number of results to return for each query.

Advanced settings

Configure the advanced settings as required.

Batch size*

 

Default Value0
Example10

Integer

Specify the number of documents to return per batch.

Timezone hours offset

 

Default Value0
Example12

Integer

Specify the timezone hour offset to apply to all DateTime fields.

The values for this field should be in the range of (-12)-14.

Timezone minutes offset

 

Default Value0
Example1

Integer

Specify the timezone minutes offset to apply to all DateTime fields.

Snap Execution

 

Default Value
Example: Validate & Execute

Dropdown list

Select one of the following three modes in which the Snap executes:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Troubleshooting

Error

Reason

Resolution

Error

Reason

Resolution

The vector search operation failed.

The specified vector might be invalid.

Specify a valid vector and try again.

Unable to process the input document.

The input document might be invalid.

Specify valid input and try again

The collection provided cannot be found.

The specified collection is not available.

Specify a valid collection and try again.

The index is not specified.

The index field is mandatory.

Specify an index and try again.

The input document does not have a 'vector' specified in the structure.

The input document must have a valid vector.

Add the vector to query the input document and try again.

Examples

Execute a vector search on a specified field

This example pipeline below demonstrates how to use MongoDB - Atlas Vector Search Snap to perform an ANN (approximate nearest neighbor) search on a vector in the specified field.

mongodb-atlas-vector-search-pipeline-overview.png

Download this pipeline.

Step 1: Configure the Edit JSON field in the JSON Generator Snap with the prompt “List movies.”

Step 2: Configure the Azure OpenAI Embedder Snap as follows to create a vector embedding for the list of movies.

Azure OpenAI Embedder configuration

Azure OpenAI Embedder output

Azure OpenAI Embedder configuration

Azure OpenAI Embedder output

 

Step 3: Configure the Mapper Snap to pass the vector embedding to the MongoDB - Atlas Vector Search Snap.

Step 4: Configure the MongoDB - Atlas Vector Search Snap to run a search query on 10,000 nearest neighbor records for a single-matched record with a similar plot_embedding vector field.

On validation, one record is populated with a similar plot-embedding vector field.

Downloads

 

  File Modified

File example-mongodb-avs.slp

May 02, 2024 by Lakshmi Manda