MongoDB - Execute

In this article

Overview

You can use this Snap to execute all Data Manipulation Language (DML) and Data Definition Language (DDL) commands on the MongoDB database. It supports CRUD operations, index search, and aggregate functions.

mongodb-execute-overview.png

Snap Type

The MongoDB - Execute Snap is a Write-type Snap. 

Prerequisites

None.

Support for Ultra Pipelines

Works in Ultra Pipelines

Limitations

  • The MongoDB driver has limited support for inline or multiline comments in a command.

  • This Snap supports only the JSON format and specific date-type formats because it uses a Mongo template. A few of the supported date-type formats are:

    • { "$date": "2020-09-01T00:00:00Z" }

    • ISODate("2020-01-15T00:00:00Z")

Known Issues

None.

Supported Versions

The Mongo-DB Execute Snap is tested against v4.2.24.

Snap Views

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Type

Format

Number of Views

Examples of Upstream and Downstream Snaps

Description

Input 

Document

 

  • Min: 0

  • Max: 1

  • Mapper

  • JSON Generator

  • JSON Formatter

Requires the command to be executed on the specified database.

Output

Document

 

  • Min: 0

  • Max: 1

  • Mapper

  • Copy

  • File Writer

The original document that was used to create the statement will be output with the status of the statement executed.

Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the pipeline by choosing one of the following options from the When errors occur list under the Views tab:

  • Stop Pipeline Execution: Stops the current pipeline execution if the Snap encounters an error.

  • Discard Error Data and Continue: Ignores the error, discards that record, and continues with the remaining records.

  • Route Error Data to Error View: Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap Settings

  • Asterisk ( * ): Indicates a mandatory field.

  • Suggestion icon (): Indicates a list that is dynamically populated based on the configuration.

  • Expression icon ( ): Indicates the value is an expression (if enabled) or a static value (if disabled). Learn more about Using Expressions in SnapLogic.

  • Add icon ( ): Indicates that you can add fields in the field set.

  • Remove icon ( ): Indicates that you can remove fields from the field set.

  • Upload icon ( ): Indicates that you can upload files.

Field Name

Field Type

Description

Field Name

Field Type

Description

Label*

 

Default ValueMongoDB - Execute
ExampleExecute Lookup for Customers

String

Enter a unique name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

Database name

 

Default Value: N/A
Example: mydatabase

String/Suggestion

Specify the database name where the query is executed.

  • If you do not specify a database name, the database configured in the MongoDB Account is used for Snap processing.

  • If you provide a database name that is not available in the dropdown list, MongoDB automatically creates a new database.

Command*

 

Default Value: N/A
Example: { find: "collectionName"}

String/Expression

Specify the MongoDB command to execute on the MongoDB instance using the runCommand() operation. Learn more.

The runCommand() method enables you to execute MongoDB commands that might not be readily available through the standard MongoDB query language, such as CRUD operations. The syntax of the runCommand() method in MongoDB is as follows:

db.runCommand( { <command>: <value> } )

  • <command>: The command or operation you want to execute.

  • <value>: Optional. Additional parameters or options specific to the command being executed.

Timezone Offset

Use the field set to configure the timezone to apply to date fields. The default value is to read the date in UTC (00:00 offset). A value of -2 for hours and 30 for minutes produces the offset -2.30

Hours Offset

Default value0
Example -7

Integer

Specify the hours to offset for the timezone. MongoDB returns dates in the UTC timezone. If another timezone is desired, an hour's offset can be used. If a negative value is used for the hours, then the timezone offset will be in the format -7:00.

 

Minutes Offset

 

Default value0
Example: 1

Integer

Specify the minutes to use for the time zone offset in the date returned by MongoDB.

 

Number of retries

Default Value0
Example: 4

Integer

Specify the maximum number of retry attempts the Snap must make in case there is a network failure and is unable to read the target file. The request is terminated if the attempts do not result in a response.

  • If the Number of retries value is set to 0 (the default value), the retry option is disabled, and the Snap does not initiate a retry. The pipeline will not attempt to retry the operation in case of a failure—any failure encountered during the database operation will immediately result in the pipeline failing without any retry attempts to recover from the errors.

  • If the Number of retries value is greater than 0, the Snap initiates a download of the target file into a temporary local file. If any error occurs during the download, the Snap waits for the time specified in the Retry interval and then attempts to download the file again from the beginning. After the download is successful, the Snap streams the data from the temporary file to the downstream pipeline. All temporary local files are deleted when they are no longer needed.

Retry interval (seconds)

Default Value: 1
Example: 5

Integer

Specify the time interval between two retry requests.

Snap Execution

Default ValueExecute only
Example: Validate & Execute

Dropdown list

Select one of the following three modes in which the Snap executes:

  • Validate & Execute: Performs limited execution of the Snap and generates a data preview during pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during pipeline runtime.

  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.

  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Use the runCommand()

You can use the runCommand() to perform the following:

  • Administrative tasks: Check server status, do database profiling, replication-related operations, user management, and more.

  • Access internal commands: Access and execute the internal commands.

  • Diagnostic Commands: Execute diagnostic commands to gather information about the database, collections, indexes, or the server status.

Examples:

  • Checking server status:
    db.runCommand({ serverStatus: 1 })

  • Listing databases:
    db.runCommand({ listDatabases: 1 })

  • Checking the current operation in progress:
    db.runCommand({ currentOp: 1 })

  • Document insertion into a collection:
    db.runCommand( {
    insert: "Test1",
    documents: [ { _id: 7, title: "Mary test", body: "Execute snap for MongoDB snappack" } ]
    } )

  • Document updation into a collection:

    db.runCommand({ update: "Test1", updates: [ { q: { title: "abc123" }, u: { $set: { body: "Toy Story" }, $inc: { points: 1 } } } ], ordered: false, writeConcern: { w: "majority", wtimeout: 5000 } })
  • Document deletion from collection:

    db.runCommand({ delete: "Test1", deletes: [ { q: { title: "Mary test" }, limit: 1 } ], ordered: false, writeConcern: { w: "majority", wtimeout: 5000 } })
  • Creating indexes for a collection:

    db.runCommand( { createIndexes: "collectionName", indexes: [ { key: { field1: 1, field2: -1 }, name: "indexName", unique: true }, { ... }, { ... } ], writeConcern: { w: "majority" }, commitQuorum: 1, comment: "Optional comment" } )

Example

Use Common Commands in MongoDB

This example demonstrates how to use the common (Create, Insert, and Find) commands in MongoDB.

ex-pipeline-overview.png

Download this pipeline.

Configure the pipeline with three MongoDB Execute Snaps to perform the create, insert, and find operations. Each Snap in this pipeline performs the following operations:

  • Create employee details collection (collection indicates a table in MongoDB)

  • Insert the employee data into the collection

  • Retrieve the employee data

Step 1: Configure the Execute Snap with the create command to create a collection named employees_data with id, name, and salary fields.

Step 2: Configure the Execute Snap with the insert command to insert three records of employees (id:1, id:2, id:3) with their salary details.

Step 3: Configure the Execute Snap with the findcommand to retrieve the employee details.

On validation, you can view the inserted records of the employees in the employee_data collection.

 

  File Modified

File Example-MongoDB-Execute-Create-Insert-Find.slp

Apr 30, 2024 by Kalpana Malladi