MongoDB - Group

On this Page

Overview

The MongoDB Group Snap groups input documents by specified expressions. The Snap then generates one output document for each distinct grouping. Each of the output documents contains an _id field which contains the distinct group by key.

Input and Output

Expected input

  • A document stream that contains information to construct a query condition or grouping condition.

Expected output

  • A document stream that contains documents that are a result of the specified grouping or query condition.

Expected upstream Snaps

  • A Snap that generates documents. For example, Mapper, JSON Generator, CSV Generator.

Expected downstream Snaps

  • A Snap that accepts documents. For example, Mapper, Filter, and JSON Formatter.

Prerequisites

None.

Configuring Accounts

This Snap uses account references created on the Accounts page of SnapLogic Manager to handle access to this endpoint. See MongoDB Account for information on setting up this type of account.

Configuring Views

Input

This Snap has at most one document input view. If the input view is defined, then values will be used to evaluate the expression in the conditions.
OutputThis Snap has exactly one document output view.
ErrorThis Snap has at most one document error view and produces zero or more documents in the view.

Troubleshooting

None.

Limitations and Known Issues

None.

Modes

Snap Settings


Label

Required. The name for the Snap. Modify this to be more specific, especially if there are more than one of the same Snap in the Pipeline.

Database name

The database that contains the documents. If you do not specify a database, then the Snap uses the MongoDB account database.

Example: assets

Default value: N/A

Collection name

Required. The MongoDB collection name to execute the grouping.

Exampleusers

Default value: N/A

Query Condition

An expression that represents the query parameter. If no query condition is defined, then MongoDB retrieves all the documents of the collection. When the expression evaluates to an object, only strict mode is supported. When the expression evaluates to a JSON string, both strict mode and mongo shell mode are supported. Click here for more information about MongoDB Extended JSON.

Example: {num:1}

Default value: N/A

Group Condition

Required. The condition to group documents and create a single document for each distinct group. Read here for details on group conditions.

Example: {"_id": "$Churn", "num_customer": {$sum: 1}, "total": {$sum: "$TotalCharges"}}

Default value: N/A

Sort Condition

The condition to order the documents in the result set. To use multiple sort orders, enter comma-separated sort conditions. Read here for details on sort conditions. 

Example: {age: -1}

Default value: N/A

Batch Size

Required. The number of documents to return in a batch. Click here for more details on how MongoDB batches documents. 

If n is the batch size, the Snap behaves as follows:

  • If n > 1, the Snap gets all the documents in the collection that match the query condition (returns n documents at a time, or as many that can fit in the batch buffer)
  • If n = 1, the Snap gets only one document in the collection that matches the query condition
  • If n = 0, the Snap gets all documents in the collection that match the query condition (returns as many documents that can fit in the batch buffer at a time)
  • If n < 0, the Snap gets n documents in the collection that match the query condition (for example, if n = -10, the Snap gets 10 documents)

In MongoDB 3.4, the batch buffer size is 16MB. In previous versions, the batch buffer size is 4MB. The initial batch always returns a maximum of 101 documents. Click here for details.

Example: 0

Default value: 0 

Timezone Offset

The timezone offset to be applied to the time fields. By default, the Snap follows UTC (00:00 offset). 

Hours Offset

The number of hours to be offset. For example, if you specify a value of -2 in Hours Offset and 30 in Minutes Offset, then the timezone is offset by -2:30 hours. 

Example:  -7

Default value: 0 

Minutes Offset

The number of minutes to be offset. For example, if you specify a value of -2 in Hours Offset and 30 in Minutes Offset, then the timezone is offset by -2:30 hours. 

Example: 1

Default value: 0

Ignore empty result

If selected, no document is written to the output view when the group operation does not produce any result. If this property is not selected and the Pass through property is selected, the input document is passed through to the output view.

Default value: Not selected

Group result

Select to group results in one single field named result, instead of an array.

Default value: Not selected

Pass through

If selected, the input document is passed through to the output view under the key 'original'.

Default value: Selected

Snap Execution

Select one of the three modes in which the Snap executes. Available options are:

  • Validate & Execute: Performs limited execution of the Snap, and generates a data preview during Pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during Pipeline runtime.
  • Execute only: Performs full execution of the Snap during Pipeline execution without generating preview data.
  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Example


This Pipeline demonstrates how the MongoDB Group Snap helps you group the number of customers based on churn and the total customer count for a telecommunication company. For each group, the Pipeline returns the average monthly charges, and total charges for the Telco customer churn dataset. 

Download the Pipeline.

 Understanding the Pipeline

In this example, we use the dataset of a telecommunication company. The dataset contains 21 fields. Each document in the dataset represents a customer record and contains data about the customer's demographics, service subscriptions, and the field $churn which indicates whether the customer is an existing customer or has quit. The Snap reads the dataset from the database location specified in the Snap configuration. The Snap configuration is as follows:

A preview of the input dataset is as follows:

The query that we use in the Group Condition field is:

{"_id": "$Churn", "num_customer": {$sum: 1}, "avg_monthly": {$avg: "$MonthlyCharges"}, "total": {$sum: "$TotalCharges"}}

This Snap runs this query on the dataset and returns the customer churn rate, the total number of customers, the average monthly charges, and the total charges. The output preview of the Snap is as follows:

The Snap groups the dataset into two: one for customer churn rate and the other for the total number of customers, represented by _id as Yes and No, respectively. And for each group, it returns the average monthly charges and the total charges.


Downloads

Important steps to successfully reuse Pipelines

  1. Download and import the Pipeline into SnapLogic.
  2. Configure Snap accounts as applicable.
  3. Provide Pipeline parameters as applicable.

  File Modified

File MongoDBGroupExample.slp

Aug 08, 2019 by Vidya Patil

See Also