concept aggregation framework in category mongoDB

This is an excerpt from Manning's book MongoDB in Action, Second Edition: Covers MongoDB version 3.0. Login to get full access to this book.

Aggregation framework performance —Improvements are made in the performance of the aggregation framework to support real-time analytics; chapter 6 explores the Aggregation framework.

In the previous chapter, you saw how to use MongoDB’s JSON-like query language to perform common query operations, such as lookup by ID, lookup by name, and sorting. In this chapter, we’ll extend that topic to include more complex queries using the MongoDB aggregation framework. The aggregation framework is MongoDB’s advanced query language, and it allows you to transform and combine data from multiple documents to generate new information not available in any single document. For example, you might use the aggregation framework to determine sales by month, sales by product, or order totals by user. For those familiar with relational databases, you can think of the aggregation framework as MongoDB’s equivalent to the SQL GROUP BY clause. Although you could have calculated this information previously using MongoDB’s map reduce capabilities or within program code, the aggregation framework makes this task much easier as well as more efficient by allowing you to define a series of document operations and then send them as an array to MongoDB in a single call.

In this chapter, we’ll show you a number of examples using the e-commerce data model that’s used in the rest of the book and then provide a detailed look at all the aggregation framework operators and various options for each operator. By the end of this chapter, we’ll have examples for the key aspects of the aggregation framework, along with examples of how to use them on the e-commerce data model. We won’t cover even a fraction of the types of aggregations you might want to build for an e-commerce data model, but that’s the idea of the aggregation framework: it provides you with the flexibility to examine your data in more ways than you could have ever foreseen.

Up to now, you’ve designed your data model and database queries to support fast and responsive website performance. The aggregation framework can also help with real-time information summarization that may be needed for an e-commerce website, but it can do much more: providing answers to a wide variety of questions you might want to answer from your data but that may require crunching large amounts of data.

The identical query in the aggregation framework would look like this:

reviews2 = db.reviews.aggregate([
    {$match: {'product_id': product['_id']}},
    {$skip : (page_number - 1) * 12},
    {$limit: 12},
    {$sort:  {'helpful_votes': -1}}
]).toArray();

copy

As you can see, functionality and input parameters for the two versions are identical. One exception to this is the find() $where function, which allows you to select documents using a JavaScript expression. The $where can’t be used with the aggregation framework $match operator.

The explain() function for the aggregation framework is a bit different from the explain() used in the find() query function but it provides similar capabilities. As you might expect, for an aggregation pipeline you’ll receive explain output for each operation in the pipeline, because each step in the pipeline is almost a call unto itself (see the following listing).

Listing 6.3. Example explain() output for aggregation framework

Although the aggregation framework explain output shown in this listing isn’t as extensive as the output that comes from find().explain() shown in listing 6.2, it still provides some critical information. For example, it shows whether an index is used and the range scanned within the index. This will give you an idea of how well the index was able to limit the query.