532x Multikey Index Insertion Performance Increase for MongoDB with Fractal Tree Indexes

Posted On November 13, 2012 | By Tim Callaghan | 2 comments

In my three previous MongoDB blogs I wrote about our implementation of Fractal Tree(R) indexes on MongoDB, showing a 10x insertion performance increase, a 268x query performance increase, and a comparison of covered indexes and clustered indexes. These benchmarks show the difference that rich and efficient indexing can make to your MongoDB workload.

Given the high performance of Fractal Tree Indexes, we’ve created a new benchmark to test our ability to handle indexing large multikey indexes. MongoDB creates a multikey index when the index field is an array of values, each value creates a separate entry in the index.

NOTE: MongoDB does not support indexes on parallel arrays (a multikey index with more than 1 array field).  Since we plan on adding support for parallel array indexes, it made sense to test our indexes in scenarios where the cartesian product of two array fields could produce a large number of index entries.  Stay tuned for those benchmark results.

The main change from our original insertion benchmark is that the URI field is now an array with 100 entries per document.  This forces 100 index operations on the URI index for every document insertion.  A sample document is as follows, the full URI is abbreviated:

{"URI" : ["ilaor5234", "robcar8823", ..., "stear83"],
"name" : "weather",
"origin"  : "497ce4051ca9ca6d3efca323",
"creation" : <nanotime>,
"expiration" : <nanotime>}

The benchmark client is available here.

Benchmark Environment

  • Sun x4150, (2) Xeon 5460, 16GB RAM, StorageTek Controller (256MB, write-back), 4x10K SAS/RAID 0
  • Ubuntu 10.04 Server (64-bit), ext4 filesystem
  • MongoDB v2.2.RC0

Benchmark Results

At 3.5 million inserted documents, the exit velocity of standard MongoDB was 2.11 inserts per second versus MongoDB with Fractal Tree Indexes exit velocity of 1,124.36 inserts per second: an improvement of over 532x.  My plan was to run this benchmark to 5 million inserts, but at 2 inserts per second it would have taken MongoDB an additional week to get there.

I am presenting our results and roadmap in a webinar today, November 13, at 2pm EST.  You can sign up for “MongoDB and Fractal Tree Indexes” via this link.

2 thoughts

Leave a Reply

Your email address will not be published. Required fields are marked *