iiBench Benchmark

iiBench – Indexed Insertion Benchmark

In addition to the well-recognized TPC benchmarks, several benchmarks such as the Star Schema Benchmark, sysbench, and the DBT series, have been proposed and implemented to measure database performance. These benchmarks simulate and measure performance for workloads and data characteristics similar to those measured by the TPC benchmarks.

We developed the iiBench benchmark to measure performance for a use case that occurs commonly in production applications related to advertising, social media, and network management. iiBench measures the rate at which a database can insert new rows while maintaining several secondary indexes, a pattern of usage required in always-on applications that:

  • require fast query performance and hence require indexes
  • have high data insert rates
  • cannot wait for offline batch processing and hence require the indexes be maintained as data comes in

Open source

We have developed iiBench as an open-source benchmark, allowing others to freely use it, extend it, and contribute their changes back. We originally unveiled the benchmark in the context of a challenge issued at the 2008 OpenSQL camp. Since then, iiBench has been downloaded and used many times.

Tokutek implemented the original iiBench in C++, and Mark Callaghan ported it to a Python version, and extended it with additional query benchmarking capabilities. He contributed his enhanced version back to the community via Launchpad.

We have further enhanced the Python version of iIBench and have contributed our changes back to Mark’s Launchpad project. Going forward, we will continue to contribute to Mark’s version, and we do not expect to maintain the C++ version of iiBench.