Measuring Load Performance

We generated several different datasets to measure the performance of TokuDB when doing a LOAD DATA INFILE … command. To characterize performance, we vary

• rows to load
• keys per row
• row length (including keys)

All generated keys, including the primary, are random, 8-byte values. The remaining data, needed to pad out the row length to specified length, is text.

Two files files are produced as part of data generation.

1. data file, containing ‘|’ separated fields
2. sql file, containing the CREATE TABLE command corresponding to the generated data

For instance, if the number of keys is 3 and the row length is 256 bytes, the following SQL statement is produced:

     CREATE TABLE load_table (
         val0 BIGINT UNSIGNED NOT NULL,
         val1 BIGINT UNSIGNED NOT NULL,
         val2 BIGINT UNSIGNED NOT NULL,
         pad VARCHAR(232) NOT NULL,
         PRIMARY KEY (val0),
         KEY valkey1 (val1),
         KEY valkey2 (val2)
         ) ENGINE=tokudb

We can make the data generation program available if anyone is interested.

Load Test

A simple shell script

• creates the test table
• performs a LOAD DATA INFILE <datafile> INTO TABLE load_table FIELDS TERMINATED BY ‘|’
• returns execution time

For the experiments to be meaningful, we created datasets that do not fit in memory.

Results

We ran our benchmark on an Amazon Web Services c1.large node with 8 cores and 7 GB of memory. The test loads 100M rows (NOT pre-sorted). The data file was on a 2 disk RAID-0, the MySQL DB files on a different 2 disk RAID-0.

TokuDB Version 3 (~single-threaded) v. TokuDB Version 4 (multi-threaded)

Other metrics

Several metrics can be used to measure performance:

• rows per second : data insert rate
• key-value pairs per second : indicates how fast the primary and secondary indexes are being created
• MB/s : how much raw data is being added to the database

Metrics for TokuDB v4:

These results show

1. significant parallelization (we believe larger CPU core count machines will see even larger benefits)
2. a significant jump in absolute load performance
3. speed-ups are not limited to tables with many keys – even the 1 key tables are 5-7x faster

We will report further results, especially speedups on larger CPU count machines, as they become available.

Background

In the first experiment, I compared recovery times when both storage engines (TokuDB and InnoDB) were inserting at maximum rates. In that experiment, following a power cord pull and server restart, TokuDB recovered in 501s, InnoDB in 18505s. In response, it was suggested that I run a patched version of InnoDB that vastly improves InnoDB recovery times. The patched version recovered in 1020s. (Quite an improvement!)

The first experiment ignored the fact that TokuDB was inserting >16,000 rows/s into the table, while InnoDB was inserting less than 200 rows/s. Given that TokuDB was doing so much more work, you would expect a longer recovery time. This post discusses an experiment where recovery times were measured when both DBs were doing the same amount of work.

The experiment

As before, I ran on the following server:

Sunfire x4150
Dual socket quad core Xeon 3.1GHz
16GiB memory
2 SAS 146GB
6 SAS 146GB hardware RAID 0 256KiB stripe
CentOS 5.1
Ext3 file system

The disks and RAID controller were configured with the write-caches OFF.

I modified iiBench to generate rows at a rate of ~200 rows per second for insertion into a MySQL database so that both storage engines would perform insertions at the same rate. After inserting 250M rows, I pulled the power cords, then measured the time to restart mysqld after the machine was rebooted. I repeated this experiment inserting ~2000 rows per second. The results:

The reduced workload meant there was far less log data to scan and recover. These results also match our findings when we recover from power plug pulls on typical applications, which don’t necessarily use all of TokuDB’s insertion bandwidth at all times.

A final note: it takes mysqld 1s to start when no recovery is required.

Looks like the upper hand, is on the other foot!

In summary, while actively inserting records into a MySQL database using iiBench, we compared the time to recover from a power-cord pull for both InnoDB and TokuDB.

This is by no means an exhaustive look at recovery performance, but does illustrate the benefits of Tokutek’s approach.

The experiment

We ran iiBench on the following server:

Sunfire x4150
Dual socket quad core Xeon 3.1GHz
16GiB memory
2 SAS 146GB
6 SAS 146GB hardware RAID 0 256KiB stripe
CentOS 5.1
Ext3 file system

The disks and RAID controller were configured with the write-caches OFF using arcconf. (Note: an experiment with the RAID controller write-cache ON resulted in a non-recoverable database, as you should expect)

We ran an insert-only iiBench test. After inserting about 250M rows and while actively inserting more, we manually pulled the plugs (2 on a Sunfire x4150). We then powered up the server and timed how long it took to start mysqld.

The results for InnoDB are not surprising – they are in line with results reported elsewhere. During recovery, InnoDB reported 1 transaction to be rolled back or cleaned up, and 68 row operations to undo.

The results for TokuDB are encouraging, in that they indicate recovery can be performed in minutes instead of hours. In this experiment, rebooting the server almost took as long as recovering the TokuDB database.

The primary reason for TokuDB’s quick recovery time is the use of regular, automatic checkpoints. This feature (introduced in version 2.0.0) ensures that work is completed and written to disk (including the necessary fsync) on a regular basis. Recovery is limited to processing uncommitted transactions and work since the last checkpoint. When not pushing performance to the absolute max we find recovery times typically take less than a minute.