Tokutek » zardosht

A Case for Write Optimizations in MySQL

zardosht — Mon, 21 Nov 2011 15:22:33 +0000

As a storage engine developer, I am excited for MySQL 5.6. Looking at http://dev.mysql.com/tech-resources/articles/whats-new-in-mysql-5.6.html, there has been plenty of work done to improve the performance of reads in MySQL for all storage engines (provided they take advantage of the new APIs).

What would be great to add is API improvements to increase the performance of writes, and more specifically, updates. For many applications that perform updates, such as applications that do click counting or impression counting, there are significant opportunities for improving write performance.

Take the following example of click counting (or impression counting). You have a website and want to save the number of times links on your website have been clicked. Your table may look something like:

create table num_clicks( link_id int, num_clicks int);

To update the number of clicks, you do something like:

insert into num_clicks (LINK_ID, 1) on duplicate key update set num_clicks=num_clicks+1;

With MySQL as it currently works, this is slower than it needs to be, as I explained here. At a high level, the reason is that MySQL forces the storage engine to check in the table if a value exists for LINK_ID. If a row is returned, MySQL performs the increment away from the storage engine, and passes a new row to the storage engine for an update. The check incurs a disk seek, which is very costly in terms of latency. Disks can do only hundreds of seeks per second. Furthermore, NoSQL solutions based on B-trees are similarly limited and can’t be significantly accelerated because updates incur disk I/O.

However, with some changes to MySQL, a storage engine can take advantage of this knowledge to improve its algorithms. All that’s needed is for the storage engine to know that the user wants to perform an insert or to perform this particular update, as opposed to getting individual handler calls of write_row, index_read, and update_row (which is the current design). Hence, what’s needed is a way for the storage engine layer to be able to apply updates on its own.

This change can help all storage engines. Although I am not an expert in MySQL Cluster, I imagine reducing these individual handler calls also helps MySQL Cluster avoid network hops to retrieve information. For in-memory databases, performance may increase due to reducing the number of calls made by the handler. InnoDB can potentially use its insertion buffer to store the “insert … on duplicate key update” operation, thereby giving the operation the same boost insertions into secondary keys get. For TokuDB, we estimate that these types of updates aided by this additional information could run much faster. In future posts, I will expand on how we think TokuDB can do this.

Understanding Indexing – NY Effective MySQL Meetup

zardosht — Fri, 07 Oct 2011 15:05:41 +0000

At next week’s NY Effective MySQL Meetup, I will give a talk: “Understanding Indexing: Three rules on making indexes around queries to provide good performance.” The meetup is 7 pm Tuesday, October 11th, and will be held at Hive at 55 (55 Broad Street, New York, NY). Thanks to host Ronald Bradford for the invitation.

Application performance often depends on how fast a query can respond and query performance almost always depends on good indexing. So one of the quickest and least expensive ways to increase application performance is to optimize the indexes. This talk presents three simple and effective rules on how to construct indexes around queries that result in good performance.

This is a general discussion applicable to all databases using indexes and is not specific to any particular MySQL^® storage engine (e.g., InnoDB, TokuDB^®, etc.). The rules are explained using a simple model that does NOT rely on understanding B-trees, Fractal Tree^® indexing, or any other data structure used to store the data on disk.

The rules are derived from these simple properties:

Point queries are slow
Range queries are fast

I hope to see you there!

Indexing: The Director’s Cut

zardosht — Fri, 15 Jul 2011 16:01:26 +0000

Thanks again to Erin O’Neill and Mike Tougeron for having me at the SF MySQL Meetup last month for the talk on “Understanding Indexing.” The crowd was very interactive, and I appreciated that over 100 people signed up for the event and left some very positive comments and reviews.

Thanks to Mike, a video of the talk is now available:

As a brief overview – Application performance often depends on how fast a query can respond and query performance almost always depends on good indexing. So one of the quickest and least expensive ways to increase application performance is to optimize the indexes. This talk presents three simple and effective rules on how to construct indexes around queries that result in good performance.

—

“Director’s Cut” Bonus Feature: During the talk, a number of people wanted the lowdown on Fractal Tree™ Indexing. After the first hour on general indexing, I was able to address this and give a whiteboard overview of Fractal Tree indexes:

Understanding Indexing – SF MySQL Meetup

zardosht — Tue, 21 Jun 2011 05:14:12 +0000

At this week’s SF MySQL Meetup, I will give a talk: “Understanding Indexing: Three rules on making indexes around queries to provide good performance.” The meetup is 7 pm tomorrow (Wednesday, 6/22), and will be held at CBS Interactive (235 2nd St., San Francisco). Thanks to hosts Erin O’Neill and Mike Tougeron for the invitation and location.

This is a general discussion applicable to all databases using indexes and is not specific to any particular MySQL storage engine (e.g., InnoDB, TokuDB, etc.). The rules are explained using a simple model that does NOT rely on understanding B-Trees, Fractal Tree™ indexing, or any other data structure used to store the data on disk.

The rules are derived from these simple properties:

Point queries are slow
Range queries are fast

I hope to see you there!

Effective MySQL, a New York City Meetup

zardosht — Tue, 26 Apr 2011 14:39:05 +0000

Kudos to Ronald Bradford for creating a new MySQL meetup group in New York city and giving MySQL related talks. The next one is tonight, titled “MySQL Idiosyncrasies That Bite”. Information on it can be found at http://ny.effectivemysql.com/events/16884850/.

We’ll have a contingent from our New York office there this evening. We went to the last one on indexing (a favorite topic of ours) in March and it was excellent.

We look forward to seeing folks there as well as at upcoming NY events, including Percona Live (May 26th) and future Effective MySQL meetups.

Understanding Indexing – MySQL Meetup

zardosht — Wed, 16 Mar 2011 22:01:21 +0000

Yesterday, at the Boston MySQL Meetup, I gave a talk on indexing. It is posted here (also goo.gl/S2LBe).

In short, indexes are used to improve query performance. As a result, good indexes are designed around queries that users find important in their application. The presentation covers three simple and effective rules on how to construct indexes around queries that result in good performance.

The rules are explained using a simple model that does NOT rely on understanding B-trees, Fractal Tree™ indexing, or any other data structure used to store the data on disk. They are derived from these simple properties:

Point queries are slow
Range queries are fast

As always, comments, questions and thoughts are welcome!

Scenarios where TokuDB’s Loader is Used

zardosht — Thu, 16 Sep 2010 17:06:21 +0000

TokuDB’s loader uses the available multicore computing resources of the machine to presort and insert the data. In the last couple of posts (here and here), Rich and Dave presented performance results of TokuDB’s loader. Comparing load times with TokuDB 2.1.0, Rich found a 2.1x speedup on a 2 core machine, and a 4.2x speedup on an 8 core machine. Comparing load times with TokuDB 3.1, Dave found an 8.2x speedup on Amazon Web Services c1.large node with 8 cores while loading a table with 256 byte rows.

This leads to these natural questions: how does one use the TokuDB loader? Under what scenarios is it used?

The loader has two purposes:

to ease migration of data from other sources (e.g. other storage engines, data files) to TokuDB.
to build newly defined indexes (that TokuDB can maintain in real time) very fast.

So, the loader is designed to operate on empty tables.

The loader is integrated into the TokuDB storage engine. Using it does not require any external binaries. Users can use the loader by inserting data into an empty TokuDB table with any of the following commands:

insert into
load data infile
alter table … engine=TokuDB
alter table … add index…

Scenarios that do not yet work are:

replace into
insert ignore

On “Replace Into”, “Insert Ignore”, Triggers, and Row Based Replication

zardosht — Wed, 11 Aug 2010 04:14:02 +0000

In posts on June 30 and July 6, I explained how implementing the commands “replace into” and “insert ignore” with TokuDB’s fractal trees data structures can be two orders of magnitude faster than implementing them with B-trees. Towards the end of each post, I hinted at that there are some caveats that complicate the story a little. On July 21st I explained one caveat, secondary keys, and on August 3rd, Rich explained another caveat. In this post, I explain the other two caveats: triggers and replication.

First, let’s look at triggers. The key to “replace into” and “insert ignore” being fast is that the uniqueness check is not done when the command is executed. The uniqueness check is deferred to a more opportune time. Triggers do not allow the uniquness check to be deferred. For triggers to properly work, we must know exactly what rows applied what modifications. For “replace into”, we must know which rows resulted in an insertion and which rows resulted in a deletion and insertion. For “insert ignore”, we must know which rows resulted in an insertion and which rows performed no operation. This information is not available unless disk seeks are performed.

Now, let’s look at row based replication. The problem here lies within mysql (as I describe in bug 53561, although that bug only addresses “replace into”, similar issues exist with “insert ignore”). Row based replication is not designed to handle anything other than normal insertions, which return an error if a duplicate key is found. So, a successful execution of “replace into” or “insert ignore” on the master maps to a normal insertion on the slave. Because normal insertions have different semantics than “replace into” and “insert ignore”, row based replication does not work if you use the altered semantics.

The problems with triggers are semantic. Their requirements incur disk seeks, which slows performance. The problem with row based replication is a design issue which will hopefully be fixed one day.

On “Replace Into”, “Insert Ignore”, and Secondary Keys

zardosht — Wed, 21 Jul 2010 20:51:55 +0000

In posts on June 30 and July 6, I explained how implementing the commands “replace into” and “insert ignore” with TokuDB’s fractal trees data structures can be two orders of magnitude faster than implementing them with B-trees. Towards the end of each post, I hinted at that there are some caveats that complicate the story a little. In this post, I explain one of the complications: secondary indexes.

Secondary indexes act the same way in TokuDB as they do in InnoDB. They store the defined secondary key, and the primary key as a pointer to the rest of the row. So, say the table foo has the following schema:

create table (a int, b int, c int, primary key (a), key(b));

And we did:

insert into foo values (1,10,100),(2,20,200);

Logically, there is one dictionary that stores all the data (this is the clustered primary key). Let us call it the main dictionary:

key  value
1    10,100
2    20,200

And there is another dictionary for the secondary key that stores the column ‘b’ and the primary key, ‘a’:

key  value
10   1
20   2

For secondary indexes to work properly, there must be a one to one correspondence between elements in the secondary index and in the primary index. If this correspondence is broken, then the table is corrupt.

Now suppose we were to execute:

replace into foo values (1,1000,1000);

This does:

in main dictionary, overwrite the value of key ’1′ and value ’10,100′ with key ’1′ and value ’1000,1000′.
in secondary dictionary, remove the key ’10′ with value ’1′.
in secondary dictionary, insert the key ’1000′ and key ’1′.

Notice that we cannot perform the second step unless we know the content of the existing row that is being replaced. Learning the content of the existing row requires a lookup in the main dictionary, which incurs a disk seek.

So, when executing “replace into” or “insert ignore” on tables with secondary keys, all engines must still incur a disk seek on the primary dictionary to learn where associated elements are in a secondary index, whereas if no secondary keys exist, then TokuDB’s fractal trees can avoid this disk seek.

Even with secondary indexes, fractal tree indexes are preferred. B-trees still incur additional disk seeks on insertions into secondary indexes that fractal trees do not. However, with no secondary indexes, fractal trees can do away with the mandatory disk seek whereas B-trees do not.

Why “insert … on duplicate key update” May Be Slow, by Incurring Disk Seeks

zardosht — Wed, 14 Jul 2010 19:17:49 +0000

In my post on June 18th, I explained why the semantics of normal ad-hoc insertions with a primary key are expensive because they require disk seeks on large data sets. I previously explained why it would be better to use “replace into” or to use “insert ignore” over normal inserts. In this post, I explain why another alternative to normal inserts, “insert … on duplicate key update” is no better in MySQL, because the command incurs disk seeks.

The reason “insert ignore” and “replace into” can be made fast with TokuDB’s fractal trees is that the semantics of what to do in case a duplicate key is found is simple. In one case, you ignore, and in the other, you overwrite. With specific tombstone messages defined for these simple semantics, we defer the uniqueness check to a more opportune time.

The semantics of “insert … on duplicate key update” are not simple:

if the primary (or unique) key does not exist, insert the new row
if the primary key does exist, perform some update as defined in the SQL statement

The problem is we do not have a way of encoding the SQL update function into a message, the way we are able to encode “replace into” as an ‘i’ and “insert ignore” as an ‘ii’. If we did, we could similarly make “insert … on duplicate key update” fast.

I am not claiming that this is not theoretically possible, just that the storage engine API in MySQL does not allow for the encoding of updates as messages. Instead, what MySQL does is the following:

call handler::write_row to attempt an insertion, if it succeeds, we are done
if handler::write_row returns an error indicating a duplicate key, outside of the handler, apply the necessary update to the row
call handler::update_row to apply the update

The storage engine API does not have any access to the function that applies an update to the existing row. This is why the storage engine has no way of encoding any SQL update function (even some simple ones, such as “increment column a”).

So, in the meantime, to implement these semantics, B-trees and Fractal Tree data structures both:

look up the primary (or unique) key to verify existence
take the appropriate action based on whether the primary (or unique) key exists

The first step incurs a disk seek on large data sets with an ad-hoc primary (or unique key). And that is why it is slow.

So, the moral of the story is this. In MySQL, “insert … on duplicate key update” is slower than “replace into”. Although the sematics are slightly different in the case where the primary key is found (the former is defined as an update, whereas the latter is defined as a delete followed by an insert), if possible, the simpler semantics of “replace into” allow it to be faster than “insert … on duplicate key update”.