The content comes from my personal experiences over many years benchmarking and testing databases, usually focusing on performance.  It was an opportunity to see how far my personal benchmark infrastructure has evolved, but even better has inspired me to improve it in several areas.

I never had a chance to to my own post-conference wrap-up regarding the Percona Live show.  While waiting for my flight home at SFO airport I concluded that it was by far the best technology conference I’ve ever attended.  The MySQL community is truly amazing and finally meeting many of the people that I’ve only “known” online was great.  I’m really looking forward to next year’s event and ways I can contribute to existing benchmarking and testing projects.

I’ll give an overview on how Fractal Tree® indexes work, and then get into specific product features that Fractal Trees enable in MySQL and MongoDB.  Some benchmarking and customer use-cases will be discussed, but my intent is for this to be a deep technical dive.  Several Tokutek Engineers will also be on hand, so bring any questions you’ve got.

I hope to see you there!

Following are my slide decks for all 3 events:

• Tutorial: A Guided Walk Through a TokuDB Implementation
• Performance At Scale with TokuDB
• Poor Man’s Monitoring

For those who attended the events, thank you very much for your interest. For those who didn’t, we hope to see you at today’s webinar at 2pm ET, or if you are in NY, at my colleague Tim’s presentation at Ronald Bradford’s Effective MySQL Meetup on the 14th.

Date: May 2nd
Time: 2 PM EST / 11 AM PST
REGISTER TODAY

In the past TokuDB has been free for evaluation; the new TokuDB Community Edition extends free use to deployed environments. With this release Tokutek is also planning on making available a TokuDB Enterprise Edition, which includes technical support, initial customer onboarding services, and advanced tools for backup and recovery.

We look forward to having you join the webinar.

About the Presenters

Martín Farach-Colton is the Co-founder & Chief Technology Officer at Tokutek. Prof. Farach-Colton is an expert in algorithmic and information retrieval. He was an early employee at Google, where he worked on improving performance of the Web Crawl and where he developed the prototype for the AdSense system. Prof. Farach-Colton is also a Professor of Computer Science at Rutgers University.

Gerry Narvaja leads Technical Services at Tokutek. Gerry has spent more than 25 years in the software industry, most of them working with databases for different kinds of applications, from embedded to large-scale web products. Gerry worked first at MySQL, and then Sun Microsystems supporting the Sales teams. In 2008 he transitioned into being a Senior MySQL DBA. For over a year he has been co-hosting the popular MySQL Community podcast, OurSQL, which was given the MySQL Community Contributor of the Year 2012 award at the recent Percona MySQL Users Conference.

Here are some of the highlights of our first week of open source:

We started hearing back from the community almost immediately after the announcement with discussions in multiple forums. We even reached #2 on Hacker News for a bit.

On Tuesday, a friend of mine in China wrote up some notes based on a first reading of our Fractal Tree Indexing code. I’ve been talking to Bohu for a while about our algorithms and I know he was eager to dive into the code. I can’t read it myself, but he tells me it explains some of our internal mechanisms for durability and concurrency, and that there is a lot of interest in China.

On Wednesday, we heard that Vadim Tkachenko built TokuDB and our MySQL patches into an experimental build of Percona Server, and people have already started playing with it. We also saw that the MariaDB developers started working on integrating TokuDB into the official MariaDB source tree.

And yesterday, we were named Corporate Contributor of the Year from the MySQL community, and fittingly started receiving our first pull requests on github.

Who knows what the next few days will bring, but if you happen to be in the Bay Area, please swing by to see us today at the MySQL & Cloud Database Solutions Day or Monday/Tuesday at OSBC.

To learn more about our open source resources, click here.

And for those who want some behind the scenes photos, see here and here. (courtesy of @seattlegaucho).

The MySQL Community Awards are given annually to the people and companies that support the MySQL ecosystem. The MySQL Community Award for Corporate Contributor of the Year recognizes a company or other organization or entity that has made valuable contributions to the MySQL ecosystem either in terms of open source code, knowledge, funding or other resources or sponsorship. The winners are selected by an independent community panel.

“Open Source is about collaborating and contributing to build and benefit from something great together. We created the Corporate Contributor category for the MySQL Awards in order to recognize that this collaboration is not just between individuals in the community, but also corporate entities like Tokutek, which has participated in the MySQL community for many years,” said Henrik Ingo, secretary of the MySQL Awards committee. “On behalf of the entire panel, I’d like to congratulate Tokutek for this award and for their recently announced open sourced TokuDB Community Edition product.”

A special thanks to Erin Grace O’Malley O’Neill for the great performance. Thanks also to @NuoDB and @geobdz (whose photo is below) for tweets.

Stay tuned for the video as well….

As I browse through the archive where I keep all my code
I look at my software and see it’s very old
But that’s just perfect for a hacker like me
You know I shun fancy things like agility

At 4:30 in the morning I’m writing code
Zardosht debugs and Richard builds, fool
And I’ve been building and debugging so long
Even Monty think’s that my mind is gone

I like high performance and test discipline
I’ve got a man page in my hand and gprof on my screen
But if I finish my builds and you finish thine
Tonight we’ll party like we survived 1999.

We’ve been spending most our lives hacking in MySQL paradise
I’ve churned code base once or twice hacking in MySQL paradise
It’s hard work and sacrifice hacking in MySQL paradise
We write code at discount price hacking in MySQL paradise

We’ve been spending most our lives hacking in MySQL paradise
We’ve been spending most our lives hacking in MySQL paradise
We’ve been spending most our lives hacking in MySQL paradise
We’ve been hacking most our lives living in MySQL paradise.

MySQL Paradise is based on Weird Al Yankovic’s Amish Paradise
Which is based on Coolio’s Gangsta’s Paradise
Which is based on Stevie Wonder’s Pastime Paradise

Here are the repositories:

• ft-index. This repository is the “magic”. It contains the Fractal Tree data structures we have been talking about for years. This is also the main piece that was previously closed source. Here are some interesting directories:
• src: This directory is a layer that implements an API that is similar to the BDB API.
• locktree: an in-memory data structure that maintains transactions’ row-level locks.
• ft: the fractal tree. This directory contains many of our data structures and algorithms, some of which are:
  • the tree data structures,
  • the low level transaction objects
• mysql: this contains our copy of MySQL source code, along with several patches to enhance it for TokuDB, such as a patch to allow online schema changes and grammar for clustering keys. The source for this has always been available, so there’s nothing new here except that it is on github.
• mariadb: same as mysql, only for MariaDB.
• ft-engine: This is our storage engine plugin that allows MySQL to talk to TokuDB. All MySQL storage engines have one. This has been open source for a while.

To further discussion, we have also started the following google groups to discuss TokuDB with the community:

• tokudb-user: User and support list for TokuDB.
• tokudb-dev: Developer list for contributors to TokuDB.

We are also on IRC in #tokutek at irc.freenode.net.

We look forward to hearing from the community.

We wanted to thank everyone for the great feedback. Here is a sampling from some of the forums where dialogue is occurring:

Reddit:

BrianAtDTS: “With this update, this puts MySQL in the upper echelon of data storage applications. You don’t have anything like this in any of the other major db’s.”

Twitter:

@xaprb: “The TokuDB storage engine for MySQL is now open-source. Congratulations & thanks Tokutek!”

@rolandbouman: “The MySQL ecosystem just got a lot more interesting for the Enterprise: @mariadb merges with @skysql, and @tokutek is now open source.”

Hacker News:

mrjbq7: “TokuDB is great technology, glad to see them embracing open source.”

Colin Charles Agenda:

“I think this is really big news. Beyond just the fact that it can now be a storage engine in the main MariaDB tree, I love the work they’re doing to extend it to be an engine for MongoDB as well.”

Blog Comments:

Ronald Bradford: “That’s great news. I am looking forward into delving into some code when I have some free time. Also easier to evaluate with clients as there no barrier to entry in getting the foot in the door. Well done to the team.”

Press Release:

“We’re delighted to welcome Tokutek to the open source community of MySQL,” said Tomas Ulin, Vice President MySQL Development at Oracle. “Much of the vitality of the MySQL ecosystem comes from its open source software dynamic, and it’s great to see new innovation in products like the TokuDB storage engine.”

“I was really happy to hear that TokuDB is now Open Source. It will make it much easier for us to both integrate it deeper than ever before into MariaDB and to add it as a standard component. Being Open Source, and part of MariaDB, will greatly help accelerate the adoption of TokuDB,” said Monty Widenius, co-founder of MySQL. “Having more offerings around TokuDB is also good news as it will make TokuDB more attractive to a larger range of customers.”

“At Mozilla we’ve been using TokuDB since day one of Datazilla. We knew that we’d have large data sets and it was our storage engine of choice. By going open source, Tokutek has made it much easier for TokuDB to be used across many more projects,” said Sheeri Cabral, DBA, Mozilla.

“Tokutek’s decision to open source its TokuDB storage engine will result in tighter integration with the community of complementary partner offerings used to enhance the MySQL experience,” said Peter Zaitsev, CEO of Percona. “Overall, this will accelerate the acceptance of TokuDB as an important tool for improving overall MySQL performance.”

“Tokutek open sourcing its TokuDB storage engine is in line with Continuent’s practice of promoting high-quality open source offerings for MySQL users,” noted Robert Hodges, CEO of Continuent. “A vibrant, open MySQL ecosystem is a real benefit to DBAs and IT architects who are looking to build innovative new businesses and applications that leverage the popular LAMP stack. We welcome the news and look forward to seeing TokuDB work even better with Continuent Tungsten as a result.”

And this is just sample. Thanks again to all for the great feedback, and thanks to those who came by the booth to express their excitement about the offering.

It is nice to be interesting!