Fetch as Much as You Can March 24, 2010

In my “Everything DBAs Need to Know About TCP/IP” presentation, I discussed the issue of fetch size – tuning the number of rows your application will get from Oracle in a single “fetch” call.

The problem is interesting when you want to get a large number of rows from the DB to the application. Cary Millsap would say that no one should want to do that, but Cary obviously never had to work with statistical analysis application. These things demand large amounts of blood raw data to do their thing.

If this is too slow for your users, you should look at statistics like “round trips”. If the number of round-trips looks completely off the wall, like 200,000 round trips for 2M rows, someone might have left JDBC’s default fetch size of 10.

In my presentation, I mentioned that there is really no problem with setting a very high fetch size. I recommended setting it as high as possible while taking into account the amount of memory each row will consume.

Charles Hooper blogged last week with a new reason to use large fetch sizes: Turns out that it allows Oracle to pin the block for the length of the fetch. He references an OTN discussion where Jonathan Lewis says:

“…what it the array fetch size ? If the array fetch size is large enough the number of block visits would be similar to the number of physical block reads. If the arraysize is one then the number of block visits could be close to the number of rows fetched as a block has to be released at the end of each fetch and the reacquired for the next fetch.”

So a higher fetch size ends up with fewer consistent gets. Even if you are not tuning any specific issue that requires lowering the number of consistent gets, I hope you’ll agree that getting the same results with less work is a good thing.

One more reason to set fetch size to as many rows as your memory allows.

What a Difference a Month Makes March 23, 2010

Its March 22. Exactly one month ago, I came back from few days spent in Colorado. I gave presentations, met amazing people, enjoyed skiing and drank a lot of beer, wine and whiskey. I also barely made it back, but that’s a different story.

Obviously I drank too much, or maybe I gave too many presentations. I don’t remember.

What I do know is that in the month since that eventful weekend, my life has taken a sharp turn.

I’m about to start working for Pythian. There will be a separate post about that, where I explain how amazing Pythian is and how to continue to follow my blog once it is merged with the official Pythian blog. Same hypothetical blog post will also include comments about how much I love my current colleagues and how sorry I am to leave them.

But more important – I’m now an Oakie. Seriously. Check it out Can you tell I’m stoked?

This is not really a blog post, more of a tweet really. But I had to share the news.

VMWare Hires Redis Key Developer – But Why? March 17, 2010

My friend MosheZ alerted me to the fact (which few hours later appeared all over the net) that VMWare hired Redis key developer. Which is as close to acquisition as you can get with an open source project.

What is Redis? Redis is yet-another-NoSQL. A key-value store, somewhat similar to Tokyo Cabinet. Except that Redis does persistence differently, which makes it faster in many cases. Redis started as a Memcached replacement, so a lot of the documentation describes it as follows: Redis is like Memcached, except it supports more data types, it is persistent to some degree and it is not distributed.

But the more interesting question is – Why does VMWare need Redis?

VMWare says: “As VMware continues its investments in the context of cloud computing, technologies such as Redis become key for future cloud based apps, whether private or public cloud, and the cloud infrastructure itself.”

So Redis is cloud and VMWare is a major cloud player, therefore VMWare needs redis. Two discrepancies stand out in this story:

1. Redis is not distributed system. Unlike Cassandra, where you can scale by quickly adding more Cassandras to the party, Redis is just one (very fast) server, only supporting master-slave replication. VMWare is all about adding new machines quickly. Something doesn’t fit.
2. While key-value stores are  cloudy and VMWare is cloudy, there is no natural match between their cloudiness. VMWare itself can’t use Redis – because Redis technology is a natural match for big-data websites which VMWare clearly isn’t. Some VMWare customers can benefit from Redis, but most can’t. What’s going on here?

Clearly, the place to look is not in existing value but in the future. So here are my predictions:

1. Redis will become distributed. It can certainly be done. Perhaps it can even be done without losing it’s performance edge.
2. VMware will announce an Amazon-like, cloud-for-rent service. They have the technology for this, and Redis will help them manage the “huge website” part of it.
3. They may also offer Redis on top of the virtual servers, as something built in. Like Amazon’s Oracle servers.
4. VMWare can also offer storage for rent. They can do it with EMC storage (since VMWare is an EMC company), but I’m betting that they’ll do it with Netapp – their favorite cloud partner.  I can totally imagine a near-future Netapp-Vmware offering that is similar to Amazon’s EC2+ S3+AWS.

Predicting is very difficult (especially about the future) and I’m very much ready to regret ever posting my day dreams in public, but these are exciting possibilities. I wonder if they make sense to anyone else.

*********************************

And speaking of MosheZ, he is a prolific song writer, and he wrote a song about DBAs! I’m thinking of performing it live during one of my presentations. Actually I’m thinking of writing a presentation “How to win arguments or influence users” just to have an excuse to sneak this song in