Tuesday, January 30, 2007

Flash HDD

Most bottlenecks when processing large sized data exists in file I/O. Two most notable bottleneck locations are HDD and networks.

Google must've known this problem better than anyone and it uses its specially designed data processing algorithms (Map and Reduce) on it s powerful distributed computing environment which also runs distributed file system (Google File System). Google's algorithm tries its best to do all the data processing in one machine so that there will be minimal amount of data transactions over the networks.

But more serious botttleneck problems exist with the current HDD. The fastest HDD out there cannot even match data trasaction rate of the gigabit ethernet which is common nowadays, not to mention fibre channel which moves data at the speed of light (in theory!) Speed isn't the only problem, either. HDD has got many mechanical parts that fail quite frequently as well.

Semiconductor manufacturers know this problem, too, and they know there are demands for faster HDD. So they come up with Flash HDD, like the one pictured above!

Someone at Samsung once said something like conventional HDD is dead and the age of Flash is upon us. I totally agree. If they can get the cost down, which they will surely do, it will be everywhere.

Maybe it's time to buy Flash manufacturers' stocks...

By the way, I wonder if Google replaced all its HDD with Flash HDD how much faster their search and other services will be.

No comments: