The Petabyte Era

I was just reading an article that proposes that we are now entering the “Petabyte Era”.

What is a petabyte (PB)? A petabyte is 1,024 terabytes of computer data storage. For those that missed it, a terabyte (TB) is 1,024 gigabyte (GB), which in turn is 1,024 megabyte (MB), which, as almost everyone knows these days is 1,024 kilobyte (KB).

To write it correctly—with all its digits—a petabyte is 1,125,899,906,842,624 bytes, although people who don’t care much for accuracy may just write it as 1,000,000,000,000,000 or 1.0E15 bytes; figuring that the chopped-off 125,899,906,842,624 bytes, which is a mere 125 terabytes, don’t really count for much. A bit the same was as you often hear a terabyte referred to as 1,000 gigabytes when it is actually 1,099.5 gigabytes.

According to the article I was skimming we are entering the petabyte era. By 2015 petabyte storage will start to become the norm. This, apparently, is around when a petabyte hard disk will become commercially available and when petabytes of storage will become ‘common’. As common as terabytes of storage are now.

Apparently all of the pictures stored on Facebook at the start of 2011 came to 1.5 petabytes. Every 24 hours the Google search crawlers trawl through 20 petabytes of data refreshing their search indexes. Assuming this is correct then that means Google indexes about 47 terabytes of data every hour.

Just to bring the size of a petabyte a bit more into focus, on a single petabyte hard disk you could store approximately:

• 13.3 years of continuous 24x7 uncompressed HD-TV video.
• 1,324,500 episodes of your favourite 1 hour TV show in 15 precent compression 720p high-quality (remembering that a 1 hour TV show only has about 40 minutes of content after you remove the commercials).
• That works out to about 110,000 years (seasons) based on 12 episodes per year.
• In the case of music you could store about 321,685,000 CD-quality 3 minute tracks, which is about 1,800 years of music if you listened to each track right through once.

My view is that people will not end up with petabyte, or multi-petabyte, storage devices in their houses in the same what that we have (mostly) all ended up with terabytes of storage in our houses—whether they are network connected ‘servers’ or just USB disks laying around. If “the cloud” concept takes off, as the experts tell us is going to be the case, then we are all going to move more and more to storing our data “in the cloud”—which means on the Internet somewhere (if you missed it than you can check my posting “Microsoft’s Office Web Apps: Part 1” for a little more about ‘the cloud’). So, this being the case, then we are more likely to end up storing less data at home and paying a monthly fee to store all our stuff “in the cloud”.

The following snippet comes from the a paper presented by the Utilities Industries.

As an aside—In my house I have an online Acer EasyStore Windows Home Server configured with 4TB. At this point it is about half full.