Speed versus Size

The astounding record of ever-faster computer processors often leads to a mistaken generalization: everything about computers is getting significantly faster every year. If you believe this, you won't be able to understand an important trend in backups.

While many components in computers are improving every year, virtually none of them have been getting faster at the rate that that the CPU (central processing unit, e.g. Pentium) has been getting faster. To your CPU, everything else in your computer is terribly, terribly slow. Even accessing data in main memory is very slow for the CPU, compared to accessing data it has already loaded into its internal CPU cache. The key issue here is that ever-faster computer processors are not going to help the fact that the devices we use to store backup data are not seeing such rapid improvements in speed. No matter how fast your comptuer processor is, if you have a lot of data to back up, it's going to take a lot of time.

Hard Disk Speed Versus Size

Moore's law says that CPU speed doubles about every 18 months. Moore's law has held true for two decades now, resulting in a rate-of-improvement curve for CPUs that has single-handedly altered the landscaping of computing.

Despite the astounding improvement rate of CPU speed, the rate of improvement of hard disk capacities has outstripped it. Hard disk space capacity is doubling roughly every 12 months! As I write this (early 2004), hard disk space can be easily had for around $?? per gigabyte.

In 1990, an 8GB hard disk would have seemed wastefully large for most computer users. Predictably, however, this explosion of available disk space began to spawn applications that had been unthinkable in the days of sub-gigabyte hard disks. In particular, the ability to store, edit, and view video on computers (combined with the affordability of high-quality consumer digital camcorders) promises to ensure that this embarassment of disk space riches will be spent at an equally embarassing rate for many computer users.

Hard Disks to Backup Hard Disks?

Are hard disks really becoming the best solution for backing up other hard disks? This is perhaps more unintuitive to experienced computer professionals, who have historically always had some somewhat slower media that was more cost-effective for backing up data than a hard disk.

If this unintuitive solution is becoming the best solution, it is only because of the vicious pace of capacity improvements in hard disks. If we can write to a hard disk roughly as fast as we can read from one, then no other backup media can offer a real speed advantage -- the bottleneck is the speed of reading a hard disk. And if a commodity product like the hard disk is experiencing a blistering pace of improvement and cost reduction, it's difficult for any new alternative technology to get the foothold needed to grow.

The history of the CPU provides an analogous situation. The CPU has various bottlenecks that could, logically, be overcome by adding other independent processors to the computer. For example, a simple processor attached to the hard disk could create a "smart" hard disk that could offer features such as searching for data while the main CPU was doing other work.

But the attempt to improve the computer by offloading work from the CPU has largely been obliterated by the incredible rate of improvement of CPUs. Floating-point number calculations that used to be provided by a separate chip have been folded into the CPU. Attempts to offload network protocol work onto "smart" network cards have failed to achieve much market penetration -- just wait 12 months and the CPU will speed up significantly anyway. Printer manufacturers have transferred image processing work off of the printer and into the software that drives the printer -- the result is cheaper printers and printing that gets faster as CPU speeds increase without any investment by the printer manufacturer.

Tape Speed Versus Size

The capacity of backup tapes has shown steady incremental improvement, but it simply has not kept up with the improvement curve of hard disks. Compared to hard disks, backup tapes have been improving only slowly.

The most interesting

Hard Disk Versus Tape Backup

Tape backup began its life when hard disk space was very, very expensive. However, as I write this in early 2004, we are seeing the effects of the magnetic tape improvement curve failing to keep up with the hard disk improvement curve. The cost of a gigabyte of backup tape is now more than the cost of a gigabyte of hard disk space, which is a stunning development. Cost of media was one of the most compelling reasons to use it for backing up data, but that motivation has been nearly eliminated.

Removability

One advantage of tape over hard disk backup has not been removed: removability. Magnetic tape has long been designed to be portable and removable. The tape is in protective shell cartridges that can be easily inserted and removed from the tape drive.

Hard disks, however, have long been designed to be mounted permanently in a computer, and not removed. There are hardware solutions that allow you to make an ordinary IDE hard drive removable, but they suffer from a variety of niggling problems: keeping the now-enclosed hard disk cool enough, not having the extra set of pins and connectors cause problems in data transfer rates, sometimes requiring that the computer be turned off in order to safely remove the drive, and so on.

For hard disks to really match the ease of removability of magnetic tape, the hard disk would have to have an enclosure designed for portability, and the PC would have to have an affordance for easily inserting and removing the portable hard drive without having to reboot the computer or risk losing data.

In fact, such portable hard disks have begun to appear, in the form of devices that compete with flash-memory-based USB backup devices. The first crop of such devices is too small for image backups (around 1GB), and more expensive per gigabyte than traditional-sized IDE hard drives. However, it is not hard to imagine such devices reaching a point where their cost and capacity matches that of magnetic tape cartridges.

Conclusion

Overall, backup device sizes are increasing much more rapidly than backup device speeds. Also, the proliferation of activities that involve audio, bitmaps, video, and the like mean that the amount of data to be backed up has been roughly keeping pace with the amazing increases in storage capacity.

One result of this trend is that image backups, backing up a full and complete copy of each hard disk containing important data, are becoming less practical. A really safe image backup requires that the disk being backed up be taken out of operation during the backup, and if the backup itself requires several hours, that is a less acceptable requirement. Thus, the pressure increases to move away from image backups to more incremental methods, only backing up what data has changed.

Another result of this trend is that it creates a market for addressing the problem of slow backup devices. The obvious solution, simply making the devices faster, has proven non-trivial. That leaves open less direct solutions, such as using multiple slow devices in parallel to achieve a much faster overall speed. Exactly what solutions might develop or become successful is impossible to predict, but there will be considerable interest in attacking the problem for the foreseeable future.

One advantage that tape has over hard disk backup is simply portability. Tape cartridges are relatively small, easy to remove and transport, and (barring heat, moisture, and magnetic fields), fairly robust. However, we may be seeing the advent of a new breed of portable hard disks. Already, you can purchase a 1GB miniature hard disk that plugs into a USB port and then drops into your pocket (and is not prone to the limited rewrite difficulties that similar devices based on flash memory have). 1GB is useless when you want to make an image backup of a 40GB disk, but it's hard to rule out the evolution of a compact, portable, and high-capacity device that lies somewhere between a full-sized hard disk and the drop-in-your-pocket USB disk.

Finally, the astounding increase in capacity and decrease in cost of hard disks compared to tape will continue to make tape backups less and less palatable. At this point, tape technology needs a breakthrough leap that offers faster speed, larger capacities, and lower cost per gigabyte. It's hard to see how continued modest incremental improvements in tape technology can stave off the continued move to abandon tape in favor of hard disk for backups.

MediaBackup Trends