Are you backing up your data? Are you sure?

(Originally written and posted on June 5 2011)

This past week, I was reminded of the importance of having redundant backups of all of your data. Data redundancy should never be an after though, and as our industries become more and more reliant on digital media, this is becoming even more important. In the event of data loss from events such as a fire, flood, theft, or even a simply media failure such as a hard drive crash, an entire practice could crumble. The recovery efforts can be very expensive, and often not be successful.

A professional in her field, of whom I have done periodic IT work for over the years, phones me in a panic. Her database, which contained all client and other imperative data used in her day-to-day operations became corrupted. The database file size was reading 0 bytes , and none of her backups were working. Before phoning me, she spoke with the tech support of the company who develops the CRM software her company was using to get their assistance in recovering the data. All their tech support was able to tell her was “you must have done something wrong...” Also before phoning me, they dropped off the backup disks at a local IT shop in an attempt to recover the backups.

Now, although this company has their front end computers networked so that they assist more than one client at a time, these machines, including the database, are completely disconnected from the internet. This means that backups must be physically removed from the building at the end of the day to ensure that they are kept safe. They were diligently taking nightly backups, and taking turns taking these backups home, so that over the course of a week, there were 5 backups scattered across 5 different locations. However, the media they were using to store this data, and the method they were using to handle the media, was somewhat concerning.

You see, this particular location was using CD-RWs to hold their backups. They had a box full of CD-RWs stored on a shelf under their computer storing the database. However ... none of these CD-RWs had cases. It was simply a box full of random rewritable disks without cases. Their process at the end of the day was to reach into this box and grab a random disk, burn the latest data file to the disk, put the disk into their purse (still without a case), and take it home with them. A week later, they would put the disk back in their purse, and bring the disk back to drop back into the box. Over the years, these disks have gotten so scratched that they were virtually unusable. In addition to using disks that were scratched, these backups were obviously never being verified.

Now this particular company has two offices in two different communities. A few years ago, a similar problem occurred, where their database had been lost. We were able to recover the data to a point where very little data was lost. At that time, I reworked their backup strategies and suggested new practices that best suited the technical ability of the people who would be doing the work, and the technical limitations of their restrictive architecture. This new system was a replacement for the CD-RWs that they were using, so I was surprised to learn that they were still using these disks. When I asked why they were still using the old media for their backups, they informed me that they only adopted the new strategies at one of their locations. The other location, where she was calling me from, never changed.

This is a horrible situation to be in. Any time data is lost, especially data that is vital to your day-to-day business, it can cost your business a great deal of money and lost income. However, there are things that can be done to mitigate the risk and impact of data being lost. These include:

1. Backup, backup, backup. If the data doesn’t exist in at least three places, it might as well not exist at all. 
2. Store at least one of your backups off-site. This will help prevent the chances of data being lost to fire, flood, theft, and natural disasters. 
3. If possible, consider using cloud storage for your backups. This has a number of advantages such as having a low-cost professional team responsible for protecting your data, as well as having the data stored in another city, further reducing the chance of loss from natural disasters. 
4. Test your backups frequently. Make sure that you can recover if an emergency arises. Don’t just assume what you are doing will work tomorrow. 
5. Don’t use damaged media to store your backups!!! 

In this particular situation, after nearly an entire day of wasted labour, expense, and stress, they managed to find a backup from 3 days earlier that was readable. So they lost 3 days worth of data. But this is worlds better than what could have happened. The IT shop they delivered their most recent damaged CD to was unsuccessful in recovering any data.

I suggest that each of you revisit your data backup strategies, test them, and make sure that they are actually effective. Do this now, and do this regularly. Otherwise, it is only a matter of time before disaster strikes.

- Mark Bass