Data Backups

This is VITAL to every person.

People don't realize what they will lose until their hard disk crashes. Companies have gone out of business over just a centralized server hard disk crash with no backup (but none of my customers!). You risk losing all customer email addresses and contact info, all email, accounting, contracts by email, etc.

I have been thru this sad situation with so many people. Often, that's why they called me in the first place. The traumas and dramas of losing everything, like having the house burn down with all your photos and documents ...

There are two parts to this: 1. Choosing backup media, such as CDs, DVDs, and extra hard disks. 2. Choosing backup program.

Media:

Over time, backup media change. I use CDs and now DVDs as the final media, though not day-to-day (see next paragraph). In contrast, I never used tapes nor zip drives for myself. In the 1990s, I started using the CD medium. I have data going back to the 1980s which is now on CD though I did lose some things, long story. Before CD, I did backups mainly using extra hard disks, normally two backup hard disks for redundancy for my own documents, email, and configuration files, but only one backup hard disk for everything publicly available such as downloadable programs.

Unlike tape and zip and other specialized media, a CD can be read by any computer without any special hardware or software, especially if the files were copied using simple XCOPY (see below), no compression, nothing fancy. It is as simple, bug-proof and timeless as you can get now. Recently, I've switched from CD to DVD, which has about 7 times the storage capacity of CD, but it's the same principle. However, the process can be slow and laborious, so I do archiving to DVD only once in a while.

For quick and convenient backups, I have always used multiple computers over a LAN, copying files over the LAN to a \backups directory. I still use at least two backup hard disks for redundancy, and now I also use external hard disks by USB for backups. Since hard disks have become so cheap, I have many backups now, with backup hard disks both on-site and off-site.

Backup program:

So, how to copy the files between hard disks? Still today, and going back to pre-Windows MS-DOS days to around 1990, I have always used simple XCOPY (and before XCOPY existed, a similar shareware utility maybe copied by Microsoft). I never use compressed files nor esoteric programs with potential bugs. Just proven and simple XCOPY. XCOPY is still my primary program.

Then I use a second backup process, a synchronization between two hard disks, file by file (again, no storage of data files in any proprietary format, and no compression).

There are many backup programs on the market, but I find risks and faults with practically all of them, especially those which use compression (bug risk), their proprietary storage format (more bug risks), and their own system of organization (you must learn a new system...). I have seen people rely on one backup program, but it turns out to have bugs. You don't know how good (or bad) your backup is until you need it -- and then it's too late! I am a strong believer in double checking and even triple checking when it comes to vital personal data.

For myself and my past customers, first of all, I put all original data onto a separate partition of the hard disk such as D: or a network drive. Then just XCOPY all of D: to your other disk (external hard disk or over your LAN) such as from D: to X: or Z: or whatever, and not copy your operating system and installed programs. Never keep a backup on the same partition (such as a drive E: of the same disk), only a different physical disk.

XCOPY is a DOS program which has been with DOS and Windows since around 1990. It is extremely reliable, bug free and simple. It simply copies files. Nothing fancy.

I do the XCOPY by a batch file with something like this, whereby only the first line is all you need:

xcopy d:\*.* z:\ /m /s /c /i /y /h /r
if errorlevel 1 goto errors
@echo There were no errors.
goto end
:errors
@echo There was at least one error. Maybe rerun it and sit and watch it this time, or report it to your techie.
:end
@echo Program finished.
pause

The /m means copy only modified and new files, /s means all subdirectories, and the rest are just to keep it simple and be complete. The Z: drive is just my example, and you can specify whatever drive you wish. If you plug in your external hard disk and it's drive F: then substitute F: for Z:

The second and lower lines have nothing to do with the backup, and are optional. They only report what happens. The "pause" command is important so you can see the result, to show you any errors or that it was complete. Otherwise, the window will automatically close itself upon completion and you won't see the final result.

This batch process can be just be put into a .BAT or .CMD file and a shortcut icon put onto the desktop. I do this for novice staff who keep some data on their workstation. They don't see the commands. They only see an icon on the desktop and click on it. In the end, they just look to see if there were any errors. (You must keep things as simple, quick & easy as possible for most people. I also have a way of reporting errors automatically to the techie, but that's beyond the scope of this article.)

I do a similar "second stage XCOPY" batch file to copy newly backed up files from my backup hard disk to DVD or CD, or to an additional backup hard disk. That's beyond the scope of this article but if there's interest I can elaborate.

Having backup files on write-once CDs protects you in case you accidentally overwrite a data file, or if a data file gets corrupted, whereby you can always go back to find an earlier version.

So you have a stack of CDs in chronological order. How to find out which file is on which CD? I do a "dir e: /s>d:\backups\index\08-04-16.txt" on every CD (where e: is the CD drive letter in this example, and d: is my primary data location), which will list all files on the CD in a text file. The 08-04-16 means 2008 April 16. I keep all those index files in one directory on my hard disk (which also gets backed up). I also handwrite 08-04-16 onto the CD. If I need to look for an earlier version of a file, then I can just search for the filename in that directory. I actually have a simple way of doing this. (Notably, if adding to an index file, use >> not > so it appends instead of overwrites.)

The CD backup is kept offsite. If the office or house burns down, or a mother nature or a thief comes in and cleans me out, or whatever, I still have all my life's data. There are a lot of bad and crazy people in this world, as well as just human error in technical things.

I actually have backup hard disks and stacks of CDs in multiple places. You can get 150 GB for just $100 so why not make extra redundant backups of all your data and stash them away at various sites. Just encrypt your passwords and usernames very well (a modification to the above, but I won't go into that here).

I also keep an 8 GB USB memory stick in my pocket, and another xcopy icon on my desktop to copy new files to the memory stick. This is convenient for quick backups on the road after a major writing session, without needing to burn a CD or carry along the backup hard disk.

My backup hard disk is always stored in a separate place from the computer! Backups must stay "off-site". I duplicate it occasionally at a second "offsite".

There are three disadvantages to this XCOPY method:

First, if you move files and folders on your data drive, then you get duplicates on your backup. Also, if you delete or rename files, you still have the old one on your backup hard disk. This is why the XCOPY backup is not my only backup, but only my "extra" backup.

For my primary backup, I use a program called Folder Match or sometimes Total Commander to synchronize between two different disks. Either program, both seem fairly bug free in their synchronization modules. This is my primary backup, but the xcopy backup is actually my redundant backup #2 which is available in case there is a bug in these two synchronizing programs. You can also use two different synchronizing programs to double-check each other for bugs, using two different backup hard disks. I do this occasionally, then wipe clean the xcopy backup and run the command without /m to get a fresh new copy (overnight).

However, the second stage XCOPY process is vital for the CD backups, cannot use a synchronize program for that!

I also use the ATTRIB program occasionally to smooth over custom tweaks, such as ATTRIB -A or ATTRIB +A on directories and files.

Complications:

Programs like MS Outlook create one big file for all email by default. Therefore, if your email file is now 1 gigabyte (many business associates of mine have multi-gigabyte Outlook files now), with lots of email and file attachments added up over the years, then add one little email and it backs up the entire gigabyte file again. I have various workarounds with Outlook, such as splitting up PST files, or the command "attrib -a *.pst /s" before running XCOPY. (You must also reconfigure Outlook to put all PST file(s) onto D:, but that's not the only issue with Outlook backups.)

Outlook Express and Thunderbird are better that Outlook in that they split mail by folder and/or account, so you back up only the folders which have changed. I use Thunderbird, not Outlook, for reasons other than backup, so the Outlook megafile is a hassle I can avoid. Thunderbird is better for other things, too, but for backups it's best.

The above XCOPY (or FolderMatch or Total Commander) method will get your main data, but it may not back up your Outlook email phone book and other things, and it surely won't back up your programs and their configuration settings. You can use ABF Outlook Backup and ABF Outlook Express Backup if you use Outlook, which are great tools but they are not bug free. (I use Thunderbird, so I have none of these problems.)

For programs and settings, such as Windows and all your installed programs, plus all your configuration settings, the only quick and easy way to do this is by a sophisticated third party global backup program, such as by Norton Ghost or PowerQuest DriveImage or Acronis Workstation. The problem is that you cannot restore a Windows setup to a new set of hardware using any of these programs. For example, if your workstation hard disk crashes, then you can just put in another hard disk and restore everything exactly the way it was. However, if you upgrade your computer or if your mainboard dies and must be replaced, then the restore will not work. (Actually, it worked fine with Windows 95 and 98, but this has always been a problem with the Windows 2000 and XP lineage of Microsoft products.)

That means after a hard disk crash you must reinstall all your software and registration keys, and reconfigure it all.

Acronis has its Universal Workstation Restore add-on which claims to have a solution for exactly this problem, but when my portable notebook PC died, my Acronis Universal Restore failed miserably. I used the Universal Workstation Restore, but XP bluescreened immediately upon bootup. Not after awhile but immediately. Obviously, it wasn't even close to working. It makes me wonder whether some of the positive reviews for this product on the web were planted by its employees... However, Acronis looks as good as Norton Ghost and PowerQuest DriveImage for normal restores, and Acronis has never failed when restoring to the same hardware, but with this important extra feature advertised, and paying more for this add-on extension, it was disappointing. I used DriveImage until it was bought up by another company and started getting troublesome. Acronis is not cheap. I spent about $150 on Acronis. But maybe I just need to explore it some more to learn its ideosyncracies and workarounds for Universal Restore. Acronis also creates bootable media to a USB memory stick and CD.

Maybe someone out there knows something I don't, about how to restore a Windows backup to new hardware. Or how to get Acronis to do this. Or another program which offers this. I searched hard up thru mid-2007, when Acronis was the first and last.

Email programs - Mozilla Thunderbird vs. MS Outlook and Outlook Express