Before Dreamhost began offering personal backup space countless users were (in violation of Dreamhost’s Terms of Service) simply helping themselves. It was simply too easy for a Dreamhost customers to abuse the generous hard drive space provided buy the discount shared-hosting service.

Rather than continue to fight a growing, unpopular war with its customers, Dreamhost has offered up a sanctioned alternative for personal backups. The caveat is that personal backups may only be accessed from a separate FTP account and they can’t be accessed by a webserver.

This is likely to curb the amount of piracy taking place on Dreamhost’s servers and reduce the number of DMCA requests their office receives. Dreamhost should be commended for making the best of the situation for both their users and the copyright holders involved involved.

Turns out there is a bug in Leopard’s Disk Utility whereby you must use the command line to rebuild degraded RAID sets. If you are getting the infamous ‘Unrecognized Filesystem’ error when you try to rebuild your RAID set follow the instructions below.

1. Erase the RAID disk storing degraded slice in Disk Utility. It will be the one labeled in red.
2. Open up Terminal. Run:

   diskutil checkRAID

   you will see something like this:

   RAID SETS
   ---------
   ===============================================================================
   Name:                 thriller
   Unique ID:            B63E8B4C-97D7-4A90-960B-B5DCB9030731
   Type:                 Mirror
   Status:               Degraded
   Size:                 499972669440 B
   Device Node:          disk3
   Apple RAID Version:   2
   -------------------------------------------------------------------------------
   #   Device Node       UUID                                   Status
   -------------------------------------------------------------------------------
   0   -none-            68269FE2-0C68-4C2D-B129-FFF1DA5999EC   Missing/Damaged
   1   disk1s3           108564B3-F353-4740-BF21-72F89E881E46   Online
   ===============================================================================

   Note the RAID’s Unique ID. (in this case it’s B63E8B4C-97D7-4A90-960B-B5DCB9030731)

3. Next run:

   diskutil list

   it will spit out something like:

   /dev/disk0
      #:                       TYPE NAME                    SIZE       IDENTIFIER
      0:      GUID_partition_scheme                        *111.8 Gi   disk0
      1:                        EFI                         200.0 Mi   disk0s1
      2:                  Apple_HFS dangerous               111.5 Gi   disk0s2
   /dev/disk1
      #:                       TYPE NAME                    SIZE       IDENTIFIER
      0:     Apple_partition_scheme                        *465.8 Gi   disk1
      1:        Apple_partition_map                         31.5 Ki    disk1s1
      2:                 Apple_Boot Boot OSX                128.0 Mi   disk1s2
      3:                 Apple_RAID                         465.6 Gi   disk1s3
   /dev/disk2
      #:                       TYPE NAME                    SIZE       IDENTIFIER
      0:     Apple_partition_scheme                        *465.8 Gi   disk2
      1:        Apple_partition_map                         31.5 Ki    disk2s1
      2:                  Apple_HFS Untitled                465.6 Gi   disk2s3
   /dev/disk3
      #:                       TYPE NAME                    SIZE       IDENTIFIER
      0:                  Apple_HFS thriller               *465.6 Gi   disk3
   

   here we are looking for the Identifier of the freshly erased drive. (in this case it is disk2s3)

4. The last step is to run

   diskutil addToRAID member <drive identifier> <RAID Unique ID>

   so for me the command looks like:

   diskutil addToRAID member disk2s3 B63E8B4C-97D7-4A90-960B-B5DCB9030731

5. Hold on tight. When I ran the command above I got a progress window indicating that the rebuild process would take 18 hours. It actually took 22 hours. If you don’t get the progress window you can check the progress with

   diskutil checkRAID

Hopefully this saves you a bit of trouble. This is the second time my RAID set has been degraded and I’m beginning to doubt the wisdom of running software RAID at all but for now I’m back on the horse.

This article will focus on how to setup a personal backup using one of the suddenly ubiquitous shared hosting companies like DreamHost, LunarPages or BlueHost. If you already have a account with one of these companies it won’t cost you anything to start backing up your data to a remote server. If you don’t already have an account and want to setup online backup you have two choices:

1. Forget RSync, save yourself some trouble - sign up with one of the online backup services like Mozy which is priced nearly identically to a shared hosting account.
2. Buy a hosting account and forge ahead - if you’re reading this you probably want to get your hands a little dirty and a shared hosting account lets you do all sorts of neat things like host your own domain, web apps, etc.

Note that you will need a user account on your server with SSH (somtimes called Shell) access permission. This is offered by most but not all shared hosting services.

Now that you have all the pre-requisites let’s outline the setup process.

1. Set up key-based authentication. We do this so you don’t have to enter your password every time your backup runs.
2. Write a backup script.
3. Automate the execution of your backup script.

Not too bad right? At this point the instructions fork into Windows and Mac OS X branches. Because Windows users outnumber Mac users they get a write up first. I promise the Mac write up is coming soon.

Windows Rsync Backup Instructions

Again, you’ll need to download and install cwrsync.

1. Set up key-based authentication
Use PuTTY to login to your account via SSH. If you don’t have PuTTY you can get it here for free. After you’ve logged in, run these commands:

mkdir ~/.ssh
cd ~/.ssh
touch authorized_keys
chmod go-w $HOME $HOME/.ssh $HOME/.ssh/authorized_keys

Next open up a command prompt on your local computer. (Start -> Run -> cmd) When it opens paste this command into it:

ssh-keygen -t rsa

Hit enter every time you are prompted to accept the default values. (three times)

The next step is the trickiest part of the process but it’s not too bad. You need to copy your newly minted public key to your remote hosting account. We are going to use rsync for the first time to do it, how exciting! Replace the <username> and <yourdomain.com> with your hosting account username and domain and run the command below.

rsync -avrz –progress "/cygdrive/c/Documents and Settings/Administrator/.ssh/id_rsa.pub" <username>@<yourdomain.com>:/home/<username>/.ssh/authorized_keys

If you get an error when you run this command, double check your paths. Maybe your Windows username isn’t ‘Administrator’ or maybe your shared host doesn’t put your files in /home/<username>/

When you get it to work you’re all set up to securely transfer files to your hosting account without a password. Congratulations! Time for step 2.

2. Write your backup script
Below is the command for backing up a local folder.

rsync -avrz –progress “/cygdrive/c/foldertobackup" <username>@<yourdomain.com>:/home/<username>/backup

So by now you are hopefully beginning to figure out how this works.

CwRsync comes with a sample script to demonstrate what a backup script should look like. Open C:\Program Files\cwRsync\cwrsync.cmd in your favorite text editor (notepad works fine) and add a line using the command above as the formula for each folder you’d like to backup. e.g.

rsync -avrz –progress “/cygdrive/c/Documents and Settings/Administrator/My Documents" <username>@<yourdomain.com>:/home/<username>/backup
rsync -avrz –progress “/cygdrive/c/Documents and Settings/Administrator/Desktop" <username>@<yourdomain.com>:/home/<username>/backup

3. Automate the execution of your backup script
Luckily this step is built into Windows so we’re done with the command line. Hooray! Click Start -> Programs -> Accessories -> System Tools -> Scheduled Tasks

Double click “Add Scheduled Task” to launch the wizard. When it asks you which program you’d like to run click browse and find the backup script you just made @ C:\Program Files\cwRsync\cwrsync.cmd

Give the task a name, choose your backup interval and you’re done! Hopefully you sleep a little better at night knowing that your data is safe and sound.

Why do you care? You should care if you are a computer user with large data storage requirements who uses a single hard drive for archiving your files. Obviously if your drive fails you are going to be in a world of hurt. With a RAID system, you set it up once and then don’t think about it again until a drive fails, at which point you won’t lose any data, you just replace the failed drive and move along.

RAID is an umbrella term for a few distinct systems which are identified by number. We will only discuss the RAID configurations that a typical computer user might use to achieve data redundancy and therefore fault tolerance.

RAID 1
Also known as ‘mirrored’ disks, RAID 1 uses two hard drives to store identical sets of data. If one disk in the array dies or is corrupted no data is lost because the other disk contains a ‘mirror’ of the failed disk’s data. The working disk will act as a single drive until the failed disk is replaced. This is simplest and most practical flavor of RAID for the average computer user. All that is required for RAID 1 is pair of identical hard drives.

RAID 5
RAID 5 requires a minimum of three hard drives but offers an efficiency advantage over RAID 1 while still providing fault tolerance. It accomplishes this by spreading data across 3 or more disks instead of mirroring it on two drives.

e.g.
2 x 150GB drives in RAID 1 = 150GB
3 x 100GB drives in RAID 5 = 200GB

There are two approaches to implementing RAID:

1. Software RAID usually takes the form of a utility application in your operating system. All modern operating systems, including Windows and Mac OS X, provide everything you need to setup and use a RAID.
2. Hardware RAID controllers physically connect to multiple disk drives and make them appear as one to the operating system. By providing dedicated resources to the duty of implementing RAID, hardware-based RAID offers a performance advantage over software RAID.

Hardware RAID comes in many forms. From sleek and sexy drives to enclosures evoking UFO aesthetics. From motherboards with baked-in RAID to RAID controller cards.

RAID offers a unique proposition in the backup world by providing real-time high-speed data backup with next to no downtime in the event of a failure.

It is not a pure backup in the sense that if you accidentally delete a file it will be deleted on both drives and irretrievable.

RAID fills a niche for users who need to backup very large data sets. For users who need to backup gigabytes, not megabytes, spending weeks sending files to an online backup service over their cable modem simply isn’t an option, and having a hot copy of their data when a drive fails is a godsend.

Their pricing model @ $49.95 / year for unlimited backup is less expensive that most of their competitors. Carbonite’s software only runs on Windows XP and Windows Vista, so Mac users will need to look to another serivce, like Mozy. For Windows users, Carbonite offers a level of integration that its competitors can’t match. Their software allows you to see right from Windows Explorer (i.e. Desktop, My Documents, etc.) each file or folder’s backup status.

Carbonite’s security model is very similar to the other major players in the market, that is to say, very good. All files are encrypted twice; you may optionally manage your own private key to prevent even Carbonite employees from decrypting your data.

All-in-all Carbonite’s backup service is a very appealing one to Windows users who need simple online backup. The simplicity of their service combined with the depth of its integration into the Windows operating system and low price create a perfect storm for a great number of users. Odds are, one of those users is you.

Click the banner below to get started started with Carbonite.

Installation on my Macbook Pro only took a few minutes. When it started up, it showed me a list of things I might like to backup. I simply checked the items I wished to backup, clicked start, and Mozy started backing me up.

Transfer rates were respectable, if not spectacular. The transfer rates would probably be higher if Mozy didn’t encrypt everything. Twice.

That’s right, your files are encrypted once locally with 448-bit Blowfish encryption and then again with 128 bit SSL when they are transferred. Mozy clearly wants you to feel that your data is safe. You have the option of using a Mozy encryption key, or a personal encryption key. If you go with a personal key, be sure to keep it somewhere safe because if you lose it you’re out of luck. Most people use the Mozy key, which in the event of an emergency means Mozy can help you decrypt your data.

If you outgrow your 2GB free account (easy to do for music or photo buffs) you can upgrade to an unlimited account for $4.95 / month. A steal when you consider the alternatives:

1. Praying that your hard drive won’t crash and you won’t get a virus (doesn’t work)
2. Backing up to hard drives or DVD-R and keeping them off-site. Safe from fires and burglars (works but a hassle)

I’ll be counting on my Mozy account to save my butt if (when) something bad happens to my computer.

Click the banner below to sign up for a free Mozy account.

The service is based on their Microsoft Windows software application which I tested this week. I should note that I am a Macintosh user and BeInSync doesn’t have a Macintosh compatible version of their software so I tested it in a virtualized Windows environment using Parallels. That said, setup was a snap. I signed up for their free trial and downloaded the BeInSync application which is only 9.5MB. I was up and running in under 5 minutes.

When you first launch their application you are greeted by the Launch Pad. The launch pad presents you with four basic tasks: Sync, Share, Access and Backup. For power users, there is also a button to launch directly into the full featured app.

The Sync feature keeps it pretty simple, you are presented with a list of folders you might want to sync across multiple computers like My Documents, your Desktop, your browser bookmarks, your email and your contacts. Unfortunately, the email and contact sync only works with Outlook and the bookmarks only work with Internet Explorer. Users of Firefox or other browsers and email clients are out of luck.

The Backup feature is also straightforward to use. The interface looks a lot like the the Sync feature interface with a few more options. First, you are presented with a tree view of your computer’s hard drive(s). You simple check a box next to a folder to at it to the list to backup. After you have identified all the folders you would like backed up, you schedule the backup. The default schedule is called “Hot Backup” which “automatically keeps all your files backed up all the time.” You can also set daily, weekly or manual backups. The last setting on the schedule page allows you to set the number of file versions to keep. I can see this feature being very powerful in combination with the Hot Backup setting. The last step in setting up your backup is to give it a name and begin the first upload.

Upload speeds were very steady and adequate, they came just shy of maxing out my cable modem upload speed. BeInSync is an incremental backup tool, so the first time you upload your documents takes a little while but subsequent backups go much quicker.

All-in-all the backup feature, which is BeInSync’s flagship feature, measures up quite well. Set up is simple, even for a beginner. Speeds are good, and the 14-day free trial lets you test it out before you spend anything but a little time.

The Share feature works but it isn’t as feature rich as other features of this product. To share files you choose the files you want to share in the application, type in the email address of the person you wish to share with and they are sent an email containing a link to get the files. Before the recipient can download the files they must sign up with BeInSync. This isn’t much of a problem though because BeInSync makes it dead simple to sign up. I literally did it in a matter of seconds.

A bigger potential pitfall here is that the files you share are not stored on BeInSync’s servers. That means your computer must be on and signed into BeInSync for your friend to download the files you sent them. So if, for example, you turn off or restart your computer during the file transfer, it is interrupted and must be restarted. Your friend is also limited by the upload speed of your internet connection which is almost certainly less than that of a typical server. Ultimately BeInSync’s approach to file sharing is comparable to the file sharing built into AOL Instant Messenger, it works, but not that well. You are probably better of using one of the numerous free file sharing services like ZShare or DropSend if you need to share large files with friends or coworkers.

Various settings and preferences in BeInSync’s application let you fine-tune its behavior to match your needs and taste. Setting let you specify the event you would like to be notified about and allow you to throttle both the upload and download speed. Limiting the bandwidth the app is allowed to use is essential for any respectful user of a shared internet connection. It’s also worth noting that BeInSync uses SSL encryption when it transfers your data across the internet which prevents a third party from intercepting your files.

BeInSync isn’t the most flexible online backup solution available but it may be one of the easiest to use. If you are a Windows user without a lot of requirements BeInSync is probably a good fit for you.

Click the banner below to sign up for the free 14-day trial.

Complete backups seem, on the face, inarguably better. So why would anyone choose to go with partial backups? The main reason is that it’s somtimes just not necessary. Your operating system and other programs take up a lot of space, rarely change, and can easily be restored from installation CDs. Operating systems are also very fragile things. In many cases if you experience major hardware failure you will need to reload your operating system anyway. If you are keeping multiple versions of your backup (and you should be), complete backups can quickly become a hefty amount of data to store.

Even though complete backups aren’t always necessary, they are very powerful tools. If you do have catastrophic data loss, a complete backup can get your machine up and running in its previous working state in a matter of hours. If you have a lot of software that requires installation and configuration you could spend days getting it all set up again. Users for whom downtime is very expensive are the best candidates for a complete backup plan.

The best of both worlds
It’s very possible (and quite prudent!) to implement a hybrid strategy. A bi-monthly or weekly complete backup combined with a daily partial backup will keep you covered in most cases while keeping total backup size at a manageable level.

The answer to this question depends on a number of factors including how much data you need to secure and the rate at which you create new data. Each backup type has unique strengths and weakness. There are more backup options available to you than you have time to consider so it helps to narrow your choices down to a manageable size. The major types of backups are:

1. Online Backups (also called Remote Backups)
Online backup services use the Internet to transfer data securely from your computer to a server managed by a service provider. There are many advantages to this type of backup. Because the server that stores your files is in a geographically separate location, your data is secure from environmental dangers like flooding, fires, earthquakes and burglars. Also, because you are leasing the hardware that is storing your files, it if fails, your service provider is responsible for replacing it.

Of course, by outsourcing those responsibilities you give up some control. It is unlikely, but entirely possible that a backup company could disappear overnight, taking your backups with them. Putting your data in someone else’s system also forces you to trust them not to make your data available to identity thieves or other crooks. In some cases you may be legally required to use a service provider that complies with regulations placed on certain industries, e.g. HIPAA for the Health Industry.

2. Local Backups
Local backups work via a large variety of methods including hard drives, tapes, CDs and DVDs. The one thing all these methods have in common is that if you choose to use them, you take on full responsibility for administering the backup process. This maybe isn’t as bad as it sounds because you have a huge number of tools to choose from that will help you do the job.

By taking on more responsibility you gain complete control of the process. If you have unique requirements, this is the route you will have to take. If your requirements are small, online backup is probably where you need to be.

We’re only getting started
In future articles I will dig into the specifics of both Online Backups and Local Backups. The good news is, because the average person’s backup needs are increasing every year, backup utilities have improved greatly in recent years and there are now more options than ever before.

• They don’t realize the risk of data loss is great
• They don’t fully understand the consequences of data loss
• They are intimidated by the process and don’t know where to begin
• Data backup is not sexy

This site will explore all of these issues in some depth with the goal of helping people understand the importance of backing up their data. It will also examine various approaches to securing your data.