jwz - PSA: backups [entries|archive|friends|userinfo]
jwz

  www.jwz.org
  userinfo
  archive
  rss

Links
[»| [DNA Lounge] [Blog] [iCal] ]
[»| [DNA Lounge Legal Defense Fund] ]
[»| [WebCollage] [LJ WebCollage] ]

PSA: backups [Sat, 29-Sep-2007 1:55 AM]
Previous Entry Add to Memories Tell a Friend Next Entry
[Tags|, ]
[music |Boris Mikulic -- Secret Knowledge]

Dear Lazyweb, and also a certain you-know-who-you-are who should certainly know better by now,

I am here to tell you about backups. It's very simple.

Option 1: Learn not to care about your data. Don't save any old email, use a film camera, and only listen to physical CDs and not MP3s. If you have no posessions, you have nothing to lose.

Option 2 goes like this:

  • You have a computer. It came with a hard drive in it. Go buy two more drives of the same size or larger. If the drive in your computer is SATA2, get SATA2. If it's a 2.5" laptop drive, get two of those. Brand doesn't matter, but physical measurements and connectors should match.

  • Get external enclosures for both of them. The enclosures are under $30.

  • Put one of these drives in its enclosure on your desk. Name it something clever like "Backup". If you are using a Mac, the command you use to back up is this:

    sudo rsync -vaxE --delete --ignore-errors / /Volumes/Backup/

    If you're using Linux, it's something a lot like that. If you're using Windows, go fuck yourself.

  • If you have a desktop computer, have this happen every morning at 5AM by creating a temporary text file containing this line:

    0 5 * * * rsync -vaxE --delete --ignore-errors / /Volumes/Backup/

    and then doing sudo crontab -u root that-file

    If you have a laptop, do that before you go to bed. Really. Every night when you plug your laptop in to charge.

  • If you're on a Mac, that backup drive will be bootable. That means that when (WHEN) your internal drive scorches itself, you can just take your backup drive and put it in your computer and go. This is nice.

  • When (WHEN) your backup drive goes bad, which you will notice because your last backup failed, replace it immediately. This is your number one priority. Don't wait until the weekend when you have time, do it now, before you so much as touch your computer again. Do it before goddamned breakfast. The universe tends toward maximum irony. Don't push it.

  • That third drive? Do a backup onto it the same way, then take that to your office and lock it in a desk. Every few months, bring it home, do a backup, and immediately take it away again. This is your "my house burned down" backup.

"OMG, three drives is so expensive! That sounds like a hassle!" Shut up. I know things. You will listen to me. Do it anyway.



Update: Mac users: for the backup drive to be bootable, you need to do two things:

  • When you partition the drive, use GUID, not Apple Partition Map;
  • Get Info on the drive and un-check "Ignore ownership on this drive" under "Ownership and permissions."
You can test whether it's bootable by holding down Option while booting and selecting the external drive.
linkReply

Comments:
Page 1 of 2
<<[1] [2] >>
[User Picture]From: [info]jwz
Sat, 29-Sep-2007 8:59 AM (UTC)

RAID

(Link)

P.S., RAID is a waste of your goddamned time and money. Is your personal computer a high-availability server with hot-swappable drives? No? Then you don't need RAID, you just need backups.
(Replies frozen) (Thread) (Expand)
[User Picture]From: [info]aaronlehmann
Sat, 29-Sep-2007 9:05 AM (UTC)

Re: RAID

(Link)

100% agree. I've had too many close calls with broken RAID implementations to even consider bothering with it anymore. I just back up using a similar rsync command to what you proposed. I'm planning to buy an external 750GB hard drive specifically for backups when they get just a bit cheaper.
(Replies frozen) (Parent) (Thread)
[User Picture]From: [info]barbedwirekiss
Sat, 29-Sep-2007 9:26 AM (UTC)

(Link)

The universe tends toward maximum irony. Don't push it.

Somewhere along the line I can see that being included in the set of most memorable quotes of all time.
[User Picture]From: [info]ghewgill
Sat, 29-Sep-2007 9:56 PM (UTC)

(Link)

The irony is, it won't.
[User Picture]From: [info]duskwuff
Sat, 29-Sep-2007 9:34 AM (UTC)

(Link)

As a counter-PSA, I'll point out that rsync doesn't behave entirely correctly on HFS. (Yes, even with Apple's improved version. It still screws up some random bits of metadata in the backup.) If this matters to you, you can either use ditto and live with the fact that it isn't incremental, or shell out $30 for SuperDuper, which is totally worth it.
[User Picture]From: [info]jwz
Sat, 29-Sep-2007 9:36 AM (UTC)

metadata and wires

(Link)

What does it screw up?
[User Picture]From: [info]33mhz
Sat, 29-Sep-2007 10:01 AM (UTC)

(Link)

Strangely, with the laptops I have, the display is way more prone to outright fail than the hard drive. I only wish there were a way to matter-compile a back-up of one of those.
From: [info]edge_walker
Sat, 29-Sep-2007 12:27 PM (UTC)

(Link)

Not voodoo at all.

[User Picture]From: [info]autopope
Sat, 29-Sep-2007 10:33 AM (UTC)

(Link)

I don't do this: I have a different backup policy.

I work on laptops.

When I buy a new laptop, I probably sell the last but one. The old one then becomes the hot backup machine. I never have less than two similarly-configured work machines.

(If the new laptop's drive is much larger than the old laptop, consider buying the old lappie a new hard disk.)

Use rsync to backup data from the current work machine to the old one every week. Meanwhile, every day (or more often) use rsync to backup critical data -- work files, email archives, not random MP3s -- onto an 8Gb thumb drive that lives on a keyring. Every couple of days, use rsync to backup the critical data to a server 500 miles away.

When you boil it down, this is essentially the same level of cover as [info]jwz's spare drive policy; it just uses a different kind of drive enclosure (the laptop).

Oh, as for Windows ... use rsync. (You did install Cygwin, right?)

Horrible confession time: my most recent laptop is running vista. And it's staying that way for, oh, another month or so (until Ubuntu Gusty Gibbon is stable and I have a spare few days to spend getting the wifi drivers to work). Again: rsync is available for Windows.
From: [info]wilecoyote
Sat, 29-Sep-2007 11:24 AM (UTC)

(Link)

Alternatively, if you don't want to fight with Cygwin (I mean, isn't that the Windows equivalent of "recompile your kernel"?), you can try cwrsync.

From: [info]everybody42
Sat, 29-Sep-2007 11:48 AM (UTC)

size matters

(Link)

Go buy two more drives of the same size or larger. If the drive in your computer is SATA2, get SATA2. If it's a 2.5" laptop drive, get two of those. Brand doesn't matter, but physical size and connectors should match.

Well, brand matters sometimes, as the typical user won't find out that models with seemingly identical sizes (e.g. 400 GB) but of different brands have different capacities (off by some bytes). That is no problem for isync, but if you backup your disk with dd (or any other partition copy tool) so you have a replacement to plug in, you should have the same capacity (or more).

Also, raid: I use a raid 1 setup on my main workstation (WinXP) and on a server (debian). Pro: backup disk is always up to date, reads are double as fast. Cons: If your disk does not die 'naturally', but from any physical cause (PC falls down, electrical problems), probably both disks are gone.

Any chance you tell us what prompted you to write this post?
(Replies frozen) (Thread)
[User Picture]From: [info]jwz
Sat, 29-Sep-2007 6:12 PM (UTC)

Re: size matters

(Link)

Which is exactly why my advice does not include the use of "dd".

I say again: RAID is a waste of your goddamned time and money.
(Replies frozen) (Parent) (Thread)
[User Picture]From: [info]allartburns
Sat, 29-Sep-2007 1:46 PM (UTC)

(Link)

(Option++;): Buy a DDS-4 tape drive, a copy of Retrospect and a bunch of DDS-4 tapes. Divide the tapes in half -- one set is the current backup set the other is the "keep at a friend's house" backup set. Swap between the two.

I backup four systems this way (two laptops, two desktops) and have saved my ass many, many times.
[User Picture]From: [info]nrr
Sat, 29-Sep-2007 4:02 PM (UTC)

price points

(Link)

This fails miserably because the price point for the tape drive and tapes is far higher than that of the two hard drives and their external enclosures. Really.

The idea is to do this sanely and cheaply when confronted with the problem as a home user.
[User Picture]From: [info]cowsandmilk
Sat, 29-Sep-2007 1:56 PM (UTC)

(Link)

is there a reason why rsync would be better than using carbon copy cloner?
[User Picture]From: [info]jwz
Sat, 29-Sep-2007 6:27 PM (UTC)

(Link)

Carbon Copy Cloner uses rsync. They are identical.
[User Picture]From: [info]malokai
Sat, 29-Sep-2007 3:10 PM (UTC)

Apart from rsync + resource forks being broken

(Link)

The `ditto` command can also be used to make one-off bootable backups, preserving the resource forks (for fonts, I guess, or any mac os classic applications you may be using if you're still running that ancient copy of quark?)

Other utilities like the carbon copy cloner can be used for this (useful for the 'third' backup drive option that you leave in your desk).

The 10.5 'time travelling' thing seems to just wander your filesystem looking for modified files and backing up previous revisions of the files, so at least things are looking up in the automated Mac OS X backup world(since this post is obviously inspired by a friend or relative's drive crashing?).

http://www.bombich.com/mactips/image.html
[User Picture]From: [info]houdini_cs
Sat, 29-Sep-2007 3:53 PM (UTC)

Re: Not that I disagree

(Link)

I'm awesome. Oh well, you can see what I meant to link to.
[User Picture]From: [info]decibel45
Sat, 29-Sep-2007 4:30 PM (UTC)

(Link)

Personally, I use rdiff-backup instead of rsync so that I get incremental backups, and eventually I'll by rsyncing that backup offsite.

But yeah, what he said. :)
[User Picture]From: [info]keimel
Mon, 1-Oct-2007 2:21 PM (UTC)

(Link)

Semantics aside, how is rsync not ending up being incremental. If I use the switches indicated in the original post, it says "only copy the new shit" and "delete all the old shit I deleted". It doesn't have to do what I would normally think of as a Level 0 anymore than one time, when the drive is synchronized the first time.

Seriously asking, not just yanking your chain or playing. Am I missing something?

Is it just because rsync automatically looks at the entire list of files first before determining what has changed? Or is there some other thing?

Thanks
[User Picture]From: [info]eliot
Sat, 29-Sep-2007 4:46 PM (UTC)

online backup useful too

(Link)

I do a complete backup at my office on to an external drive with SuperDuper! and then an rsync to StrongSpace of my Documents, Mail, Address Book, and Pictures. I've found this to be useful because not only do I have a completely off-site secondary storage, but I also can browse from a friend's house and find a photo or document I have on StrongSpace.
From: [info]tongodeon
Sat, 29-Sep-2007 5:11 PM (UTC)

Old and busted: cron / New hotness: launchd

(Link)

[User Picture]From: [info]jwz
Sat, 29-Sep-2007 6:48 PM (UTC)

Re: Old and busted: cron / New hotness: launchd

(Link)

That looks like a lot more typing for zero benefit.
[User Picture]From: [info]smin
Sat, 29-Sep-2007 5:40 PM (UTC)

Bless

(Link)

Does one no longer have to bless a Mac backup drive before it's bootable? Did that come in with Tiger?
From: [info]wootest
Sat, 29-Sep-2007 6:34 PM (UTC)

Re: Bless

(Link)

A bit-by-bit copy of every data on the drive will include the blessing. It's not in the partition table.
[User Picture]From: [info]ladykalessia
Sat, 29-Sep-2007 6:32 PM (UTC)

(Link)

I hereby invoke the [info]jwz-fanboy-advice=baiting.

I've got an iBook with a tiny hard drive, which I've been cloning to an external drive using CarbonCopy Cloner whenever I remember and/or whenever Apple Software Updater runs. Not a lot of important data here, but every once in awhile I dump my documents folder onto a CD and put it away somewhere.

I remember hearing horror stories of a graphic artist somewhere who had impeccable backup technique, except that they were all sitting by his computer when his house burned down. Leetle paranoid after hearing of that.

Eventually I'll upgrade my desktop paperweight and have something running OSX, at which point I will come back here and follow the above advice.
[User Picture]From: [info]lohphat
Sat, 29-Sep-2007 6:34 PM (UTC)

Fire, schmire.

(Link)

From: [info]ausfish
Mon, 1-Oct-2007 10:20 AM (UTC)

Another example of why you need off-site backups

(Link)

$20,000 reward for stolen memories

She said her husband had made back-ups of the computer files but they were left in the laptop bag, which was also stolen.
[User Picture]From: [info]lohphat
Sat, 29-Sep-2007 6:40 PM (UTC)

*yawn*

(Link)

Basic rules of IT:

1. Information wants to be free.
2. It's not a matter of "if" but "when" it will fail.
3. People do what they can do, not what they should do.

My desktop system is hardware RAID1 to buy me time when then the first drive does fail.

I backup to one of these weekly. "RAID-X" which allows my to grow the volume size dynamically just by popping in a larger drive at a time and letting the volume re-balance without having to do a full backup and restore.

It handles CIFS (Supports AD auth), NFS (UDP only for now), AFP, FTP, Rsync, http, has built-in iTunes and Slimserver streaming and more..

Comes in a rack mount model too.
[User Picture]From: [info]ptz
Sat, 2-Feb-2008 10:26 PM (UTC)

Re: *yawn*

(Link)

Just some thoughts:
1. "My desktop system is hardware RAID1 to buy me time when then the first drive does fail."

RAIDs tend to use the same disks. Actually, identical types, sometimes taken from the same production lot.
Exercise to the reader: Go figure about the probability of them failing at similar times. Also take into account damage caused by over-current, etc.

2. ""RAID-X" which allows my to grow the volume size dynamically just by popping in a larger drive at a time and letting the volume re-balance without having to do a full backup and restore."

Do a disaster recovery exercise. Do it while your primary hard drive is OK (onto a new, spare, empty drive).
Better yet, do it after doing another backup onto some other computer/drive at a friend's house 50 miles away.

Couple of years a friend of mine got a RAID controller for his work PC, some disks and - before production use - tried out its balancing capabilities, result of a simulated hardware fail, drive exchange, etc.
It was not a cheap RAID solution, and the vendor claimed everything would be fault-tolerant and even hot-pluggable.

Net result: five trials, five complete data losses.
[User Picture]From: [info]awooster
Sat, 29-Sep-2007 6:40 PM (UTC)

Or

(Link)

Option 3

When Leopard comes out, use Time Machine, the thing that's been making my weekends disappear for the past few months.
From: [info]cjensen
Sat, 29-Sep-2007 7:18 PM (UTC)

Re: Or

(Link)

Time Machine looks cool and all.... but I'm supposed to trust a version 1.0 of a really-complex backup utility with all my precious bits?

I'm thinking I need to get a real backup scheme working BEFORE switching to Leopard and its time machine.
[User Picture]From: [info]mattlazycat
Sat, 29-Sep-2007 6:42 PM (UTC)

(Link)

Most people seem to think that they don't need backups. It might not be productive to call them idiots, but masturbating on drugs isn't productive (except in the white sticky sense) either, and that's totally worth it too.

As someone else pointed out, SuperDuper really is a lovely bit of backup software for those allergic to doing things in the Terminal, and having a bootable backup is supremely useful as well as wise.

As an additional tip, paranoia (and with backups, every little helps) has me keeping my backup drives switched off whenever I'm not actually doing a backup. No one can wipe or mess up a drive remotely if it's not connected.
[User Picture]From: [info]sircyan
Sat, 29-Sep-2007 7:09 PM (UTC)

(Link)

As a former computer store owner, I can safely say that virtually nobody backs up their data. I know this, because data recovery was my #2 source of income after virus removal. Then you get bizarre happenings, like the woman who bought a 250GB external drive, plugged it in, and thought it would 'magically' backup her data. Her primary hard disk failed, she brought it in all smiles expecting me to restore it from the backup USB drive. Boy, she was sure unhappy to find out that it was empty!

Personally, I use an online backup service for anything important. I just rsync directly to them via cron every morning at 5am, and it's done. I think it's something like $5.00 a month or so for a couple of gigs. Takes care of the "keep your backups off-site" problem, too.

-RS.
[User Picture]From: [info]khedron
Sat, 29-Sep-2007 10:37 PM (UTC)

(Link)

That's one of the things Maxtor's "OneTouch" line is meant to solve -- once you've plugged the drive in and configured it, you just push the button on front whenever you want a backup. You can also do scheduled backups too, of course.

I just ordered another of these from Fry's. 750 GB, $185.
[User Picture]From: [info]rstevens
Sat, 29-Sep-2007 7:35 PM (UTC)

(Link)

Learned this the hard way. Can't agree enough.
[User Picture]From: [info]ghewgill
Sat, 29-Sep-2007 9:19 PM (UTC)

(Link)

You sir, are wise. I had a "maximum irony" episode back in february, where a software installation/config failure was followed by intermittent hardware failure followed by complete drive failure on my colo server followed by a stolen laptop.

Now, amazon s3 and rdiff-backup are my daily friends. I have learned my backup lesson many times before, but I had become lax. That's when the universe strikes.
[User Picture]From: [info]boggyb
Mon, 1-Oct-2007 10:09 AM (UTC)

(Link)

Hear hear!
From: [info]darkengobot
Tue, 2-Oct-2007 3:34 PM (UTC)

(Link)

I more or less do exactly this. I do use OSX's software RAID-1 on some file servers, but this is backed up still using this procedure. The software RAID is there merely to prevent losing a day's worth of data, which is decent value for your $100 SATA drive.

You always have to back up. And you always have to have an off-site backup. Good post.
Page 1 of 2
<<[1] [2] >>