| Comments: |
![[User Picture]](http://l-userpic.livejournal.com/5887295/515656) | From: jwz Sat, 29-Sep-2007 8:59 AM (UTC)
RAID | (Link)
|
P.S., RAID is a waste of your goddamned time and money. Is your personal computer a high-availability server with hot-swappable drives? No? Then you don't need RAID, you just need backups.
100% agree. I've had too many close calls with broken RAID implementations to even consider bothering with it anymore. I just back up using a similar rsync command to what you proposed. I'm planning to buy an external 750GB hard drive specifically for backups when they get just a bit cheaper.
The universe tends toward maximum irony. Don't push it.
Somewhere along the line I can see that being included in the set of most memorable quotes of all time.
As a counter-PSA, I'll point out that rsync doesn't behave entirely correctly on HFS. (Yes, even with Apple's improved version. It still screws up some random bits of metadata in the backup.) If this matters to you, you can either use ditto and live with the fact that it isn't incremental, or shell out $30 for SuperDuper, which is totally worth it.
![[User Picture]](http://l-userpic.livejournal.com/5887295/515656) | From: jwz Sat, 29-Sep-2007 9:36 AM (UTC)
metadata and wires | (Link)
|
What does it screw up?
![[User Picture]](http://l-userpic.livejournal.com/94077102/302422) | From: 33mhz Sat, 29-Sep-2007 10:01 AM (UTC)
| (Link)
|
Strangely, with the laptops I have, the display is way more prone to outright fail than the hard drive. I only wish there were a way to matter-compile a back-up of one of those.
I don't do this: I have a different backup policy. I work on laptops. When I buy a new laptop, I probably sell the last but one. The old one then becomes the hot backup machine. I never have less than two similarly-configured work machines. (If the new laptop's drive is much larger than the old laptop, consider buying the old lappie a new hard disk.) Use rsync to backup data from the current work machine to the old one every week. Meanwhile, every day (or more often) use rsync to backup critical data -- work files, email archives, not random MP3s -- onto an 8Gb thumb drive that lives on a keyring. Every couple of days, use rsync to backup the critical data to a server 500 miles away. When you boil it down, this is essentially the same level of cover as jwz's spare drive policy; it just uses a different kind of drive enclosure (the laptop). Oh, as for Windows ... use rsync. (You did install Cygwin, right?) Horrible confession time: my most recent laptop is running vista. And it's staying that way for, oh, another month or so (until Ubuntu Gusty Gibbon is stable and I have a spare few days to spend getting the wifi drivers to work). Again: rsync is available for Windows.
Alternatively, if you don't want to fight with Cygwin (I mean, isn't that the Windows equivalent of "recompile your kernel"?), you can try cwrsync.
Go buy two more drives of the same size or larger. If the drive in your computer is SATA2, get SATA2. If it's a 2.5" laptop drive, get two of those. Brand doesn't matter, but physical size and connectors should match.
Well, brand matters sometimes, as the typical user won't find out that models with seemingly identical sizes (e.g. 400 GB) but of different brands have different capacities (off by some bytes). That is no problem for isync, but if you backup your disk with dd (or any other partition copy tool) so you have a replacement to plug in, you should have the same capacity (or more).
Also, raid: I use a raid 1 setup on my main workstation (WinXP) and on a server (debian). Pro: backup disk is always up to date, reads are double as fast. Cons: If your disk does not die 'naturally', but from any physical cause (PC falls down, electrical problems), probably both disks are gone.
Any chance you tell us what prompted you to write this post?
![[User Picture]](http://l-userpic.livejournal.com/5887295/515656) | From: jwz Sat, 29-Sep-2007 6:12 PM (UTC)
Re: size matters | (Link)
|
Which is exactly why my advice does not include the use of "dd".
I say again: RAID is a waste of your goddamned time and money.
(Option++;): Buy a DDS-4 tape drive, a copy of Retrospect and a bunch of DDS-4 tapes. Divide the tapes in half -- one set is the current backup set the other is the "keep at a friend's house" backup set. Swap between the two.
I backup four systems this way (two laptops, two desktops) and have saved my ass many, many times.
![[User Picture]](http://l-userpic.livejournal.com/15168285/422568) | From: nrr Sat, 29-Sep-2007 4:02 PM (UTC)
price points | (Link)
|
This fails miserably because the price point for the tape drive and tapes is far higher than that of the two hard drives and their external enclosures. Really.
The idea is to do this sanely and cheaply when confronted with the problem as a home user.
is there a reason why rsync would be better than using carbon copy cloner?
![[User Picture]](http://l-userpic.livejournal.com/5887295/515656) | From: jwz Sat, 29-Sep-2007 6:27 PM (UTC)
| (Link)
|
Carbon Copy Cloner uses rsync. They are identical.
![[User Picture]](http://l-userpic.livejournal.com/39554298/562816) | From: malokai Sat, 29-Sep-2007 3:10 PM (UTC)
Apart from rsync + resource forks being broken | (Link)
|
The `ditto` command can also be used to make one-off bootable backups, preserving the resource forks (for fonts, I guess, or any mac os classic applications you may be using if you're still running that ancient copy of quark?) Other utilities like the carbon copy cloner can be used for this (useful for the 'third' backup drive option that you leave in your desk). The 10.5 'time travelling' thing seems to just wander your filesystem looking for modified files and backing up previous revisions of the files, so at least things are looking up in the automated Mac OS X backup world(since this post is obviously inspired by a friend or relative's drive crashing?). http://www.bombich.com/mactips/image.html
![[User Picture]](http://l-userpic.livejournal.com/19883661/4534609) | From: houdini_cs Sat, 29-Sep-2007 3:53 PM (UTC)
Re: Not that I disagree | (Link)
|
I'm awesome. Oh well, you can see what I meant to link to.
Personally, I use rdiff-backup instead of rsync so that I get incremental backups, and eventually I'll by rsyncing that backup offsite.
But yeah, what he said. :)
Semantics aside, how is rsync not ending up being incremental. If I use the switches indicated in the original post, it says "only copy the new shit" and "delete all the old shit I deleted". It doesn't have to do what I would normally think of as a Level 0 anymore than one time, when the drive is synchronized the first time.
Seriously asking, not just yanking your chain or playing. Am I missing something?
Is it just because rsync automatically looks at the entire list of files first before determining what has changed? Or is there some other thing?
Thanks
![[User Picture]](http://l-userpic.livejournal.com/66605916/116108) | From: eliot Sat, 29-Sep-2007 4:46 PM (UTC)
online backup useful too | (Link)
|
I do a complete backup at my office on to an external drive with SuperDuper! and then an rsync to StrongSpace of my Documents, Mail, Address Book, and Pictures. I've found this to be useful because not only do I have a completely off-site secondary storage, but I also can browse from a friend's house and find a photo or document I have on StrongSpace.
From: tongodeon Sat, 29-Sep-2007 5:11 PM (UTC)
Old and busted: cron / New hotness: launchd | (Link)
|
![[User Picture]](http://l-userpic.livejournal.com/5887295/515656) | From: jwz Sat, 29-Sep-2007 6:48 PM (UTC)
Re: Old and busted: cron / New hotness: launchd | (Link)
|
That looks like a lot more typing for zero benefit.
![[User Picture]](http://l-userpic.livejournal.com/43706661/4281866) | From: smin Sat, 29-Sep-2007 5:40 PM (UTC)
Bless | (Link)
|
Does one no longer have to bless a Mac backup drive before it's bootable? Did that come in with Tiger?
From: wootest Sat, 29-Sep-2007 6:34 PM (UTC)
Re: Bless | (Link)
|
A bit-by-bit copy of every data on the drive will include the blessing. It's not in the partition table.
I hereby invoke the jwz-fanboy-advice=baiting. I've got an iBook with a tiny hard drive, which I've been cloning to an external drive using CarbonCopy Cloner whenever I remember and/or whenever Apple Software Updater runs. Not a lot of important data here, but every once in awhile I dump my documents folder onto a CD and put it away somewhere. I remember hearing horror stories of a graphic artist somewhere who had impeccable backup technique, except that they were all sitting by his computer when his house burned down. Leetle paranoid after hearing of that. Eventually I'll upgrade my desktop paperweight and have something running OSX, at which point I will come back here and follow the above advice.
![[User Picture]](http://l-userpic.livejournal.com/26104787/6145587) | From: lohphat Sat, 29-Sep-2007 6:34 PM (UTC)
Fire, schmire. | (Link)
|
From: ausfish Mon, 1-Oct-2007 10:20 AM (UTC)
Another example of why you need off-site backups | (Link)
|
![[User Picture]](http://l-userpic.livejournal.com/26104787/6145587) | From: lohphat Sat, 29-Sep-2007 6:40 PM (UTC)
*yawn* | (Link)
|
Basic rules of IT: 1. Information wants to be free. 2. It's not a matter of "if" but "when" it will fail. 3. People do what they can do, not what they should do. My desktop system is hardware RAID1 to buy me time when then the first drive does fail. I backup to one of these weekly. "RAID-X" which allows my to grow the volume size dynamically just by popping in a larger drive at a time and letting the volume re-balance without having to do a full backup and restore. It handles CIFS (Supports AD auth), NFS (UDP only for now), AFP, FTP, Rsync, http, has built-in iTunes and Slimserver streaming and more.. Comes in a rack mount model too.
![[User Picture]](http://l-userpic.livejournal.com/7321084/1353421) | From: ptz Sat, 2-Feb-2008 10:26 PM (UTC)
Re: *yawn* | (Link)
|
Just some thoughts: 1. "My desktop system is hardware RAID1 to buy me time when then the first drive does fail."
RAIDs tend to use the same disks. Actually, identical types, sometimes taken from the same production lot. Exercise to the reader: Go figure about the probability of them failing at similar times. Also take into account damage caused by over-current, etc.
2. ""RAID-X" which allows my to grow the volume size dynamically just by popping in a larger drive at a time and letting the volume re-balance without having to do a full backup and restore."
Do a disaster recovery exercise. Do it while your primary hard drive is OK (onto a new, spare, empty drive). Better yet, do it after doing another backup onto some other computer/drive at a friend's house 50 miles away.
Couple of years a friend of mine got a RAID controller for his work PC, some disks and - before production use - tried out its balancing capabilities, result of a simulated hardware fail, drive exchange, etc. It was not a cheap RAID solution, and the vendor claimed everything would be fault-tolerant and even hot-pluggable.
Net result: five trials, five complete data losses.
Option 3 When Leopard comes out, use Time Machine, the thing that's been making my weekends disappear for the past few months.
From: cjensen Sat, 29-Sep-2007 7:18 PM (UTC)
Re: Or | (Link)
|
Time Machine looks cool and all.... but I'm supposed to trust a version 1.0 of a really-complex backup utility with all my precious bits?
I'm thinking I need to get a real backup scheme working BEFORE switching to Leopard and its time machine.
Most people seem to think that they don't need backups. It might not be productive to call them idiots, but masturbating on drugs isn't productive (except in the white sticky sense) either, and that's totally worth it too. As someone else pointed out, SuperDuper really is a lovely bit of backup software for those allergic to doing things in the Terminal, and having a bootable backup is supremely useful as well as wise. As an additional tip, paranoia (and with backups, every little helps) has me keeping my backup drives switched off whenever I'm not actually doing a backup. No one can wipe or mess up a drive remotely if it's not connected.
As a former computer store owner, I can safely say that virtually nobody backs up their data. I know this, because data recovery was my #2 source of income after virus removal. Then you get bizarre happenings, like the woman who bought a 250GB external drive, plugged it in, and thought it would 'magically' backup her data. Her primary hard disk failed, she brought it in all smiles expecting me to restore it from the backup USB drive. Boy, she was sure unhappy to find out that it was empty!
Personally, I use an online backup service for anything important. I just rsync directly to them via cron every morning at 5am, and it's done. I think it's something like $5.00 a month or so for a couple of gigs. Takes care of the "keep your backups off-site" problem, too.
-RS.
That's one of the things Maxtor's "OneTouch" line is meant to solve -- once you've plugged the drive in and configured it, you just push the button on front whenever you want a backup. You can also do scheduled backups too, of course.
I just ordered another of these from Fry's. 750 GB, $185.
Learned this the hard way. Can't agree enough.
You sir, are wise. I had a "maximum irony" episode back in february, where a software installation/config failure was followed by intermittent hardware failure followed by complete drive failure on my colo server followed by a stolen laptop.
Now, amazon s3 and rdiff-backup are my daily friends. I have learned my backup lesson many times before, but I had become lax. That's when the universe strikes.
I more or less do exactly this. I do use OSX's software RAID-1 on some file servers, but this is backed up still using this procedure. The software RAID is there merely to prevent losing a day's worth of data, which is decent value for your $100 SATA drive.
You always have to back up. And you always have to have an off-site backup. Good post. | |