Mirroring protects against drive failure but not data corruption. If one copy becomes corrupted the corrupt data is mirrored to all the others and they all become corrupted.
And how is backup different? The proper way to prevent data corruption is the revision control. Backup cannot detect the original data corruption because the latter is indistinguishable from a cromulent edit. If a file becomes corrupt and then gets backed up in a corrupt state, now the backup is corrupt. If you have an older snapshot, you may be in luck, but what are the chances of that in your case? With mirrors, one can mirror, say, to target 1 on every 1st and to target 2 on every 15th of a month, and that guarantees a 2-to-4-week stale snapshot, so the situation is basically the same.
Having a single backup isn't something I'd recommend because, as you point out, it's really no better than mirroring. But most backup solutions do more than that.
I can only speak for Time Machine (the backup solution built in to OSX) but it does hourly backups for the past day, daily backups for the past month, and weekly backups for every previous week for as long as you have the disk space (mine go back to January 2013 when I first switched it on). It also lets you swap in multiple backup drives and will alternate backups between them if you have them connected at the same time.
If a file has become corrupted you skim back through the copies until you find the last good copy (it has a very nice interface for doing this).
It's seriously a good thing you're backing up at all, but if your data is valuable to you I'd look in to a better strategy than backing up when you feel like it. Something that keeps automatic backups can save your skin.
I solve that problem by having an infinite army of monkeys with typewriters. All my data is always guaranteed to be backed up, even before I generate it.
A long time ago, before I had backups, before btrfs was in wide use; I tried putting everything on the drive, in a source control system.
Some source control systems, create lotsa duplicate copies, to make it easier to detect changes. For example, SVN places an extra copy of the source tree, in .svn/
Anyway, figured that I didn't need duplicate copies like that; removing them immediately would just result in some extra free disk space and a tiny loss of performance.
Ran something like
opencm blah blah; opencm delete
to insert the root directory into source control, and immediately delete the unnecessary duplicate copy that would be created.
Something went wrong, and I lost about half the data on the drive; about 5 gig (this was very long ago, when drives were tiny), before noticing the problem and killing the deletion process.
Thus, deeply burning onto my psyche, that reliability is more important than saving disk space, and to never, ever, dedup.
(Score: 2) by WizardFusion on Friday October 30 2015, @01:10PM
But I have RAID, surely I don't need a backup too.???
/sarcasm
(Score: 0) by Anonymous Coward on Friday October 30 2015, @08:01PM
by Anonymous Coward
on Friday October 30 2015, @08:01PM (#256634)
The problem with RAID is that it also backs up your commands, so if you run rm -rf / on one of the hard disks, it'll run it on them all. If you want to avoid this behaviour what you should do is pull out one or more of the disk drives after you've run a potentially problematic command.
Don't laugh - I went into one customer to do a piece of work (about 12 yrs ago now...) and found that their backup procedure was to yank one of the drives from their RAID 1 mirror and stick it in the safe before they left the office in the evening, then replace it when they returned in the morning. Quite why they thought that something would only happen at night (even if they didn't understand the technical stuff that makes that wrong on so many levels) I'll never know.
Usually, before I install any new software or upgrade, I run CloneZilla on the disk, and store the disk image files on one of those Western Digital "Elements" USB drive.
I have one subdirectory for Eagle, LTSpice, and MathCad files which I have been working on a lot, so I usually keep a couple of copies of that subdirectory on the drive as well, and update those every week or so, more often if I have been doing a lot of serious and new work.
I have two "Elements" drives and ping-pong between the two, hoping that in the event of big-time trouble, if one is damaged, hopefully the other one will bail me out.
Being I image the complete drive, if I do get a nasty - I intend to go ahead and replace the HDD on my machine with a larger brand new one and re-image the drive from the CloneZilla files on the Elements drive. That should put me back in the state the machine was in when the image was taken.
My biggest fear is one of those Microsoft updates makes its way into my machine and corrupts some critical driver files, like those FTDI drivers covered several months ago here.
Unfortunately, as long as I am connected to the internet, there is only a "gentlemen's agreement" between me and Microsoft that they will not mess with my machine if I have turned automatic updating off. I know full good and well about the ADVAPI32.DLL backdoor, and know full good and well no Microsoft machine is truly secure if online. But I do need it in order to communicate with businesses that require proprietary Microsoft protocols in order to communicate with them - as simple .txt files are often inappropriately dressed for business, where image and presentation trump substance and integrity.
I have strong suspicions that many companies are "working with" ( aka "bribing") Microsoft to have their digital policemen uploaded into everyone's machine, and once these policemen start observing and reporting everything somebody does something that somebody else does not want them to do, all sorts of legal crap - probably 1% real, 99% phish - will erupt into the email streams.
-- "Prove all things; hold fast that which is good." [KJV: I Thessalonians 5:21]
I recall there were some clowns running a cloud storage service that believed that RAID was as good as a proper backup system. It bit them on the arse too. Tried looking for them on Google but nothing, sure it happened, if anyone here can recall/find citations...?
(Score: 0) by Anonymous Coward on Friday October 30 2015, @09:07PM
by Anonymous Coward
on Friday October 30 2015, @09:07PM (#256669)
Back in my Windows 98 days, I used to backup my MS Money database to a floppy regularly. It seemed like a safe, reliable system, and never gave me any errors, until... I decided to upgrade my operating system and wanted to do a fresh install on a new, larger, faster hard drive. No problem, I have the database backed up to floppy. Surprise surprise, the floppy kept spitting out "unable to read, disc read error" or something. All this time Windows never told me there was a problem with the floppy. Luckily, I still had the old hard drive, shoved it in a USB enclosure, and dug through the files to find my last money database. I've since converted to Linux and do time stamped backups to a few flashdrives.
(Score: 2) by LoRdTAW on Friday October 30 2015, @10:35PM
Back in ye olde days, my father's 386 couldn't read 3.5" disks from other computers and his 3.5's couldn't be read on other PC's. His friend, an old unix hacker, finally figured out what the problem was: the floppy drive had a slight misalignment of the heads. He had to buy a new drive, install it and painstakingly xcopy a:\* b:\ for over a hundred disks. Oh, right. He had 10yo me do it. Fun times.
(Score: 0) by Anonymous Coward on Saturday October 31 2015, @04:05AM
by Anonymous Coward
on Saturday October 31 2015, @04:05AM (#256789)
Mmm. When I used to burn dvds, I tried to check readability in a different drive, than the original. Showed me that crap-cheap dvd drives handle DVD+R better than DVD-R (could read only the first 800 meg . . . weird); more often that it caught burn errors.
I have a bit of a mess on my hands now. About 100 dvds from 2006 - 2009.
(Score: 0) by Anonymous Coward on Saturday October 31 2015, @03:05PM
by Anonymous Coward
on Saturday October 31 2015, @03:05PM (#256899)
Did you ever run into the problem where Windows permanently handicaps the CD/DVD drive if it encounters a read error x number of times with a disk? It makes DVD video playback very slow and/or choppy. The only way to fix it is by shutting down the PC, disconnecting the drive, reboot, shutdown again, re-connect the drive, reboot again... Or hack the registry. That used to royally piss me off.
When you switched to Linux, did you find a suitable replacement for Money? I think my mother might *still* be using an ancient Dell desktop solely for Money, because she can't find anything else that will track investments as well as that 13 year old copy of Money 2002. We tried a bunch of iPad apps, and various Linux finance apps, and nothing worked for her. I think the main problem was directly integrating with all the various financial institutions. There's plenty of apps that will connect to your bank and pull your savings/checking info and break it all down, but apparently none of them can do the same for investments. Or they can't do both at once or something.
(Score: 0) by Anonymous Coward on Saturday November 07 2015, @02:04AM
by Anonymous Coward
on Saturday November 07 2015, @02:04AM (#259767)
I don't do investments any more like I did when I was using MS Money, I only use it for a checking/savings register. I'm using kmymoney for that, it's the closest thing to MS Money I've found. It does have an investments account section... https://kmymoney.org/ [kmymoney.org]
I've said it before, and I'll say it again: Nobody cares about backups! . . . But everyone cares about restores!
Seriously, I've found it better to start people talking about what circumstances they want to be able to get their data back than how it should be backed up - and that should apply here too - the question should be ...last tested your restores? It's no good having a well verified copy if it won't restore. But a test restore validates the backups too (and is, in fairness, what the author meant.).
(Score: 2) by GungnirSniper on Sunday November 01 2015, @07:55PM
The only time I do a full backup is when I put in a larger hard disk. The old one is cloned to the new, and then the old is stored away. So maybe every other year this is done. Really important things go on CD-R or DVD-R, but I don't consider saved games or torrents to be a required backup typically.
Yeah, a time machine is a great way to recover information; just go back into the past, copy the data and try not to change anything while in the past so you don't run into a time paradox. However you should always have a second time machine as backup, in case the first one fails. And of course, you should regularly test that backup.
-- The Tao of math: The numbers you can count are not the real numbers.
(Score: 0) by Anonymous Coward on Tuesday November 03 2015, @07:45PM
by Anonymous Coward
on Tuesday November 03 2015, @07:45PM (#258081)
Wouldn't going to the past to retrieve the data create a paradox because now you'll have the data and no longer have an incentive to go back to the past in the first place?
I occasionally check the rsync* logs and make sure the files exist on the remote server, but that's about it. Rarely I'll open a random file to ensure it's good. But I've never checked the full backup. I figure that's probably good enough for my uses though; I might add a bit more once I spin up some new servers over the next couple months. At the very least I'll be adding RAID to the backup server, which should help. I was considering a second parallel backup server for some of the more critical stuff, and some automated checks of the logs. Still need to figure out an offsite strategy though. And perhaps actual restore testing.
But since my backup is rsync, not total disk mirroring...my strategy is if I ever hose my disk or accidentally delete something, I'll be retrieving the individual files I need rather than imaging the entire damn thing. So even if I only occasionally spot-check a few individual files at random then I know I won't lose *everything*, though I might lose a file or directory somewhere if something goes seriously wrong. Or a few days work since I don't check on a daily basis. Not great, but quite an improvement from the 'dd to a bigger drive when I get a new PC' method I was using until a few months ago :)
*And yes, I'm using rsync to do nightly incrementals, not just a sync :)
(Score: 2, Insightful) by zoefff on Thursday October 29 2015, @07:22PM
It's just a bunch of files scattered around on several disks
(Score: 3, Insightful) by Subsentient on Thursday October 29 2015, @08:52PM
Most critical data of mine is mirrored on my main desktop, laptop, netbook, and server. If one fails, I got it on the others.
"It is no measure of health to be well adjusted to a profoundly sick society." -Jiddu Krishnamurti
(Score: 5, Informative) by basicbasicbasic on Sunday November 01 2015, @07:14PM
Mirroring protects against drive failure but not data corruption. If one copy becomes corrupted the corrupt data is mirrored to all the others and they all become corrupted.
(Score: 3, Insightful) by melikamp on Thursday November 05 2015, @05:07AM
And how is backup different? The proper way to prevent data corruption is the revision control. Backup cannot detect the original data corruption because the latter is indistinguishable from a cromulent edit. If a file becomes corrupt and then gets backed up in a corrupt state, now the backup is corrupt. If you have an older snapshot, you may be in luck, but what are the chances of that in your case? With mirrors, one can mirror, say, to target 1 on every 1st and to target 2 on every 15th of a month, and that guarantees a 2-to-4-week stale snapshot, so the situation is basically the same.
(Score: 2) by basicbasicbasic on Friday November 06 2015, @04:56PM
Having a single backup isn't something I'd recommend because, as you point out, it's really no better than mirroring. But most backup solutions do more than that.
I can only speak for Time Machine (the backup solution built in to OSX) but it does hourly backups for the past day, daily backups for the past month, and weekly backups for every previous week for as long as you have the disk space (mine go back to January 2013 when I first switched it on). It also lets you swap in multiple backup drives and will alternate backups between them if you have them connected at the same time.
If a file has become corrupted you skim back through the copies until you find the last good copy (it has a very nice interface for doing this).
(Score: 2) by Subsentient on Friday November 06 2015, @03:28AM
If I scheduled routine mirrors, that'd be a legitimate concern, but I only back up when I feel like it or it comes to mind.
"It is no measure of health to be well adjusted to a profoundly sick society." -Jiddu Krishnamurti
(Score: 2) by basicbasicbasic on Friday November 06 2015, @05:17PM
It's seriously a good thing you're backing up at all, but if your data is valuable to you I'd look in to a better strategy than backing up when you feel like it. Something that keeps automatic backups can save your skin.
(Score: 3, Funny) by bob_super on Friday November 13 2015, @01:51AM
I solve that problem by having an infinite army of monkeys with typewriters.
All my data is always guaranteed to be backed up, even before I generate it.
(Score: 2) by basicbasicbasic on Friday November 13 2015, @11:16PM
Ah - the Library of Babel backup strategy;
https://en.wikipedia.org/wiki/The_Library_of_Babel [wikipedia.org]
https://libraryofbabel.info/About.html [libraryofbabel.info]
(Score: 2, Funny) by Anonymous Coward on Friday October 30 2015, @12:12AM
Back up my files? Are you kidding? Is that a real thing you have to do?
I always thought that that was just like a figure of speech. Y'know, like "wake up and smell the coffee", or "See ya later, alligator!"
(Score: 1) by Osamabobama on Friday October 30 2015, @04:24PM
It's more like "I've gotta go see a man about a horse."
Appended to the end of comments you post. Max: 120 chars.
(Score: 3, Interesting) by throwaway28 on Friday October 30 2015, @02:35AM
A long time ago, before I had backups, before btrfs was in wide use; I tried putting everything on the drive, in a source control system.
Some source control systems, create lotsa duplicate copies, to make it easier to detect changes. For example, SVN places an extra copy of the source tree, in .svn/
Anyway, figured that I didn't need duplicate copies like that; removing them immediately would just result in some extra free disk space and a tiny loss of performance.
Ran something like
to insert the root directory into source control, and immediately delete the unnecessary duplicate copy that would be created.
Something went wrong, and I lost about half the data on the drive; about 5 gig (this was very long ago, when drives were tiny), before noticing the problem and killing the deletion process.
Thus, deeply burning onto my psyche, that reliability is more important than saving disk space, and to never, ever, dedup.
(Score: 2) by WizardFusion on Friday October 30 2015, @01:10PM
But I have RAID, surely I don't need a backup too.???
/sarcasm
(Score: 0) by Anonymous Coward on Friday October 30 2015, @08:01PM
The problem with RAID is that it also backs up your commands, so if you run rm -rf / on one of the hard disks, it'll run it on them all. If you want to avoid this behaviour what you should do is pull out one or more of the disk drives after you've run a potentially problematic command.
(Score: 4, Funny) by Adrian Harvey on Sunday November 01 2015, @08:39AM
Don't laugh - I went into one customer to do a piece of work (about 12 yrs ago now...) and found that their backup procedure was to yank one of the drives from their RAID 1 mirror and stick it in the safe before they left the office in the evening, then replace it when they returned in the morning. Quite why they thought that something would only happen at night (even if they didn't understand the technical stuff that makes that wrong on so many levels) I'll never know.
(Score: 2) by WizardFusion on Sunday November 01 2015, @06:40PM
+1 OMG
(Score: 1) by anubi on Monday November 02 2015, @07:06AM
Fail. I reverse snorted my coffee.
Usually, before I install any new software or upgrade, I run CloneZilla on the disk, and store the disk image files on one of those Western Digital "Elements" USB drive.
I have one subdirectory for Eagle, LTSpice, and MathCad files which I have been working on a lot, so I usually keep a couple of copies of that subdirectory on the drive as well, and update those every week or so, more often if I have been doing a lot of serious and new work.
I have two "Elements" drives and ping-pong between the two, hoping that in the event of big-time trouble, if one is damaged, hopefully the other one will bail me out.
Being I image the complete drive, if I do get a nasty - I intend to go ahead and replace the HDD on my machine with a larger brand new one and re-image the drive from the CloneZilla files on the Elements drive. That should put me back in the state the machine was in when the image was taken.
My biggest fear is one of those Microsoft updates makes its way into my machine and corrupts some critical driver files, like those FTDI drivers covered several months ago here.
Unfortunately, as long as I am connected to the internet, there is only a "gentlemen's agreement" between me and Microsoft that they will not mess with my machine if I have turned automatic updating off. I know full good and well about the ADVAPI32.DLL backdoor, and know full good and well no Microsoft machine is truly secure if online. But I do need it in order to communicate with businesses that require proprietary Microsoft protocols in order to communicate with them - as simple .txt files are often inappropriately dressed for business, where image and presentation trump substance and integrity.
I have strong suspicions that many companies are "working with" ( aka "bribing") Microsoft to have their digital policemen uploaded into everyone's machine, and once these policemen start observing and reporting everything somebody does something that somebody else does not want them to do, all sorts of legal crap - probably 1% real, 99% phish - will erupt into the email streams.
"Prove all things; hold fast that which is good." [KJV: I Thessalonians 5:21]
(Score: 1) by malloc_free on Monday November 02 2015, @06:33AM
I recall there were some clowns running a cloud storage service that believed that RAID was as good as a proper backup system. It bit them on the arse too. Tried looking for them on Google but nothing, sure it happened, if anyone here can recall/find citations...?
(Score: 3, Funny) by mhajicek on Tuesday November 03 2015, @05:13AM
The article wasn't properly backed up...
The spacelike surfaces of time foliations can have a cusp at the surface of discontinuity. - P. Hajicek
(Score: 0) by Anonymous Coward on Friday October 30 2015, @09:07PM
Back in my Windows 98 days, I used to backup my MS Money database to a floppy regularly. It seemed like a safe, reliable system, and never gave me any errors, until... I decided to upgrade my operating system and wanted to do a fresh install on a new, larger, faster hard drive. No problem, I have the database backed up to floppy. Surprise surprise, the floppy kept spitting out "unable to read, disc read error" or something. All this time Windows never told me there was a problem with the floppy. Luckily, I still had the old hard drive, shoved it in a USB enclosure, and dug through the files to find my last money database. I've since converted to Linux and do time stamped backups to a few flashdrives.
(Score: 2) by LoRdTAW on Friday October 30 2015, @10:35PM
Back in ye olde days, my father's 386 couldn't read 3.5" disks from other computers and his 3.5's couldn't be read on other PC's. His friend, an old unix hacker, finally figured out what the problem was: the floppy drive had a slight misalignment of the heads. He had to buy a new drive, install it and painstakingly xcopy a:\* b:\ for over a hundred disks. Oh, right. He had 10yo me do it. Fun times.
(Score: 0) by Anonymous Coward on Saturday October 31 2015, @04:05AM
Mmm. When I used to burn dvds, I tried to check readability in a different drive, than the original. Showed me that crap-cheap dvd drives handle DVD+R better than DVD-R (could read only the first 800 meg . . . weird); more often that it caught burn errors.
I have a bit of a mess on my hands now. About 100 dvds from 2006 - 2009.
(Score: 0) by Anonymous Coward on Saturday October 31 2015, @03:05PM
Did you ever run into the problem where Windows permanently handicaps the CD/DVD drive if it encounters a read error x number of times with a disk? It makes DVD video playback very slow and/or choppy. The only way to fix it is by shutting down the PC, disconnecting the drive, reboot, shutdown again, re-connect the drive, reboot again... Or hack the registry. That used to royally piss me off.
(Score: 0) by Anonymous Coward on Saturday October 31 2015, @11:53PM
No. I switched to linux (redhat 9 / fedora 2 / knoppix 3.3 ) on my primary systems in 2004 - 2005, before I started burning dvds.
(Score: 2) by urza9814 on Friday November 06 2015, @03:32PM
Totally off topic, but...
When you switched to Linux, did you find a suitable replacement for Money? I think my mother might *still* be using an ancient Dell desktop solely for Money, because she can't find anything else that will track investments as well as that 13 year old copy of Money 2002. We tried a bunch of iPad apps, and various Linux finance apps, and nothing worked for her. I think the main problem was directly integrating with all the various financial institutions. There's plenty of apps that will connect to your bank and pull your savings/checking info and break it all down, but apparently none of them can do the same for investments. Or they can't do both at once or something.
(Score: 0) by Anonymous Coward on Saturday November 07 2015, @02:04AM
I don't do investments any more like I did when I was using MS Money, I only use it for a checking/savings register. I'm using kmymoney for that, it's the closest thing to MS Money I've found. It does have an investments account section... https://kmymoney.org/ [kmymoney.org]
(Score: 0) by Anonymous Coward on Saturday October 31 2015, @04:54PM
The last time I verified my Bitcoin private keys was about a year ago.
However, the majority of my data is either "what are are backups?" or "verify?"
Seems I am more willing to do things when money is on the line. (I hate money.)
(Score: 4, Touché) by Adrian Harvey on Sunday November 01 2015, @08:55AM
I've said it before, and I'll say it again: Nobody cares about backups!
.
.
.
But everyone cares about restores!
Seriously, I've found it better to start people talking about what circumstances they want to be able to get their data back than how it should be backed up - and that should apply here too - the question should be ...last tested your restores? It's no good having a well verified copy if it won't restore. But a test restore validates the backups too (and is, in fairness, what the author meant.).
(Score: 2) by GungnirSniper on Sunday November 01 2015, @07:55PM
The only time I do a full backup is when I put in a larger hard disk. The old one is cloned to the new, and then the old is stored away. So maybe every other year this is done. Really important things go on CD-R or DVD-R, but I don't consider saved games or torrents to be a required backup typically.
Tips for better submissions to help our site grow. [soylentnews.org]
(Score: 2) by Alfred on Monday November 02 2015, @10:54PM
(Score: 3, Funny) by maxwell demon on Tuesday November 03 2015, @12:15AM
Yeah, a time machine is a great way to recover information; just go back into the past, copy the data and try not to change anything while in the past so you don't run into a time paradox. However you should always have a second time machine as backup, in case the first one fails. And of course, you should regularly test that backup.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 2) by mhajicek on Tuesday November 03 2015, @05:18AM
That which is measured is changed...
The spacelike surfaces of time foliations can have a cusp at the surface of discontinuity. - P. Hajicek
(Score: 2) by fliptop on Friday November 13 2015, @04:16PM
By taking the second back with you in the first? [imdb.com]
To be oneself, and unafraid whether right or wrong, is more admirable than the easy cowardice of surrender to conformity
(Score: 0) by Anonymous Coward on Tuesday November 03 2015, @07:45PM
Wouldn't going to the past to retrieve the data create a paradox because now you'll have the data and no longer have an incentive to go back to the past in the first place?
(Score: 0) by Anonymous Coward on Friday November 06 2015, @04:30PM
Not if you take the copy of the data with you when going back to the future.
(Score: 2) by Non Sequor on Wednesday November 04 2015, @02:39AM
I decided to live with the fact that I could write a small program that would generate a set of files including my own.
Write your congressman. Tell him he sucks.
(Score: 4, Interesting) by bradley13 on Thursday November 05 2015, @06:51PM
Every few months, someone destroys a file and I have to go fetch it from backup. So...they work.
I've never tested a full restore. Not that paranoid. If the individual files are there, then a full restore ought to also be possible.
Everyone is somebody else's weirdo.
(Score: 2) by urza9814 on Friday November 06 2015, @04:35PM
I occasionally check the rsync* logs and make sure the files exist on the remote server, but that's about it. Rarely I'll open a random file to ensure it's good. But I've never checked the full backup. I figure that's probably good enough for my uses though; I might add a bit more once I spin up some new servers over the next couple months. At the very least I'll be adding RAID to the backup server, which should help. I was considering a second parallel backup server for some of the more critical stuff, and some automated checks of the logs. Still need to figure out an offsite strategy though. And perhaps actual restore testing.
But since my backup is rsync, not total disk mirroring...my strategy is if I ever hose my disk or accidentally delete something, I'll be retrieving the individual files I need rather than imaging the entire damn thing. So even if I only occasionally spot-check a few individual files at random then I know I won't lose *everything*, though I might lose a file or directory somewhere if something goes seriously wrong. Or a few days work since I don't check on a daily basis. Not great, but quite an improvement from the 'dd to a bigger drive when I get a new PC' method I was using until a few months ago :)
*And yes, I'm using rsync to do nightly incrementals, not just a sync :)