Incremental backups using rsync onto a NAS...which also serves as my steambox and media center. Still a work in progress though. I'm planning to add RAID and a pair of 4TB hot swap drives and eventually do some offsite rotation of those. But that's still a few thousand bucks away.
Starting Score:
1
point
Moderation
+1
Interesting=1,
Total=1
Extra 'Interesting' Modifier
0
Karma-Bonus Modifier
+1
Total Score:
3
(Score: 5, Informative) by WizardFusion on Friday August 28 2015, @02:17PM
Wait, please tell me you are not planning on removing a hot-swap drive and using that as the offsite backup.? That would be a massively bad idea.
When you put a drive back into the slot the existing drive (assuming RAID-1) will have to undergo a near stress test for the next 8+ hours while it copies all it's data to the new drive. Doing this a few times and the drives will quickly wear out.
Hmm, that's a good point. I was thinking two drives that I rotate in and out so it would only have to sync one or two days worth of changes, but I guess if it's RAID it still probably has to read and compare the entire drive. Maybe I'll just have to do something with rsync...though I was hoping for some small performance gain as well through RAID. That's probably not happening until early next year though so I haven't put THAT much thought into the details yet. Thanks for the advice!
Generally, what you're describing isn't actually desirable anyway. I understand your motivations, but you typically want to keep multiple historical versions. For example: What if you delete a file, but don't notice it for a week -- by which time you will have gone through 7 backup cycles? That deleted file will have been gone for 6 backups.
What you can do to achieve something sort of similar is to get a rotation of NASes. The first dump will be costly, and your incrementals will be more expensive than with your original plan, but still not as bad as the full. Plus, this gives you the option of RAID on the NAS, increasing the chances that your data will survive movement to offsite.
It's expensive, so I don't take my own advice (yet). Right now I just back up to FreeNAS, which is a 4-disk raidz of 1TB "enterprise" drives. But it's my eventual goal to do it right.
Generally, what you're describing isn't actually desirable anyway. I understand your motivations, but you typically want to keep multiple historical versions. For example: What if you delete a file, but don't notice it for a week -- by which time you will have gone through 7 backup cycles? That deleted file will have been gone for 6 backups.
No, that's what my backup server already does. Every night all of my systems take an incremental backup (using rsync) to the backup server. So if my laptop gets stolen or its hard drive fails or I accidentally delete a file, I've got the backups right there. Right now all those backups live on a single drive in a single server though. Which isn't the worst thing ever since that drive has no unique data, so two separate drives in two separate systems would need to fail before I lose anything current -- which I figure is fairly solid since this is personal, not business. But since several of those systems are always in my apartment together, and therefore would likely fail together in the event of a fire or a hurricane or something, I want to duplicate the entire drive -- with all of those incremental backups -- to an offsite location. When I talk about syncing incremental changes, I'm talking about incremental changes to the backups, not to the main systems.
What you can do to achieve something sort of similar is to get a rotation of NASes. The first dump will be costly, and your incrementals will be more expensive than with your original plan, but still not as bad as the full. Plus, this gives you the option of RAID on the NAS, increasing the chances that your data will survive movement to offsite.
So, get a second NAS and...do what with it? Disconnect the entire thing and haul it out to my car every morning? That's a bit much. I don't really have anywhere that I could store a second NAS either. I certainly don't have anywhere that I could have it live online at an offsite location. So a second drive is a lot more feasible than a second NAS. Don't really have the budget for a second NAS either.
Just unplug it. A one-inch air gap is enough to protect you against the most common class of problems: botched firmware upgrades, mistakenly replacing the wrong drive, clicking the wrong button in some GUI, and so on.
Yeah, if your house burns down you're screwed. But... if your house burns down you're screwed. I doubt the integrity of your data will be at the forefront of your mind.
Just unplug it. A one-inch air gap is enough to protect you against the most common class of problems: botched firmware upgrades, mistakenly replacing the wrong drive, clicking the wrong button in some GUI, and so on.
That's what the incremental backups are for. The backups of those backups that I'm planning now is specifically for if my house burns down or something like that. These are very different systems running different distros so I doubt they're running much common firmware and they never update together, and I won't be replacing the drive on multiple systems at the same time, and I don't use a GUI tool for backups -- I have an automated script which I don't screw with. Sure, I guess you could come up with some stupid way I could nuke every drive in my home all at once, but it would be pretty difficult. I think I'm more likely to burn the place down.
Yeah, if your house burns down you're screwed. But... if your house burns down you're screwed. I doubt the integrity of your data will be at the forefront of your mind.
I can get a new place to live, I can get new computers and new furniture and everything else -- it'd be some work, it'd be expensive, but it would be a simple enough process. But I can't just go on Amazon and buy my data back. That's the one thing in my home that cannot ever be replaced. And there are thousands of hours of work on there. Believe it or not, my data might actually be the most valuable thing I own.
(Score: 3, Interesting) by urza9814 on Friday August 28 2015, @01:49PM
Incremental backups using rsync onto a NAS...which also serves as my steambox and media center. Still a work in progress though. I'm planning to add RAID and a pair of 4TB hot swap drives and eventually do some offsite rotation of those. But that's still a few thousand bucks away.
(Score: 5, Informative) by WizardFusion on Friday August 28 2015, @02:17PM
Wait, please tell me you are not planning on removing a hot-swap drive and using that as the offsite backup.? That would be a massively bad idea.
When you put a drive back into the slot the existing drive (assuming RAID-1) will have to undergo a near stress test for the next 8+ hours while it copies all it's data to the new drive. Doing this a few times and the drives will quickly wear out.
(Score: 2) by urza9814 on Monday August 31 2015, @12:06PM
Hmm, that's a good point. I was thinking two drives that I rotate in and out so it would only have to sync one or two days worth of changes, but I guess if it's RAID it still probably has to read and compare the entire drive. Maybe I'll just have to do something with rsync...though I was hoping for some small performance gain as well through RAID. That's probably not happening until early next year though so I haven't put THAT much thought into the details yet. Thanks for the advice!
(Score: 2) by schad on Saturday September 05 2015, @05:38PM
Generally, what you're describing isn't actually desirable anyway. I understand your motivations, but you typically want to keep multiple historical versions. For example: What if you delete a file, but don't notice it for a week -- by which time you will have gone through 7 backup cycles? That deleted file will have been gone for 6 backups.
What you can do to achieve something sort of similar is to get a rotation of NASes. The first dump will be costly, and your incrementals will be more expensive than with your original plan, but still not as bad as the full. Plus, this gives you the option of RAID on the NAS, increasing the chances that your data will survive movement to offsite.
It's expensive, so I don't take my own advice (yet). Right now I just back up to FreeNAS, which is a 4-disk raidz of 1TB "enterprise" drives. But it's my eventual goal to do it right.
(Score: 2) by urza9814 on Tuesday September 08 2015, @03:11PM
No, that's what my backup server already does. Every night all of my systems take an incremental backup (using rsync) to the backup server. So if my laptop gets stolen or its hard drive fails or I accidentally delete a file, I've got the backups right there. Right now all those backups live on a single drive in a single server though. Which isn't the worst thing ever since that drive has no unique data, so two separate drives in two separate systems would need to fail before I lose anything current -- which I figure is fairly solid since this is personal, not business. But since several of those systems are always in my apartment together, and therefore would likely fail together in the event of a fire or a hurricane or something, I want to duplicate the entire drive -- with all of those incremental backups -- to an offsite location. When I talk about syncing incremental changes, I'm talking about incremental changes to the backups, not to the main systems.
So, get a second NAS and...do what with it? Disconnect the entire thing and haul it out to my car every morning? That's a bit much. I don't really have anywhere that I could store a second NAS either. I certainly don't have anywhere that I could have it live online at an offsite location. So a second drive is a lot more feasible than a second NAS. Don't really have the budget for a second NAS either.
(Score: 2) by schad on Tuesday September 08 2015, @04:56PM
Just unplug it. A one-inch air gap is enough to protect you against the most common class of problems: botched firmware upgrades, mistakenly replacing the wrong drive, clicking the wrong button in some GUI, and so on.
Yeah, if your house burns down you're screwed. But... if your house burns down you're screwed. I doubt the integrity of your data will be at the forefront of your mind.
(Score: 2) by urza9814 on Tuesday September 08 2015, @07:02PM
That's what the incremental backups are for. The backups of those backups that I'm planning now is specifically for if my house burns down or something like that. These are very different systems running different distros so I doubt they're running much common firmware and they never update together, and I won't be replacing the drive on multiple systems at the same time, and I don't use a GUI tool for backups -- I have an automated script which I don't screw with. Sure, I guess you could come up with some stupid way I could nuke every drive in my home all at once, but it would be pretty difficult. I think I'm more likely to burn the place down.
I can get a new place to live, I can get new computers and new furniture and everything else -- it'd be some work, it'd be expensive, but it would be a simple enough process. But I can't just go on Amazon and buy my data back. That's the one thing in my home that cannot ever be replaced. And there are thousands of hours of work on there. Believe it or not, my data might actually be the most valuable thing I own.