from the a-one-and-a-two-and-a-three-and-a dept.
FOSS developer Michael Stapelberg has started a four part blog post on Rsync and how it works. He wrote the i3 tiling window manager, among other projects, and is a former Debian developer. Now he has written about three scenarios for which he has come to appreciate Rsync, specifically in DokuWiki transfers, software deployment, and backups. Then he looks at at integrating it into various work flows, and then at what the software and protocol actually do. The fourth section is to be announced.
Rsync is an algorithm and a utilty, both initially developed by Andrew Tridgell as part of his PhD dissertation work, and by Paul Mackerras. It is used for updating files on one machine so that they become identical to a file on another machine while at the same time transferring the minimal amount of data to effect the update, saving on time and bandwidth. Rsync is the underlying component in a great many backup utilities and routines. With the right settings it can even do incremental backups. Andrew is also well-known for having worked on Samba, and won in the EU against M$ in order to get the required interoperability specifications needed to share files using CIFS/SMB.
Previously:
(2014) Ask Soylent: Suggestions for Remote Backup
(2014) How Do You Sync Your Home Directory?
Related Stories
digitalderbs writes:
"A perennial problem facing computer users is how to keep documents, pictures, music and other personal files synchronized between computers. Robust uni-directional solutions, like rsync, and bi-directional solutions, like unison, have existed for a long time. However, these tools require some degree of manual intervention on a periodic basis. Simplified tools like Dropbox and bittorrent sync have emerged as popular, useful and automated alternatives, but these rely on closed-source software, which could be subject to backdooring. Open source solutions, like OwnCloud, are gaining traction, but are these open source platform robust and easy enough to maintain for routine and daily use? Moreover, distributed and encrypted file systems, like Ceph, are increasingly easy to use, but many of these do not work between Linux and OS X or Windows operating systems. What are your experiences and thoughts?"
This is probably one of those topics that gets regurgitated periodically, but it's always good to get some fresh answers.
The small consultancy business I work for wants to set up a new file server with remote backup. In the past we have used a Windows XP file server and plugged in a couple of external USB drives when space runs out. Backups were performed nightly to a USB drive and taken offsite to a trusted employees home.
They are looking to Linux for a new file server (I think more because they found out how much a new Windows file server would be).
I'm not a server guy but I have set up a simple Debian-based web server at work for a specific intranet application, but when I was asked about ideas for the new system the best I could come up with was maybe ssh+rsync (which I have only recently started using myself so I'm no expert by any means). Using Amazon's cloud service has been suggested, as well as the remote being a dedicated machine at a trusted employee's home (probably with a new dedicated line in) or with our local ISP (if they can offer such a service). A new dedicated line out of the office has also been suggested, I think mainly because daily file changes can potentially be quite large (3D CAD models etc). A possible advantage of the remote being nearby is that the initial backup could be using a portable hard drive instead of having to uploading terabytes of data (I guess there is always courier services though).
Anyway, just thought I'd chuck it out there. A lot of you guys probably already set up and/or look after remote backup systems. Even if anyone just has some ideas regarding potential traps/pitfalls would be handy. The company is fairly small (about 20-odd employees) so I don't think they need anything overly elaborate, but all feedback is appreciated.
(Score: 3, Insightful) by Barenflimski on Tuesday August 30 2022, @04:38AM (2 children)
I like these articles that further the education of those that haven't spent days reading the man page.
A seminar is great.
If I would have tried to download a seminar when I was learning rsync, it would have taken me 3 years.
rsync --help
May the force be with you.
(Score: 0) by Anonymous Coward on Tuesday August 30 2022, @10:48AM (1 child)
rsync is one of those wonderfully useful tools that just blows the minds of people who have not heard of it, especially Windows people.
(Score: 4, Informative) by RedGreen on Tuesday August 30 2022, @03:41PM
"rsync is one of those wonderfully useful tools"
Most certainly is I have used it for god knows how long now it has been that long and it has never failed me even a single time. I use it many times a day though not always with me personally doing it, but at least 24 times a day with my cron job scripts that run every two hours backing up my home and / directories, my personal version of backing up my machine. I tried some of those other programs but found they were rather convoluted and limited in their application. My scripts do the backups the restore script allows me to select any of those multiple backups I have made to restore onto new drive and boot, it always does without fail. Same as the script I have that clones the running system to a boot able internal or external drive for testing or transfer onto a backup machine(s), those always boot too, all thanks to that wonderful utility that Just Works like a champ.
"I modded down, down, down, and the flames went higher." -- Sven Olsen
(Score: 3, Disagree) by bzipitidoo on Tuesday August 30 2022, @03:57PM
One improvement I'd like to see in rsync is better handling of unsupported attributes. Too often, when copying to FAT, rsync spews a bunch of warnings that it has simply dropped the attributes that FAT can't support. If rsync has an option to save that info somehow, like to a text file, seems like it should be default behavior.
(Score: 4, Interesting) by ElizabethGreene on Tuesday August 30 2022, @04:38PM
The normal upvote moderation system doesn't give me a way to express appropriate appreciation for this post. It's worth the price of admission. 10/10, 5*, would read again.