SoylentNews is people
SoylentNews
Canonical gave up on operating its Ubuntu One cloud storage service more than a year ago, but this week it released the system's file-syncing code under an open source AGPLv3 license.
Though Canonical is primarily known for its open source Ubuntu operating system, it also has some closed source products and services, including Ubuntu One.
Ubuntu's desktop "on its own will die"—Shuttleworth [said, explaining] why Canonical must expand. "Today, we're happy to be open sourcing the biggest piece of our Ubuntu One file syncing service," Canonical Director of Online Services Martin Albisetti wrote. "The code we're releasing is the server side of what desktop clients connected to when syncing local or remote changes. This is code where most of the innovation and hard work went throughout the years, where we faced most of the scaling challenges and the basis on which other components were built upon."
Canonical hopes the code will be "useful for developers to read through, fork into their own projects or pick out useful bits and pieces."
What project ideas would you use this code for?
(Score: 4, Insightful) by AndyTheAbsurd on Thursday August 13 2015, @08:14PM
What use is this when we already have rsync [die.net]? Is this code in any way superior to rsync, or keeping files in git (and using the fast-forward merge to combine copies), or any of half a dozen other file synch solutions?
Please note my username before responding. You may have been trolled.
(Score: 3, Interesting) by Anonymous Coward on Thursday August 13 2015, @08:24PM
I for one want something 'like' dropbox. But uses my own at home harddrives. There are some products like that. But none seem to be exactly what I want or have some other 'pay me x per year' service in the middle of it all.
Just something where I drop a file in and it pops out on 3 other boxes. With no space/size limits other than what space I have available.
No tinkering together a bunch of rsync scripts or whatever. Just something that syncs a set of folders together and lets me set particular permissions to particular users. And lets me limit the bandwidth with some sort of QoS.
(Score: 3, Interesting) by Zinho on Thursday August 13 2015, @09:02PM
I for one want something 'like' dropbox. But uses my own at home harddrives.
Amen, brother!
How long before setting up a personal cloud is considered a normal thing that vendors would support? I've got an android phone, and it seems my choices for off-phone archive is synch to Google or nothing. I've heard that it's the same for Apple and iCloud. Seriously, how hard should it be to let me designate my own server for this?
If having an open-source cloud implementation out there is step one for making this happen, then I wish the community lots of success in carrying the development forward.
"Space Exploration is not endless circles in low earth orbit." -Buzz Aldrin
(Score: 3, Insightful) by c0lo on Thursday August 13 2015, @09:31PM
How long before you get a static IP address and your ISP allows you to run public servers in your home net?
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 3, Informative) by NCommander on Thursday August 13 2015, @09:49PM
Get a Hurricane Electric IPv6 tunnel, which gives you a static endpoint + /64 to use as you please. I wasn't involved with Ubuntu One development, but I'm fairly sure the codebase is IPv6 clean. I do know most of it is written in python, so I'd be rather surprised if it choked.
Still always moving
(Score: 2) by Dunbal on Thursday August 13 2015, @11:11PM
They will never let it happen on an asymmetrical residential subscription when they can get you to pay more for a "business" subscription (for virtually exactly the same service).
(Score: 2) by Zinho on Friday August 14 2015, @02:01PM
How long before setting up a personal cloud is considered a normal thing that vendors would support?
How long before you get a static IP address and your ISP allows you to run public servers in your home net?
You're solving the wrong end of the problem. The networking aspect of this is trivial; my issue is that none of the phone vendors nor any of the app vendors are willing to give an option to use a personal cloud instead of a company-owned one. There is literally no option for this in the software configuration.
I suspect that, beyond the fact that there's not a standard for personal clouds to code against, many of them are monetizing the data, and they can't do that if you are storing it locally instead of dropping it in their laps. Kinda hurts their motivation level.
I also suspect that, regardless of how much of a privacy win it would be, the general consumer is going to look at the operating requirements for a personal cloud, respond "That looks hard!" and not bother. Either that, or they'll buy it pre-packaged and then leave the default admin password in place.
You know that saying about leading a horse to water? It's like I'm the horse, I'm thirsty, and the only places I'm being led want to fill my trough with opiate-laced beer. All the other horses there are stoned out of their minds, and looking at me like I'm crazy, because "thish shtuff ish the BESHT!"
"Space Exploration is not endless circles in low earth orbit." -Buzz Aldrin
(Score: 0) by Anonymous Coward on Thursday August 13 2015, @09:45PM
The only one I have that has come 'close' is the one built into my synology. But even that is 'not quite right' and only works in the synology eco system.
https://www.synology.com/en-us/dsm/5.2/cloud_services [synology.com]
(Score: 2) by Marand on Thursday August 13 2015, @09:44PM
I for one want something 'like' dropbox. But uses my own at home harddrives. There are some products like that. But none seem to be exactly what I want or have some other 'pay me x per year' service in the middle of it all.
Exactly this. User-friendly, personal cloud storage isn't a mature area of software development yet, so there's still plenty of room for more FOSS options. More projects in this area is good, because right now it seems like the best options are "roll your own" -- which isn't something most people want to do -- or use ownCloud, which is unfortunate. Whenever this sort of topic comes up, people mostly seem to suggest ownCloud, but using it requires setting up your own httpd (apache suggested) with PHP on it, so you're practically begging to have your server hacked if you decide to use it.
(Score: 2) by NCommander on Thursday August 13 2015, @09:51PM
Someone should just write an AppArmor profile for it if you're concerned about the exploitability of mod_php (we use AppArmor for this site as an additional line of defense). While PHP's design lends it to making bad code much more easier than other languages, it is perfectly possible to write secure and stable PHP code.
Still always moving
(Score: 2) by Marand on Friday August 14 2015, @04:50AM
Someone should just write an AppArmor profile for it if you're concerned about the exploitability of mod_php (we use AppArmor for this site as an additional line of defense).
Because adding apparmor to the mix makes the entire thing even more user-friendly and less error-prone for your average user...
While PHP's design lends it to making bad code much more easier than other languages, it is perfectly possible to write secure and stable PHP code.
It's possible to write secure, good code in any language. That doesn't mean some languages don't go out of their way to make it harder, though, which is the problem with PHP and security. The track record of PHP itself has done a good job of showing that secure PHP basically requires fighting against the language itself and hoping that the devs don't do something bone-headed or make some fucked up change between minor versions that makes previously-safe code unsafe. The situation gets worse when you stray from Apache+mod_php, too. For example, I prefer lighttpd, which means using PHP gets even riskier if I choose to run it, because PHP's largely tested against mod_php and you get a bunch of fun edge cases using it other ways (fastcgi, php-cgi)
I wasn't trying to turn this into an excuse to attack PHP, though. The point was (and still is) that it's an area that should be welcoming new code, because the choices are fairly limited right now, and the most common solutions are practically a minefield for the average user. Instead of "cool, new code!" the news is greeted with a snarky remark about rsync (original poster), a "make your own" suggestion (another comment farther down), and "all you need is ownCloud" (also farther down).
I found it interesting because, at least based on the brief code-browsing I did, it looks like the entire thing is in Python, and it seems to be a case of "create one thing to do one job" rather than trying to piggyback on top of browsers. Between the server and client code, there's a chance to have an actual, personal dropbox-esque service with multiple clients. Ubuntu One failed as a commercial service, but having its source and the option of running your own has the potential to be a great thing.
(Score: 5, Informative) by number6 on Thursday August 13 2015, @10:15PM
__How to Sync Files Between Computers Without Storing Them in the Cloud
__http://www.howtogeek.com/166424/how-to-sync-files-between-computers-without-storing-them-in-the-cloud/
__Jan 2013
So you have multiple computers and you want to keep your files in sync, but you don’t want to store them on someone else’s servers. You’ll want a service that synchronizes files directly between your computers.
___________________________________
BitTorrent Sync [bittorrent.com]
BitTorrent Sync uses BitTorrent to transfer files — in private and in encrypted form, so no one can snoop on them. Just install it, select a folder, and generate a secret. Provide that secret to anyone — either another computer you own or a friend you would like to sync files with — and your folder will be automatically kept in sync across all configured PCs. This happens directly — either over a local network or over the Internet — using the powerful and fast BitTorrent technology.
BitTorrent Sync offers clients for Windows, Mac, and Linux, so you can use it to sync your files with computers running any popular operating system. Unlike many other services, its features are completely free and it doesn’t require you run a separate server.
___________________________________
AeroFS [aerofs.com]
AeroFS is free, assuming you don’t need its more advanced features. It creates a Dropbox-like folder on your computer and files sync automatically between the computers you set up. You can share each folder with one additional person, but you’ll need the paid version to share with additional people after that. It doesn’t use BitTorrent and relies on a user account system — there’s a central server that manages user accounts and sharing, but files aren’t hosted on AeroFS’s servers. They’re only stored on your computers. AeroFS promises that it “can’t even see your file names.”
Its interface is very similar to Dropbox’s, even down to the tour that appears when you install it. It’s a very Dropbox-like solution, but it skips the cloud so you can sync unlimited files. Like Dropbox, it supports Windows, Mac, and Linux.
___________________________________
Cubby [cubby.com]
LogMeIn’s Cubby offers cloud storage, but it also offers a “DirectSync” feature. DirectSync allows you to synchronize an unlimited number of files directly between computers, skipping the cloud. Microsoft’s Windows Live Mesh used to do this, but Live Mesh has been discontinued. Cubby is available for both Windows and Mac OS X; there’s no Linux support.
You will have to create an account, and cloud storage is enabled by default in Cubby. While DirectSync was previously free when we recommended it as an alternative to Windows Live Mesh, DirectSync is now a paid feature. Unless you really love Cubby, you’re probably better off with another solution.
___________________________________
Roll Your Own Server
These are the two biggest options. However, these aren’t the only ways to sync files directly between your own computers. There are other options you have, although those solutions aren’t as easy to use and will require more manual configuration:
There are many other options you can use. Anything with a self-hosted server component or any sort of solution that automatically creates incremental backups and uploads to a remote server will do, but you’ll have to host your own server software in both cases. Solutions like BitTorrent Sync and AeroFS are the most convenient because they’re not made to require a separate server — they just run on your existing computers.
___________________________________
Disadvantages
Of course, there are a number of disadvantages to doing it this way. You’ll have to ensure you have backup copies of your files, as there’s no central backup copy in the cloud on someone else’s servers. There’s also no way to access these files from your phone or tablet with a mobile app, as you can with the Dropbox, Google Drive, or SkyDrive mobile apps. They’re not stored on a central server the apps can pull from; they’re just automatically synced between your computers.
And, of course, your computers must be powered on at the same time or they won’t be able to sync directly with each other.
In return, you get the ability to sync an unlimited number of files and keep them entirely under your control. It’s up to you which tradeoffs you want to make.
(Score: 2) by number6 on Friday August 14 2015, @01:39AM
__How I Created My Own Personal Cloud Using BitTorrent Sync, Owncloud, and Raspberry Pi
__http://blog.bittorrent.com/2013/05/23/how-i-created-my-own-personal-cloud-using-bittorrent-sync-owncloud-and-raspberry-pi/
__May 2013
I’ve had a Raspberry Pi lying around the house for a few months now. Every once in a while I’ll take it out and tinker around with it. When BitTorrent released the BitTorrent Sync Alpha a few weeks back, I decided to see what kind of options I had for using the program with my Raspberry Pi.
BitTorrent Sync [bittorrent.com], an awesome software allowing you to sync folders between machines using P2P. No cloud necessary. I’ve been using it for syncing several gigabytes of RAW photos and video across my various machines. There is the occasional scenario however, where I’ve wanted to grab a few files, but my other machine is turned off. To solve this problem, I’m using a Raspberry Pi as a low power, always-on device with Sync installed.
Just for kicks, I’m also using the open-source Owncloud [owncloud.org] to provide me with a web interface for accessing my files from any computer, including my mobile phone. Sound interesting?
If you have a DIY attitude and aren’t afraid of a few terminal commands, then read on to learn about how I did it.
First, you’ll need the following:
_______________________________________________
Step 1: Setting up Raspbian
Fire up your favorite torrent client and download the Raspbian “wheezy” image. Click here for the link [raspberrypi.org].
I won’t go in to full detail on how to write the images to the SD card. There are plenty of resources available for that. If you are using Windows, check out this link [elinux.org]. If you are on a Mac, check out this link [elinux.org].
When that is done, plug the SD card into your RasPi and fire it up. For the initial configuration screen that appears I recommend setting your username and password, expanding the root partition, setting your boot_behavior to start desktop on boot and enabling SSH. Exit the configuration screen and you’ll eventually be presented with the Wheezy desktop.
_______________________________________________
Step 2: Installing BitTorrent Sync
Make sure you have an ethernet cable plugged in to your RasPi, and fire up lxterminal (there is a shortcut on the desktop). We’ll need to download the binary for Sync. To do this, type in:
Then change the download directory:
extract the tar.gz:
And finally, run the program:
Now, we want to make sure Sync will run at startup. This requires an extra step. In terminal, change your directory to the config folder:
Create a new folder called “autostart”
Now create a new text document. You can use the leafpad program that comes pre-installed. The contents of the text document need to be as follows:
Save this file in the newly created ~/.config/autostart as syncstart.desktop
Sync is now up and running on your tiny Raspberry Pi. You can access the web GUI by opening a browser and typing in [YOUR IP ADDRESS]:8888/gui (example: 192.168.1.2:8888/gui). You can find the IP address for your RasPi by opening terminal and typing in ifconfig
At this point, you can stop right here. If your SD card is big enough, you can set up Sync and have your always-on Sync device….but why stop there? Lets add a thumb drive so that we can have more space.
_______________________________________________
Step 3: Adding a thumb drive
As mentioned, you have plenty of options for adding hard drive space to your Raspberry Pi. If you want to, you can plug in an external hard drive and go crazy. For this blog post however, I’m just using a micro-sd thumb drive. I love how small it is.
Plug the thumb drive in to the Raspberry Pi. (if you are short on usb ports, I would suggest setting up remote desktop [jeremymorgan.com]).
Open up terminal and type in:
Find your thumb drive. Mine was at the bottom and was called /dev/sda1. Take note of the name. You’ll need it. Format your hard drive by doing the following (replace sda1 with the name of your drive):
If you aren’t interested in Owncloud, then you can stop here and simply set up Sync to point to /media/SYNC. If you are interested in Owncloud, then keep reading.
_______________________________________________
Step 4: Setting up Owncloud
Setting up Owncloud on linux is a mess of various terminal commands. Thankfully, Github user Petrockblog has made it very simple. He created a shell script [github.com] for installing and updating Owncloud. Simply do the following:
Install Github:
Download the latest setup script:
And execute the script:
...and wait. This will take a while. Grab a coffee. Watch this [youtube.com] video. When it is done, you can navigate to [Your IP Address]/owncloud to set everything up----It is very important that you click on “advanced” and set your data folder to /media/SYNC before finishing setup.
_______________________________________________
Step 5: Combining BitTorrent Sync and Owncloud
First we need to clear up a few file permissions. Add sync to the www-data group:
Allow sync to write to the thumb drive:
Now open up the BitTorrent Sync WebUI ( [YOUR IP ADDRESS]:8888/gui ) and add a folder. You’ll want to add /media/SYNC/USERNAME/files where "username" is the name you selected when setting up Owncloud. Generate a key and you are done. You can now use that key to set up a folder on any computer you want. Any files added to that folder will be synced to your RasPi and be available through the Owncloud web interface. While this is pretty cool, there’s just one more little step to make this much more useful.
_______________________________________________
Step 6: Combining BitTorrent Sync and Owncloud
To access our files from anywhere, we’ll need to first set up a static IP address. If you used the wifi dongle I recommended, you can simply plug it in and it will work. Open terminal and type in:
You’ll need to make a few changes in order to get a static IP address. Below is what I ended up using after a bit of trial and error. Your address, gateway, net mask, SSID, and PSK will need to be changed according to your network. The settings may be different if you used a different wireless adaptor, or if you are only using an ethernet cable.
Finally, you’ll need to forward a port so that owncloud can be accessed from outside networks. I just forwarded port 80 for the IP address of my RasPi. In this example, my IP address when using wifi is 192.168.1.22. The procedure for forwarding ports is different for every router. If I grab my real IP address from a site such as this one [whatsmyip.org], I can now access Owncloud and my Sync files from anywhere by going to [MyRealIpAddress]/owncloud (where the IP address is what you find from the website I linked to). You can also set up Owncloud with an Android [google.com] or iOS [apple.com] client for mobile access.
Phew. If you made it this far, congrats. I had fun creating my own personal BitTorrent-powered cloud and I hope you found this walkthrough helpful. There are so many uses for something like BitTorrent Sync, and I think we’ve only begun to scratch the surface.
If you have found any creative uses for BitTorrent Sync, then share with us on our forums [bittorrent.com]. We’d love to hear about it.
-- EOF --
(Score: 2) by Phoenix666 on Saturday August 15 2015, @01:50AM
Thanks for posting this. I have a RPi laying around at home. I'll give this a shot when I get back to town.
Washington DC delenda est.
(Score: 0) by Anonymous Coward on Friday August 14 2015, @04:43AM
The 3 first options are proprietary, therefore unusable.
(Score: 1) by ptman on Friday August 14 2015, @09:21AM
Is this the same ac that posted the original desire? So hard to keep the track of the discussion with anonymous users
(Score: 2) by Appalbarry on Thursday August 13 2015, @11:15PM
Yes to this. As much as I'm growing to despise Google, the bottom line is that my e-mail archives*, address books** select essential backups, and browser*** passwords and bookmarks all wind up on Google's server.
The reason is simple: it just works, has reliable 99.99% uptime, and I know that I can access any of this from any device I'm likely to use.
Thus far none of the roll your own options are as good.
I'd pay for a preconfigured box, or for a package that I can stick in our Dreamhost web space, that would do the same things,
* OK, I do use Gmail backup to back it up locally too. I'm actually watching MailPile with interest.
** Although Google has managed to mess up my contact lists enough times that I really want to be rid of it.
*** OK, I'm actually using Firefox, but it still has that same functionality.
(Score: 2) by NCommander on Thursday August 13 2015, @11:43PM
Basically the same boat I'm in. Not too many places have my underlying @gmail.com address so I could theorically move it without *too* much issue. I know I could easily set up a mail server of my own, or just do a private cloud. While I don't trust Google, for the most part, they've kept to their philosphy of "Do No Evil", and I continue to be an Android user, so no matter what happens, they'll always have some data on me.
Still always moving
(Score: 3, Informative) by meisterister on Thursday August 13 2015, @11:58PM
I'm pretty sure that OwnCloud meets just about all of your use cases.
(May or may not have been) Posted from my K6-2, Athlon XP, or Pentium I/II/III.
(Score: 1) by ptman on Friday August 14 2015, @09:25AM
Take a look at https://syncthing.net/ [syncthing.net] (modeled after bittorrent sync, but open source) and https://git-annex.branchable.com/assistant/ [branchable.com] (more like dropbox, but powered by git-annex which gives a great deal of flexibility). People also seem to have good experiences with OwnCloud, but it is much more than just a replacement for dropbox, and quite a complex beast.
(Score: 0) by Anonymous Coward on Friday August 14 2015, @09:09PM
https://owncloud.org/features/ [owncloud.org]
https://gigaom.com/2014/03/01/how-to-set-up-your-own-personal-home-cloud-storage-system/ [gigaom.com]
http://www.computerworld.com/article/2497495/private-cloud/private-cloud-here-there-everywhere-3-personal-cloud-storage-systems.html [computerworld.com]