Your own private dropbox with free software

There are lots of file storage and sharing software packages out there that make it easy for a group of people to share files. Dropbox is perhaps the most well known of the group, it provides an easy way for a group of people to share files. The downside of Dropbox is that it is not a private service, just like any cloud-based service. Dropbox has total access to your files that you store there. That means its likely that the NSA and its collaborators do too.

Dropbox also knows where the computers are that are accessing their service because they can see the IP address of the incoming connections. To help with this, it is possible to use use Dropbox over Tor, thankfully they have provided proxy settings.

For our shared files, we use SparkleShare. It provides an experience very similar to Dropbox: you have a SparkleShare folder that is synced up with the service, and in turn any other users who are also linked up to it. Once its setup, it is as easy to use as Dropbox, but setting it up takes a bit more work than Dropbox. It builds upon two software packages well known for security and reliability: git and ssh, and works with Tor Hidden Services. It runs on Windows, Mac OS X, and GNU/Linux, and there is an Android client in the works.sparkleshare-onion

You can use any git service as the server for SparkleShare, including github, bitbucket, gitorious, etc. But these have the same issues as putting your files on Dropbox: that service has complete access to your files. For extra protection, SparkleShare can encrypt the files on the client-side, have encrypted shared folders with SparkleShare, so that the server does not have access to the files. For the last piece of setting up a private SparkleShare, you need a computer that you can ssh to, and has git and Tor on it. This computer could even be an old Android device running Lil’ Debi, it only needs enough disk space for your SparkleShare files and a steady network connection. Running it on your own computer means it can use a Tor Hidden Service, and that all of the metadata related to who is editing what files remains private.

To start, setup a Tor Hidden Service to the sshd port. You can read all about that process in the Tor instructions, but basically, you need to add something like this to the torrc configuration file:

HiddenServiceDir /var/lib/tor/ssh_hidden_service/
HiddenServicePort 22 127.0.0.1:22

Then restart tor, and it will generate two files in /var/lib/tor/ssh_hidden_service/, open the file called hostname to see your new .onion address. We’re going to use fakefakefakefake.onion as our made-up one for this HOWTO. That is the address you will use in SparkleShare as the server address.

Next ssh needs to be setup to use Tor to access the .onion address of the Tor Hidden Service. To do that we need the wonderful Netcat tool (nc). On Debian/Ubuntu, run sudo apt-get install netcat-openbsd to get it, its included with Mac OS X by default. Now edit your SSH config file, its usually in ~/.ssh/config, and add this section:

Host *.onion
     Compression yes
     ProxyCommand nc -X 5 -x 127.0.0.1:9050 %h %p

For Windows, you need to use connect proxy, which is thankfully included in SparkleShare. You can optionally use connect instead of Netcat/nc on Mac OS X (fink install connect or brew install connect) and GNU/Linux (e.g. apt-get install connect-proxy or yum install connect-proxy). Instead of the snippet above, use this snippet in ~/.ssh/config to use connect:

Host *.onion
     Compression yes
     ProxyCommand connect -5 -S 127.0.0.1:9050 %h %p

Now its time to set up the git repo on the server that will be the conduit for sharing files between the different users. Basically, all you need to do is create a new folder, then make it a “bare” git repo (you can read all about it in the git book):

ssh git@fakefakefakefake.onion
mkdir /home/git/MyPrivateShare
cd /home/git/MyPrivateShare
git init --bare

For sshing to the git, we set up a single account called git, then to grant access, we add the SSH key (SparkleShare calls this the Client ID) to the git account’s ~/.ssh/authorized_keys file.

Now everything should be ready to start adding clients! In SparkleShare, go to Add Hosted Project…, choose On my own server, then enter your username and .onion address (ssh://git@fakefakefakefake.onion) in Address and the path to the git repo (/home/git/MyPrivateShare) in Remote Path:

SparkleShare Setup

Now the client will download the entire git repository from the server, and you’ll then have a working shared dropbox! If there are a lot of files in it, then the first sync can take a long time before any files show up. This is because git first downloads the entire history first, then it checks out the files. After that initial setup, then the new files show up quite quickly.

So this SparkleShare setup keeps your files on computers that you control, it prevents information and metadata from being leaked to the network while people are using this SparkleShare setup. When using Client Side Encryption, even less data is leaked. The server cannot access the content of the files at all since they are encrypted. The the server in this case would only be able to see the network traffic, and which ssh key was used to access the server. If everyone accessing this setup used the same user account (i.e. git) and ssh key, then the server would not even know which user is making the changes. This is about as private as you could hope for in a shared dropbox folder.

One last nice feature of this setup is that git server does not need a domain name, static IP or even a public IP, it just needs a working internet connection. As long as it can connect to Tor, then the Hidden Service will work. So if this private dropbox is for extra sensitive stuff, it could be stashed anywhere it can get power and wifi.

11 comments for “Your own private dropbox with free software

  1. Anymous
    2013/11/12 at 6:27 pm

    The SSH-key to add to authorized_keys can be found in your local Sparkleshare folder.

  2. Hans-Christoph Steiner
    2013/11/29 at 9:32 pm

    I forgot to add: if you need up-to-date SparkleShare packages for Ubuntu/Mint, you can get them from the Guardian Project PPA (fingerprint: F50E ADDD 2234 F563):
    https://launchpad.net/~guardianproject/+archive/ppa

    sudo add-apt-repository ppa:guardianproject/ppa
    sudo apt-get update
    sudo apt-get install sparkleshare
    
  3. KLP
    2014/09/28 at 2:11 am

    Security wise, how does this method compare to Bittorrent Sync?

    • Hans-Christoph Steiner
      2014/09/29 at 1:43 pm

      There a number of ways that it is a improvement, privacy-wise:

      • full Free Software stack, so it can be audited (BTSync is proprietary and closed)
      • all network communications can be via Tor
      • you can control which machines are running it, to leak less metadata
  4. 2014/12/18 at 2:16 pm

    Actually there is an easier way to do so if you have a computer or storage that has SFTP (SSH) enabled. You can use odrive (www.odrive.com, a free software currently in beta) as your sync folder to access whatever storage you already have, set up is fast and easy: http://blog.odrive.com/blog/2014/12/16/create-your-own-dropbox-to-nas-mac-linux

    • Hans-Christoph Steiner
      2014/12/19 at 7:26 am

      Do you mean that odrive is Free Software, as in free as in freedom, or that you don’t charge money for the software? If odrive is free, open source software, it could be quite interesting. If it is yet another proprietary software service, then it is not at all interesting to us. The only way that a user can be sure that software is doing what it is supposed to be doing is by having a fully free, open source client so that people can inspect the source code, and even build and run their binaries.

  5. Max
    2015/12/20 at 1:56 pm

    This sounds very interesting to me as I was looking for a self-hosted
    cloud storage with encryption and Tor “support” or atleast Tor friendly,
    compatible with a Hidden Service and with the user-friendliness of
    Dropbox (the dream).

    The thing is Sparkleshare doesn’t seem to be supported anymore. Last
    release is from Apr 20, 2014 (1.4) and the github project page isn’t
    very active. Aswell, the Sparkleshare version available in Debian repo
    is only 1.2.

    So, not complaining about that, I just wanted to know if you still use
    this setup, and if so, do you find it stable enough?

    If not, have you find any alternative compatible with a Hidden Service?
    I would be interested if that’s the case.

    • Hans-Christoph Steiner
      2015/12/21 at 8:38 am

      We do still use our Sparkleshare, but not heavily. v1.2 has been stable, and as far as I can tell, the newer versions are just to fix issues on Windows and OSX. If you are looking for a similar solution that is under active development, I recommend checking out ownCloud. I think SparkleShare will be more secure, since it is based on ssh and git, while ownCloud is PHP.

  6. joedoe
    2016/06/27 at 6:44 pm

    I hate to post on a really old thread but sparkleshare is still good to use imo

    1) since it uses Git and SSH, what else can you do as far as updates? (a lot of the stuff is handled by other standard applications)
    – Access control: SSH+gitolite/gitlab/gogs/etc, VC: Git, Client: sparkleshare (what can you do? its literally only the UI)

    2) they are working on adding Git-LFS support (which is cool imo but LFS not as “mature” as git annex)

    I honestly got really into git annex because it did the whole git-lfs thing like a few years ago, its an awesome alt to sparkle share if you are a cli person. I’m starting to look at sparkleshare now because now its adding LFS support, also because I need allow people to see some of my projects. (git annex gets complicated when sharing with groups)
    Anyways sparkleshare isn’t dead and seems as reliable as an old AK-47.

    Also owncloud over tor sucks. Its really slow if you only have APC enabled, you may want to use memcached or something else.

Leave a Reply

Your email address will not be published. Required fields are marked *