BitTorrent Sync device in the Google Cloud

I use BitTorrent Sync to distribute copies of my personal files (documents, pictures, GoPro videos and so on) and to share files with my friends. For the later to work reliably, the data needs to be stored in a device which is always turned on. I have a Mac Mini at home and a personal laptop, both of which are asleep during much of the day. I run a few instances in the Google Cloud, so I was looking into putting a BitTorrent Sync device there so it can serve as a central hub.

Google Cloud Storage

At first I looked into storing the files in Google Cloud Storage (a bucket similar to S3, if you’re familiar with Amazon AWS). This is by far the cheapest option, when using the Durable Reduced Availability (DRA) storage class, it costs 20$ / TB / month. Also, there is no need to provision space, and you pay for what you use. However the BitTorrent Sync client needs to have the files on a filesystem. I found gcs-fuse, an implementation of FUSE which stores files in Google Cloud Storage. But the btsync client refused to work with it. I suspect the high latency and the fact that the gcs-fuse implementation is not particularly efficient when applications continuously write small blocks into large files (as is the case with any BitTorrent client). I could not get BitTorrent Sync to run reliably, I had to give up.

Google Compute — Persistent Disks

Persistent Disks is a type of block storage which you attach to instances. You need to provision a fixed size and you pay for that, not just for what you use. Persistent Disks are twice as expensive as Google Cloud Storage DRA (40$ / TB / month). I did not want to overprovision the space, and still have the option to grow in the future as needed. My idea was to provision space in small blocks (~ 100GB) and add more disks as needed.

I briefly considered using ZFS to manage the disk pool. I’ve had good experience with ZFS, but setting it up on my instances would be difficult: I run CoreOS on my instances and everything is packaged up in Docker containers. CoreOS supports btrfs out of the box, and it is trivial to mount a btrfs pool and expose it as volume to a Docker container.

Initial BitTorrent Sync Device setup

Once I had the disk pool mounted and a Docker container with BitTorrent Sync up and running, I needed configure the new device. The whole setup is done through a web UI, which is nice.

The easiest way to share folders with the new device would be to link it with my existing identity. But instead of that, I created a completely new identity. The reasons are so that I can treat it as a untrusted device, individually select which folders I want to share with it, and most importantly, so that I can use the Encrypted Folder feature to encrypt my sensitive files. I can even offer my friends a backup service if they give me the Encrypted key of their folders.

I named the new identity Tomas Carnecky (Google Cloud):