Various projects from the last couple of years

Tahoe

http://allmydata.org

I wrote the majority of the code in Tahoe, a robust distributed secure filesystem. It allows you to upload files and directories into a "cloud" of servers, retrieve them later, and securely share them with others using a fine-grained capability-based access control model. Encryption and cryptographic integrity checks are used to prevent the servers from examining or corrupting the files, and erasure-coding is used to enable recovery of files even if a significant number of servers are not responding correctly. Clients thus do not rely upon servers for confidentiality or integrity, merely for a certain level of reliability, making it possible to use untrusted hosts for storage.

Tahoe is an open-source product of the AllMyData consumer backup service, which uses a Tahoe backend to provide backup services for their customers. It is also very easy for a group of friends to create a private grid and store files on each others machines, providng an easy, secure, and robust backup solution.

Tahoe runs on Linux, MacOS (and other unix derivatives), and Windows. Graphical frontends have been developed for Windows, the iPhone, and web browsers (both a pretty javascript-based UI and a bare-bones non-javascript version), all of which use Tahoe's RESTful webapi. Tahoe is implemented mostly in Python, using Twisted and Foolscap.

Buildbot

http://buildbot.net/

I am the author (but no longer the primary maintainer) of Buildbot, a build/test automation system which helps software projects perform builds or run unit tests every time the codebase is changed. The idea is to make it cheap to constantly test the software or make sure it still compiles, so that simple bugs are discovered sooner, before they become expensive. Build status is delivered through a variety of means, including web pages, email, and an IRC bot.

There are several hundred Buildbot installations world-wide, both for free-software projects and proprietary in-house code. It happens to be implemented in python, but this is mostly hidden: many of the publically visible Buildbot instances are compiling and testing C++, Perl, and other languages. I started Buildbot in the fall of 2002, first announced it at PyCon 2003 with a paper and a talk abstract, and the first public release was made in April 2003. Buildbot is in active use by many sites and has a busy mailing list.

Petmail

http://petmail.lothar.com

Petmail is a spam-resistant communication system, intended as a replacement for SMTP. It combines a plausible economic service model with a strong receiver-grants-permission security framework. Sender and receiver anonymity are options: nothing in the design prohibits anonymity (unlike many other current proposals), and the agent can provide a convenient frontend for modern remailer networks.

By writing and promoting Petmail, I have also met most of the criteria for being an anti-spam kook on rhyolite.com.

I presented this at CodeCon 2004, and continued code development for a year or so afterwards, but haven't put much time into it since then. An article was written about it in a japanese magazine named Software Design by Takashi Matsuo, published around May of 2006 (unfortunately the article itself is not available online).

Twisted Python

http://twistedmatrix.com

I'm an occasional contributor to Twisted, a framework for doing event-driven network programming in python.

Foolscap

http://foolscap.lothar.com/trac

For the last few years I've been working on a new RPC layer for Twisted. The intention is to make it easier and safer to expose local functionality to remote parties. In addition I'm trying to bring even more of the concepts from the E Language and the object-capability world into python, specifically:

Halloween 2005

For halloween last year, I dressed up as an RFID-enabled passport, complete with alarmist sayings on the back. The little radio tower on my helmet had a small microcontroller and a few LEDs that I threw together to blink out my name, address, and phone number in morse code all night. (the code and conversion script were pretty small).

Alas, nobody got it.

Mailcrypt

http://mailcrypt.sf.net/

Mailcrypt is an emacs interface to encryption programs that's been around forever (since the first days of PGP 1.0). It makes it pretty easy to encrypt/decrypt and sign/verify email and news messages. It also serves as an outbound anonymous-remailer frontend. I added support for GPG to it around 1998, and wound up becoming the maintainer for a while with Len Budney. Neither of us have had any time for it for a long while, and it is in need of a new maintainer.

If I ever get around to putting some time into it again, I'd like to add unit tests and frontends for modern remailers (mixminion and some nymservers).

EGD

http://egd.sf.net/

I wrote EGD: The Entropy Gathering Daemon back in 2000 (or earlier) because I wanted to use GPG on a solaris box that didn't have /dev/random . It is a perl script which gathers system-wide statistics (from commands like 'ps' and 'iostat') and hashes their output into an entropy pool. On a busy system, this provides a moderately good source of random numbers. The daemon runs in the background, and you connect to it over a TCP or UNIX-domain socket to read out the entropy.

GPG and OpenSSL added code to read from this socket interface, and for a little while it seemed to be the entropy source of choice on non-linux/bsd platforms. I haven't touched EGD since 2002, and as a recovering perl programmer, I'm fairly embarrassed by the quality of the code by now. Fortunately there are several other tools that provide the same interface these days, and most of the world has moved on to platforms that provide a sensible /dev/random in the kernel (where it has access to much better data anyway). So EGD is no longer as relevant as it was back then.

Microcontrollers

I've built a number of small projects with AVR microcontrollers. I used to use PICs, but I switched to AVRs because of the wider range of chips, from 8 pins to hundreds, some cheaper than a dollar, all of which are flash-based. The fact that GCC has an AVR target makes development much easier.

CarMP3

I have a small laptop in the trunk of my car with a copy of my music collection (mostly encoded to Ogg Vorbis). There's a joystick up by the drivers seat to control it, and the audio feeds into the car stereo's head unit. It arrives over the same audio bus used by the CD changer, through a simple passive mixer I built, but does not participate in the digital control channel, which means the head unit is only willing to play the sound from the audio bus when it thinks there is a CD playing. Rather than disconnect the CD changer's audio completely, I burned a CD full of zeros (entitled "The Sound Of Silence") and leave it playing on track-repeat.

The laptop has an 802.11 card, and a special sequence of button presses causes it to wake up, associate with my home network, and signal a daemon on my desktop machine to push over new music to it (via rsync and ssh).

Phidgets

Phidgets are small, convenient USB interface boards. I've got some patches for their linux C library, and a new python binding.

Miscellaneous

I have a huge list of projects that haven't yet gotten off the ground. One of the meta-projects is to publish these half-started ideas on the web to solicit comments, inspire others, and goad myself into making progress on them.


Brian Warner <warner@lothar.com>
Last modified: Sun Mar 29 15:11:16 CDT 2009