Brian's weblog

I was thumbing through some of my old del.icio.us bookmarks today, and came across sparkfun electronics again. Man, their coolness doubles in size every six months. $25 for a half-inch square self-contained radio data link, serial interface that you can run with a microcontroller, 3V, built-in antenna. Wow. $6 for a white Luxeon 1W LED ($8 for 3W, $25 for 5W). $5 for a 1W Luxeon that's TWO FRIGGING MILLIMETERS on a side. Holy crap.

And $20 for a color LCD like the ones from a cellphone. And speaking of cellphones, $184 gets you a quad-band cellphone module with a GPS receiver, camera driver, and a python interpreter. Add an antenna, a battery, a serial port, and a SIM card, and you've got a mobile data node. And I think you can even get prepaid SIM cards that can be topped-off online.

(note to self, places like this sell such cards, generally 5 to 20 cents per minute, which can be recharged with scratch-off coupons. And it looks like you can buy them from retail cellphone shops too. They all come with a phone number.. no wonder the phone numberspace is getting so crowded, you can buy them from vending machines in some countries..)

Each time I visit these folks (or browse through the digikey catalog, or just look through my old notebooks), I feel such a strong drive to build something. The delay involved in actually getting the parts usually means I don't get around to doing it. But maybe if I just keep buying stuff and stocking my workbench then the next time I'm in a construction mood I'll have everything I need already at hand and I can just start soldering away...

Oh happy day! The buildbot.net trac instance just recently got visited by the link spammers. They haven't caused any actual damage yet, just a user account created with advertising in the profile text, but I'm afraid it's only a matter of time before the bots descend upon us and we're smothered by a wave of sentient AIs dedicated to filing mass buildbot bug reports containing nothing but links to offshore casinos and faux designer watches.

sigh.

I guess I should add some sort of "prove you can read" test to the account-creation page, just barely enough to make the script kiddies work for a living. Something like "what is 1+2?" or "type the word 'please' in here" or something.

Reminds me of a suggestion someone made to me while I was working on petmail: you don't need super-clever CAPTCHA techniques if you can manage to have a whole bunch of different requirements instead, like each user creating their own simple technique. A bot could be written to mass-solve any particular one, but since everybody is creating their own, the bot-writers job is that much harder.

And sometimes, just raising the bar a bit is good enough for now. As the joke goes, I don't have to outrun the lion.. I just have to outrun you :-).

I just finished building a Trac instance for Foolscap, now online at http://foolscap.lothar.com/trac . It's got a (mercurial-based) code browser, tickets, and a wiki.

Setting it up required some twisted.web hacking, because my setup puts a twisted.web server out front, and reverse-proxies certain requests to a separate Xen virtual machine which handles all CGI (for multiple sites, like buildbot.net and foolscap.lothar.com). That CGI host is running apache, and since URLs inside returned pages are not being rewritten, I had to use named virtual hosts to distinguish between, say, http://buildbot.net/trac and http://foolscap.lothar.com/trac .

But the normal twistd.web.proxy ReverseProxyResource clobbers the Host: header when it forwards the request (setting it equal to the new host being targeted). I suppose this is to hide the presence of the proxy from the new host, but in my situation is has the effect of making it impossible to use vhosts on the apache side to distinguish between requests that were received for different hostnames.

So I subclassed and commented out that line, and apache is happy. Now that I can have more than one trac instance on this box, I'm creating Tracs for everything. Whee!

Wow, so long since I updated this. Each time I remember that I do have a technical blog, and think to add something to it, I am tempted to start by rewriting the whole blog system in some brand new way that will make it easier to post to (and, the theory goes, therefore make me more likely to write in it). The process of writing more code creates something that I'm even less likely to understand next time, and code begets more code. It's like a depth-first search through an infinite design space. Bad idea.

And speaking of technical distractions, I've been playing with Mercurial recently. I like it. I moved Foolscap from Darcs to Mercurial last week, mostly to learn more about it, and I've been pleased. My main reason was to make it easier for folks to hack on Foolscap: darcs is all fine if you're running debian and someone else has compiled it for you, but if you have to build it yourself you have to start by building GHC, which is a non-trivial adventure.

Mercurial's plugin architecture is pretty nice: one line in the .hgrc file tells it to import a .py file, which registers a set of new subcommands with the main /usr/bin/hg entry point. Which reminds me that I want to adapt Trac's plugin mechanism (which lets you drop an .egg file in a specific directory and then reference modules inside it from the config file) to Buildbot, to make it easier for users to get interesting code into their master.cfg files. Not that huge of a change, but it would make the installation instructions for that code to get simpler; no need to change sys.path from within master.cfg .

And because the plugin approach makes it easy, people are writing fun plugins. The Tk-based graphical revision browser is great (and has a little tram-line-style graph of which revisions got merged into which, very cute). The 'bisect' extension helps you do an efficient binary search for the revision which introduced (or fixed) a bug.

I'm still trying to figure out the "forest" extension, though. I think it's what I want for tracking a couple dozen separate small projects (things I've been doing in CVS for years, since I can update just one at a time, or commit the whole lot of them and push the work from my laptop to my desktop). But for the life of me I can't figure out how to use it, and the documentation is heavy on the per-subcommand reference and light on the big-picture descriptions.

And mercurial is fast. The cgi-based web server lets them speed up the initial checkout: for the full Foolscap repository, doing a 'darcs get' through the naive (twisted.web) server took 22 seconds (of which probably 17 was network), whereas doing the equivalent 'hg clone' from a hgwebdir.cgi server (under apache) took 6 total. Mercurial manages to store the history more compactly too: the tree with full history under darcs was 4.4MB, and 2.9MB in hg.

I've been using Darcs for a year or two now, and we've been using it extensively at work, and it's fun (the incremental commit feature is amazing, and I miss it in hg, and it wouldn't be impossible to add). But every once in a while something explodes (possibly because we've used 'darcs oblit' more than once, and that seems to be an underexplored corner of the darcs jungle). I really like the append-only and cryptographically-secure nature of hg revisions, and regret that you can't securely and concisely name a specific darcs revision the way you can with mercurial. Having spent a lot of time defining sha-256 hash-based identifiers recently, I'm coming to be wary of any system that doesn't let me create strong references like that.

So I'm looking forward to playing with it more. Commuting patches is nifty, but for things like Buildbot and Foolscap I'm not really creating crazy branches with patches that need to be held out of trunk for months at a time. So I think hg has a lot of promise.