The new Sync protocol

Last time I described the user difficulties we observed with the pairing-based Sync we shipped in Firefox 4.0. In late April, we released Firefox 29, with a new password-based Sync setup process. In this post, I want to describe the protocol we use in the new system, and their security properties.

(For the cryptographic details, you can jump directly to the full technical definition of the protocol, which we've nicknamed "onepw", since there is now just "one password" to protect both account access and your encrypted data) more…

Pairing Problems

we learned that our pairing implementation in Firefox Sync had some problems. Some were shallow, others were deep, but the net result is that a lot of people were confused by Sync, and we didn't get as many people using it as we'd hoped. This post is meant to capture some of the problems that we observed.

more…

remote entropy

Can you safely deliver entropy to a remote system?

Running a system without enough entropy is like tolerating a toothache: something you'd really like to fix, but not quite bothersome enough to deal with. more…

urllib3

Today I learned about the urllib3 module. The biggest feature (from my point of view) is that this one can properly validate SSL sessions. more…

New Blog Software

Just finished moving the web site to a new host, and switching (yet again!) to new blog software in the process. I wanted to get rid of CGIs on the new host, so I switched to a static blog-site generator named Pelican. more…

Zombie T-Shirts

Just wanted to say hi to Dave and mention his nerd t-shirt store at http://www.nerdkungfu.com . more…

emacs command of the day

C-x 4 c : clone-indirect-buffer-other-window more…

phishing training

I stopped by the bank this morning to make a deposit. While fussing with the ATM machine, I was listening to a nearby bank employee making a phone call. more…

projects

Must.. write.. more. I'm trying to get over the temptation to rewrite my blog software again (probably using Jekyll). My blog-yak-shaving process works like this: "Oh, here's an interesting idea, I should blog about it. But my blog software is kind of annoying, I should really rewrite it ...

more…

darcs-fast-export

So idnar just turned me on to darcs-fast-export, which can be used with git-fast-import to quickly convert a repository from darcs to git. I've been using Git more and more in the last few months, and I'm growing quite fond of it. Tahoe is managed in darcs, and ...

more…

Foolscap-0.4.2 released

I've released foolscap-0.4.2 .. download it from http://foolscap.lothar.com/trac . more…

moved blog to git

I just finished moving this weblog to be managed in a Git repository, using the scheme described in http://joemaller.com/2008/11/25/a-web-focused-git-workflow/ . Plus, I'm running the connection over Foolscap.. more on that in a moment if this update actually works..

more…

web updates

I finally updated the system that hosts http://buildbot.net and http://foolscap.lothar.com (a dedicated VM that just runs apache for CGIs, needed to make trac and mod_python work well). Upgrading it from edgy to anything newer was a hassle, because the "update-manager" package that I wanted to ...

more…

pastebinit

Another package that appeared in debian today: pastebinit, which is a command-line tool to upload bits of code to some of the various pastebin web servers out there (handy when you want to discuss some code over IRC and don't want to jam the whole thing into the channel ...

more…

Mutation Testing

I've often thought that it would be a great idea to test your test suite by randomly changing bits of code and seeing if the tests catch it. It turns out that other people feel the same way: I just saw a Ruby library named "Heckle" show up in ...

more…

Emacs Trick of the Day

There are a few million gems hidden inside emacs. The two that I ran into most recently are:

C-x r m, C-x r b, C-x r l : these create named bookmarks, each of which records the file that you're visiting and a position within that file. When I need ...

more…

Levenshtein Distance

A library just showed up in debian ("python-levenshtein") to measure the Levenshtein Distance between two strings: the minimum number of edits (inserts, changes, deletes) necessary to turn one string into another.

I've been thinking about ways to implement efficiently-edited large mutable files for Tahoe, and it seems like a ...

more…

sparkfun toys

I was thumbing through some of my old del.icio.us bookmarks today, and came across sparkfun electronics again. Man, their coolness doubles in size every six months. $25 for a half-inch square self-contained radio data link, serial interface that you can run with a microcontroller, 3V, built-in antenna. Wow ...

more…

trac spam

Oh happy day! The buildbot.net trac instance just recently got visited by the link spammers. They haven't caused any actual damage yet, just a user account created with advertising in the profile text, but I'm afraid it's only a matter of time before the bots descend ...

more…

foolscap.lothar.com

I just finished building a Trac instance for Foolscap, now online at http://foolscap.lothar.com/trac . It's got a (mercurial-based) code browser, tickets, and a wiki.

Setting it up required some twisted.web hacking, because my setup puts a twisted.web server out front, and reverse-proxies certain requests ...

more…

mercurial

Wow, so long since I updated this. Each time I remember that I do have a technical blog, and think to add something to it, I am tempted to start by rewriting the whole blog system in some brand new way that will make it easier to post to (and ...

more…

forgetfulness-based development

You're probably familiar with eXtreme Programming, and branch-based development, and agile development. But I've discovered that I've been using a new technique recently, that I call Forgetfulness-Based Development. The way it works is this: I come up with something insanely complicated, that takes me weeks to get ...

more…

PyCon2007, Buildbot

I just got back from PyCon. Highly inspirational as always, saw some fascinating projects and some thought-provoking keynotes. r0ml's talk in particular has me thinking about how to structure code as a narrative, trying to bring the world of human-to-human communication and the world of human-to-machine communication closer together ...

more…

Trac

I've been setting up a Trac instance for Buildbot, to make it easier for people other than me to publish advice and tips in a persistent and easily-searchable fashion, also to make the Buildbot web page a little bit less ugly. Trac is quite spiffy, and I've been ...

more…

utilities

/usr/bin/watch is a little utility that will erase the screen, run a command, sleep for a few seconds, then repeat. You can use it to follow files in /proc without continually re-typing the command.

This program has been around since 1991. How is it that I've been ...

more…

promise syntax

Zooko's in town, and already I feel 20% smarter. I roped him into a discussion about the Promise syntax I'm developing for Foolscap, and he suggested an alternative that has some good properties.

I'll illustrate with an example where promise-pipelining actually does you some good. (many of ...

more…

new microcontrollers

I've been playing with Phidgets recently, having a lot of fun. They're great for prototyping, but they would be too expensive to use for most of the production purposes I have in mind. I've been thinking that for gadgets I plan to make more than one of ...

more…

Promises

Aaaagh! Promises are hurting my brain.

I'm trying to figure out how to provide a useful subset of E's reference mechanics in newpb/foolscap. Specifically, one of the clever things that E does is to provide Promise Pipelining, a limited form of remote code execution, in which I ...

more…

newpb-0.0.2 released

I finally got some twisted time this weekend, so I fixed ticket #1999 and moved newpb out of the Twisted subdirectory entirely, renaming it to Foolscap in the process. I also released version 0.0.2, so there's a complete tarball ready to install and play with.

Having it ...

more…

antispam

I ran some stats on my spambuckets tonight, comparing which of my email addreses get a lot of spam now versus 6 months ago, and noticed a few addresses that had stopped getting spam altogether. This gives me hope that by making my 10-year-old primary address less harvestable, the 500-plus ...

more…

new kernel options

I'm in the process of upgrading my systems to linux-2.6.14, and noticed a couple of neat patches that made it into the kernel this time around.

One is that FUSE (http://fuse.sourceforge.net) has finally gotten in. One thing I'd like to use this for ...

more…

concurrency

Had a great chat with Donovan today, about newpb and E and secure python and concurrency management. It turns out we have some of the same ideas about interesting things to do with these kinds of tools. He pointed me at a language named Io that's doing some neat ...

more…

happy birthday!

% whois lothar.com
...
domain:         LOTHAR.COM
person:         Brian Warner
nic-hdl:        BW116-GANDI
address:        The Castle Lothar
...
reg_created:    1995-07-29 00:00:00

Ten years ago today, I registered my little personal domain, with a woman at best.com named Pandora, who was nicely amused by the "company name". In the intervening time ...

more…

hacking

The last few weeks have been mostly filled with hacking hacking. I'm neck-deep in the implementation phase of a big new set of features, and it's taking forever. But I think I'm finally past the hardest part, the design issues that remain to be solved are at ...

more…

Go Tools

I was talking with my brother-in-law about a gadget to make playing Go online a bit more like playing it in person. The feel of the board and the THWACK! as you plunk down stones adds a lovely touch to the game, but you don't get that when clicking ...

more…

Twist-E

Spent another great day down at HP, talking about implementing E and web-calculus concepts within Twisted and newpb. Tyler Close was kind enough to spend the entire afternoon with me, explaining how his web-calculus works and the design decisions behind it. I'm really excited about implenting this stuff in ...

more…

books

I started in on Alastair Reynolds' Century Rain last night, got about halfway through before I finally succumbed to sleep. It's a good read: finally he gets to have at least a few chapters that don't involve pervasing nanotechnology or uploaded personality constructs or galaxy-spanning machine intelligences.

I ...

more…

and a calendar too

Hey, that wasn't too bad. I also added some CSS to make everything a tiny bit less ugly.

Now all I need is auto-completion on the category elisp..

more…

adding subcategories

I think I've gotten my elisp code to handle pyblosxom categories now. pyblosxom was easy, but I have to add the glue to let you choose a category. Unfortunately creating new categories requires manual work (registering the CVS directory).

Next step: find a pyblosxom plugin to create that spiffy ...

more…

great week

Man, what a great week. I spent a couple of days working with Donovan at his office on a couple of issues: making py.test capable of running Twisted test cases, improving LivePage event notification, and setting up a BuildBot for their in-house test suite.

Thursday night was the BayPIGgies ...

more…

SPF

I've been trying to decide whether to publish an SPF record for lothar.com or not. The last few days have seen an absolute deluge of spam from some german bastards, much of which is being forged in my name. The only real solution is, of course, to sign ...

more…

iButtons

I was talking with Pavel (aka PenguinOfDoom, on #twisted) last week about iButtons, and mentioned the JavaButton I picked up years ago that I haven't really managed to do anything with yet. That prompted me to poke around the web site (was dalsemi.com, since bought by http://www ...

more…

sparklines

My friend Drew just sent this one along:

http://bitworking.org/news/Sparklines_in_data_URIs_in_Python

I'm pondering things I might do with this. I've been using Data: URIs for one of my projects, they're pretty handy and both Firefox and Safari are more than happy to take ridiculously large ...

more…

pyblosxom-noindex

After some amount of perseverance, I finally figured out how to make pyblosxom insert "noindex" meta tags in the top-level index page. This was the last barrier keeping me from linking this blog to the main site, since I didn't want Google indexing a page that's going to ...

more…

buildbot versus windows

I just spent several hours getting a reasonable python environment working under Windows, something I had hoped to never have a need for. The Buildbot is having some.. disagreements.. with Windows, and it became clear that being able to reproduce the problem locally was the only sane way to fix ...

more…

buildbot hacking

I'm pushing to get a new BuildBot release out on monday, so the last few days have been a flurry of commits (and the weekend will probably be the same). I was very pleased to hear that the Boost crew have implemented a Buildbot to run their (very large ...

more…

twisted talk

So I think the talk went really well. I spoke for about an hour before the room was needed for another meeting, to about 10 or 15 OSAF developers. I managed to cover the reactor, Protocols, Factories, building higher-level protocols, Failures, Deferreds, reactor.run() vs twistd -y vs mktap/twistd ...

more…

OSAF Twisted talk

This is a rough outline of the talk I'll be giving at the OSAF tomorrow.

definition of Twisted, resources:
 http://www.twistedmatrix.com
  svn://svn.twistedmatrix.com/svn/Twisted/trunk
  http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
  http://twistedmatrix.com/bugs/
  http://twistedmatrix.com/buildbot/
 #twisted, #twisted.web on ...
more…

emacs

I set up a few tools to post blog entries from emacs. All entries are kept in CVS, and the whole tree is rsync'ed over to the web server. The elisp which actually publishes the entry looks like this:

(defvar pyblosxom-entry-dir "~/stuff/Projects/WebLog/entries")

;; adapted from http://wiki ...
more…

blog startup

I've been trying to get my project notes online for years now, and I finally realized that I need to start smaller. After a week of intermittent activity, I finally got PyBlosxom set up and behaving fairly well.

In the process, I discovered that the CGI specification doesn't ...

more…