OSAF Twisted talk
This is a rough outline of the talk I'll be giving at the OSAF tomorrow.
definition of Twisted, resources: http://www.twistedmatrix.com svn://svn.twistedmatrix.com/svn/Twisted/trunk http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python http://twistedmatrix.com/bugs/ http://twistedmatrix.com/buildbot/ #twisted, #twisted.web on freenode relationship of subprojects, dependencies: core, names, mail, web, words, conch, trial zope.interface, python2.2 optional: pyopenssl, db stuff directory overview: twisted.python: usage.Options, Failure, log twisted.internet: reactors, base classes for Protocol+Factory, Deferred twisted.protocols: simple protocols: finger, socks, telnet subproject directories doc/*/howto doc/core/howto/tutorial/listings/finter/*.py motivation: simple client simple server not-so-simple server client+server need for a generalized solution threads, processes, event loop event loop: asyncore reactor picture: reactor with select() call, sockets in .readers/.writers sockets have .doRead, .doWrite, are scheduled with .addReader/etc timers different kinds of reactors, using other event loops: gtk, kqueue picture: Protocol with Transports, reactor Protocol: connectionMade, dataReceived, connectionLost, transport.write how do those Protocols get created? reactor.listenTCP(port, factory) picture (server): Protocols, Factory listening socket (Port) points to Factory, creates new Protocols Factory gets startFactory, stopFactory, buildProtocol Protocols generally have .factory reactor.connectTCP(host, port, factory) picture (client): Factory gets startedConnecting, clientConnectionFailed, clientConnectionLost as well as startFactory, stopFactory, buildProtocol Connector is responsible for getting a connection to host+port+factory possibly multiple times, for ReconnectingClientFactory skip over Connector stuff writing Protocols, using existing ones picture: t.p.finger.Finger overridable methods for getUser, getDomain, forwardQuery subclass, override method make a Factory which instantiates your new subclass attach to listenTCP Protocols are used for both clients and servers state machine return one-shot results with Deferreds return multi-shot results by overriding methods larger protocols have more complex setup names: protocol parses the query, hands to factory factory does self.handleQuery, asks self.resolver, calls self.sendReply # good example of API, use of deferred: t.n.server.py:120, dns.py:1050 web: basic HTTP protocol creates Requests, then does req.process twisted.web.site implements a Resource tree picture(web): root, getChild(), isLeaf, render(req) specialized subclasses provide CGI processing, static.File, distrib imap: involves cred, Mailbox objects, Message objects top-level invocation: __main__, reactor.run() connectTCP, listenTCP or, creating an Application, then using twistd motivation: daemonization, logging, setuid/chroot, reactor, profiling think /etc/init.d picture: trees of Service/MultiService objects each gets startService, stopService t.a.internet.TCPServer(port, factory), TCPClient twistd -y foo.tac, script which creates an Application object sidebar: python as a configuration language serialize the Application, then launch it again later: twistd -f foo.tap shortcuts for common applications: mktap mktap plugins: Options, makeService(), register with plugins.tml threads: nothing here needs threads where are they useful? wrapping blocking APIs: adbapi in particular integrating with other code threadpool: run a function in a thread, tell me when it is done t.p.log: log.msg(msg, msg) emits a log log.err() emits the current exception log.err(f) emits a Failure object log output goes to an observer running from twistd: goes to twistd.log, or syslog running from __main__: log messages are discarded log.startLogging() Failure: encapsulates a python exception can be serialized, printed, queried about what caused it Failure() inside an except: block wraps the current exception Deferred: callback management use web.client.getPage as an example synchronous style: a=foo() b=bar(a) baz(b) asynchronous style: d=foo(); d.addCallback(bar) d.addCallback(baz) callback vs errback, ladder diagram fire-before-addCallback is safe callbacks can return Deferreds: sub-ladders usage.Options: create subclass, attributes indicate valid options optFlags, optParameters, subCommands define opt_foo(self,str) to implement --foo=str methods can customize processing further parseArgs, postOptions str() provides usage message Options implements the dict interface, opts['foo'], opts['v'] usually invoked with opts.parseOptions(), which grabs sys.argv why? mktap plugins use the 'Options' class from the plugin to parse argv lore: turn .xhtml into .html (or .latex, others) inline listings, pretty-print python code links to epydoc-generated API docs pb: translucent RPC f=pb.PBServerFactory(root); reactor.listenTCP(port, f) cf=pb.PBClientFactory(); reactor.connectTCP(host, port, cf) d=cf.getRootObject(); d.addCallback(dostuff) ref.callRemote("method", args) def remote_method(self, args) cred: howto is really good avatar, portal, realm, credentials, checker, mind portal has a set of checkers checker gets credentials, decides if they're ok, provides an avatarID realm gets avatarID and desired interfaces, returns an avatar protocol gets back the avatar, does stuff with it interfaces: PEP245-style twisted/python/components.py zope.interface, tiny portion of Zope3 many APIs want "object that can be adapted to IFoo" rather than an instance of a specific class some systems use it extensively: nevow's 'context': IRequest,ISession,ISite