Using app.Application
Motivation
Calling reactor methods (like .listenTCP
and
.run
) directly, as in the examples in Writing Servers, is a good way to immediately
demonstrate the use of Factories and Protocols. But you would ask for more
from a fully-fledged, easy-to-run, easy-to-configure Internet Server (with
capital I and S). To be precise, your users (defined as someone who wants to
install your server without knowing all the details of how it works) will
ask for more from it. Twisted provides this for you.
What more could we want from our little test program? Well:
-
configuration arguments:
suppose your QOTD server behaves a bit more like the normal port 17 server and pulls a random line from /usr/share/fortunes. Your QOTDFactory() might take a filename to indicate where the QOTD protocols should pull these lines. It would be nice if the person installing your quote server didn't have to modify any Python code to change where this file should be found.
Likewise, what if they want it to listen on some other port? That shouldn't require editing the code.
-
starting/stopping and persistence:
If your protocol demands that you keep some state from one invocation of the server to the next, you'll need to save some information before the server shuts down, and to restore it again when you start back up.
Suppose your protocol's purpose in life is to generate one-time keys, and that people can connect to it to retrieve a single-use key. (Don't ask me why they might want to do this. Security is such a weird big thing that chances are somebody out there will want to do something that's probably pretty dumb when you think about it carefully). The important thing is that you never give out the same key twice. So you have to remember a sequence number, and each time you give out a key, you bump up the number. Before you shut down, you save the number to a file somewhere; at start up, if the file exists you read the number from it, if it doesn't exist, you start at 0. (an example is included below)
This kind of persistent data is a common need, and many kinds of servers require it. Hence Twisted provides an easy way to record and reload this data.
This functionality is provided by the Application class (defined in twisted/internet/app.py). You create an Application with a constructor like any other object. Then you tell the app to listen to ports (just like you told the reactor to in the previous example), providing a Factory on each one. The difference is that the App won't starting listening on those ports right away, but will wait until it starts to run.
When you're done setting up the ports, you have two options: you can start running the app immediately, by calling the .run() method, or you can save the Application out to a file by calling the .save() method. The saved application can then be started later by using the twistd utility.
Example Application
Here is a short example of the first option, running the server immediately. This example uses the pre-defined Daytime protocol, which simply sends the current time to each client.:
#! /usr/bin/python import twisted.internet.app from twisted.protocols.wire import Daytime from twisted.internet.protocol import Factory app = twisted.internet.app.Application("daytimer") f = Factory() f.protocol = Daytime app.listenTCP(8813, f) app.run()
This program will start listening to port 8813 in the app.run() call, and won't return from that call until the server is terminated (probably when you send it SIGINT).
To use the second option and launch the server later, just use .save() instead of .run(). The .save() method takes a base name for the generated .tap file:
...
app.listenTCP(8813, f)
app.save("start")
When you run this program, it will create a file called daytime-start.tap, and then exit. (The name is obtained by combining the application name with the argument to .save()). To start the server from the "freeze-dried" .tap file, use twistd:
% ./app2.py Saving daytimer application to daytimer-start.tap... Saved. % twistd -f daytimer-start.tap % tail twistd.log 30/09/2002 01:38 [-] Log opened. 30/09/2002 01:38 [-] twistd 0.99.2 (/usr/bin/python2.2 2.2.1) starting up 30/09/2002 01:38 [-] license user: Nobody <> 30/09/2002 01:38 [-] organization: No Organization 30/09/2002 01:38 [-] reactor class: twisted.internet.default.SelectReactor 30/09/2002 01:38 [-] Loading daytimer-start.tap... 30/09/2002 01:38 [-] Loaded. 30/09/2002 01:38 [*daytimer*] twisted.internet.protocol.Factory starting on 8813 30/09/2002 01:38 [*daytimer*] Starting factory <twisted.internet.protocol.Factory instance at 0x81ac9fc> %
That will "thaw out" the .tap file, create the Application, and then run it just as if you'd invoked app.run() yourself. It forks the new server off into the background (so twistd itself completes instead of waiting for the server to die), writes the server's process ID to a file called "twistd.pid", and directs all the server's stdout messages to a file called "twistd.log" (these file names can be changed by appropriate arguments to twistd: see twistd -h for a list).
When you try this example, be aware that twistd returns right away, but it takes a second or two for the server to actually start. The twistd.pid file won't be created until it does. Wait a moment before doing ls or netstat, or you'll think that the server failed to start. If it persists in failing, look in twistd.log for details. Remember that trying to bind to a reserved port will fail unless you're root, and the exception will be listed at the end of the log file.
To kill the server, just do:
% kill `cat twistd.pid`
When the server is shut down, you'll notice that it creates a file called "daytimer-shutdown.tap" in the directory it was run from (again, the name is derived from the application name and the word "shutdown"). This .tap file is just like the "daytimer-start.tap" created by your original setup program, except that it represents the state of the Application object as it existed just before shutdown, rather than when it was freshly created by your code.
Also note that the twistd.pid file is automatically deleted when the application shuts down.
Saving State Across Sessions: Adding Persistent Data
You can add persistent data (like that sequence number described above) to the protocol Factory object, and it will get saved in the -shutdown.tap file. Then, if you restart the server with twistd -f daytimer-shutdown.tap, the new server will get the data saved by the old server, and it can pick up where the old one left off, as if the server had been running continuously the whole time.
To take advantage of this, simply add the attributes you want to the Factory, or to your subclass of Service (see the docs on Perspective Broker for details about Services). When the application terminates, it simply pickles up the whole Application (and everything it references, including Factories and Services). Any attributes or objects you have added will be saved and later restored.
Here is an example:
#! /usr/bin/python from twisted.internet.protocol import Protocol, Factory class OneTimeKey(Protocol): def connectionMade(self): key = self.factory.nextkey print "giving key", key self.factory.nextkey += 1 self.transport.write("%d\n" % key) self.transport.loseConnection() def main(): # namespaces are weird. See the comment in doc/examples/echoserv.py import app1 from twisted.internet.app import Application f = Factory() f.protocol = app1.OneTimeKey f.nextkey = 0 app = Application("otk") app.listenTCP(8123, f) app.save("start") if __name__ == '__main__': main()
To demonstrate this, do the following:
% ./app3.py Saving otk application to otk-start.tap... Saved. % twistd -f otk-start.tap % % nc localhost 8123 0 % nc localhost 8123 1 % nc localhost 8123 2 %
Note that the stdout of the process is being directed into the log file, contained in twistd.log. Now stop the server, verify that it is no longer running, then restart it from the saved-at-shutdown .tap file:
% kill `cat twistd.pid ` % nc localhost 8123 localhost [127.0.0.1] 8123 (?) : Connection refused % twistd -f otk-shutdown.tap % nc localhost 8123 3 %
Notice how the saved .nextkey attribute was restored, and the application picks up where it left off.
Configuration arguments
To do this right, you'll want to follow the sequence described by the "writing plugins" document. Instead of writing a short program that creates a .tap file (by creating an Application, doing various .listenTCPs on it, then calling .save), you will write a subroutine called updateApplication(). This subroutine should take a bunch of config arguments (using the usage.Options class described in the plugins document) and use them to create Factories and feed them to .listenTCP on an existing Application instance.
With that in place, and a few files to register this new server you've created, a utility program called 'mktap' can relieve you of the business of gathering user arguments and creating the app instance. mktap can use the Options subclass you define in your build-a-tap class to figure out what arguments are legal (--port taking a number, --quotes taking a filename, etc), provide --help with a list of valid arguments, and parse everything the user passes in argv[]. It creates the Application, then passes the app and the parsed options to your updateApplication() method, where you do the server-specific creation of a Factory and the various listenTCP calls. Then mktap saves out the .tap file, ready for starting by twistd.
The end result is that installing your new server is simplified to the following steps:
- Unpack your server module (including the classes and plugin glue files) into somewhere on your PYTHONPATH, perhaps /usr/local/lib/python.
- Run the standard mktap program, giving it the name of your module and whatever configuration arguments it requires. Watch it create a .tap file.
- Use twistd to start the server contained in the .tap file.
Pretty easy. At least your users will think so.
And, once your application is defined by the .tap file, there are other tools that can be used to configure it. coil is one that provides a nice web-based UI to change the various parameters. tap2deb is a tool that creates installable Debian .deb packages from your .tap file, making installation even easier.
The Application object has some other features designed to solve common server needs:
- logging is controlled, through the log.Logger class
- delayed events can be scheduled, with the .addDelayed() method
- the process can switch to a different uid/gid after binding reserved ports
- styles.Versioned allows old saved copies of an object to be upgraded when new versions of the class are available
- Applications have Authorizers, used to authenticate client connections
- Applications have Services, which can be accessed by PB clients
$Id: application.html,v 1.10 2002/10/04 05:16:19 warner Exp $
Brian Warner <warner@lothar.com> Last modified: Thu Oct 3 22:13:16 PDT 2002