We’ve Got Options

The peak.running.options module is finished in CVS, and there’s now a nice Options How-to available, courtesy of myself and the doctest module. As I was saying last Friday, “Doctest is cool!” It reminds me of when I first discovered literate programming, and I would write the documentation and code at the same time. I quit using literate programming tools once I moved to Python, partly because they added an extra compilation step, but also because they didn’t provide nearly as much benefit – Python is already quite readable!

But now, after a multi-year hiatus, I’m effectively back to literate programming, and boy am I glad. I really hate having to write documentation after the fact, especially documentation that then has to be kept up-to-date when the code changes, and that I might forget to update.

But I digress. The options package is cool, not just because it has a nice how-to, but also because it’s built on three brand new PEAK features, “dispatchers”, “map combiners”, and “attribute metadata” – all of which I “literately tested” in the last week. Dispatchers are basically the machinery underlying PyProtocols’ predicate-dispatch generic functions; I just refactored the dispatching part out into a base class that provides a mapping interface.

Map combiners are basically something you can plug into a dispatcher or generic function so it can figure out what to do when more than one rule applies. Specifically, they implement a strategy for merging keyed metadata – such as a set of command line options defined for a base class and its subclasses.

Finally, the attribute metadata subsystem lets you declare arbitrary “metadata” about attributes in a class. The semantics of the metadata are up to you; PEAK just makes sure your metadata implementation gets told about the declarations. This is a really big advance in extensibility for PEAK, because previously the various kinds of metadata (such as security permissions, parsing syntax, etc.) were all hard-wired into the various attribute descriptor implementations. The new system allows you to just say, “here’s some metadata for this attribute”, so it’s extensible to any new kind of metadata somebody can come up with. They just decide what semantics they want it to have, and they write a little “glue” snippet like this:

[binding.declareAttribute.when(AbstractOption)]

def _declare_option(classobj,attrname,option):
    OptionRegistry[(classobj,)] = tuple(
        [(optname,(attrname,option)) for optname in option.option_names]
    )

in order to receive notification from any PEAK API or descriptor that encounters their new kind of metadata. (Command-line options, in the case shown.) Generic functions make great framework extensibility hooks, so much so that I’m starting to think they’re more “Pythonic” than interfaces, for a great many purposes.

But anyway, the options framework basically consists of a bunch of Option objects that are usable as metadata, and the semantics defined for them is that the Options get registered in a dispatcher, keyed by target class, and merged by a map combiner. When you go to parse options or display help, it just looks up the target class in the dispatcher, which spits out a mapping with all the applicable option information.

Normally, using the optparse module alone, you don’t get option inheritance, and you have to type out the attribute names and actions. But here, because the attribute metadata system knows what class and attribute you specified the options on, you not only don’t have to type that stuff, but the option object doesn’t even have to know… which means it can be reused in multiple classes, under different attribute names.

Now, all I’ve got to do is go add nice options to all of the existing PEAK commands… but first, dinner!