It seems like those wily TurboGears folks are at it again, talking about actually using stuff I wrote (RuleDispatch, in this case). What is it with them, anyway? 🙂 Whatever it is, I hope they keep it up. In all fairness, Django actually started using setuptools before TurboGears did, but nobody uses as many eggs for as many projects as TurboGears does, with over 75,000 eggs served and counting. And, from an IRC log I stumbled across this evening, it sounds like the Django developers recently considered using RuleDispatch too, although it also sounds like they decided not to bring in the extra dependency.
Maybe I’m biased, but I think you should just bite the bullet, guys; TurboGears depends on nearly a dozen eggs in order to install, but that hasn’t stopped them from getting thousands of downloads. (The fact that everything but the bootstrap script gets downloaded automatically helps.) Plus, think of all those TurboGears users who now already have RuleDispatch installed, and could now “easy_install Django” to try your framework too, if you only added a download link to your PyPI project page!
But that’s okay, you can wait to try RuleDispatch later. 🙂 I’ve started some work on refactoring it so it can actually generate Python bytecode for generic functions on the fly, which basically means that it will end up being able to run at least as fast as hand-written Python in the limit case, and might make it possible to use Psyco (or some future PyPy JIT) to do further optimization. This is part of a general refactoring to separate the policy aspects of generic functions (i.e. what rules are in a given function and how should they be combined) from the dispatch implementation aspects (indexing, tree building, and actual execution).
There may be some minor API shakeups from all this, but probably not much. Mainly it should just make it possible to choose alternative implementations for a generic function’s execution, independently of its method combination strategy and any other special features. This should be particularly useful for creating incrementally-updated generic functions, which would be useful for doing stuff like what PyDispatcher does, or for content-based routing engines, enterprise messaging systems, and all that sort of thing. Mainly, though, it should just simplify some of the existing code, get rid of some cruft, and (I’m hoping) be faster than the current engine at call time, using less C code.
It’s all just a part of keeping up with the reputation I seem to be getting lately. In that IRC log I mentioned, after all, I saw this amusing comment:
rjwittams: I think eventually ‘Eby’ will be an adjective that means ‘fucking cool, but insane’. ‘That is a really eby trick!’
So, I guess I’d better head back to my magic workshop and get with the Eby-ing already. 🙂
Hey, I’m rjwittams 😉
Yeah, there are some bits where RuleDispatch would be really useful… for now we can get away with PyDispatcher/louie. At least it would be easy to switch in future.
I think when we come to redo security, it’ll be really attractive. Dependencies are really a hard sell when you currently don’t have any though…
I’ve been using RuleDispatch in some new code that parses and generates several different binary formats. What I’d like to see most is a reduced startup overhead.
I haven’t profiled it yet, but it definitely has a noticable import time.. that’s gotta be either setuptools or RuleDispatch, and I’m relatively sure it’s not setuptools.
Currently there are 25 generic functions, with 108 “when” statements…
Bob, I recently fixed an O(N**2) sys.path length issue with pkg_resources… so if you haven’t updated that in the last few days, it might actually *be* setuptools. The issue was that with 20 or so eggs on sys.path my PC would pause by as much as *six seconds* upon importing pkg_resources. The problem was a high number of calls to “normalize_path()”.
I kind of doubt your delay is due to your function definitions, because the worst-case generic function conditions will, on my PC, allow compiling and indexing at about 2500/conditions per second. You’ve got a lot less than that, but I don’t know how the speed compares on a Mac. My current refactoring plan calls for moving most of that overhead from when() time to “first call” time, though. I’m not 100% sure how that will play out though.
rjw: You guys already use setuptools; just add ‘install_requires=[…]’ and you’re golden. 🙂
A couple days ago, I finally got around to checking in use of jsonify, which uses RuleDispatch for choosing how to convert an object to JSON. Further, if you check out my latest devcasts, you’ll see that TurboGears uses RuleDispatch to map SQLObject columns to widgets:
http://www.turbogears.org/docs/devcasts/index.html
I do suspect we’ll see some use of it in security at some point.
All this reminds me that I need to ask you about how best to implement “override” behavior where TurboGears comes with some standard behavior but a user wants to replace some part of the standard behavior. (That’s worth an email message rather than a blog comment, I think.)
The way that peak.security does it is to have a “security context” object that the developer can control/access/override. Then, simply specifying an “and” condition between the to-be-overridden condition, and an isinstance() or other check on the context object, is sufficient to be “more specific” than the existing condition.
In TurboGears, you might just want to include the controller and/or model classes in the function parameters somewhere, since those are already controlled by the developer.
I’m having a problem with RuleDispatch, and I hope you can shed some light on it. Here’s the code:
import dispatch
@dispatch.generic()
def foo(x):
pass
@foo.when(dispatch.strategy.default)
def foo_default(x):
print x, “: default”
@foo.when(“6 < = x <= 8")
def foo_from_6_to_8(x):
print x, “: 6 < = x <= 8" if __name__ == “__main__”: for x in range(10):
foo(x)
This code produces this output:
0 : default
1 : default
2 : default
3 : default
4 : default
5 : default
6 : 6 < = x <= 8
7 : 6 < = x <= 8
8 : default
9 : default
The problem is with the value ‘8’. It should have tiggered the ‘6 <= x <= 8' rule. What happened?
What happened is I had a bug in the code that handles intersecting open inequalities. I’ve added a test for this now. Thanks for the bug report.
That was quick! How do I get the fixed code?
easy_install svn://svn.eby-sarna.com/svnroot/RuleDispatch
Nevermind, got it from your subversion repository.
My god… it seems like overnight, installing and keeping a python package up-to-date went from a nightmare to easy-peasy. Eggs, easy_install, setuptools. Nerdvana.