Well, we’re about as prepared as we’re going to get. Nothing to do but wait, and take advantage of the simple amenities like TV, air conditioning, and internet service while they last. I’ve been doing some OSAF work, and answering emails generally.
It looks like the storm isn’t slowing down, which means it’ll get here sooner but be weaker. The NHC wind forecast from 11am shows only about a 50% chance it’ll be a hurricane this evening, and I believe that was when they were still expecting it to slow down and be offshore longer.
(Hurricanes, in case you didn’t know, are kind of a solar-powered steam engine. They’re powered by warm water, so being on land hurts them, although obviously not as much as it hurts us!)
It’s funny how the mind sometimes doesn’t really start working until you take “your” mind off of things. I repeatedly had this experience yesterday. For example, I was out in the garage looking for the power cord for my portable mini-refridgerator, as Leslie wanted to use it in her office at the store. (This was before we knew about the hurricane situation.) Anyway, I was looking high and low for that darn thing. Every now and then, I would try to do my Jedi mind trick, but it just wasn’t working. (In fact, it hasn’t worked that often since the first time I discovered it.) Finally, I took a deep breath and relaxed, emptying my mind as completely as I could, and sort of “let myself be”.
At that precise moment, my eyes fell upon a box that was partially hidden under a pile of wood. So I opened the box, and there were the cables that I’d spent the last 15 minutes tearing up the place for. I get the impression that when you’re sitting there in your conscious mind going, “Do the trick, find me the cables, yeah, do it now,” the subconscious mind can’t hear itself think.
Actually, I think it’s more like this… when you focus consciously on a thing, the metadata you’re tagging it with is more like “focus on this”, not “do this”. And those are not really the same thing! To “just act”, you need to quiet and empty the conscious mind a bit, to sort of take the consciousness filter offline. While the conscious mind is “in the loop”, the system’s bandwidth is more constrained. I suspect that one of the most valuable spinoffs of “bompu” (secular) Zen is practice at clearing the conscious mind. Maybe hypnosis works by letting the hypnotist bypass that nosy consciousness filter and have a chat directly with the subconscious, who knows?
Anyway, I had the same experience with a bunch of other things yesterday, just not so much in the physical looking-for-stuff sense. For the last few days I’d been chewing on the problem of designing a configuration syntax for WSGI deployment, trying to get something that in the simplest case could specify an application to find, download, and deploy with maybe one line of configuration text, something like “someapp from SomeProject” as a minimal “hello world” sort of configuration for some WSGI application.
I’d been bouncing some ideas off of Ian Bicking on the Web-SIG list, and made some progress, but the whole thing still seemed too complicated to me, and there were lots of annoying little parsing ambiguities in the grammar I had come up with. The whole thing also seemed way too WSGI-specific in some ways, even though the syntax seemed to have potential for doing other kinds of configuration and component assembly, from logger configuration to GUI widget hiearchies, or to create aXML-free ZCML or schema-free ZConfig variants.
So, having spent many a spare hour struggling consciously with the design, I of course solved the essential problem yesterday while my conscious mind was distracted by worrying about the hurricane. The light that came on in my head as I was getting in the car to drive to the grocery store was blinding, so much so that I was tempted to go back inside and try out the idea. But luckily, sanity prevailed. 🙂
I’ve dubbed the idea SCALE, which stands for Syntax for Configuration and Language Extensions. Thus, files created using it can be thought of as “Python scales”, which is a nice double entendre suggesting both the skin of a snake, and the idea that Python is a scalable language. And if we apply it usefully in the web field, we can perhaps say that the alternative to Ruby On Rails is “Python that Scales”. 😉
Seriously, though, the concept is more or less a user-defined statement facility, implemented via runtime execution of “scope” objects (similar to PEP 343, but different). There are some very small similarties to Logix, except that SCALE is actually for the most part a subset of Python rather than a superset. At a lower level of the SCALE system, it’ll be possible to have scopes in which the language isn’t actually Python any more, but that will be a non-trivial thing to implement, which should help to keep most people from doing it. 🙂
SCALE’s main purpose is to make it easy to textually specify some kind of hierarchical object structure, especially ones that may involve cross-references or need to be able to refer to an arbitrarily large assortment of potential object types. Thus, I don’t see a point to enabling widespread mini-language proliferation of the sort that Logix supports. While Logix turns all Python statements into expressions, SCALE simply does away with non-expression statements altogether, but allows any expression to have a nested block beneath it. There are a couple of other little twists on the basic syntax for dealing with assignment and context, but basically, SCALE is Python assignments or expressions, with the ability to have an indented block under a given assignment or expression. This means that creating nested components and their configurations are easy. Each block has its own scope, nested lexically as in Python functions.
The execution model is rather like SAX parsing for XML; each scope is an object that receives events describing the expressions and
[3 hour hiatus as the power goes out; my UPS kept this PC going long enough for me to “Save as Draft” to blogger’s servers.]
Wow, I’m absolutely amazed that our power is back already, since when I called FPL they gave me a spiel (the same as they’re repeating almost hourly on every TV and radio station) about how, once the winds die down, they’ll go out and assess things, and then 24 hours later tell everyone when they might get power back. So, the fact that it’s back on (at least for now) 3 hours after it went out is very nice. During the intervening time, I thought of lots of interesting things to talk about in this post, none of which I now remember. Oh well. I’m going to go ahead and publish this now, before any chance of the power going back off again. We’re still in a zone with a high probability of tropical storm force winds. The counterbalance is that we’re very near to both a fire station and an FPL regional staging area for hurricane response, which means we’re frequently among the first to get power restored during these things. Still, the lights are flickering every now and then as I type. I’d like to think we’re past the worst of it, but the hurricane’s stronger eastern side is still coming on shore.
Will this address class statement abuse, for those of us who find it the only palatable way (at the moment) for defining deep structure?
A while back I speculated on a declarative extension; what you describe sounds vaguely similar. So I think it could be quite useful.
Yeah, it’s like that, except that the entire base SCALE syntax is something like:
stmt: testlist (‘=’ testlist)* (“from” test)? ((“:” NEWLINE INDENT stmt+ DEDENT) | NEWLINE)
However, any time you have a nested code block, you can switch to whatever syntax you want, as long as it’s parsed in terms of Python logical lines and blocks (i.e., matching parens, line continuations, indent/dedent). This is different from Logix, which actually has non-Python syntax rules for that sort of thing.
When using the base syntax, the expressions are evaluated in a namespace determined by the “scope” object, at a time of the scope object’s choosing. (i.e., the scope object is given closures that eval the expressions, given a namespace.)
The expressions themselves do not create scope objects, unless the currently-executing scope object wants them to. In effect, a scope class totally defines the language syntax and semantics for a given block. But the base syntax of Python assignments with an optional “from” clause is flexible enough that syntax variations will be rare.
Since scope objects receive SAX-style events containing code, they are free to determine whether or when to execute a given piece of code, so the semantics can be very declarative. I think we should probably be able to implement a WSGI deployment language with just one Scope subclass, and one special syntax (for the “from” clauses).
Although actually I think the special “from” syntax will be part of the SCALE-defined core syntax, because being able to use something “from” an egg or URL is a generally useful configuration language feature. So, the WSGI scope will really just be an adapter to convert scope events into a tree of factory objects.
Anyway, if you wanted to implement your declarative syntax, you’d just need to create a scope that could recognize the syntax you wanted (using the SAX-like scope events), and adapt the events to the interface of your “declarative” objects. You’d also need to recognize regular Python syntax, of course, and I suppose I might want to include such a scope class in the library, since I imagine that for mini-languages like parser generators and mock object test scripts, being able to treat blocks as Python code would be handy. Certainly I’m going to need to have a way to turn a token stream back into something that Python’s regular parser can handle, even for expressions.
Anyway, all of this is a lot simpler than it sounds; I’ve already done a little experimentation, and really the toughest part is dealing with certain quirks of the tokenizer. And by “tough”, I simply mean that it’s tedious and annoying, not that it’s complicated. At the meta-language level, there are only three scope event methods (start_statement, end_statement, and end_scope), which is considerably simpler than SAX!
The core Scope class will then translate those events into slightly higher-level events that receive expression closures instead of token streams. You then subclass it to implement the high-level methods that determine the semantics of the received expressions.
Of course, executing a SCALE block will be slower than executing the equivalent Python, since Python has .pyc files and SCALE doesn’t. For mini-languages where a .pyc would be useful, though, it’s always possible to make a scope object that generates either bytecode or Python source that implement the desired semantics, and then save that. But I expect that will be a rather specialized set of use cases. For example, a parser generator would probably want to “compile” code in that way, but for configuration it’s probably not really useful.