PEP 342 – Coroutines via Enhanced Generators. The name pretty much says it all. This is syntactical support for doing in a “forwards” way, what Twisted does in a “backwards” way, what Stackless Python does by reimplementing the Python interpreter, and Greenlets do by messing with the C stack in assembly. (And peak.events does with a magic function and some syntactic ugliness.)
The PEP was originally written by Guido, but after I implemented the important parts of his draft (and his other generator enhancements from PEP 343), he asked me to take co-authorship. Since then, I’ve fleshed out the “Motivation” and “Examples” sections to help show what these (proposed) new features can really do.
Happily, this proposal is no longer speculative as to whether it can be implemented; last weekend I put together a patch on Sourceforge that implements the functionality shown now in the PEP. So, the only real question is whether there’s any backlash on comp.lang.python or the Python-Dev list, which I don’t really expect. Hopefully, this or something like it will find its way into Python 2.5.
(And while you’re reading PEPs, it’s probably also worth taking a glance at PEP 343 too, if you haven’t already. It’s a nice way to encapsulate try-finally usage patterns as reusable objects.)
Firstly, having to have a blogger account in order to post a comment is quite a hassle to go through.
Secondly, I hope this goes in. I am tired of having to be concerned with the Stackless being practically abandoned, greenlets having strange garbage collection issues I don’t understand. Having a standard python based solution will be so much better.
Nice job.
Hi,
I’ve been working with this style of generator use in a fairly large project for a while now.
The implementation I’ve been using is around 80 lines of python source:
http://wiretooth.thaumagen.com/python/sloop/sloop-0.1.0/hub.py
In the above source ‘resume’ is analagous to send in pep 342 with the addition of the parameter identifying the sender.
Whats the prospect of having something like ‘send(value, from_=None)’ become the signature for send in pep 342 ?
It also supports any iterable as a send target.
I quite often need a generator-iterator that acts as a completion point for many, simpler, async things.
they tend to look something like:
me = thisgeneratoriterator
def aggregation(values):
….print “all values”, values
for iterable in asyncthings.keys():
….send(iterable, value, me)
values = []
while 1:
….try:
……..yield send(None) # sleep until I get next result.
……..value, sender = receive()
….except ThrownException, value:
……..pass
….values.append(value)
For control flow there are two cases I encounter a lot.
1. something further down the stack needs to wait on some intermediate result.
2. something up the stack needs to pass a ‘seed’ value down the stack.
I’ve been using this for a while to do this:
http://wiretooth.thaumagen.com/python/sloop/sloop-0.1.0/control.py
FWIW, PEP 342 and 343 have now been accepted! They will go in.
So this PEP would still require that
the user of a coroutine ‘know’ that
it is a coroutine and thus require
the use of ‘yield’, is that correct?
In other words, if you have
res = obj.my_method()
And you want to switch my_method
to use a coroutine, you will need
to change the call to:
res = yield obj.my_method()
so that the ‘trampoline’ can manage
the new call. Is that right?
“””So this PEP would still require that the user of a coroutine ‘know’ that it is a coroutine and thus require the use of ‘yield’, is that correct?”””
Yes.