Bob Ippolito and I hacked out a lot of the Python Eggs implementation today. (My new shorthand for explaining them is “An egg is to a Python as a jar is to Java”.) I wrote a “bdist_egg” command for setuptools, which is a Python sandbox project where I prototype potential new distutils features. (Two of my previous setuptools features made it into Python 2.4; I’m hoping eggs will be a no-brainer for inclusion in 2.5, especially since the bdist_egg command doesn’t really depend on anything else in setuptools.)
Anyway, if you want to play with it, you’ll need to check out python/nondist/sandbox/setuptools, and change your setup.py to import the “setup” function from setuptools instead of from distutils. Then, you can use “setup.py bdist_egg” or “setup.py bdist –formats=egg” or any of the other ways you can specify that same thing.
The resulting .egg file can be added to PYTHONPATH or sys.path, as long as it contains only pure Python modules, or if you have the “pkg_resources” runtime installed. Bob has been working on “pkg_resources”, but due to various hitches (including a weird bug in Python’s zipimport.c), it doesn’t actually work yet. But the idea is that .egg files, unlike normal Python zipimport files, can include C extensions as well as pure Python. The “bdist_egg” command automatically generates pure Python stub modules that request extraction of the extensions when the stub is loaded by the normal zipimport machinery. Extensions (and any other contained files that absolutely have to be “real” files) are extracted to a (configurable) cache directory.
Which reminds me. Remember a few paragraphs ago when I mentioned a couple of setuptools features that made it into Python 2.4? Well, one of those features is a way to specify “package data” files, which are data files that get installed within a package’s target directory. “bdist_egg” supports this feature by packing those data files into the egg, where they can be accessed via the standard PEP 302 “loader.get_data()” facility. To simplify your use of this, the “pkg_resources” module offers a resource access API that doesn’t care whether the data you’re accessing is in a “real” directory, an egg, or a regular zipfile. So, we’ll be advocating the addition of pkg_resources to Python 2.5, at least once it’s working and stable. In the meantime, we’ll probably add an option to the bdist_egg command to bundle pkg_resources inside generated eggs, to ensure that it’s available when the egg is used.
There’s still a fairly long list of features we want to implement in the runtime, but I’ll be pretty happy once we get C extensions and “namespace packages” working right, because that’ll give us the same zero-install convenience as Java’s “jar” files.
Just FYI, the Chicken Scheme compiler also uses the term “egg file” for its module distribution format.
http://www.call-with-current-continuation.org/
Well, I’ve rewritten zipimport in Python (pyzipimport.py), and it passes the Python zipimport test suite.
However, it should have the same bug as the Python implementation as-is. I need to figure out how to write the test for the meta_path problem, should be easy to fix the Python one after that. Do you happen to have a broken egg lying around?
actually, as an implementation detail of how I wrote it, pyzipimport *does* actually work for the trivial breakage we found with the setuptools egg (I remembered how to reproduce).
It still doesn’t respect __path__ yet, though.