fbpx
dirtSimple.orgwhat stands in the way, becomes the way
Python Interfaces are not Java Interfaces

Python Interfaces are not Java Interfaces

My “Java is not Python, either” article seems to have raised a few hackles. In retrospect, I realize that the fault is my own. Although I myself said in the article that “frameworks like Zope, Twisted, and PEAK all have interfaces, but since they’re not part of the language or standard library, for most Python developers it’s as if they don’t exist,” I then proceeded to write about them as if everyone would know what I was talking about. Duh! So, let’s take a look at the subject that I should have covered first, about what interfaces are used for in Python today, and how their use differs from interface usage in Java.

First of all, in Python, interfaces are much more dynamic than their counterparts in Java. For example, Java doesn’t let you change your mind on the fly about what interface a class or an instance supports, but Python does. Java requires you to write your interface in the form of a class, while in Python you can use a class, an object, generate it on the fly, pretty much whatever you want.

In Java, if you don’t get the interface or implementation just so, your code doesn’t compile. In Python, as long as your code still “quacks like a duck”, it’s still a duck. (Unless you pass it to a function that needs to introspect or adapt the interface; more on this later.)

So, if you’re accustomed to Java interfaces, you may at this point be wondering, if Python interfaces are so darn optional and don’t do hardly any of the things Java interfaces do, then what the heck do we need them for?

I’m so glad you asked that question. (Okay, I’m so glad I put that question into your mouth. Whatever.) There are three things that interfaces are useful for in Python, in roughly descending order of importance:

  1. Documentation
  2. Adaptation
  3. Introspection (A very bad idea, in my opinion, but hey, consenting adults and all that.)

Now, the next thought that I’m going to pretend you’re having is, “Why do I need interfaces for documentation? Why not just write documentation?”. Well, that depends a lot on who and what the documentation is for.

If you’re writing a library or a script, all someone needs to know is how to use it. They’re going to be calling your code; you don’t call theirs. They aren’t modifying or extending your code, and if they do, they’re on their own. So if they break it, screw ‘em. They really don’t need the formality of interfaces.

But, if you’re creating a framework, the situation is different. A framework is a library that calls the framework user’s code. Now, somebody will be writing code that – in effect – is part of your library, and they need to know things about how their code will be called. With a library, it’s okay if the author or maintainers are the only ones who know the internal dependencies of their code. But with a framework, the users are effectively co-authors and co-maintainers. They need more concrete information.

For example, consider PEP 234, the iterator protocol. In the context of iteration, we may view Python itself as the “framework”, which will be calling the user’s code to perform iteration. Let’s look at what the iterator protocol looks like, as a pair of Python interfaces, using the currently popular idioms for Python interfaces:

class IIterable(Interface):
    """An object that can be looped over with 'for'"""

    def __iter__():
        """Return an IIterator for this iterable/container,
           or return self if this is an iterator"""

class IIterator(IIterable):
    """Iterator representing the current position of a 'for' loop"""

    def next():
        """Return the next item, or raise StopIteration
           if finished"""

If you compare with PEP 234, this is roughly the same amount of text as is used there to describe the methods expected of Python iterators. But, it has some advantages that the original text does not. For one thing, it’s closer in shape to an actual implementation; you can cut and paste from it in order to begin implementing your methods, rather than having to translate “takes no arguments” into a method signature. Textual documentation can refer to “IIterable” or “IIterator” objects, instead of talking about “protocol 1” and “protocol 2” (as PEP 234 does), or coming up with other ways to specify what’s being talked about. You can easily skim to the definition of a specific method. It’s clear that one kind of thing is a special case of the other. And so on.

Now, the point of this is not to say that PEP 234 should have had interfaces in it. That would be a silly thing to say. However, there were times when I was tempted to use interfaces in PEP 333, and then backed off because of a simple issue: by the time I got done explaining the conventions for specifying interfaces, it would have taken more writing than just putting up with using less precise (and concise) ways of specifying things. So, right now, it doesn’t make sense for PEPs to include interfaces, because they’re foreign to most of the intended audience. For any given PEP author to attempt it would kill the PEP, because the author would spend more time answering questions about interface definition than about the contents of the PEP.

Which reminds me, I didn’t explain those conventions, although most of them can be reverse-engineered from the example I gave. First off, common practice among the interface-using Python community is to name interfaces with a leading ‘I’. This drives some people up the wall, but most swear by it, because it makes it unambiguosly clear whether something is an interface. And for documentation and discussion, unambiguous clarity is a very good thing to have.

Second, by convention, the methods in an interface do not include ‘self’. They reflect the signature of the method from the caller’s point of view, and you do not pass ‘self’ to methods. This is something of a pain when copying and pasting to implement them, but then it’s also easier to cut and paste when you’re writing code that calls the methods. So, either way somebody’s got to either add or delete the ‘self’, and current convention is to make the implementer add ‘self’ when implementing.

Third, inheritance between interfaces is used to denote that the inheriting interface requires all of the same behavior that the base interface does, but may provide additional capabilities or loosen restrictions on its callers. For example, a method overridden in a subclass interface may take additional arguments, as long as they are optional. Thus, the call signature of the base interface is still usable with the derived interface. If an interface has any calling signatures that are incompatible with those of its base, the interface should not be derived from the base interface. Instead, the common methods should be factored out into a common base, or duplicated in a new interface.

Interfaces can also specify attributes that are expected to appear on an object. One easy way to do this is just to include something like:

anAttribute = property(doc="""Describe the attribute here""")

in the interface definition. Both PyProtocols and zope.interface also offer Attribute objects you can use instead.

Finally, functions or other callable objects are described using the __call__ method. For example, a quick sketch of a PEP 333 application object would look like this:

class IWSGIApplication(Interface):
    """A WSGI (PEP 333) "Application" object"""

    def __call__(environ, start_response):
        """Return an iterable of the application's output body

        The server will call the application with an IEnviron
        (the 'environ' argument) and an IHeaderOutput (the
        'start_response' parameter).  The application should
        first invoke 'start_response' to set the status and
        headers, then return an iterable.  This can also be
        implemented as a generator, in which case 'start_response'
        should be called before the first 'yield' statement
        is executed."""

This is a lot shorter than the detailed explanations in PEP 333 about callables that call callables, but it means the same thing, since this calling signature may be implemented by a class, function, method, or any callable object.

Now, while it’s not a substitute for an example, the interface above provides a compact overview of the concept of a WSGI “application object”. Also, notice how I can refer to other interfaces as I describe the parameters, which allows me to have absolute precision in the specification, but without having to immediately explain what those other concepts are. You get a general sense of them as some kind of environment and header output, but you also know you can skim ahead looking for ‘class IEnviron’ or ‘class IHeaderOutput’ if you want the details.

This is the sort of precision and flexibility in specifying that I missed dearly when writing PEP 333. But I really didn’t want to cloud issues that were often contentious to begin with by bringing interfaces into the mix. I especially didn’t want people to possibly turn their backs on the PEP just because “interfaces are bad and shouldn’t be in Python.”

But, when I talk about Python needing interfaces, this is what I’m mainly talking about: conventions for documentation. Especially documentation that supports specifications that require interoperability, like PEP 234 or PEP 333. Over time, I think we’ll see an increasing number of projects in Python that will need to be able to talk about what their interfaces are. It would certainly be nice if we could all talk about them in the same way.

(P.S. I didn’t forget about adaptation and introspection; I just don’t have time to cover them in today’s post. Look for a follow-up at a later date.)

Join the discussion
13 comments
  • Interesting. I haven’t yet gotten around to reading 333 yet, because it hasn’t been relevant to my coding. But when I heard about it, I just automatically assumed it would be presented as a set of (python) interfaces. I guess that shows which python hangouts I’ve been spending most of my time in the past couple years 🙂

  • “A framework is a library that calls the framework user’s code.”

    I think this is the most useful and concise definition of the difference between frameworks and libraries I’ve seen. I googled around a bit, and it seems to be original to you.

    Definitely one for the quote-file.

    – Michael Bernstein

  • But now you’re mixing implementation and documentation! More, this “documentation” (which I also question the value of, outside of the actual documentation) is in itself another protocol forced upon the programmer who uses the framework, for extending or using both. That it’s more flexible is of questionable value (is it more flexible or just more confusing to learn).

    I would also not buy the argument that a manifest Interface would help a programmer looking at the source code; or rather, it clears things up the way “foo(level:int, domain:set) -> str” does for a function. It’s good documentation, limited as it is, but it should stay documentation, or optional optimization.

    It is mixing implementation and documentation, and forcing users of the code to follow the same process of implementation, that has me continuing to doubt that this, in the end, is the best way.

  • I would agree that interfaces are immensely useful and that there should be a standard for expressing them in any OO language .. I’m just not sure how it can/should be done in Python ..

  • “””It is mixing implementation and documentation, and forcing users of the code to follow the same process of implementation, that has me continuing to doubt that this, in the end, is the best way.”””

    The best way of *what*?

  • “The best way of *what*?”

    The best way of expressing the constraints of an interface.

    I’m not sure this gets through, but I really do like the clarity of thought and procedure that a defined interface (or perhaps better: a specfication) represents. I’m all behind what you’re doing, and what it accomplishes, but not how you’re doing it.

    (I should really try to sign posts …)
    //Simon

  • Any chance you might give a talk on interfaces at Pycon? I’ve pondered it after some discussions with Barry Warsaw produced the (to me) blinding insight that “interfaces are for static type checking” (which is why I previously had a hard time thinking about them in Python), but I think Barry is too busy to do it with me, and more importantly it seems that you’ve given it much more thought. (I think the deadline is December 25 for submissions).

  • I don’t know yet if I’m going to PyCon, but I hope to know in the next week or so.

    Also, interfaces aren’t really for static type checking in Python, unless you think of documentation as static type checking. 🙂

  • I’m quite new python and am coming from the ‘java world’. When I first saw it I quite liked what I was presented but really missed the interfaces. Then I was told by the documentations (and by some blog entries) that this is because I don’t think pythonic enough.

    I haven’t worked with python too much (I only used jython in a project together with java) but I couldn’t get away without using interfaces. Now it’s good to see that I’m not a lone idiot. At least there are some others as well.

    My mains concern against python was tha lack of using at leats some form of interface definitions. And the main problem that a well practiced python user might not observ when only using his well known libraties is that interfaces serve as a FORMAL and COMPACT way of documentation.

    Why use plain english for something that can be expressed in an easier, more compact and more concrete way using python? I would disagree that you only need interfaces for frameworks and not for libraries. And the reason is that libraries DO call your code as you pass your objects to them. Or they call someone else’s code (you pass an object from a library as a parameter to a method from another library). Now how do you know if it has all the methods needed? Is it ‘file-like’ enough or not? 🙂

  • I think this is a great explanation, but what I take away from it is this :

    1) Interfaces are for documenting how two pieces of code interact.

    2) Documenting using code is easier than a text description because you can cut and paste.

    To which the obvious response from is : “well, wouldn’t good boilerplate / examples be EVEN BETTER for this”

    Because I still have to bend my mind a certain way (that I consider Javic, rather than Pythonic) when I try to read your spec.

    First I get confused by the two levels of inheritance.

    What’s this “Interface” which IIterable is inheriting from? Why is __iter__() defined in IIterable whereas next() is defined in IIterator.

    Particularly because the documentation actually keeps refering to all these newly defined terms : __iter__ in IIterable refurns an IIterator … aaargh! This is NOT easy to read and follow.

    What would be pretty straightforward is this :

    class DemoIterator() :

  • doh! Tab submitted the last before I finished.

    What would be straightforward to me is this :

    class DemoIterator :

    def __iter__(self) :
    “””returns the Iterator for this class, might return self if the class is it’s own iterator”””

    def next(self) :
    “””any iterator must support this methos. returns the next item from the iterator”””

  • Aarrgh! more typos as I try to rush the above in.

    To finish …

    Good example are a million times better as a way of explaining how things interact. If you don’t have them, any documentation is still a pain to read and interpret. And if you do have them, you probably don’t read the spec. anyway.

dirtSimple.org

Menu

Stay In Touch

Follow our feeds or subscribe to get new articles by email on these topics:

  • RSS
  • RSS
  • RSS

 

Get Unstuck, FAST

Cover photo of "A Minute To Unlimit You" by PJ Eby
Skip to toolbar