Thursday, February 04, 2010

Don't use CGIHandler on Google App Engine

Update (March 12th): Guido says this problem only happens in the AppEngine SDK, and doesn't happen in production.  "In production, wsgiref.handlers is imported in an earlier stage,  before the request-specific environment is set. So the scope of the problem is much smaller."  Yay!

It seems that an early documentation example recommended that people use wsgiref.handlers.CGIHandler to run WSGI apps on Google App Engine, instead of the correctly-functioning google.appengine.ext.webapp.util.run_wsgi_app() function.

If you are doing this in your application or your web framework, you have a potentially-exploitable security hole and you should fix it at once.  (Or maybe not; see Guido's comment.)

The specific problem is that one of CGIHandler's base classes caches a copy of os.environ, for non-CGI use cases, and this makes it possible for certain CGI variables to "leak" from the request that started the process, into every subsequent request.

Of course, CGIHandler was never intended to be capable of handling long running processes like GAE, because CGI is not a long-running process.  The idea is that if you have a new kind of long-running process, you subclass BaseCGIHandler for your specific use case.

See, in a "traditional" long-running web app protocol (like FastCGI), process startup is distinct from request handling.  Even if a FastCGI app is started because there's a request ready for processing, there is still a separation between application initialization and the actual request processing.  (And wsgiref tries to cache the "startup" os.environ, separate from the "request" os.environ.)

App Engine, however, jams these two phases together, such that the "main" script is being re-run for each request, so there's no distinction between "startup" and "request".  This makes things convenient for people used to a CGI environment, but brings up problems for the CGIHandler, which expects that it will only be used once per process invocation, and so inherits a cached version of os.environ that also contains request content.

The fix is straightforward: switch from using wsgiref.handlers.CGIHandler to google.appengine.ext.webapp.util.run_wsgi_app().

However, if for some reason you can't do that, a quick monkeypatch fix is to add this line:

CGIHandler.os_environ = {}

somewhere in your code before the first use of CGIHandler.

It is possible that Google has already implemented the patch I provided them to fix this, but if so, the bug opened for App Engine is still open, more than two months later, and some of the documentation is still recommending CGIHandler.  Don't know whether that means it's fixed and the docs are okay, or that it's unfixed and they're still recommending people use it.

Either way, though, recommending CGIHandler for use in the GAE environment was never a good idea, since GAE is not really CGI.  If it ain't CGI, don't use CGIHandler.  Subclass BaseCGIHandler instead, and make a GAEHandler or AWSHandler or whatever, and take advantage of the branding opportunity provided thereby.  ;-)