Saturday, January 29, 2005

The Prisoner's Dilemma

Politics-Oriented Software Development: "Also remember that someone who points out a problem early is a troublemaker; someone who fixes a problem at the last minute is a hero."

The Core Protocols (page 3): "When I believe I have a better idea than the currently prevailing idea, I will state it at once to any and all of those who can perfect, reject or implement it. Once I state an idea, I will maintain it until: i. I learn that it is not the best idea after all, ii. Someone else improves it, and states it anew, or iii. It is officially adopted or is decisively rejected."

I've spent nearly twenty years living by most of the Core Commitments, using operating practices rather similar to the Core Protocols. They weren't written down then, of course. Roger taught them to us by example and principle, and I shared them with others in the same way. (The McCarthy Tech folks have done a good job of figuring out how to define more purely behavioral rules than the rough practices Roger taught; but in truth the principles are more important than the procedures. Procedures can always be changed to better fit the situation, but principles cannot. The Core Protocols do look like a good way to learn the principles, though.)

While at Verio, however, I also had to deal with an organization that (outside of my own team) ran mostly according to the POSD rules. So I see both of the above documents as well worth reading, as they give expert advice within their respective spheres of influence -- cynicism, and idealism. The necessity at times to use one set of rules, however, should not be cause for us to doubt in the efficacy -- and indeed the sheer necessity -- of the other set. It should also not deter us from striving to change the rules in play, from one kind to the other.

I think that most successful open source projects run on rules that are much closer to the second set of rules than the first, though, and that this is a big reason why such projects operate far more efficiently than projects run under "traditional" structures (regardless of whether the source code is available to the public, which is irrelevant to the team's vision and productivity). A culture that values ideas and consensus accomodates everyone's use cases, or tells them why that case is out of scope up front. That which is not officially adopted, is decisively rejected. And that which is neither, is carried forward by the proponent until it is, or until they learn something. In all three outcomes, something of value is achieved.

By contrast, in a POSD process, all forces move towards individual outcomes that indirectly conspire to prevent any value from being created anywhere, even in the form of learning. (Unless you consider learning to be cynical a valuable skill!) In POSD, everyone focuses on themselves as individuals, so there is no hope of creating something larger than the individual.

Those who make the core commitments, on the other hand, dedicate themselves to creating something that is bigger than what any one of them can create, and willingly sacrifice a little personal ego and a lot of personal comfort zone in order to do so.

Until I read both of these documents, I had previously been puzzled by the conflict between my attempts to become more effective in one kind of culture, and my desire to continue in (and spread the influence of) the other kind of culture. Today, it seems obvious - you can't mix and match, within a given sphere of influence.

And yet, in a way, it's always a mix. The truth is that most of us aspire to live in one culture, but are afraid that they live in the other. And if you really do live in an everyone-for-themselves, dog-eat-dog environment, then you really do need to "look out for number one," and aspiring to something greater than yourself is a dangerous illusion. Yet, if everybody else around you is thinking in the exact same way, the greater tragedy is that we all could have done something different. But how could we have known?

For there to be any possibility of knowing, somebody has to actually stand up and say, "I believe in something, and I will not back down." The only way out of the Prisoner's Dilemna is for somebody to not defect, to not sell out their dream because they're afraid of being sold out first. While Richard Stallman and Mahatma Gandhi differ significantly in the scope of their achievements' impact on mankind, they do have one interesting thing in common: they were both willing to say, if this is to be, it begins with me. Somebody has to go first, and be willing to accept the consequences. To dream the impossible dream, because it's only impossible so long as everybody still thinks they're the only one who has that dream.

I have that dream, and I know that it's possible, because I've been lucky enough to live quite close to it at times, even if I didn't always appreciate it then. To me, it's always been just how things were supposed to be. It's never really struck home to me so dramatically before, that it's really not that common in the so-called "real world". So sometimes I get frustrated, thinking, "What's wrong with people?", or, "What's wrong with me?"

But there's nothing wrong with anybody. In the Prisoner's Dilemma, it's rational to defect if you think others will defect, and rational to co-operate if you think others will join you. People just differ on how likely they think each outcome is, and in whether they think the possible gain is worth the risk.