The New Bug Economy
Published 07 December 08 07:41 PM | carlcs
I sure picked an interesting time to start learning more about economics. Everything I've read, and experienced since my first post a while ago has gone against my first tentative steps. The first book I read the contradicts classical economics was The Black Swan, which argues against the linear Gaussian assumptions behind much classical economic theory. Next came Predictably Irrational which shows how people are systematically irrational. Most people understand that we aren't all rational in the economic sense of the word, but we expect it to average out in a 'wisdom of the crowds' sort of way. "Predictably irrational" shows that the way in which we are irrational breaks economic models.
Finally I plowed through The Origin of Wealth. This book tries to reconstruct economic theory on the basis of evolutionary theory, network theory, and automata. And then finally, we had real life impose itself in the form of a rather severe economic crisis.
Seeing the direction economics is going reminds me of the history of physics. Starting in the 17th and 18th centuries we developed analytical tools that allowed us to solve exactly for many physical phenomena. They work well, but they require very artificial circumstances to be solvable. That's why physics students have to solve problems like describing the electrical field around a charged sphere with stripes or a semicircular waveguide. Of course these things don't really exist in real life but since they can be solved in terms of Bessel functions and other things they can be written in a homework assignment. It's easy enough to solve for the motions of a pendulum, but only if you assumed a small angle.
Then came chaos. It turns out that there are lots of things you can't solve for. Sure we knew this all along, but now we have computational tools to show the behaviors that emerge from this. And even many of the things you can solve for exactly are better off dealing with numerically. But we also had the broad realization that you can't know the initial conditions well enough to predict results with any specified accuracy.
Now computational and storage power are moving us away from all of these abstractions. You just simulate based on first principles. This is clearly the way that micro economics can be unified with macro economics and there are those that believe that the same applies to physics as well. I was talking to an astrophysicist friend of mine and he said that for some projects they no longer think of light in the classical sense at all but rather count and ray-trace each and every photon. The computational power is there, and it's actually easier and more accurate given how dim the objects are. The telescope becomes the business end of a high-energy accelerator at the edge of the universe.
What does this mean for policy, both for economic policy and and for planning complex projects like software?
The first stage of classical economics and 18th century physics corresponds to "laissez faire" free markets. When you hear an Ayn Rand follower like Greenspan talk about "irrational" investors, it's used in a pejorative sense. You can hear the echos of "Atlas Shrugged" and get the sense that all that's needed is for the market apply it's corrective power.
Once you realize that markets can get into a serious, non-linear state it becomes obvious that laissez faire is not the best solution. You institute damping functions in the form of regulations and fed action to prevent the markets from getting into this state. In software development you slow down progress through the use of check-in criteria, static code analysis, stabilization periods, and other overhead. But these are crude tools. You slow down the overall economy to minimize your risk on the downside. It also treats developers and individual actors as fungible, interchangeable actors.
Economics is using simulation to move to the next step, but clearly it's a long way away from policy. The closest thing we have in software development is agile methods. Ultimately I think what this all comes down to is better tools with extremely fine grained tracking and integration of everything you know about a project. Rather than trying to remember to integrate vacations and meeting over head, integrate the calendar system and account for actual working hours per developer. With this data you can estimate the code churn introduced by feature as well as the expected bug count per LOC churned. Given the back and forth between churn and bugs, you can even plot out a reasonable glide path.
The problem with most project planning (and economic models for that matter) is that there are too many externalities. You can be enormously precise in your estimate to develop a particular module, but you have only rough swags on the external factors that really make or break a project. The answer to those external factors is to make them internal, based on actual data, and predictive.
The funny thing is that pretty much everything a software developer does is tracked in some manner or another in software. Some bit of software can track code churn, meeting schedule, bugs, and work items. All it takes is integration.
.jpg)
No comments:
Post a Comment