The Modeling Problem: Garbage In, Gospel Out

Planners by definition deal with complicated problems, and the only way to handle complicated problems is with models. Some models are computer programs. Others are simply in the heads of the people doing the planning. Either way, they are simplifications of reality.

For some purposes, simplifications can be useful. But when planning something as complicated as a national forest, urban area, or regional transportation system, planners fun up against what I call the Law of Modeling Limits:

Before a model becomes complicated enough to be useful for planning, it becomes too complicated for anyone to understand.”

National forest planners ran into this problem in the 1980s. Each national forest averages nearly two million acres that often range from lush wetlands to tundra. Competing uses for the forests include timber, livestock grazing, mining, recreation, wildlife, fisheries, and numerous (often incompatible) forms of recreation. Each use had positive or negative effects on the other uses.

Computer models allowed planners to divide their forests into just a few hundred kinds of land. This meant such important factors as soil productivity and erodability were usually left out. Models usually allowed planners to include only a handful of resources and the modeled relationships between those resources were crude.

There tadalafil cialis from india check this site out is no product that best express our performance in the games. Bladder cancer is another one of the dreaded conditions often free sample cialis handled by the urologists of the city. Place one hand on cialis generic wholesale your chest and one hand on your chest and one hand on your stomach. Some people get in touch with their unconscious minds through long distance cheap cialis canada running, practicing yoga, listening to music, but at low volume. The oversimplified models often produced results that were inane, such as the model that assumed that grizzly bear numbers in a roadless area would increase if the Forest Service decided to keep the area roadless; or the model that assumed that trees could grow 650-feet tall, nearly twice as tall as the tallest trees in the world. As one forest modeler told me, their motto was, “garbage in, gospel out.”

Despite these oversimplifications, the resulting models were so complicated that only a handful of people in the country really understood them. Certainly, the officials who based their decisions on the models did not understand them.

For example, most of the models allowed the computer to “high grade” the forest, cutting the most valuable timber first and leaving the crummy timber for future generations. No national forest manager would consider this a respectable practice, they they endorsed plans based on models that did exactly this.

“In urban planning,” warns Yale political scientist James Scott in his book, Seeing Like a State, “it is a short step from parsimonious assumptions to the practice of shaping the environment so that it satisfies the simplifications required by the formula.” In other words, planners who rely on oversimplified models are more likely to try to impose the model’s results on reality than to build more accurate models.

As absurd as this sounds, some planning advocates actually endorse such a policy. “If economic reality is so complex that it can only be described by complicated mathematical models,” says planning guru Herman Daly, “then the reality should be simplified.” Under this ideal, planners should regulate choice and complexity out of existence and require everyone to adopt the lifestyle choices that planners think best.

Bookmark the permalink.

About The Antiplanner

The Antiplanner is a forester and economist with more than fifty years of experience critiquing government land-use and transportation plans.

7 Responses to The Modeling Problem: Garbage In, Gospel Out

  1. pdxf says:

    “The oversimplified models often produced results that were inane, such as the model that assumed that grizzly bear numbers in a roadless area would increase if the Forest Service decided to keep the area roadless”.

    Why is this inane? This also wouldn’t be an “assumption” it would be a conclusion from the model (although not necessarily correct). What information do you have that shows this conclusion from the model to be incorrect?

    “Planners by definition deal with complicated problems, and the only way to handle complicated problems is with models…Either way, they are simplifications of reality…
    Before a model becomes complicated enough to be useful for planning, it becomes too complicated for anyone to understand.”

    It would seem, since you are defining a model as a simplification, that if a model becomes to complicated to understand, then it isn’t truly a model and perhaps the model would need to be further reduced to make it simple (and thus a model). Also tied up in your definition, it would seem that you believe a model has to complicated to be useful (therefore a model cannot exist). Do you think our models of physics aren’t useful? Newtonian physics is a simplification of reality. It isn’t even correct for a lot of the happenings in the universe, but for our everyday lives it does a good job of modelling real-world behavior. Our models in physics are merely simple representations of reality, yet they allow us to develop cars, computers, etc…It would seem that the idea “for a model to be useful it needs to be complicated” is more of an assumption than a fact, and possibly incorrect.

    “Despite these oversimplifications, the resulting models were so complicated that only a handful of people in the country really understood them. Certainly, the officials who based their decisions on the models did not understand them.”

    Models don’t need to be understood by everyone, only those making the decisions from them. I’m not sure where you get the information that the decision-makers didn’t understand them. This would imply either that you heard from someone else that they didn’t understand, or that you understand the models better, and can see that the officials did not understand them. A third possibility is that you are basing this on a result which is different than the aim of the planning, but this could be from incorrect starting data going into the model, resulting in consistent but incorrect conclusions. You could be right, that the officials didn’t understand, but from the information you give, it doesn’t necessarily follow that your conclusion that they didn’t understand is correct.

  2. Dan says:

    As absurd as this sounds, some planning advocates actually endorse such a policy. “If economic reality is so complex that it can only be described by complicated mathematical models,” says planning guru Herman Daly, “then the reality should be simplified.” Under this ideal, planners should regulate choice and complexity out of existence and require everyone to adopt the lifestyle choices that planners think best.

    Planning guru.

    Can you please share with us Daly’s planning credentials, and then contextualize the quote you used [hint for readers: it’s in the context of disciplinolatry [including esp Neoclassical economics] esp wrt Neoclassical economics fetishizing?

    Thank you ever so much.

    DS

  3. pdxf:

    “Why is this inane?” Are you serious? Do you think that grizzly bears read planning documents and base their population growth on decisions made in those documents? Planners were projecting that no changes in on-the-ground management would lead to population increases.

    “Newtonian physics is a simplification of reality.” No it is not. It is an accurate picture of reality within certain limits. (Einstein showed that beyond those limits, other rules apply.)

    “Models don’t need to be understood by everyone, only those making the decisions from them. I’m not sure where you get the information that the decision-makers didn’t understand them.” I knew hundreds of planners and scores of decision-makers who did forest planning in the 1980s. I understood the models better than most — even the person who wrote the modelling software told me that I had analyzed more models than he had ever seen.

    In my conversations with planners and decision makers, as well as in the decision statements themselves, it was clear that few planners and none of the decision makers understood the models or why the models made the decisions they made. The decision makers were letting the computer make the decisions, writing decision statements that set the computer’s decisions in stone, yet they did not have the foggiest idea how the computer came to those decisions. I have documented this in detail in my book, Reforming the Forest Service.

  4. Dan,

    Herman Daly is a public policy professor and could probably be credited with the original ideas behind “sustainable development” (if not the term itself). The quote I used was from his 1977 book, “Steady-State Economics,” which is a sort of “small is beautiful” book. I find most of it to be incomprehensible, which is one characteristic of someone I call a guru.

  5. Dan says:

    I have the book by the ecological economist and former Senior Economist at the World Bank, Randal, and thank you for agreeing that I contextualized the quote properly and that Daly has no planning credentials to conflate an out-of-context quote with/to an entire profession.

    And I’m shocked that an ‘economist’ found an economics textbook incomprehensible – lemme guess: you’re a macro Neoclassical guy, right? No micro for you I guess.

    DS

  6. pdxf says:

    “Why is this inane?” Are you serious? Do you think that grizzly bears read planning documents and base their population growth on decisions made in those documents?”
    Surely you can think of other reasons for this other than literate bears? Perhaps bears don’t like crossing roads, where they could meet cars, people, hunters, etc…, thus reducing their range and their populations. There could be numerous other possibilities that could be plausible, but let’s just stop at the most absurd answer.

    “Newtonian physics is a simplification of reality.” No it is not. It is an accurate picture of reality within certain limits. (Einstein showed that beyond those limits, other rules apply.)”
    newtonian physics is an accurate representation of a piece of reality = simplification of reality. Actually Newtonian physics is just not as accurate of a model as Einstein’s. This is why Newtonian doesn’t work at high velocities, but Einstein’s models both low and high velocities accurately. I also must note that relativity and all of Eintein’s work is a model and simplification of reality as well. There are other model’s that fill in areas that Einstein’s models don’t represent.

    “I understood the models better than most — even the person who wrote the modelling software told me that I had analyzed more models than he had ever seen. “
    Did you analyze them correctly? Are you sure you understood them better than most? If you can only think of literate bears as conclusions to to the models, I’d certainly question your understanding.

  7. Pingback: Seven Reasons Why Government Planning Cannot Work » The Antiplanner

Leave a Reply