As the Antiplanner recently noted, rail and smart-growth advocates are fond of using touchy-feely arguments for their costly policies, and when presented with evidence that a preponderance of their projects are unquestionable failures, they simply respond that critics “lie with statistics.” The Antiplanner’s response is that you have to rely on data to figure out of policies are working or not, and if you are afraid someone is lying with statistics, you had best learn enough about data to watch for the signs of such lying.
Of course, an entire book was written on this subject way back in 1954. But here are a few ways of lying with statistics that Antiplanner readers should watch out for.
Correlation Equals Causation
We hear this all the time: People living in dense inner-city neighborhoods drive less than people living in low-density suburbs, so therefore increasing suburban densities will lead people to drive less. The problem is that the densities are not the reasons why inner-city people drive less. Instead, there is a strong self-selection component: people who want to drive less choose to live in dense neighborhoods. As economist David Brownstone found in a literature review of the question commissioned by the Transportation Research Board, once you factor out self selection, the effect of density on per capita driving is “too small to be useful” (page 7).
Of course, the Antiplanner relies on correlation analyses as well, and some people have charged that I am guilty of the correlation-equals-causation error. But (aside from the care I take in trying to identify causation), I am not the one arguing for massively expensive transportation projects and hugely coercive social engineering programs based on those correlations.
Looking at Data without Looking at Trends
Europeans live in denser cities and they drive less, so therefore increasing the density of American cities will lead to less driving. That was the major conclusion of Peter Newman’s and Jeff Kenworthy’s book on auto dependency.
However, they failed to mention — even though their own book documented it in excruciating detail — that European cities were becoming more “auto dependent” every decade. From 1960 through 1990, the book revealed for cities from Amsterdam to Zurich, urban densities steadily declined, per capita auto driving rapidly grew, and transit ridership was stagnant. This clearly meant that the density and transit policies that European nations instituted after World War II didn’t work.
Inadequate Sample Size
In arguing that rail transit is successful, advocates will often point to one or two cities where rail construction has been followed by a surge in ridership. Or they point to New York as a wonderful rail system, ignoring the fact that New York City is about ten times denser than most cities that are considering building new rail lines.
For example, leaders and managers can arrange professional development donssite.com cialis cheap online opportunities, but staff must actually apply the new ideas in the workplaces. Apart from this, they are highly beneficial since they not only save time but levitra vardenafil generic a significant amount of money as well. Erectile dysfunction comes across in men only due to a number of emotional and physiological reasons such as high blood pressure, nervousness, diabetes, cardiac disorder and depression that intensifies the danger of this donssite.com levitra online grave situation in men. The view for more now purchase cialis is owned by Eli Lilly and ICOS production you need to contact the medical practitioner to avoid any type of adverse happening that may lead to accidents. The late Paul Weyrich often mentioned St. Louis as a successful light-rail system. It is true that the city’s first light-rail line led to an increase in transit ridership. But when the city doubled rail miles with a second line, bus ridership fell and light-rail ridership did not increase.
Some people have accused the Antiplanner of selectively using data. It is true: I select and use all of the data I can find. My recent report on rail transit presents data for every single rail system reported by the Federal Transit Administration (and one or two that were not). My analysis of the effects of smart growth on home prices reviewed data for more than 400 urban areas.
Using the Wrong Indicators
Here is an actual conversation: “Rail transit costs far more than buses and in many cities has led to a reduction in transit ridership as transit agencies cannibalize buses to pay for rail.” “Yes, but ridership on the Minneapolis light-rail line was greater than projected.”
The first problem is that the sample size (one) is too small, but the second problem is that the success of a rail project shouldn’t be judged by comparing ridership with projections. Instead, the benefits of ridership have to be compared with the costs. How do we measure this? My recent transit report offers six alternative measures, and most rail systems fail them all.
Logical Fallacies
Many developers today feel burned by the planners who told them about the huge demand for downtown condos and other high-density housing. The logic behind this supposed demand was, “Most households in high-density housing have no children. Most households in America have no children. So therefore most Americans want to live in high-density housing.”
I call this the “all dogs have four legs” fallacy: All dogs have four legs; Seabiscuit had four legs; so therefore Seabiscuit was a dog. Such fallacies permeate smart-growth thinking.
Non-Random Sampling
Planners love to hold charrettes (pronounced “charades”) and then tell elected officials, “239 people helped us write our plan and so therefore the public support it.” Yet the people who self-select to attend these meetings tend to be people who believe in planning in the first place, so what they say is hardly representative of what the half-million or so people who will be affected by the plan think of it.
I am sure you can think of more. And if you can find any specific examples where the Antiplanner was guilty of these or other statistical deceptions, please bring them to my attention.
Instead, there is a strong self-selection component: people who want to drive less choose to live in dense neighborhoods.
I’ve already pointed out the latest Orenco study shows how this is wrong. The SMARTRAQ somewhat disagrees with you as well, Randal. You either cannot speak to the issue or are distracting away from the full context.
One might call such arguments cherry-picking.
Nonetheless, one wonders yet again why offering increased choices is such a bad thing. Unless kindamaybe you are lobbying for the dinosaur industries.
And speaking of logical fallacies, yet again we see Randal employing conflation and hasty generalization.
That’s OK, I guess. You have to have something to go on.
DS
This is bound to drive antiplanners and planners nuts, but I think we give way to much power to quantitative data, particularly when outside the realm of physics. When taking several stats classes over the last ten years or so, I was always amazed at how most of the equations had some component that is relatively arbitrary. But after doing the math, you have a concrete and often precise number that people can sink their teeth into. We live in a “show me the numbers” society, but most don’t realize that the numbers are bullshit.
Qualitative arguments are bullshit too, but everyone knows that, so they are usually dismissed. The same is NOT true for numbers. Probability has to be the most misused statistical methodology. As my stats teacher cleverly put it,
“If you are swimming of the gulf coast of Florida in the springtime, what is the probability that you will get bit by a shark? Who cares, you either get bit or you don’t.”
On this blog we’ve often seen two separate groups use the same data set, sometimes the same methodology, and come up with drastically different conclusions. My point is, that whether or not the methodology is sound, and the reporting is ethical, we give too much credence to statistical arguments, particularly in the social sciences.
When planning a highway expansion, or new light rail line we seem to automatically dismiss the subjective arguments of those who will have their lives impacted the most, simply because the arguments are subjective. I think subjectivity matters. I think we rely on numbers too much.
This is bound to drive antiplanners and planners nuts, but I think we give way to much power to quantitative data,
Everyone in the sciences uses quantitative data.
DS
Dan,
I know. And I’m not saying we shouldn’t. In fact I’ve been on SPSS all morning. I guess I saying that there is an overall lack of ethics when reporting statistical data. We give the number, but little explanation on how we got it. Put another way, stats are not objective, we just pretend that they are.
I’m beginning to question whether your blog title accurately reflects your views, Mr. O’Toole. I’m not so sure you’re anti-planning or anti-planner. You seem to be anti-commonly held views by many planners, but then you advocate alternative ways of coming to conclusions for planning, especially transportation planning. In essence, you seem to be forcing planners to step off their soapbox of ideals and go back to their basics learned in the first year of planning school about the process itself and the public-service nature of the profession.
We give the number, but little explanation on how we got it. Put another way, stats are not objective, we just pretend that they are.
If you are talking empirical literature, the Ed. will reject the manuscript without explanation in the Methods. If you are talking the gray literature, that happens. If it is not called out, then the paper may be too obscure to notice.
DS
I agree with Bennet. The Antiplanner post lists some common problems with statistics, which is helpful. A bigger problem is that most of the data used in urban planning is trivial observations and meaningless for public policy.
Bennet’s example is good. For another example, any ground truthing of average urban density would reveal that dense inner cities are largely a mix of very poor and very rich people without children, so averaging those two disparate groups is meaningless. Even the TV show Seinfeld observed that wealthy yuppies who tend to live in the urban core “have to” move to the suburbs when they marry and have children. So all this debate about average densities and average automobile use is not grounded in any meaningful data.
An even greater problem is the interpretation of what the data means, but that is a topic for another day.
Both Belltown in SEA and Vancouver report ‘stroller congestion’. Maybe wealthy educated professionals living in the urban core is a Cascadia phenomenon, perhaps there is an affordable housing requirement to give diversity beyond wealthy empty-nesters?
————–
A bigger problem is that most of the data used in urban planning is trivial observations and meaningless for public policy.
Evidence please. Some papers from JAPA, Trans Res A/B, JPL, Urb Studies, etc. showing how this is true.
TIA.
DS
Let me be gentle. “Social science” is hardly “science.” And “planning” is barely “social science.” And the yuppies with children living downtown is small compared to the yuppies with children living in the suburbs.
If wealthy yuppies want to live in the urban core – great for them! Revealed economic preferences are real data, not preferences calculated by planners.
DS, consider this one of those rare times where the Autoplanner is honest about his dishonesty.
”
Here is an actual conversation: “Rail transit costs far more than buses and in many cities has led to a reduction in transit ridership as transit agencies cannibalize buses to pay for rail.†“Yes, but ridership on the Minneapolis light-rail line was greater than projected.â€
The first problem is that the sample size (one) is too small, but the second problem is that the success of a rail project shouldn’t be judged by comparing ridership with projections.
”
And on a minor note, even if it’s about doing better than projections, which projections? The Hiawatha lines initial # of trips was higher than the projections they used when getting funding. But they weren’t any higher than they were just a few years earlier when they were unable to secure state funding during the Ventura administration.
Borealis, do you have any evidence to back your claim, italicized @8?
TIA.
DS
Dan, as I said, it is a topic for another day. I said that because it is a debate about the role of urban and transportation planning in public policy, and the role of government in planning a society. That is too much to debate on this discussion thread.
It’s rather strange that Dan asks for evidence. He stands down to so many challenges to his comments. His double standards & hypocrisy, continue to show, in addition to faulty logic.
For example, on this thread, he typed that pointing out the fact of people who don’t like driving as choosing high density, is cherry-picking. It is not–How is that taking certain selective data?
He also claims that Randal has marketing motives in analyzing urban issues. That is ridiculous–low density does not have to be pushed on people & most businesses make products & service, regardless of density.
I said that because it is a debate about the role of urban and transportation planning in public policy, and the role of government in planning a society
Yes. Yet you falsely claimed that most of the data used in urban planning is trivial observations and meaningless for public policy and cannot back this claim. Usually false assertions such as this are made to mischaracterize the context for debate.
DS
Dan,
As I said, now for a third time, that is a topic for another day. Planning is apparently a religion to some people and I do not want to debate the religion at the end of an old thread.
Yet, for a third time, you brought it up early on in this two-day-old thread and can’t explain why it is false. Not that I’m claiming anything specific, mind you.
Jus’ sayin’.
DS
The Antiplanner makes a good point about charrettes. I remember attending one for a major land use-transportation issue – scheduled, of course, in at a time inconvenient for the vast majority of people who work for a living. And there was a robust attendance of about 25 people (in a metro region of 200,000). Since I arrived late, I got to sign the attendance roster last. Typical name, address, email and *profession* columns. Turns out that of the 25, only 3 were actually private citizens. Everyone else on the roster were urban planners, transportation planners, public works staff, city management, county management, etc.
Later, the charrette results were presented to the local governing body, which subsequently adopted the plan, and the facilitator repeatedly made the point that the charrette was the “community’s vision” because the “community” had participated and determined the results.
Charade is right!
CO,
I would argue that a good Charrette has to last at least 5 days, and be open to the public with events scheduled during normal business hours, during the evening, and on the weekend. I like the Fri-tues format with the kick off Friday evening, the main hands-on event being on Saturday afternoon, sun-mon is production time open to the public, and a tues evening final presentation. The charrette team will be working about 16 hr days and for the most part the doors should always be open.
A charrette should also be combined with other outreach efforts to obtain data from groups that do not participate in the charrette events.
If plans are being implemented off the results of 1 meeting, one, that’s not a charrette, and two, it’s “bad” planning.
BTW, I’ve been to chareettes that have filled convention center expo floors with hundreds of participants.