Statistics and Public Policy

Message Bookmarked

Bookmark Removed

What should be the role of statistics in formulating public policy?

This question was prompted by an LA Times commentary Terror Threat May Be Mostly a Big Bluff by USC professor Bart Kosko. (Link requires registration.) He argues that the relatively low number of deaths caused by terrorism can be interpreted as negative evidence for the conclusion that terrorism may be a weaker threat than imagined.

Another example is election polling. Say that polling showed a correlation between polling results and voter participation, as suggested here, or voter confidence, as discussed here.

Another example is having minority candidates on ballots.

Do you think it is right for policy makers to use this type of information? Would you put any limits on formulating policy based upon the odds for achieving the greatest public good? Or is there something wrong with the way I framed this question?

― youn, Monday, 20 September 2004 03:24 (twenty-one years ago)

there's a reason why every respectable public policy/administration and economics program in US colleges teaches its students statistical methods. it's a gauge of how well a given policy is working -- is there a problem, and if so what are the statistical/probabalistic chances of the problem's being resolved under different scenarios? it isn't the ONLY tool available to policymakers or policy advocates, but it can be a very powerful one.

am i understanding this question correctly, then?

― Eisbär (llamasfur), Monday, 20 September 2004 03:34 (twenty-one years ago)

Would you put any limits on formulating policy based upon the odds for achieving the greatest public good?

that's really a political question, isn't it? that turns on a policymaker's philosophical beliefs, as well as society's political mechanics. any way, it comes down to particular issues -- "greatest good for the greatest number" may be sound for some policies (e.g., national defense or budgetary priorities), and not sound for others (e.g., how to rectify past injustices to a discrete minority of a population, or an individual).

― Eisbär (llamasfur), Monday, 20 September 2004 03:38 (twenty-one years ago)

He argues that the relatively low number of deaths caused by terrorism can be interpreted as negative evidence for the conclusion that terrorism may be a weaker threat than imagined.

This is certainly true. However, policy with regards to the war on terror is very definitely not driven by figures. Most of the reports now coming out of think tanks, government and military analysis pools on terrorist threats are notable for -not- using figures or any method for determining probability and likelihood in the analysis of threat and formulation of policy for said threat.

In other words, they're completely unscientific although they may appear so to the layman.

For example, the government is tossing money to a couple of small pretty much useless biopharmaceutical firms to develop a vaccine that would immunize an individual against ricin. But there is no practical need for such a thing. No citizens will want it and the only way it will be possible to get people to take it, if it can be made, is to
either mandate it as part of duty -- as in a requisite for being in the military, or by instituting a prefab panic. Statistics and probabilities on poisoning by ricin have nothing to do with this policy.

― George Smith, Monday, 20 September 2004 14:10 (twenty-one years ago)

I should get some of the professors here to thread. Stat classes are pretty big around here, and happily the instructors also include books like How to Lie With Statistics and Statistical Tricks and Traps as part of the required reading. V. smart.

― Ned Raggett (Ned), Monday, 20 September 2004 14:19 (twenty-one years ago)

I know a lot of people who would prefer that the government were managed with facts, rather than emotions. But that would require research, and like, fuck that.

― dave225 (Dave225), Monday, 20 September 2004 14:23 (twenty-one years ago)

Yeah, and people tend to glaze over when confronted with facts. A nice bogeyman or a scapecoat however, works a charm.

― Kevin Gilchrist (Mr Fusion), Monday, 20 September 2004 14:25 (twenty-one years ago)

Stat classes are pretty big around here, and happily the instructors also include books like How to Lie With Statistics

Terror assessments would be more interesting if at least the people writing them would attempt to lie with statistics. But the rigor is much less than that, just relying on statements delivered by "authorities."

Generally, there is a formula that is followed. (1) Hypothesize a threat or capability on the part of the terrorists that is plausible. It doesn't have to be probable -- just remotely plausible.

(2) Collect apocrypha from the media using Lex-Nex; season with
circle-jerk citations on the same subject written by other organizations or agencies in the exact same business.

(3)By the end of the report, states that what was merely a hypothesis at the beginning is now probably and imminent, or use a "not a matter of if, but when" statement.

― George Smith, Monday, 20 September 2004 14:50 (twenty-one years ago)

UCL has a man who I believe is the UK's leading risk analyst on staff, named John Adams if I remember rightly. I went to one of his lectures and exchanged a couple of emails with him. He is scathing and very accurate on the abuse of statistics and probability to manufacture scares, and the way money is thrown at the wrong problems.

― Martin Skidmore (Martin Skidmore), Monday, 20 September 2004 15:51 (twenty-one years ago)

yeah the best thing i learned in college is how to not be fooled by scientific-sounding language/charts/graphs that simply illustrates bogus logic

― amateur!!st, Monday, 20 September 2004 15:53 (twenty-one years ago)

My grad school management program throws a lot of weight behind using statistical and total cost/annualized loss expectancy models for risk managment decisions. The funny thing is, you get through all the formulae and the 3D matrices and all that shit and then they pop out with "of course, there exist no OBJECTIVE measurements to base these conclusions on" because the effects of policy are nigh-impossible to quantify when the environment changes daily and observable results are extremely few and far between.

Knowledgeable application of stats and formulae to support educated policy guessmaking is great, it's how things should be done. Those who aren't familiar with great books like How To Lie With Statistics (from when, 60 years ago now?) are the ones who give the practice a bad name.

Crimefighting with statistical models applied to the geography of an urban area has done wonders for several major cities in the US. But trying to make the same methods work for rare, catastrophic things like espionage or massive terrorist attacks is exactly the WRONG thing to do and it's stupid and dumb and it makes me hate people.

― TOMBOT, Monday, 20 September 2004 16:12 (twenty-one years ago)

Applying stats to rare events like terrorist attacks sounds like a job for Bayesian statistics, although I'm unable to elaborate further about it (can anyone else?).

― MindInRewind (Barry Bruner), Monday, 20 September 2004 16:16 (twenty-one years ago)

The current way of doing things in the US government re terror threats often involves a subtle mirroring. That is, the people who draw up the analyses go to those portions of the defense infrastructure who were in the business of creating terror weapons during the Cold War. And these characters are consulted with regards to how easy or difficult -they- think it is to launch a chemical or biological or nuclear attack. And, invariably, the assessment mirrors what US capabilities would be -- not what a terrorist's capabilities might be.

So what you get are assessment which claim great ease in manufacturing weapons of mass destruction based upon capabilities worked out by the US military over decades during the Cold War. And with no expense spared or restrictions from the real world applied
which may not even be close to being the case with terrorist organizations. The general tone of such assessments then becomes very paranoid and apocalyptic.

The way to assess capability is to actually talk to terrorists or get inside their organization and determine what their know how may be, not their desire, which is what is more often mapped. The analytic function within the military and intelligence infrastructure is currently very bad at this.

― George Smith, Monday, 20 September 2004 16:23 (twenty-one years ago)

Applying stats to rare events like terrorist attacks sounds like a job for Bayesian statistics,

No. There is no magic method. It requires a collection of many intellectual skills and good human -- not technical -- intelligence methods.

― George Smith, Monday, 20 September 2004 16:25 (twenty-one years ago)

Thanks. I appreciate your responses. I should read that book.

― youn, Tuesday, 21 September 2004 02:30 (twenty-one years ago)

It requires a collection of many intellectual skills and good human -- not technical -- intelligence methods.
Er, do you know what Bayesian Statistics are?

― MindInRewind (Barry Bruner), Tuesday, 21 September 2004 02:35 (twenty-one years ago)

You must be logged in to post. Please either login here, or if you are not registered, you may register here.