Quotulatiousness

May 24, 2011

“Why does dubious social science keep showing up in medical journals?”

Filed under: Economics, Media, Science — Tags: , , , , — Nicholas @ 10:08

William Easterly and Laura Freschi have determined the decision tree for publishing crappy social science research:

Aid Watch has complained before about shaky social science analysis or shaky numbers published in medical journals, which were then featured in major news stories. We questioned creative data on stillbirths, a study on health aid, and another on maternal mortality.

Just this week, yet another medical journal article got headlines for giving us the number of women raped in the DR Congo (standard headline: a rape a minute). The study applied country-wide a 2007 estimate of the rate of sexual violence in a small sample (of unknown and undiscussed bias). It did this using female population by province and age-cohort — in a country whose last census was in 1984.

We are starting to wonder, why does dubious social science keep showing up in medical journals?

The medical journals may not have as much capacity to catch flaws in social science as in medicine. They may desire to advocate for more action on tragic social problems. The news media understably assume the medical journals ARE vetting the research.

H/T to Tim Harford for the link.

April 25, 2011

Majority in reach for Harper, if Ontario numbers hold up

Filed under: Cancon, Politics — Tags: , , , — Nicholas @ 11:17

The latest Nanos/Globe & Mail poll shows that the Conservatives might just be able to reach majority status if the Ontario numbers stay steady for the final week:

Nationally, the NDP is statistically tied for second place with the Liberals. But another story is emerging out of Ontario, where they are running third; the party’s surge in Quebec and other regions has not given it a similar bump in Ontario.

Pollster Nik Nanos sees this as a boon for the Tories — one that could well provide the Conservatives a route to that coveted majority.

“Talk about Jack Layton in Stornoway has actually helped the Conservatives in Ontario,” Mr. Nanos said Monday. “The Conservatives best chance to win a majority is in Ontario. If their numbers hold or start ramping up in Ontario that could be good news for the Conservatives. One of the scenarios that we could be looking at now is a squeaker of a majority government.”

Of course, the regional numbers are subject to a larger margin of error:

There is a margin of error of plus or minus 64 percentage points, 19 times out of 20, for the provincial sample.

I hope there’s a decimal place somewhere in that MoE.

April 18, 2011

The Magic Washing Machine, by Hans Rosling

Filed under: Economics, Environment, Health, Technology — Tags: , , , — Nicholas @ 09:49

H/T to Jon for the link.

February 24, 2011

They’re called “factoids”, not “facts”

Filed under: Bureaucracy, Education, Government, USA — Tags: , , , — Nicholas @ 07:46

There’s a factoid in common circulation at the moment that, measured by SAT scores, the states that ban collective bargaining for teachers rank almost dead last, while Wisconsin ranks 2nd. Neal McCluskey explains that this is not particularly true:

Now, aside from the factoid, if true, providing no real insight into whether collective bargaining is good or bad for education — there are myriad variables at work other than collective bargaining, none of which does this control for — but the factoid itself is highly dubious. Again, it is hard to find the original source for this, but I looked up 2009 ACT and SAT state rankings, and at the very least it seems highly unlikely that Virginia ranks 44th out of all states. According to the ACT ranking, for instance, Virginia places 22nd, and on the SAT (assuming the linked to list is accurate — I’m doing this fast), it ranked 33rd. It’s hard to see how those would be combined for a 44th place overall finish.

How about the Wisconsin second place-finish? Well, that is accurate for the SAT, but notably only 5 percent of Wisconsin students took the SAT — a negligible rate. On the ACT, which is the main test taken in the Badger State, Wisconsin finished 13th — not bad, but hardly great.

So what does this tell you? Not that collective bargaining is educationally good or bad — like I said, you just can’t get there from here — but that you have to be very careful about your sources of information. Unfortunately, that seems especially true when you’re dealing with education.

January 23, 2011

Lawrence Solomon on the coming crash in China

Filed under: China, Economics, Government — Tags: , , — Nicholas @ 12:17

If you think I’ve been too chipper and dismissive on the medium-to-long term issues that could cause a Chinese meltdown, you’ll enjoy Lawrence Solomon’s article:

In China, the resentments are palpable. Many of the 300 million people who have risen out of poverty flaunt their new wealth, often egregiously so. This is especially so with the new class of rich, all but non-existent just a few years ago, which now includes some 500,000 millionaires and 200 billionaires. Worse, the gap between rich and poor has been increasing. Ominously, the bottom billion views as illegitimate the wealth of the top 300 million.

How did so many become so rich so quickly? For the most part, through corruption. Twenty years ago, the Communist Party decided that “getting rich is glorious,” giving the green light to lawless capitalism. The rulers in China started by awarding themselves and their families the lion’s share of the state’s resources in the guise of privatization, and by selling licences and other access to the economy to cronies in exchange for bribes. The system of corruption, and the public acceptance of corruption, is now pervasive — even minor officials in government backwaters are now able to enrich themselves handsomely.

[. . .]

The corruption extends to the enforcement of regulatory standards for health and safety, which few in China trust. In recent years China has endured a tainted milk scandal and a tainted blood scandal, each of which implicated corrupt officials in widespread death and debilitation. In a devastating 2008 earthquake, some 90,000 perished, one-third of them children buried alive in 7,000 shoddily built “tofu schools” that skimped on materials. Nearby buildings for the elites that met building standards, including a school for the children of the rich, were largely unscathed.

[. . .]

China is a powder keg that could explode at any moment. And if it does explode, chaos could ensue — as the Chinese are only too well aware, the country has a brutal history of carnage at the hands of unruly mobs. For this reason, corrupt officials inside China, likely by the tens of thousands, have made contingency plans, obtaining foreign passports, buying second homes abroad, establishing their families and businesses abroad, or otherwise planning their escapes. Also for this reason, much of the middle class supports the government’s increasingly repressive efforts.

Compared to my rather milder criticisms, this is strong stuff indeed.

H/T to my former virtual landlord for the link, who referred to this as my “hobby horse in full gallop”.

January 21, 2011

Remaking Red Dawn as a metaphor for US fear of China

Filed under: China, Economics, Media, USA — Tags: , , , , — Nicholas @ 12:59

David Harsanyi notes the remake of the 1980’s movie Red Dawn with the Chinese taking the place of the original film’s Soviet and Cuban troops:

Doubtlessly, the remake will be entertaining and offer a far more plausible plot line than the original — seeing that the Chinese, well, they have a proper army. Producers will almost certainly capitalize on a growing alarmism regarding China’s growth. Few issues, in fact, can bring right and left together in this polarized world of ours than a shared knowledge that China is bad news.

Now, the American populace can typically be divided into two categories: 1. Those who don’t care one whit about foreign policy. 2. Newspaper editors.

So before Chinese President Hu Jintao was here meeting with the president, Andrew Kohut of the Pew Research Center took to the pages of The Wall Street Journal and explained what we think about the topic.

Apparently, 47 percent of those he surveyed cited China as the world’s top economic power. (Only 31 percent properly identified it as the U.S., which has an economy nearly three times the size.) Another Pew survey from last year found that 47 percent of us consider China’s growth a “bad thing” for the United States. A new CNN poll found that 58 percent of us believe that China’s “wealth and economic power” are a threat to the U.S.

I’m certain our relationship with China is layered with international complexity and fraught with danger. But why would we fear the aspects of China’s ascendancy — its “wealth and economic power” — that pose the least threat to United States? Unlike ideological clashes, economic competition can be mutually beneficial. A country with real economic wealth is typically free and doesn’t look kindly on radical behavior. Suicide bombers rarely drive top-of-the-line BMWs.

I have a long history of doubting the stated size and growth of the Chinese economy and therefore feeling that the “threat” they pose is overstated. Overall, the economic growth in China is a good thing, both for China and for the world economy, but there’s still too much malignancy from the “bad old days” of the command economy that haven’t been properly dealt with. China is big, and getting bigger, but will face severe problems the longer these historical artifacts remain unexamined and unresolved.

January 17, 2011

Another reason to view self-reported study data with caution

Filed under: Britain, Science — Tags: , , , — Nicholas @ 09:30

There’s a reason that studies that depend on direct observation/measurement often differ in their results from studies that depend on self-reporting by the group being studied — because people lie:

Many mothers are under so much pressure to appear like perfect parents that they cover up how much television their children watch or what they cook their families, according to a survey.

Such “white lies” also extend to how much “quality time” mothers spend with their partner, website Netmums said its survey of 5,000 people suggested.

The parenting site said mothers often made each other feel “inadequate”.

[. . .]

Almost two-thirds of those surveyed said they had been less than honest with other mothers about how well they were coping and almost half covered up financial worries.

Almost a quarter of mothers admitted to downplaying how much television their children actually watched — and one in five “span a yarn” over how long they played with their children.

January 3, 2011

Healthy skepticism about study results

Filed under: Bureaucracy, Media, Science — Tags: , , , , — Nicholas @ 13:30

John Allen Paulos provides some useful mental tools to use when presented with unlikely published findings from various studies:

Ioannidis examined the evidence in 45 well-publicized health studies from major journals appearing between 1990 and 2003. His conclusion: the results of more than one third of these studies were flatly contradicted or significantly weakened by later work.

The same general idea is discussed in “The Truth Wears Off,” an article by Jonah Lehrer that appeared last month in the New Yorker magazine. Lehrer termed the phenomenon the “decline effect,” by which he meant the tendency for replication of scientific results to fail — that is, for the evidence supporting scientific results to seemingly weaken over time, disappear altogether, or even suggest opposite conclusions.

[. . .]

One reason for some of the instances of the decline effect is provided by regression to the mean, the tendency for an extreme value of a random quantity dependent on many variables to be followed by a value closer to the average or mean.

[. . .]

This phenomenon leads to nonsense when people attribute the regression to the mean as the result of something real, rather than to the natural behavior of any randomly varying quantity.

[. . .]

In some instances, another factor contributing to the decline effect is sample size. It’s become common knowledge that polls that survey large groups of people have a smaller margin of error than those that canvass a small number. Not just a poll, but any experiment or measurement that examines a large number of test subjects will have a smaller margin of error than one having fewer subjects.

Not surprisingly, results of experiments and studies with small samples often appear in the literature, and these results frequently suggest that the observed effects are quite large — at one end or the other of the large margin of error. When researchers attempt to demonstrate the effect on a larger sample of subjects, the margin of error is smaller and so the effect size seems to shrink or decline.

[. . .]

Publication bias is, no doubt, also part of the reason for the decline effect. That is to say that seemingly significant experimental results will be published much more readily than those that suggest no experimental effect or only a small one. People, including journal editors, naturally prefer papers announcing or at least suggesting a dramatic breakthrough to those saying, in effect, “Ehh, nothing much here.”

The availability error, the tendency to be unduly influenced by results that, for one reason or another, are more psychologically available to us, is another factor. Results that are especially striking or counterintuitive or consistent with experimenters’ pet theories also more likely will result in publication.

December 30, 2010

Cartographic explanation for the order of secession

Filed under: History, USA — Tags: , , , — Nicholas @ 00:20

A fascinating NYT post looks at one of the most influential maps of the US Civil War period:

The 1860 Census was the last time the federal government took a count of the South’s vast slave population. Several months later, the United States Coast Survey — arguably the most important scientific agency in the nation at the time — issued two maps of slavery that drew on the Census data, the first of Virginia and the second of Southern states as a whole. Though many Americans knew that dependence on slave labor varied throughout the South, these maps uniquely captured the complexity of the institution and struck a chord with a public hungry for information about the rebellion.

The map uses what was then a new technique in statistical cartography: Each county not only displays its slave population numerically, but is shaded (the darker the shading, the higher the number of slaves) to visualize the concentration of slavery across the region. The counties along the Mississippi River and in coastal South Carolina are almost black, while Kentucky and the Appalachians are nearly white.

H/T to Walter Olson for the link.

December 20, 2010

Once again, correlation is not causation

Filed under: Britain, Media, Science — Tags: , , — Nicholas @ 12:50

An excellent example of what statistical analysis can and cannot show:

Do mobile phone towers make people more likely to procreate? Could it be possible that mobile phone radiation somehow aids fertilisation, or maybe there’s just something romantic about a mobile phone transmitter mast protruding from the landscape?

These questions are our natural response to learning that variation in the number of mobile phone masts across the country exactly matches variation in the number of live births. For every extra mobile phone mast in an area, there are 17.6 more babies born above the national average.

This was discovered by taking the publicly available data on the number of mobile phone masts in each county across the United Kingdom and then matching it against the live birth data for the same counties. When a regression line is calculated it has a “correlation coefficient” (a measure of how good the match is) of 98.1 out of 100. To be “statistically significant” a pattern in a dataset needs to be less than 5% likely to be found in random data (known as a “p-value”), and the masts-births correlation only has a 0.00003% probability of occurring by chance.

Part of the problem is that our brains have evolved to detect patterns and relationships — even when they’re not really there:

Mobile phone masts, however, have absolutely no bearing on the number of births. There is no causal link between the masts and the births despite the strong correlation. Both the number of mobile phone transmitters and the number of live births are linked to a third, independent factor: the local population size. As the population of an area goes up, so do both the number of mobile phone users and the number people giving birth.

The problem is that our first instinct is to assume that a correlation means that one factor is causing the other. While this does not cause a problem when using pattern-spotting as an evolved survival tool, it does cause severe problems when assessing possible health scares based on a recently uncovered correlation. For the majority of cases, correlation does not indicate the presence of causality.

H/T to Maggie Koerth-Baker for the link.

December 17, 2010

Gay and lesbian couples’ income levels

Filed under: Economics, Education, Randomness, USA — Tags: , , — Nicholas @ 12:59

As this article asserts, I don’t remember where I heard the “fact” that gay couples had higher incomes than heterosexual couples, but it seemed likely to be true. Apparently not:

The myth of gay money holds that “gays” (really just gay males) are high-income or rich. Why? Mostly because they don’t have kids, especially not when two guys live together. (That would make them DINKs.)

This myth was relentlessly propagated through the 1990s and persists today. Maybe you couldn’t put your finger on where you heard it (perhaps in a newspaper article?), but the stereotype is out there. And it isn’t true.

[. . .]

Why do gay males have generally lower incomes than straight males?

  • Gay males have more education than straight males, but they do not choose male-dominated professions as often as straight males do. In fact, they choose female-dominated and/or service professions much more often. Male-dominated professions (like construction) have generally higher wages than female-dominated professions (like secretarial).
  • Gay males work fewer hours than straight males.

Why do lesbians generally have higher income than straight females? It’s almost the inverse of the gay-male trend.

  • Lesbians also have more education than straight females, but they work longer hours — because, generally speaking, they are less likely to have children to take care of at home.
  • Lesbians are overrepresented in male-dominated professions that pay better than female-dominated professions.

What about discrimination? It’s a ready excuse to explain away the “few” gays who don’t meet the stereotype of being affluent. (That’s what press coverage would tell you – that just a few of us aren’t affluent. In reality, it’s most of us.) But the statistical evidence for discrimination as a cause of lower gay incomes is weak at best, and of course falls down completely in the case of lesbians, who, most studies agree, have higher incomes than straight females. Discrimination is clearly a factor sometimes; it just isn’t a credible explanation for the whole effect, which doesn’t apply to half of the population we’re talking about.

H/T to Freakonomics blog for the link.

December 7, 2010

Chinese official acknowledged that official data is unreliable

Filed under: Bureaucracy, China, Economics, Railways — Tags: , , , , — Nicholas @ 07:32

I’ve been saying this for years now: China’s official GDP and associated economic numbers are just not reliable:

A senior Chinese official said in 2007 that much of the country’s local economic data are unreliable, according to a leaked diplomatic cable published by the WikiLeaks website.

The official, Li Keqiang, was at the time Communist Party secretary of the northeastern province of Liaoning, and has since been promoted to vice premier. Since landing that position, he has overseen many of the central government’s efforts to improve the quality of its economic statistics, which continue to face many questions over their accuracy and consistency.

[. . .]

China’s Foreign Ministry has said it will not comment on the content of the diplomatic cables published by WikiLeaks. The leaked cable reports comments Mr. Li made in a dinner in Beijing with then-U.S. Ambassador Clark Randt on March 12, 2007. His remarks focused on the challenges of administering the province of Liaoning, which because of its legacy of failed state-owned enterprises was burdened with a large number of unemployed workers.

“When evaluating Liaoning’s economy, he focuses on three figures: 1) electricity consumption, which was up 10% in Liaoning last year; 2) volume of rail cargo, which is fairly accurate because fees are charged for each unit of weight; and 3) amount of loans disbursed, which also tends to be accurate given the interest fees charged,” the cable says.

“By looking at these three figures, Li said he can measure with relative accuracy the speed of economic growth. All other figures, especially GDP statistics, are ‘for reference only,’ he said smiling,” the cable reads. “GDP figures are ‘man-made’ and therefore unreliable,” the cable paraphrases Mr. Li as saying.

As I said back in February, the reason for the made up numbers is inherent in the Chinese system:

In this way, the PLA stopped being just the customer/end user. They cut out the middleman and absorbed the entire supply chain. The PLA became a significant economic player in the Chinese industrial economy . . . and this is still true today. The generals aren’t formally in charge, but they own the companies that do military production.

So what? So let’s look at how a civilian corporation’s incentives differ from one owned directly by the army. In a civilian corporation, the CEO runs the business with an eye to generating the largest profit possible while staying (for the most part) within the law. A CEO who deviates from this to ride a favourite hobby horse will eventually face the wrath of the stockholders who want that maximized profit. There are natural limits on how much freedom to invest in uneconomic activity any CEO will be given. Sensible stockholders don’t try to micromanage the firm, but do raise questions if too much of the company’s efforts are devoted to things clearly not related to the company’s long term benefit. Company accounts can be rigged, for a time, to show misleading results, but eventually (Enron, Worldcom, etc.) the truth will out.

A Chinese firm that’s owned by the army? Profit may be nice, but the “CEO” reports to a different master: the guys with the guns. The company accounts will show exactly what the guys with the guns want them to show . . . and the oversight and auditing committee members carry submachine guns. You’re told that your target is 10% growth? Don’t you think that the reported result will be at least 10%? Because your life may depend on the reported results being acceptable.

As my former virtual landlord says, this is one of my hobbyhorses:

December 3, 2010

Pay no attention to the statisticians behind the curtain

Filed under: Britain, Bureaucracy, Environment, Media — Tags: , , , — Nicholas @ 09:23

James Delingpole has a handy guide to assure you that man made global warming is still happening:

“It’s all actually a sign that man made global warming is very much a live issue and that there’s more of it happening than ever,” says a top scientist, who holds the British record for securing grant-funding for global warming research projects so he must know what he’s talking about.

“Look at the Met office,” the scientist goes on. “They’ve just told us that 2010 is the hottest year since records began in 1850 and even though the stupid Central England Temperature record tells us something quite different and even though the year hasn’t actually finished yet they must know what they’re talking about and they definitely can’t have fiddled the data because the Met office is part of the government and they wouldn’t lie or get things wrong which is why that barbecue summer was such a scorcher.”

The big problem is, the scientist said, is that the public are really stupid. They think just because Dr David Viner of the Climatic Research Unit said in the Independent in 2000 that soon there’d be no snow because of global warming, when what he actually meant was that soon there’d be lots of snow and that this would be “proof” of global warming. The interviewer just missed out the word “proof” that’s all because journalists are lazy that way.

Yes, yes, confusing mere “weather” with climate again, I’m sure.

October 10, 2010

Calculate your odds of winning the lottery

Filed under: Randomness — Tags: , — Nicholas @ 10:09

By way of the ever-helpful Roger Henry, here’s a site that lets you simulate your lottery habits to determine how much you’ll win and how much it’ll cost:

Incredibly Depressing Mega Millions Lottery Simulator!

It is really hard to win the Mega Millions lottery. So hard that it can be difficult to comprehend what long odds confront its players.

Why not try for free on this Mega Millions lottery simulator? You’ll be able to try the same numbers over and over, simulating playing twice a week for a year or 10. You’ll never win.

I ran 1040 simulations. It would have cost me $1040. I would have won $243. Not a good return on investment. I actually “win” more by not buying a ticket at all: I avoid the loss of that $797 in ticket purchases.

July 27, 2010

Four weeks in Canadian history

Filed under: Cancon, Humour — Tags: , , — Nicholas @ 07:59

Scott Feschuk has been out of the country for a while. He’s delighted to find that something actually happened in Canada while he was away, and provides a useful summary for those of us not paying attention:

I’ve been away from Canada for four of the past five weeks, and it’s always fun to return and see what’s been missed. A comprehensive review:

1. The dominant domestic news story of the past month hinges on the intricacies of statistical analysis.

2. Finally demonstrating a populist touch, Michael Ignatieff has started production on his own Speed sequel: If his party’s popularity in opinion polls falls below 25 per cent, the Liberal Express explodes! (Subplot: If the bus keeps stopping for Timbits, the occupants of the Liberal Express explode!)

3. Conrad Black has apparently tunneled out of prison and escaped.

4. Upon being informed of No. 3, David Radler has soiled himself.*

* Not reported, but a safe assumption.

Don’t ever change, Canada.

« Newer Posts — Older Posts »

Powered by WordPress