Too often these days when people want to use a scientific study to bolster a political position, they utter the phrase, “It was peer reviewed” like a magical spell to shut off any criticism of a paper’s findings.
Worse, the concept of “peer review” is increasingly being treated in the popular discourse as synonymous with “the findings were reproduced and proven beyond a shadow of a doubt.”
This is never what peer review was intended to accomplish. Peer review functions largely to catch trivial mistakes and to filter out the loons. It does not confirm or refute a paper’s findings. Indeed, many scientific frauds have passed easily through peer review because the scammers knew what information the reviewers needed to see.
Peer review is the process by which scientists, knowledgeable in the field a paper is published in, look over the paper and some of the supporting data and information to make sure that no obvious errors have been made by the experimenters. The most common cause of peer review failure arises when the peer reviewer believes that the experimenters either did not properly configure their instrumentation, follow the proper procedures or sufficiently document that they did so.
Effective peer review requires that the reviewers have a deep familiarity with the instruments, protocols and procedures used by the experimenters. A chemist familiar with the use of a gas-chromatograph can tell from a description whether the instrument was properly calibrated, configured and used in a particular circumstance. On the other hand, a particle physicist who never uses gas-chromatographs could not verify it was used properly.
Shannon Love, “No One Peer-Reviews Scientific Software”, Chicago Boyz, 2009-11-28
December 10, 2009
QotD: Peer review
December 4, 2009
Don’t bother your pretty little heads about all this “science” stuff
The non-scientists among us (that’s you and you and you and . . .) should just take a pill, sit back, and stop trying to understand the science:
And history repeats itself with climate change. We tell you people of the imminent dangers from the earth warming, and what do you do? You mock us. You question our motives. People who can’t even convert Fahrenheit to Celsius try and tell us we did the science wrong. Now emails have leaked from the Climate Research Unit that apparently show that scientists were fixing the data and trying to suppress the scientific research of dissenters, and you people demand answers from us. I have one thing to say to that. How dare you!
You do not understand the first thing about climate research. Man-made global warming is settled science. Disaster is imminent. We know this. It is a fact. We don’t waste time on studies that say otherwise, the same way we don’t waste time on studies that assert that the earth is flat. We are very smart people, and when we say something is so, you should just accept it.
So you think what is in those emails is important? Well, what exactly do you know? Do you see the white lab coats we wear? That color symbolizes pure science. Were someone like you to wear one, within five minutes it would be stained with neon orange powdered cheese and wet with drool from you trying to comprehend the data sets people like me look at every day.
November 26, 2009
Dragon Age: Origins . . . not all that original
Darkwatermuse tries Dragon Age: Origins, and is not overwhelmed with joy and mirth:
Dragon Age: Origins enjoyment mileage will depend greatly on each gamer’s experience and familiarity with the RPG genre as well as any recent history of head injuries known to have affected their memory.
If I wasn’t familiar with the long history of RPG D&D styled games I might find myself in agreement with the more cynical rankings the game has received. Scores of 4/5 and 8.5/10 are justified only if you’ve never actually installed or played any of the game’s predecessors or have been paid handsomely through a complex financing network connected to the game’s publisher’s marketing department.
[. . .]
DWM’s biggest annoyance? Being confined to these cartoon landscapes which are clearly represented as hexes somewhere in the bowels of the game’s engine but which are not rendered in the visual 3d world we’re forced to navigate through during endless hours of click-double-clicking either the left — no, try the right one this time — button.
Christ on a dungeon ration biscuit! Whose idea was it to leave the last 15% of the game’s coding to the dyslexic?
DWM asks the question few before dared to ask, is it necessary for me to have to endure the chronic sense of shame evoked when I click on part of the screen I can only then discover can’t actually be moved to? More fool me. Again. Excuse me for having failed to recognize that empty terrain is actually harder than a young adolescent dragon’s pecker during the bumpy bus ride to school.
There’s much more . . . I had to force myself not to quote the entire review. Very reminiscent of a Yahtzee game review, with less yellow-background animation (who, interestingly enough, has also reviewed DA:O recently).
Damning with not-so-faint damns
A clearly impartial and unbiased advance review of James Cameron’s Avatar, due for release in December:
Budgeted at a reported $237m (£143m), Avatar is Cameron’s first dramatic feature since the record-breaking Titanic, back in 1997. The film is a science-fiction fantasy set on a verdant planet called Pandora and following the adventures of a US Marine played by Sam Worthington. Cameron shot the film on his own patented “fusion digital 3D camera system” and experts argue that the results take 3D techniques to a whole new level. [. . .]
The reviewer, however, begs to differ, describing the film as “alienating” and “weird”. Moreover, he/she argues that its pioneering visual technology is liable to induce nausea in the viewer. “The problem is with cutting in between 3D focus points and perspective,” the mystery critic writes. “The mind cannot adjust to it without a buffer — thus, Avatar is literally vomit inducing.”
Even the review’s praise comes with a sting in the tail. “There are some beautiful moments [in the film],” it concedes. “But overall it’s a horrible piece of shit.”
November 11, 2009
November 7, 2009
Watch for those dreaded ellipses
Patterico does some reconstructive surgery on Glenn Greenwald’s less-than-totally-honest practice of partial quoting:
Greenwald’s implication is clear: right-wing blogger Patterico shouldn’t have recommended Allahpundit’s coverage — and right-wing blogger Glenn Reynolds shouldn’t have linked Patterico’s recommendation of Allahpundit.
If only those right-wing bloggers had warned their readers to be skeptical and avoid jumping to conclusions . . .
But wait! What’s that little ellipsis in Greenwald’s quotation of my post? Why, I do believe that’s an indication that he left something out of my quote! Let’s just look at that whole quote to see what Greenwald chose to omit, shall we? I’ll put the part Greenwald omitted in bold type:
Whenever there is breaking news, it’s good to keep a few things in mind:
* Don’t jump to conclusions.
* Don’t be afraid to discuss relevant topics even if they seem politically incorrect.
* Always follow Allahpundit.
Hmmm. So Greenwald omits the part of my quote where I explicitly warn readers, as the very first thing I say, that they should not jump to conclusions in breaking news situations. And then Greenwald implies that my recommendation of Allahpundit was a poor one because Allahpundit jumped to invalid conclusions.
November 6, 2009
Contrarian opinion on the iPhone
You can’t accuse Flora Graham an Apple fangirl:
To quote a few of her sharper lines:
* Say What?
Call quality on the iPhone is pathetic, and it’s mostly because of the tiny speaker. It has to be aligned with your ear canal with the accuracy of a laser-guided ninja doing cataract surgery, or else the volume cuts down to nothing as the sound waves bounce uselessly around your ear shells.* Dropped Calls and Data Gaps
If, like Will Smith in Enemy of the State, you’re trying to avoid the eagle eye of Big Brother, the iPhone. could be for you. It drops calls, fails to connect and doesn’t even ring sometimes — not for everyone, but more often than any other phone we’re currently using.* You Can’t Answer If it Doesn’t Ring
Perhaps the worst of the iPhone’s problems is its ability to sit there stealthily and ignore incoming calls. With no ring or vibrate to clue you in, your friends and family are redirected to voicemail . . . or just treated to silence. If you’re in a two-iPhone family, it can be a case of the deaf leading the mute.* The iPhone Might Burn Your Face Off
According to our ultra-sciencey test, it is extremely unlikely that the iPhone will burn your face off… Nevertheless, pressing a large, flat surface to your cheek is always going to be sweaty . . . Thus the current trend for people to walk down the street with their phones on hands-free, yelling into the mike at the bottom while they hold the rest of the phone away from their faces.
However, she does still acknowledge the real reason I still love my iPhone (even acknowledging much of her criticism):
If the iPhone is inaudible, unconnected, on fire and out of battery, why is the thing so popular? The fact is, although the iPhone is the worst phone in the world, it’s the best handheld computer there is.
November 5, 2009
How unpopular do you have to be to have fans do this?
ESPN reports that Kansas City Chiefs fans really don’t like Chiefs running back Larry Johnson:
Embattled Kansas City Chiefs running back Larry Johnson is 75 yards from becoming the team’s all-time leading rusher.
And some Chiefs fans have a problem with that.
An online petition started by Chiefs fans asks general manager Scott Pioli to deactivate Johnson and keep him on the sideline so he cannot pass Priest Holmes for the team rushing record, or join the team’s Ring of Honor at Arrowhead Stadium.
Holmes holds the Chiefs’ rushing record with 6,070 yards; Johnson currently has 5,996 yards.
“While we are thankful for his service, we feel that Larry has been a black eye on the organization and has no business being mentioned” among the team’s all-time greats, the petition reads.
July 28, 2009
Did James Lileks like The Watchmen?
One quote from a fascinating take-down:
. . . it’s a sign of the movie that leaving in the giant squid would have made it less ridiculous.
Haven’t seen the movie myself, although Victor said he liked it.
July 21, 2009
How to respond to a Hugo list kvetch
John Scalzi is in fine form:
What makes this an error is the tangential fact Mr. Roberts is a science fiction author himself. Here’s something that we in the kvetching industry like to call a “pro tip”: If you take the time to squat and pinch off a steaming ass-loaf of condescension onto the heads of the people most committed to the genre of literature you happen to write in, you may find they will remember that fact when they see your books in the stores. As in “oh, here’s the book of that guy who thinks my taste in literature sucks.” How motivated does that make the average science fiction fan to buy a book? Well, you know: How motivated would it make you?
Now, I assume Mr. Roberts didn’t intend to come across as arrogant and hectoring to his primary audience, because very few people so willfully attempt to ankle-shoot their own career, even the ones with an academic aerie such as Mr. Roberts possesses. I suspect he believed he was being stern but fair. However, I also suspect that science fiction fandom, not in fact being comprised of students who have to sit for a lecture in order to graduate, may have its own opinions on the matter. In the real world, people don’t like being told, while being gently and paternalistically patted on the head, that they’re goddamned idiots. Especially from someone who then turns around and hopes to sell them a book.
The short form of this is to say that it’s one thing to believe a book on the Hugo shortlist (or, as is the case of Mr. Roberts, all the books on the shortlist) is or are mediocre. It’s another thing entirely as a writer to criticize a reader (and someone you’d presumably like to make your reader) for his or her taste in books. The first of these is perfectly valid; taste is subjective. The second of these makes you look like a jerk to the people upon whom you presumably hope to build your career.
Which is of course perfectly fine, if that’s what you intend to do. I’d just make sure that it is, in fact, what you intend to do.
