In Wired, Felix Salmon explains that “Quants don’t know everything”:
By now, nearly everyone from the president of the United States on down has admitted that the National Security Agency went too far. Documents leaked by Edward Snowden, the rogue NSA contractor who has since gained asylum in Russia, paint a picture of an organization with access to seemingly every word typed or spoken on any electronic device, anywhere in the world. And when news of the NSA’s reach became public — as it was surely bound to do at some point — the entire US intelligence apparatus was thrust into what The New York Times recently called a “crisis of purpose and legitimacy.”
It was a crisis many years in the making. Over the course of three decades, the NSA slowly transformed itself from the nation’s junior spy agency to the centerpiece of the entire intelligence system. As the amount of data in the world doubled, and doubled again, and again, the NSA kept up with it — even as America’s human intelligence capability, as typified by old-fashioned CIA spies in the field, struggled to do anything useful with the unprecedented quantities of signals intelligence they had access to. Trained agency linguists capable of parsing massive quantities of Arabic- and Farsi-language intercepts don’t scale up nearly as easily as data centers do.
That, however, wasn’t the computer geeks’ problem. Once it was clear that the NSA could do something, it seemed inarguable that the agency should do it — even after the bounds of information overload (billions of records added to bulging databases every day) or basic decency (spying on allied heads of state, for example) had long since been surpassed. The value of every marginal gigabyte of high tech signals intelligence was, at least in theory, quantifiable. The downside — the inability to prioritize essential intelligence and act on it; the damage to America’s democratic legitimacy — was not. As a result, during the past couple of decades spycraft went from being a pursuit driven by human judgment calls to one driven by technical capability.
