Quotulatiousness

October 21, 2017

Canada’s equivalent to the NSA releases a malware detection tool

Filed under: Cancon, Technology — Tags: , , , — Nicholas @ 04:00

At The Register, Simon Sharwood looks at a new security tool (in open source) released by the Communications Security Establishment (CSE, formerly known as CSEC):

Canada’s Communications Security Establishment has open-sourced its own malware detection tool.

The Communications Security Establishment (CSE) is a signals intelligence agency roughly equivalent to the United Kingdom’s GCHQ, the USA’s NSA and Australia’s Signals Directorate. It has both intelligence-gathering and advisory roles.

It also has a tool called “Assemblyline” which it describes as a “scalable distributed file analysis framework” that can “detect and analyse malicious files as they are received.”

[…]

The tool was written in Python and can run on a single PC or in a cluster. CSE claims it can process millions of files a day. “Assemblyline was built using public domain and open-source software; however the majority of the code was developed by CSE.” Nothing in it is commercial technology and the CSE says it is “easily integrated in to existing cyber defence technologies.”

The tool’s been released under the MIT licence and is available here.

The organisation says it released the code because its job is to improve Canadian’s security, and it’s confident Assemblyline will help. The CSE’s head of IT security Scott Jones has also told the Canadian Broadcasting Corporation that the release has a secondary goal of demystifying the organisation.

May 24, 2017

ESR presents Open Adventure

Filed under: Gaming, History — Tags: , , — Nicholas @ 04:00

Eric S. Raymond recently was entrusted with the original code for ADVENT, and he’s put it up on gitlab for anyone to access:

Colossal Cave Adventure was the origin of many things; the text adventure game, the dungeon-crawling D&D (computer) game, the MOO, the roguelike genre. Computer gaming as we know it would not exist without ADVENT (as it was known in its original PDP-10 incarnation).

Long ago, you might have played this game. Or maybe you’ve just heard stories about it, or vaguely know that “xyzzy” is a magic word, or have heard people say “You are in a maze of twisty little passages, all alike”,

Though there’s a C port of the original 1977 game in the BSD game package, and the original FORTRAN sources could be found if you knew where to dig, Crowther & Woods’s final version – Adventure 2.5 from 1995 – has never been packaged for modern systems and distributed under an open-source license.

Until now, that is.

With the approval of its authors, I bring you Open Adventure. And with it some thoughts about what it means to be respectful of an important historical artifact when it happens to be software.

This is code that fully deserves to be in any museum of the great artifacts of hacker history. But there’s a very basic question about an artifact like this: should a museum preserve it in a static form as close to the original as possible, or is it more in the right spirit to encourage the folk process to continue improving the code?

Modern version control makes this question easier; you can have it both ways, keeping a pristine archival version in the history and improving it. Anyway, I think the answer to the general question is clear; if heritage code like this is relevant at all, it’s as a living and functional artifact. We respect our history and the hackers of the past best by carrying on their work and their playfulness.

March 1, 2016

BrewDog releases all their beer recipies

Filed under: Britain, Business — Tags: , — Nicholas @ 02:00

Lester Haines on the recent decision by BrewDog to open source their entire beer recipie list:

From humble home-brewing origins, James Watt and Martin Dickie have grown BrewDog to an international craft beer operation. Along the way, they’ve claimed the “world’s strongest beer” title twice, firstly with the 41 per cent ABV Sink The Bismarck!, and then with the liver-bashing 55 per cent ABV The End of History.

The recipes for both (albeit with somewhat less lethal ABVs) are available on BrewDog’s “DIY Dog” PDF (see here), along with other tempting tipples such as Tactical Nuclear Penguin and Albino Squid Assassin.

BrewDog - Sink the Bismark

November 17, 2015

The “cult of meritocracy” in hacker culture

Filed under: Politics, Technology — Tags: , , , , — Nicholas @ 03:00

Eric S. Raymond on the demands for political correctness even within the hacker community:

I’m not going to analyze SJW ideology here except to point out, again, why the hacker culture must consider anyone who holds it an enemy. This is because we must be a cult of meritocracy. We must constantly demand merit – performance, intelligence, dedication, and technical excellence – of ourselves and each other.

Now that the Internet – the hacker culture’s creation! – is everywhere, and civilization is increasingly software-dependent, we have a duty, the duty I wrote about in Holding Up The Sky. The invisible gears have to turn. The shared software infrastructure of civilization has to work, or economies will seize up and people will die. And for large sections of that infrastructure, it’s on us – us! – to keep it working. Because nobody else is going to step up.

We dare not give less than our best. If we fall away from meritocracy – if we allow the SJWs to remake us as they wish, into a hell-pit of competitive grievance-mongering and political favoritism for the designated victim group of the week – we will betray not only what is best in our own traditions but the entire civilization that we serve.

This isn’t about women in tech, or minorities in tech, or gays in tech. The hacker culture’s norm about inclusion is clear: anybody who can pull the freight is welcome, and twitching about things like skin color or shape of genitalia or what thing you like to stick into what thing is beyond wrong into silly. This is about whether we will allow “diversity” issues to be used as wedges to fracture our community, degrade the quality of our work, and draw us away from our duty.

When hackers fail our own standards of meritocracy, as we sometimes do, it’s up to us to fix it from within our own tradition: judge by the work alone, you are what you do, shut up and show us the code. A movement whose favored tools include the rage mob, the dox, and faked incidents of bigotry is not morally competent to judge us or instruct us.

I have been participating in and running open-source projects for a quarter-century. In all that time I never had to know or care whether my fellow contributors were white, black, male, female, straight, gay, or from the planet Mars, only whether their code was good. The SJWs want to make me care; they want to make all of us obsess about this, to the point of having quotas and struggle sessions and what amounts to political officers threatening us if we are insufficiently “diverse”.

[…]

It has been suggested that djangoconcardiff might be a troll emulating an SJW, and we should thus take him less seriously. The problem with this idea is that no SJW disclaimed him – more generally, that “Social Justice” has reached a sort of Poe’s Law singularity at which the behavior of trolls and true believers becomes indistinguishable even to each other, and has the same emergent effects.

September 11, 2015

How about creating a truly open web?

Filed under: Liberty, Technology — Tags: , , , , , — Nicholas @ 02:00

Brewster Kahle on the need to blow up change the current web and recreate it with true open characteristics built-in from the start:

Over the last 25 years, millions of people have poured creativity and knowledge into the World Wide Web. New features have been added and dramatic flaws have emerged based on the original simple design. I would like to suggest we could now build a new Web on top of the existing Web that secures what we want most out of an expressive communication tool without giving up its inclusiveness. I believe we can do something quite counter-intuitive: We can lock the Web open.

One of my heroes, Larry Lessig, famously said “Code is Law.” The way we code the web will determine the way we live online. So we need to bake our values into our code. Freedom of expression needs to be baked into our code. Privacy should be baked into our code. Universal access to all knowledge. But right now, those values are not embedded in the Web.

It turns out that the World Wide Web is quite fragile. But it is huge. At the Internet Archive we collect one billion pages a week. We now know that Web pages only last about 100 days on average before they change or disappear. They blink on and off in their servers.

And the Web is massively accessible – unless you live in China. The Chinese government has blocked the Internet Archive, the New York Times, and other sites from its citizens. And other countries block their citizens’ access as well every once in a while. So the Web is not reliably accessible.

And the Web isn’t private. People, corporations, countries can spy on what you are reading. And they do. We now know, thanks to Edward Snowden, that Wikileaks readers were selected for targeting by the National Security Agency and the UK’s equivalent just because those organizations could identify those Web browsers that visited the site and identify the people likely to be using those browsers. In the library world, we know how important it is to protect reader privacy. Rounding people up for the things that they’ve read has a long and dreadful history. So we need a Web that is better than it is now in order to protect reader privacy.

December 27, 2014

ESR on the origins of open source theory

Filed under: History, Technology — Tags: , , , , — Nicholas @ 03:00

Eric S. Raymond acknowledges the strong influence of evolutionary psychology on the development of open source theory:

Yesterday I realized, quite a few years after I should have, that I have never identified in public where I got the seed of the idea that I developed into the modern economic theory of open-source software – that is, how open-source “altruism” could be explained as an emergent result of selfish incentives felt by individuals. So here is some credit where credit is due.

Now, in general it should be obvious that I owed a huge debt to thinkers in the classical-liberal tradition, from Adam Smith down to F. A. Hayek and Ayn Rand. The really clueful might also notice some connection to Robert Trivers’s theory of reciprocal altruism under natural selection and Robert Axelrod’s work on tit-for-tat interactions and the evolution of cooperation.

These were all significant; they gave me the conceptual toolkit I could apply successfully once I’d had my initial insight. But there’s a missing piece – where my initial breakthrough insight came from, the moment when I realized I could apply all those tools.

The seed was in the seminal 1992 anthology The Adapted Mind: Evolutionary Psychology and the Generation of Culture. That was full of brilliant work; it laid the foundations of evolutionary psychology and is still worth a read.

(I note as an interesting aside that reading science fiction did an excellent job of preparing me for the central ideas of evolutionary psychology. What we might call “hard adaptationism” – the search for explanations of social behavior in evolution under selection – has been a central theme in SF since the 1940s, well before even the first wave of academic sociobiological thinking in the early 1970s and, I suspect, strongly influencing that wave. It is implicit in, as a leading example, much of Robert Heinlein’s work.)

The specific paper that changed my life was this one: Two Nonhuman Primate Models for the Evolution of Human Food Sharing: Chimpanzees and Callitrichids by W.C. McGrew and Anna T.C. Feistner.

In it, the authors explained food sharing as a hedge against variance. Basically, foods that can be gathered reliably were not shared; high-value food that could only be obtained unreliably was shared.

The authors went on to observe that in human hunter-gatherer cultures a similar pattern obtains: gathered foods (for which the calorie/nutrient value is a smooth function of effort invested) are not typically shared, whereas hunted foods (high variance of outcome in relation to effort) are shared. Reciprocal altruism is a hedge against uncertainty of outcomes.

November 23, 2014

ESR on how to learn hacking

Filed under: Technology — Tags: , , — Nicholas @ 10:59

Eric S. Raymond has been asked to write this document for years, and he’s finally given in to the demand:

What Is Hacking?

The “hacking” we’ll be talking about in this document is exploratory programming in an open-source environment. If you think “hacking” has anything to do with computer crime or security breaking and came here to learn that, you can go away now. There’s nothing for you here.

Hacking is a style of programming, and following the recommendations in this document can be an effective way to acquire general-purpose programming skills. This path is not guaranteed to work for everybody; it appears to work best for those who start with an above-average talent for programming and a fair degree of mental flexibility. People who successfully learn this style tend to become generalists with skills that are not strongly tied to a particular application domain or language.

Note that one can be doing hacking without being a hacker. “Hacking”, broadly speaking, is a description of a method and style; “hacker” implies that you hack, and are also attached to a particular culture or historical tradition that uses this method. Properly, “hacker” is an honorific bestowed by other hackers.

Hacking doesn’t have enough formal apparatus to be a full-fledged methodology in the way the term is used in software engineering, but it does have some characteristics that tend to set it apart from other styles of programming.

  • Hacking is done on open source. Today, hacking skills are the individual micro-level of what is called “open source development” at the social macrolevel. A programmer working in the hacking style expects and readily uses peer review of source code by others to supplement and amplify his or her individual ability.
  • Hacking is lightweight and exploratory. Rigid procedures and elaborate a-priori specifications have no place in hacking; instead, the tendency is try-it-and-find-out with a rapid release tempo.
  • Hacking places a high value on modularity and reuse. In the hacking style, you try hard never to write a piece of code that can only be used once. You bias towards making general tools or libraries that can be specialized into what you want by freezing some arguments/variables or supplying a context.
  • Hacking favors scrap-and-rebuild over patch-and-extend. An essential part of hacking is ruthlessly throwing away code that has become overcomplicated or crufty, no matter how much time you have invested in it.

The hacking style has been closely associated with the technical tradition of the Unix operating system.

Recently it has become evident that hacking blends well with the “agile programming” style. Agile techniques such as pair programming and feature stories adapt readily to hacking and vice-versa. In part this is because the early thought leaders of agile were influenced by the open source community. But there has since been traffic in the other direction as well, with open-source projects increasingly adopting techniques such as test-driven development.

October 24, 2014

Google Design open sources some icons

Filed under: Media, Technology — Tags: , , , — Nicholas @ 07:19

If you have a need for system icons and don’t want to create your own (or, like me, you have no artistic skills), you might want to look at a recent Google Design set that is now open source:

Today, Google Design are open-sourcing 750 glyphs as part of the Material Design system icons pack. The system icons contain icons commonly used across different apps, such as icons used for media playback, communication, content editing, connectivity, and so on. They’re equally useful when building for the web, Android or iOS.

Google Design open source icons

September 30, 2014

“These bugs were found – and were findable – because of open-source scrutiny”

Filed under: Technology — Tags: , , , — Nicholas @ 08:13

ESR talks about the visibility problem in software bugs:

The first thing to notice here is that these bugs were found – and were findable – because of open-source scrutiny.

There’s a “things seen versus things unseen” fallacy here that gives bugs like Heartbleed and Shellshock false prominence. We don’t know – and can’t know – how many far worse exploits lurk in proprietary code known only to crackers or the NSA.

What we can project based on other measures of differential defect rates suggests that, however imperfect “many eyeballs” scrutiny is, “few eyeballs” or “no eyeballs” is far worse.

September 19, 2014

Doctorow – “The time has come to create privacy tools for normal people”

Filed under: Liberty, Technology — Tags: , , , , — Nicholas @ 00:03

In the Guardian, Cory Doctorow says that we need privacy-enhancing technical tools that can be easily used by everyone, not just the highly technical (or highly paranoid) among us:

You don’t need to be a technical expert to understand privacy risks anymore. From the Snowden revelations to the daily parade of internet security horrors around the world – like Syrian and Egyptian checkpoints where your Facebook logins are required in order to weigh your political allegiances (sometimes with fatal consequences) or celebrities having their most intimate photos splashed all over the web.

The time has come to create privacy tools for normal people – people with a normal level of technical competence. That is, all of us, no matter what our level of technical expertise, need privacy. Some privacy measures do require extraordinary technical competence; if you’re Edward Snowden, with the entire NSA bearing down on your communications, you will need to be a real expert to keep your information secure. But the kind of privacy that makes you immune to mass surveillance and attacks-of-opportunity from voyeurs, identity thieves and other bad guys is attainable by anyone.

I’m a volunteer on the advisory board for a nonprofit that’s aiming to do just that: Simply Secure (which launches Thursday at simplysecure.org) collects together some very bright usability and cryptography experts with the aim of revamping the user interface of the internet’s favorite privacy tools, starting with OTR, the extremely secure chat system whose best-known feature is “perfect forward secrecy” which gives each conversation its own unique keys, so a breach of one conversation’s keys can’t be used to snoop on others.

More importantly, Simply Secure’s process for attaining, testing and refining usability is the main product of its work. This process will be documented and published as a set of best practices for other organisations, whether they are for-profits or non-profits, creating a framework that anyone can use to make secure products easier for everyone.

April 23, 2014

LibreSSL website – “This page scientifically designed to annoy web hipsters”

Filed under: Technology — Tags: , , , , , — Nicholas @ 09:24

Julian Sanchez linked to this Ars Technica piece on a new fork of OpenSSL:

OpenBSD founder Theo de Raadt has created a fork of OpenSSL, the widely used open source cryptographic software library that contained the notorious Heartbleed security vulnerability.

OpenSSL has suffered from a lack of funding and code contributions despite being used in websites and products by many of the world’s biggest and richest corporations.

The decision to fork OpenSSL is bound to be controversial given that OpenSSL powers hundreds of thousands of Web servers. When asked why he wanted to start over instead of helping to make OpenSSL better, de Raadt said the existing code is too much of a mess.

“Our group removed half of the OpenSSL source tree in a week. It was discarded leftovers,” de Raadt told Ars in an e-mail. “The Open Source model depends [on] people being able to read the code. It depends on clarity. That is not a clear code base, because their community does not appear to care about clarity. Obviously, when such cruft builds up, there is a cultural gap. I did not make this decision… in our larger development group, it made itself.”

The LibreSSL code base is on OpenBSD.org, and the project is supported financially by the OpenBSD Foundation and OpenBSD Project. LibreSSL has a bare bones website that is intentionally unappealing.

“This page scientifically designed to annoy web hipsters,” the site says. “Donate now to stop the Comic Sans and Blink Tags.” In explaining the decision to fork, the site links to a YouTube video of a cover of the Twisted Sister song “We’re not gonna take it.”

April 11, 2014

Open source software and the Heartbleed bug

Filed under: Technology — Tags: , , , , — Nicholas @ 07:03

Some people are claiming that the Heartbleed bug proves that open source software is a failure. ESR quickly addresses that idiotic claim:

Heartbleed bugI actually chuckled when I read rumor that the few anti-open-source advocates still standing were crowing about the Heartbleed bug, because I’ve seen this movie before after every serious security flap in an open-source tool. The script, which includes a bunch of people indignantly exclaiming that many-eyeballs is useless because bug X lurked in a dusty corner for Y months, is so predictable that I can anticipate a lot of the lines.

The mistake being made here is a classic example of Frederic Bastiat’s “things seen versus things unseen”. Critics of Linus’s Law overweight the bug they can see and underweight the high probability that equivalently positioned closed-source security flaws they can’t see are actually far worse, just so far undiscovered.

That’s how it seems to go whenever we get a hint of the defect rate inside closed-source blobs, anyway. As a very pertinent example, in the last couple months I’ve learned some things about the security-defect density in proprietary firmware on residential and small business Internet routers that would absolutely curl your hair. It’s far, far worse than most people understand out there.

[…]

Ironically enough this will happen precisely because the open-source process is working … while, elsewhere, bugs that are far worse lurk in closed-source router firmware. Things seen vs. things unseen…

Returning to Heartbleed, one thing conspicuously missing from the downshouting against OpenSSL is any pointer to an implementation that is known to have a lower defect rate over time. This is for the very good reason that no such empirically-better implementation exists. What is the defect history on proprietary SSL/TLS blobs out there? We don’t know; the vendors aren’t saying. And we can’t even estimate the quality of their code, because we can’t audit it.

The response to the Heartbleed bug illustrates another huge advantage of open source: how rapidly we can push fixes. The repair for my Linux systems was a push-one-button fix less than two days after the bug hit the news. Proprietary-software customers will be lucky to see a fix within two months, and all too many of them will never see a fix patch.

Update: There are lots of sites offering tools to test whether a given site is vulnerable to the Heartbeat bug, but you need to step carefully there, as there’s a thin line between what’s legal in some countries and what counts as an illegal break-in attempt:

Websites and tools that have sprung up to check whether servers are vulnerable to OpenSSL’s mega-vulnerability Heartbleed have thrown up anomalies in computer crime law on both sides of the Atlantic.

Both the US Computer Fraud and Abuse Act and its UK equivalent the Computer Misuse Act make it an offence to test the security of third-party websites without permission.

Testing to see what version of OpenSSL a site is running, and whether it is also supports the vulnerable Heartbeat protocol, would be legal. But doing anything more active — without permission from website owners — would take security researchers onto the wrong side of the law.

And you shouldn’t just rush out and change all your passwords right now (you’ll probably need to do it, but the timing matters):

Heartbleed is a catastrophic bug in widely used OpenSSL that creates a means for attackers to lift passwords, crypto-keys and other sensitive data from the memory of secure server software, 64KB at a time. The mega-vulnerability was patched earlier this week, and software should be updated to use the new version, 1.0.1g. But to fully clean up the problem, admins of at-risk servers should generate new public-private key pairs, destroy their session cookies, and update their SSL certificates before telling users to change every potentially compromised password on the vulnerable systems.

April 3, 2014

ESR reviews Jeremy Rifkin’s latest book

Filed under: Books, Economics, Media, Technology — Tags: , , , — Nicholas @ 10:46

The publisher sent a copy of The Zero Marginal Cost Society along with a note that Rifkin himself wanted ESR to receive a copy (because Rifkin thinks ESR is a good representative of some of the concepts in the book). ESR isn’t impressed:

In this book, Rifkin is fascinated by the phenomenon of goods for which the marginal cost of production is zero, or so close to zero that it can be ignored. All of the present-day examples of these he points at are information goods — software, music, visual art, novels. He joins this to the overarching obsession of all his books, which are variations on a theme of “Let us write an epitaph for capitalism”.

In doing so, Rifkin effectively ignores what capitalists do and what capitalism actually is. “Capital” is wealth paying for setup costs. Even for pure information goods those costs can be quite high. Music is a good example; it has zero marginal cost to reproduce, but the first copy is expensive. Musicians must own expensive instruments, be paid to perform, and require other capital goods such as recording studios. If those setup costs are not reliably priced into the final good, production of music will not remain economically viable.

[…]

Rifkin cites me in his book, but it is evident that he almost completely misunderstood my arguments in two different way, both of which bear on the premises of his book.

First, software has a marginal cost of production that is effectively zero, but that’s true of all software rather than just open source. What makes open source economically viable is the strength of secondary markets in support and related services. Most other kinds of information goods don’t have these. Thus, the economics favoring open source in software are not universal even in pure information goods.

Second, even in software — with those strong secondary markets — open-source development relies on the capital goods of software production being cheap. When computers were expensive, the economics of mass industrialization and its centralized management structures ruled them. Rifkin acknowledges that this is true of a wide variety of goods, but never actually grapples with the question of how to pull capital costs of those other goods down to the point where they no longer dominate marginal costs.

There are two other, much larger, holes below the waterline of Rifkin’s thesis. One is that atoms are heavy. The other is that human attention doesn’t get cheaper as you buy more of it. In fact, the opposite tends to be true — which is exactly why capitalists can make a lot of money by substituting capital goods for labor.

These are very stubborn cost drivers. They’re the reason Rifkin’s breathless hopes for 3-D printing will not be fulfilled. Because 3-D printers require feedstock, the marginal cost of producing goods with them has a floor well above zero. That ABS plastic, or whatever, has to be produced. Then it has to be moved to where the printer is. Then somebody has to operate the printer. Then the finished good has to be moved to the point of use. None of these operations has a cost that is driven to zero, or near zero at scale. 3-D printing can increase efficiency by outcompeting some kinds of mass production, but it can’t make production costs go away.

April 2, 2014

Enigma’s 21st century open sourced descendent

Filed under: History, Military, Technology, WW2 — Tags: , , — Nicholas @ 09:51

The Enigma device was used by the German military in World War 2 to encrypt and decrypt communication between units and headquarters on land and at sea. Original Enigma units — the few that are on the market at any time — sell for tens of thousands of dollars. You may not be able to afford an original, but you might be interested in a modern implementation of Enigma using Arduino-based open-source hardware and software:

Actual hand-crafted Final design

Actual hand-crafted Final design

Enigma machines have captivated everyone from legendary code breaker Alan Turing and the dedicated cryptographers from England’s Bletchley Park to historians and collectors the world over.

But while many history buffs would surely love to get their hands on an authentic Enigma machine used during WWII, the devices aren’t exactly affordable (last year, a 1944 German Enigma machine was available for auction at Bonhams with an estimated worth of up to $82,000). Enter the Open Enigma Project, a kit for building one from scratch.

The idea came to Marc Tessier and James Sanderson from S&T Geotronics by accident.

“We were working on designing and building intelligent Arduino-based open-source geocaching devices to produce a unique interactive challenge at an upcoming Geocaching Mega Event,” Tessier told Crave. “A friend of ours suggested we use an Enigma type encrypting/decrypting machine as the ultimate stage of the challenge and pointed us to an Instructables tutorial that used a kid’s toy to provide some Enigma encoding. We looked all over to buy a real Enigma machine even if we had to assemble it ourselves and realized that there was nothing available at the moment. So we decided to build our own.”

[…]

“Our version is an electronic microprocessor-based machine that is running software which is a mathematical expression of how the historical mechanical machine behaved,” Sanderson told Crave. “Having never touched a real Enigma M4, we built our open version based on what we read online. From what we understand, the real electro-mechanical devices are much heavier and a little bigger.”

They took some design liberties — replacing the physical rotors with LED units and replacing the light bulbs with white LEDs. The replica can be modified by changing the Arduino code and can communicate to any computer via USB. Future versions may include Wi-Fi and/or Bluetooth.

March 3, 2014

The origins of hacking and the myth of a lost Eden of open source code

Filed under: History, Technology — Tags: , , , — Nicholas @ 09:40

Gather round you kids, ’cause Uncle Eric is going to tell you about the dim, distant days of hacking before open source:

I was a historian before I was an activist, and I’ve been reminded recently that a lot of younger hackers have a simplified and somewhat mythologized view of how our culture evolved, one which tends to back-project today’s conditions onto the past.

In particular, many of us never knew – or are in the process of forgetting – how dependent we used to be on proprietary software. I think by failing to remember that past we are risking that we will misunderstand the present and mispredict the future, so I’m going to do what I can to set the record straight.

[…]

Without the Unix-spawned framework of concepts and technologies, having source code simply didn’t help very much. This is hard for younger hackers to realize, because they have no experience of the software world before retargetable compilers and code portability became relatively common. It’s hard for a lot of older hackers to remember because we mostly cut our teeth on Unix environments that were a few crucial years ahead of the curve.

But we shouldn’t forget. One very good reason is that believing a myth of the fall obscures the remarkable rise that we actually accomplished, bootstrapping ourselves up through a series of technological and social inventions to where open source on everyone’s desk and in everyone’s phone and ubiquitous in the Internet infrastructure is now taken for granted.

We didn’t get here because we failed in our duty to protect a prelapsarian software commons, but because we succeeded in creating one. That is worth remembering.

Update: In a follow-up post, ESR talks about closed source “sharecroppers” and Unix “nomads”.

Like the communities around SHARE (IBM mainframe users) and DECUS (DEC minicomputers) in the 1960s and 1970s, whatever community existed around ESPOL was radically limited by its utter dependence on the permissions and APIs that a single vendor was willing to provide. The ESPOL compiler was not retargetable. Whatever community developed around it could neither develop any autonomy nor survive the death of its hardware platform; the contributors had no place to retreat to in the event of predictable single-point failures.

I’ll call this sort of community “sharecroppers”. That term is a reference to SHARE, the oldest such user group. It also roughly expresses the relationship between these user groups and contributors, on the one hand, and the vendor on the other. The implied power relationship was pretty totally asymmetrical.

Contrast this with early Unix development. The key difference is that Unix-hosted code could survive the death of not just original hardware platforms but entire product lines and vendors, and contributors could develop a portable skillset and toolkits. The enabling technology – retargetable C compilers – made them not sharecroppers but nomads, able to evade vendor control by leaving for platforms that were less locked down and taking their tools with them.

I understand that it’s sentimentally appealing to retrospectively sweep all the early sharecropper communities into “open source”. But I think it’s a mistake, because it blurs the importance of retargetability, the ability to resist or evade vendor lock-in, and portable tools that you can take away with you.

Without those things you cannot have anything like the individual mental habits or collective scale of contributions that I think is required before saying “an open-source culture” is really meaningful.

Older Posts »

Powered by WordPress