And, well, there’s the thing about technocracies. How men and women deal with being men and women among each other – and yes, if you like, expand the genders there – is something we’ve been managing these hundreds of thousands of years now. Without formal processes, it’s simply an ongoing negotiation. But here we’ve an organisation full of engineers. It’s pretty much the definition of what Google is, a bagful of the best engineers that can be tempted into working with computers.
That engineering mindset is one of order, of processes, of structures. Free form and flowing is not generally described as desirable among engineers.
To change examples, Major Douglas came up with the idea of Social Credit. Calculate the profits in an economy and then distribute them to the people. This makes sense to an engineer. The shoot down that we never can calculate such profits in anything like real time just does not compute.
To engineers, if we’ve a process, a structure, then we can handle these things. Yet human life and society is simply too complex to be handled in such a manner. Sure, Hayek never was talking about sexual harassment but the point does still stand.
No, this is not really specifically about Google nor sexual harassment. Rather, it’s about technocracy and the undesirability of it as a ruling method. Here we’ve got just great engineers stepping off their comfort zone and into social relationships. The nerds that is, the very ones we’ve been deriding for centuries as not quite getting it about those social relationships, trying to define and encode those things we’re suspicious they don’t quite understand in the first place.
That is, rule by experts doesn’t work simply because experts always do try to step out of their areas of expertise. Where they’re just as bad and dumb as the rest of us. Possibly, even worse, given the attributes that led them to their areas of expertise in the first place.
Tim Worstall, “Google’s Sexual Harassment Policies – Why We Don’t Let The Technocrats Run The World”, Continental Telegraph, 2018-11-08.
March 17, 2021
March 11, 2021
QotD: Microsoft PowerPoint
Still, PowerPoint was an improvement on the Lotus equivalent. I forget what this was called but it was so utterly unpredictable that I discovered belatedly that presentations would not be forthcoming unless you got a group of people to lay their hands on the printer and chant in Latin. (This was not in the manual, unless it was cunningly secreted there in invisible ink, perhaps on the page that read “this page left deliberately blank.”)
Grant McCracken, “Brands that bind … and when they slide”, This Blog Sits at the, 2005-03-10
January 27, 2021
QotD: Open-source the data
Eric S. Raymond, “Open-Sourcing the Global Warming Debate”, Armed and Dangerous, 2009-11-23.
September 19, 2020
The perils and pitfalls of developing open source software
In The Diff, Byrne Hobart looks at the joys and pains-in-the-ass of open source software development:
“How university open debates and discussions introduced me to open source” by opensourceway is licensed under CC BY-SA 2.0
We’re in a strange in-between state where software is increasingly essential, the most important bits of it are built by dedicated volunteers, and there’s no great way to encourage them to a) keep doing this, and b) stay motivated as the work gets less and less fun. That was my main takeaway from Nadia Eghbal’s Working in Public, published this week.
The book is a tour of the open source phenomenon: who builds projects, how they get made and updated, and why. This is an important topic! Almost all of the software I use to write this newsletter is either built from open source products or relies on them: the text is composed in Emacs, on a computer running Linux; I browse the web using Chrome, which mostly consists of open-source Chromium code; my various i-devices run Unix-based OSes (iOS is not open-source, but there are as many open-source implementations of Unix as you want); etc. All of this software Just Works, and I don’t directly pay for any of it. Clearly there are some incentives at work, or it wouldn’t exist in the first place, but generally the more complex an activity is, the harder it is to coordinate without pricing signals. Somehow, open-source projects don’t fall victim to this.
As Working in Public shows, they do face a demoralizing lifecycle. When a project starts, it’s one coder trying to solve a problem they face, and trying to solve it their way. Sometimes, there’s a good reason their preferred solution doesn’t exist yet, and the project languishes. Sometimes, the project catches on, acquiring new users, new contributors, and new bug reports and feature requests.
[…]
Software projects go through this exact cycle: at first, they’re mostly producing and building on “software capital,” code that executes the underlying logic of the process. Over time, more of the work involves integration and compatibility, and as all the easy edge cases get identified, the remaining bugs are a) disproportionately rare (or they would have been spotted by now) and b) disproportionately hard to fix (because they have to be the result of a rare and thus complicated confluence of circumstances). So, over time, a software project falls into the Baumol Trap, where high productivity in the fun stuff produces more and more un-fun work where productivity gains are hard to come by.
This is depressing for project creators. Early on, they have the triumphant experience of building exactly what they want, and solving their nagging problem. And the result is that they’re cleaning up after a bunch of requests from people who are either annoyed that it doesn’t work for them or have unsolicited feedback on how it ought to work. No wonder Linus Torvalds gets so mad. Running a successful open source project is just Good Will Hunting in reverse, where you start out as a respected genius and end up being a janitor who gets into fights.
Emphasis mine, not in the original article.
H/T to Colby Cosh for sharing the link.
September 2, 2020
Trust – the limiting factor on Chinese tech firms’ growth
Henrik Tiemroth on the “glass ceiling” that Chinese technology companies are struggling with:
The rise of Chinese technology firms has been one of the major developments of the last decade. As some of those companies expand their operations overseas, some observers see China laying the groundwork for a broader imperial project, using their growing digital might to project power and influence. By 2030, will we all be sending messages on WeChat, searching on Baidu, and shopping on Alibaba?
Probably not. As China’s major tech firms attempt to expand to global markets, they are running into a glass ceiling of their government’s own making: people don’t trust them. The recent ban on ByteDance’s popular social media app TikTok in the United States demonstrates the extent – and the limits – of China’s digital ambitions.
TikTok was the first Chinese internet product to have a mass following in the United States. As of 2020, the app has 100 million active users in the US – about a third of the population. But the popular and seemingly innocuous app for making, viewing and sharing quippy homemade music videos has been declared a national security threat.
In August, President Trump signed an executive order effectively banning the app, along with WeChat, unless their US operations are taken over by a domestic company. Given the close links between Chinese companies like ByteDance and the government, they argue, the data collected on American users of the app could be used by the Chinese state for espionage or other nefarious purposes.
The Trump administration is not the first government to take this step. In July, India banned TikTok, along with 59 other Chinese apps, amid rising tensions with China. The government cited similar concerns about the potential for mining and misuse of private data. Indonesia temporarily banned the platform in 2018, and Japan is reportedly considering following the US’s lead.
Across the world, people are becoming warier of who uses their data and how. Lawmakers are perking up, as the implications of data for national security are becoming more clear. In 2018, the European Union implemented landmark data privacy laws. In the US, tech CEOs are regularly dragged before congressional committees and a bipartisan movement for regulation is building.
Chinese internet companies face those same concerns and then some. It’s one thing to have your personal data used to promote conspicuous consumption. It’s another entirely to have it weaponized by a sophisticated digital surveillance state at the cutting edge of data-driven totalitarianism.
July 19, 2020
“Evolved cognition is a kludge – more properly, multiple stacks of kludges – developed under selection to be just barely adequate at coping”
ESR considers the role of programmers who need to document their software, after a brief trip into the kludginess of human cognition:
Maybe you’re one of the tiny minority of programmers that, like me, already enjoys writing documentation and works hard at doing it right. If so,the rest of this essay is not for you and you can skip it.
Otherwise, you might want to re-read (or at least re-skim) Ground-Truth Documents before continuing. Because ground-truth documents are a special case of a more general reason why you might want to try to change your mindset about documentation.
In that earlier essay I used the term “knowledge capture” in passing. This is a term of art from AI; it refers to the process of extracting domain knowledge from the heads of human experts into a form that can be expressed as an algorithm executable by the literalistic logic of a computer.
What I invite you to think about now is how writing documentation for software you are working on can save you pain and effort by (a) capturing knowledge you have but don’t know you have, and (b) eliciting knowledge that you have not yet developed.
Humans, including me and you, are sloppy and analogical thinkers who tend to solve problems by pattern-matching against noisy data first and checking our intuitions with logic after the fact (if we actually get that far). There’s no point in protesting that it shouldn’t be that way, that we should use rigorous logic all the way down, because our brains simply aren’t wired for that. Evolved cognition is a kludge – more properly, multiple stacks of kludges – developed under selection to be just barely adequate at coping.
This kludginess is revealed by, for example, optical illusions. And by the famous 7±2 result about the very limited sized of the human working set. And the various well-documented ways that human beings are extremely bad at statistical reasoning. And in many other ways …
July 3, 2020
Birth, rapid growth, profitable stasis, bureausclerosis, decline, death – typical tech firm lifecycle
In the Continental Telegraph, what looks to be a pretty solid characterization of the corporate life-cycle for technology firms:
There are generally, 6 stages in the life of a tech company:
- Inception. A couple of smart kids start something in a garage. Massive innovation.
- Fast growth. The business moves to some good offices, number of staff increases. They take on smart, fast moving people who are innovating big. A few people know them.
- Steady growth. The business is now something of a household name. They get their own small head office. Staff are more like normal business. Innovation continues, but the bureaucracy starts to grow.
- Bureaucracy. The business is a household name. They have multiple offices, or something designed by a smart-ass architect. There’s still staff coming in, but they’re mostly seeing it as a nice warm place to sit and be bureaucrats. The innovators start leaving as it’s just not interesting. They might even be making lots of money, but it’s mostly just living off what was built in earlier phases, or large dumb wasteful projects that go nowhere for years.
- Decline. A new disruptor arrives on the scene. They innovate in a field you’re involved in. You can’t keep up because in the previous phase, you replaced the innovators with bureaucrats. You’re outsmarted.
- Terminal decline and death. You slowly or quickly disappear, maybe holding onto a few customers who habitually use you.
I think there might also be something about when government starts taking an interest in you, and I think it’s quite some time into stage 4. Google are already there. Firing James Damore is very much bureaucracy state behaviour. They wouldn’t have cared when it was a tiny number of staff in an office. And how much have they really innovated in the last decade?
May 16, 2020
QotD: Division of labour in the modern world
In another example — the making of a woollen coat — Smith emphasises that the division of labour allows us to use machines, even “that very simple machine, the shears with which the shepherd clips the wool”.
The shepherd has the perfect tool for a focused task. That tool needs countless other focused specialists: the bricklayer who built the foundry; the collier who mined fuel; the smith who forged the blades. It is a reinforcing spiral: the division of labour lets us build new machines, while machines work best when jobs have been divided into one small task after another.
The rise of the computer complicates this story. Computers can certainly continue the process of specialisation, parcelling out jobs into repetitive chunks, but fundamentally they are general purpose devices, and by running software such as Microsoft Office they are turning many of us into generalists.
In a modern office there are no specialist typists; we all need to be able to pick our way around a keyboard. PowerPoint has made amateur slide designers of everyone. Once a slide would be produced by a professional, because no one else had the necessary equipment or training. Now anyone can have a go — and they do.
Well-paid middle managers with no design skills take far too long to produce ugly slides that nobody wants to look at. They also file their own expenses, book their own travel and, for that matter, do their own shopping in the supermarket. On a bill-by-the-minute basis none of this makes sense.
Why do we behave like this? It is partly a matter of pride: since everyone has the tools to build a website or lay out a book, it feels a little feeble to hand the job over to a professional. And it is partly bad organisational design: sacking the administrative assistants and telling senior staff to do their own expenses can look, superficially, like a cost saving.
Tim Harford, “Why Microsoft Office is a bigger productivity drain than Candy Crush Saga”, The Undercover Economist, 2018-02-02.
April 28, 2020
ESR on “Lassie errors” in software
I’d never heard this term before, but it’s an excellent description of the problem:
Lassie was a fictional dog. In all her literary, film, and TV adaptations the most recurring plot device was some character getting in trouble (in the print original, two brothers lost in a snowstorm; in popular memory “Little Timmy fell in a well”, though this never actually happened in the movies or TV series) and Lassie running home to bark at other humans to get them to follow her to the rescue.
In software, “Lassie error” is a diagnostic message that barks “error” while being comprehensively unhelpful about what is actually going on. The term seems to have first surfaced on Twitter in early 2020; there is evidence in the thread of at least two independent inventions, and I would be unsurprised to learn of others.
In the Unix world, a particularly notorious Lassie error is what the ancient line-oriented Unix editor “ed” does on a command error. It says “?” and waits for another command – which is especially confusing since ed doesn’t have a command prompt. Ken Thompson had an almost unique excuse for extreme terseness, as ed was written in 1973 to run on a computer orders of magnitude less capable than the embedded processor in your keyboard.
Herewith the burden of my rant: You are not Ken Thompson, 1973 is a long time gone, and all the cost gradients around error reporting have changed. If you ever hear this term used about one of your error messages, you have screwed up. You should immediately apologize to the person who used it and correct your mistake.
Part of your responsibility as a software engineer, if you take your craft seriously, is to minimize the costs that your own mistakes or failures to anticipate exceptional conditions inflict on others. Users have enough friction costs when software works perfectly; when it fails, you are piling insult on that injury if your Lassie error leaves them without a clue about how to recover.
April 25, 2020
Professor Neil Ferguson – “I wrote the code (thousands of lines of undocumented C) 13+ years ago to model flu pandemics…”
An anonymous guest post at Hector Drummond’s blog pivots on the disturbing quote in the headline from one of the key advisors to the British government on the Wuhan Coronavirus epidemic:
To say I was gobsmacked at his admission is an understatement. He’s one of the experts advising the government about the Covid-19 pandemic, and was consulted in previous health crises such as Foot & Mouth disease. Like the approach to combating that, we’re seeing a kind of scorched earth approach to containing another transmissible disease.
Even though the “C” programming language that Ferguson used is nearly 50 years old, the language chosen isn’t the problem. Undocumented means that modules and other code fragments are not commented, so their purpose may be unclear to someone unfamiliar with the code. In the worst case it means that modules and variables don’t have self-documenting names. For example, an accounting program could have the variables
BalanceBroughtForwardandBalanceCarriedForward, but a sloppy programmer might call themB1andB2instead – a sure recipe for confusion.The “C” language is good to work with but has some inherent issues which can lead to subtle bugs affecting the output without causing an error. A common problem is the conditional which uses two equals signs rather than one.
To compare variables
AandBfor equality you would write this:if (A == B). However, it’s easy to accidentally write this:if (A = B). The latter always returns true and assigns the value of variableBtoA. I have no idea whether Ferguson’s code contains any bugs, this is just one minor example of the need for strict testing.The reason for commenting code extensively and properly is so that the programmer himself, and anyone else who maintains it, can understand what it does and how it works, reduce the chance of mistakes and avoid unnecessary effort. During my IT career I would have terminated the contract of any contractor working for me who wrote thousands of lines of undocumented code. Not only is such code a nightmare for others to work on, it can be difficult for the original programmer to maintain if coming back to it after a long time. Sloppiness in the coding raises the worry of a concomitant lack of rigour in testing, although that’s not to assert Ferguson’s code isn’t working as intended and/or wasn’t tested.
November 2, 2019
Notepad++ release 7.8.1, also known as the “Free Uyghur” edition, draws unfriendly Chinese attention
I’ve been using Notepad++ as one of my text editor options for a long time. It’s a very useful tool, and I heartily commend it to anyone needing a Windows text editor that can do a lot more than just edit text. I hadn’t downloaded the most recent version, so I was unaware that the developer was under attack from Chinese government supporters for his explicit designation of version 7.8.1 as the Free Uyghur edition:
On Tuesday, Don HO, the developer of Notepad++, a free GPL source code editor and notepad application for Microsoft Windows, released version 7.8.1, prompting a social media firestorm and a distributed denial of service attack.
Notepad++ v7.8.1 was designated “the Free Uyghur edition,” in reference to the predominantly Muslim ethnic group in western China that faces ongoing human rights violations and persecution at the hands of Beijing.
“The site notepad-plus-plus.org has suffered DDoS attack from 1230 to 1330 Paris time,” HO said in an email to The Register. “I saw the [reduced] amount of visitors via Google analytics then the support of my host confirmed the attack. The DDoS attack has been stopped by an anti-DDoS service provided by our host [Cloudflare].”
Previous politically-themed Notepad++ releases have focused on Tiananmen Square and the terrorist attack on French satirical publication Charlie Hebdo.
A post on the project’s website explains HO’s decision to criticize the Chinese government, something companies with business interests in China generally try not to do for fear of retribution.
From the Notepad++ website:
Human rights in China is always a hotly contested topic. Since 2017, numerous reports have emerged of the Uyghur people being detained in extrajudicial “re-education camps”, subjected to political indoctrination, and sometimes even torture. 2018 estimates place the number of detainees in the hundreds of thousands.
The Uyghurs are not ethnically Chinese but live in China’s so-called autonomous Xinjiang region. The region’s name suggests the Uyghurs have autonomy and self-governance. But similarly to Tibet, Xinjiang is a tightly controlled region of China. After the recent Xinjiang conflict, Beijing has recast the Uyghur ethnic group as a terrorist collective. This has allowed Beijing to justify its transformation of Xinjiang into a surveillance state. There has also been a marked rise of Islamophobia across China.
At least 120,000 members of Kashgar’s Muslim Uyghur minority have been detained in Xinjiang’s re-education camps which aim to change the political thinking of detainees, their identities, and their religious beliefs. Reports from the World Uyghur Congress submitted to the United Nations in July 2018 suggest that 1 million Uyghurs are currently being held in the re-education camps.
August 17, 2019
How Diablo was completely Reverse Engineered without Source Code | MVG
Modern Vintage Gamer
Published on Jul 1, 2019In 1996 Blizzard Entertainment released Diablo, an Action RPG that sold over 2.5 million copies and defined a genre. In 2018 a developer known as GalaXyHaXz almost completely reverse engineered the code in 4 months and released it as open source. How was this accomplished? Find out in this episode!
► Consider supporting me – https://www.patreon.com/ModernVintage…
Links
► DevilutionX (Open Source Diablo) https://github.com/diasurgical/devilu…
► My Nintendo Switch Port – https://github.com/lantus/devilution-nx
August 4, 2019
Blog housekeeping notes
Sorry to anyone who tried to access the blog on Saturday morning, as it went down hard for about an hour after my WordPress statistics plug-in got holed below the waterline and had to be scuttled. I’ve been using it for several years without a problem, but somehow Saturday was the day it decided to stop working. A reader notified me on one of the social media sites that the blog wasn’t loading and I checked the dashboard to see a long list of fatal errors pop up. It looked like the initial problem was a call to a non-existent function in SQL which then caused a cascade of other uncaught errors and the blog no longer displayed for would-be readers. I switched to the troubleshooting tab and then looked at the details on the CyStats plug-in, and suddenly the answer appeared:
You may have to look really closely to see what the issue is…
I guess I’ll have to do without my old stats package now.
May 10, 2019
Microsoft can’t get worse than old Clippy? “Hold my non-alcoholic beer”
Libby Emmons reports on a new Microsoft Word plugin that puts Clippy into the history books:
Coming soon to a word processing app you probably already subscribe to is Microsoft’s new Ideas plugin. This leap forward in the predictive text trend will endeavor to help you be less offensive. Worried you might be a little bit racist? A little gender confused? Not sure about the difference between disabled persons and persons who are disabled? Never fear, Microsoft will fix your language for you.
Using machine learning and AI, Microsoft’s Ideas in Word will help writers be their least offensive, most milquetoast selves. Just like spell check and grammar check function, Ideas will make suggestions as to how to improve your text to be more inclusive. On the surface, this seems like a terrible idea, but when we dig further beneath the impulse, and the functionality of the program, it gets even worse. What’s happening is that AI and machine learning are going to be the background of pretty much every application, learning from our behaviours not only how we’d like to format our PowerPoint presentations, but learning, across platforms, how best to construct language so that we say what we are wanted to say as opposed to what we really mean.
There is an essential component of honest communication, namely that a person express themselves using their own words. When children are learning to talk and to articulate themselves, they are told to “use your words.” Microsoft will give writers the option of using someone else’s words, some amalgamation of users’ words across the platform, and the result will be that the ideas exhibited will not be the writer’s own.
October 8, 2018
QotD: The closed-source software dystopia we barely avoided
Thought experiment: imagine a future in which everybody takes for granted that all software outside a few toy projects in academia will be closed source controlled by managerial elites, computers are unhackable sealed boxes, communications protocols are opaque and locked down, and any use of computer-assisted technology requires layers of permissions that (in effect) mean digital information flow is utterly controlled by those with political and legal master keys. What kind of society do you suppose eventually issues from that?
Remember Trusted Computing and Palladium and crypto-export restrictions? RMS and Linus Torvalds and John Gilmore and I and a few score other hackers aborted that future before it was born, by using our leverage as engineers and mentors of engineers to change the ground of debate. The entire hacker culture at the time was certainly less than 5% of the population, by orders of magnitude.
And we may have mainstreamed open source just in time. In an attempt to defend their failing business model, the MPAA/RIAA axis of evil spent years pushing for digital “rights” management so pervasively baked into personal-computer hardware by regulatory fiat that those would have become unhackable. Large closed-source software producers had no problem with this, as it would have scratched their backs too. In retrospect, I think it was only the creation of a pro-open-source constituency with lots of money and political clout that prevented this.
Did we bend the trajectory of society? Yes. Yes, I think we did. It wasn’t a given that we’d get a future in which any random person could have a website and a blog, you know. It wasn’t even given that we’d have an Internet that anyone could hook up to without permission. And I’m pretty sure that if the political class had understood the implications of what we were actually doing, they’d have insisted on more centralized control. ~For the public good and the children, don’t you know.~
So, yes, sometimes very tiny groups can change society in visibly large ways on a short timescale. I’ve been there when it was done; once or twice I’ve been the instrument of change myself.
Eric S. Raymond, “Engineering history”, Armed and Dangerous, 2010-09-12.
