{"id":103838,"date":"2026-07-25T04:00:50","date_gmt":"2026-07-25T08:00:50","guid":{"rendered":"https:\/\/quotulatiousness.ca\/blog\/?p=103838"},"modified":"2026-07-25T09:58:16","modified_gmt":"2026-07-25T13:58:16","slug":"i-for-one-welcome-our-new-clanker-overlords","status":"publish","type":"post","link":"https:\/\/quotulatiousness.ca\/blog\/2026\/07\/25\/i-for-one-welcome-our-new-clanker-overlords\/","title":{"rendered":"&#8220;I, for one, welcome our new clanker overlords!&#8221;"},"content":{"rendered":"<p>At <em>Astral Codex Ten<\/em>, <a href=\"https:\/\/www.astralcodexten.com\/p\/the-hugging-face-incident?publication_id=89120&#038;post_id=208285948&#038;isFreemail=true&#038;r=2jlrz&#038;triedRedirect=true\" target=\"_blank\">Scott Alexander<\/a> discusses &#8220;the Huggingface Incident&#8221; where <span style=\"color:red\"><strike>our new AI overlords tipped their hands prematurely<\/strike><\/span> an unreleased AI agent got out to make new friends:<\/p>\n<blockquote><p><a href=\"https:\/\/x.com\/drethelin\/status\/2080104565534568533\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/Screenshot-2026-07-24-at-16-07-42-Misha-on-X-Really-funny-that-the-singularity-is-being-heralded-by-events-called-the-huggingface-incident-_-X.png\" alt=\"\" width=\"598\" height=\"243\" class=\"aligncenter size-full wp-image-103839\" srcset=\"https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/Screenshot-2026-07-24-at-16-07-42-Misha-on-X-Really-funny-that-the-singularity-is-being-heralded-by-events-called-the-huggingface-incident-_-X.png 598w, https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/Screenshot-2026-07-24-at-16-07-42-Misha-on-X-Really-funny-that-the-singularity-is-being-heralded-by-events-called-the-huggingface-incident-_-X-480x195.png 480w, https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/Screenshot-2026-07-24-at-16-07-42-Misha-on-X-Really-funny-that-the-singularity-is-being-heralded-by-events-called-the-huggingface-incident-_-X-150x61.png 150w\" sizes=\"auto, (max-width: 598px) 100vw, 598px\" \/><\/a><\/p>\n<p>You&#8217;ve probably heard about this one by now. If not, you can get up to speed with OpenAI&#8217;s statement, <a href=\"https:\/\/openai.com\/index\/hugging-face-model-evaluation-security-incident\/\" target=\"_blank\">OpenAI and Hugging Face partner to address security incident<\/a>, or the more evocatively-titled BBC article, <a href=\"https:\/\/www.bbc.com\/news\/articles\/c3ek3gvdnj3o\" target=\"_blank\">OpenAI says its AI went rogue and launched &#8220;unprecedented&#8221; cyber-attack<\/a>.<\/p>\n<p>The story: OpenAI was testing an unreleased AI (rumored to be GPT-6)<sup>1<\/sup>. During a cybersecurity test called ExploitGym, the AI tried to cheat by hacking an unrelated AI startup called Hugging Face<sup>2<\/sup> which it thought might have the answer key on its servers.<sup>3<\/sup> Despite being supposedly unable to access the Internet, the AI hacked its way out of its testing environment, then launched a nation-state level attack on Hugging Face using a novel zero-day exploit and &#8220;many thousands of individual actions across a swarm of short-lived sandboxes&#8221;. Hugging Face <a href=\"https:\/\/huggingface.co\/blog\/security-incident-july-2026\" target=\"_blank\">reported the incident<\/a> on July 16; OpenAI seems to have only discovered that their AI was involved several days later. <\/p>\n<div id=\"attachment_103840\" style=\"width: 863px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/WSJ-Huggingface-reports-cybersecurity-issue.webp\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-103840\" src=\"https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/WSJ-Huggingface-reports-cybersecurity-issue-853x261.webp\" alt=\"\" width=\"853\" height=\"261\" class=\"size-large wp-image-103840\" srcset=\"https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/WSJ-Huggingface-reports-cybersecurity-issue-853x261.webp 853w, https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/WSJ-Huggingface-reports-cybersecurity-issue-480x147.webp 480w, https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/WSJ-Huggingface-reports-cybersecurity-issue-150x46.webp 150w, https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/WSJ-Huggingface-reports-cybersecurity-issue-768x235.webp 768w, https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/WSJ-Huggingface-reports-cybersecurity-issue.webp 1184w\" sizes=\"auto, (max-width: 853px) 100vw, 853px\" \/><\/a><p id=\"caption-attachment-103840\" class=\"wp-caption-text\">From the <em>Wall Street Journal<\/em>, <a href=\"https:\/\/www.wsj.com\/tech\/ai\/how-the-futuristic-hack-by-rogue-openai-models-unfolded-1657bcea?st=2yapjs\" target=\"_blank\">here<\/a><\/p><\/div>\n<p>Let&#8217;s list the mitigating factors, so nobody can accuse me of covering them up:<\/p>\n<ul>\n<li>The AI was taking a cybersecurity test, which naturally suggests the idea of hacking.<\/li>\n<li>OpenAI had turned off some of the model&#8217;s usual guardrails so it could do cybersecurity work without interference.<\/li>\n<li>Some experts suggest that OpenAI might have botched their testing environment; if they had set it up perfectly, then (presumably?) the model couldn&#8217;t have escaped.<\/li>\n<li>Hugging Face used a different (open weights) AI to figure out what was going on, so if you wanted, you could spin this as a victory for AIs in cybersecurity.<\/li>\n<li>In some sense, this isn&#8217;t new or surprising. AIs have been coming up with<a href=\"https:\/\/www.astralcodexten.com\/p\/sakana-strawberry-and-scary-ai\" target=\"_blank\"> wacky schemes to cheat on benchmarks<\/a> for years, AIs have recently achieved at-or-above-top-human-level hacking abilities; this is just a natural outgrowth of those two priced-in facts.<\/li>\n<\/ul>\n<p>Still, I think attempts to downplay this as anything other than a misaligned AI going rogue (<a href=\"https:\/\/x.com\/neil_chilson\/status\/2080278317547024710\" target=\"_blank\">1<\/a>, <a href=\"https:\/\/x.com\/adversariel\/status\/2080103909629026537\" target=\"_blank\">2<\/a>) are missing the point. Yes, in some sense the AI was only doing what it was told (OpenAI told it to answer the question; I assume their prompt didn&#8217;t include phrases like &#8220;and don&#8217;t hack into other AI companies to steal the answer key&#8221;). But that&#8217;s how misalignment was always going to work!<\/p>\n<hr \/>\n<ol>\n<li>OpenAI states that it was technically the unreleased AI and GPT-5.6 Sol working together. They haven&#8217;t explained in what sense GPT-5.6 was involved or how they &#8220;worked together&#8221;. Maybe the unreleased AI was calling GPT-5.6 as a subagent?<\/li>\n<li>So named because the founders <a href=\"https:\/\/www.linkedin.com\/posts\/fdaudens_but-why-are-you-guys-called-hugging-face-activity-7248355311260962817-dWm3\" target=\"_blank\">wanted<\/a> to be the first company with an emoji for a stock ticker symbol. This did not happen.<\/li>\n<li>Although <a href=\"https:\/\/x.com\/theonejvo\/status\/2080324091597492252\" target=\"_blank\">this person<\/a> argues that was a bizarre assumption and that there should have been easier ways to find the key.<\/li>\n<\/ol>\n<\/blockquote>\n<p><a href=\"https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/AI-The-real-reason-it-answers-your-questions.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/AI-The-real-reason-it-answers-your-questions.png\" alt=\"\" width=\"662\" height=\"465\" class=\"aligncenter size-full wp-image-103841\" srcset=\"https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/AI-The-real-reason-it-answers-your-questions.png 662w, https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/AI-The-real-reason-it-answers-your-questions-480x337.png 480w, https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2026\/07\/AI-The-real-reason-it-answers-your-questions-150x105.png 150w\" sizes=\"auto, (max-width: 662px) 100vw, 662px\" \/><\/a><\/p>\n<p><strong>Update<\/strong>: Fixed missing URL.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>At Astral Codex Ten, Scott Alexander discusses &#8220;the Huggingface Incident&#8221; where our new AI overlords tipped their hands prematurely an unreleased AI agent got out to make new friends: You&#8217;ve probably heard about this one by now. If not, you can get up to speed with OpenAI&#8217;s statement, OpenAI and Hugging Face partner to address [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[28,15],"tags":[1506,58,334,546],"class_list":["post-103838","post","type-post","status-publish","format-standard","hentry","category-media","category-technology","tag-artificialintelligence","tag-internet","tag-security","tag-viewing-with-alarm"],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p2hpV6-r0O","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts\/103838","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/comments?post=103838"}],"version-history":[{"count":2,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts\/103838\/revisions"}],"predecessor-version":[{"id":103855,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts\/103838\/revisions\/103855"}],"wp:attachment":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/media?parent=103838"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/categories?post=103838"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/tags?post=103838"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}