<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mike Hearn on Medium]]></title>
        <description><![CDATA[Stories by Mike Hearn on Medium]]></description>
        <link>https://medium.com/@octskyward?source=rss-3748c5d6e52c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*vCXpXuEWOU7lPZ5C.jpeg</url>
            <title>Stories by Mike Hearn on Medium</title>
            <link>https://medium.com/@octskyward?source=rss-3748c5d6e52c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 12:36:13 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@octskyward/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[The future of developer tools]]></title>
            <link>https://blog.plan99.net/the-future-of-developer-tools-139cc351bd67?source=rss-3748c5d6e52c------2</link>
            <guid isPermaLink="false">https://medium.com/p/139cc351bd67</guid>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Mike Hearn]]></dc:creator>
            <pubDate>Fri, 17 Jul 2026 08:49:40 GMT</pubDate>
            <atom:updated>2026-07-17T08:56:16.259Z</atom:updated>
            <content:encoded><![CDATA[<h4>Predictions about where the industry is heading</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TVdD8aqboItiYSgJPDKxuQ.jpeg" /></figure><p>For over fifteen years my job has been building, open sourcing, evangelizing and selling productivity tools to software developers. It wasn’t a conscious career choice. More like one thing led to another, and it was fun, so that’s what I did.</p><p>Now I’m pondering what this work looks like post-LLMs. It’s on my mind a lot because, as always, I’m currently working on developer tools.</p><p>This is a thinking-through-writing essay and covers some personal predictions about what the next five years of tooling progress might look like. It doesn’t represent the views of my anonymous employer, which as far as I know doesn’t have any position on the matter.</p><h3>T+0 days: Bewilderment</h3><p>The most common reaction when I say I’m working on a new programming language is surprise. Why bother? It’s not like anyone is going to use it. If you believe that nearly all code in future will be written by LLMs (and I do) then it’s reasonable to ask:</p><ol><li>How will they learn a new tool without examples in the training set?</li><li>Who cares what tools models use anyway? It’s not like they’re going to complain, or go work for a competitor because your tech stack is obsolete.</li></ol><p>But I think there are good answers to these questions.</p><p>Let’s start with the training set. Is there really a catch-22 here? No, for two reasons. The first is obvious enough: LLMs support in-context learning where you load skills into the context window. This increases costs because the skill has to be processed on every single server round-trip (cached tokens are only cheap, not free). But there’s clearly a break even point here: for some skills the token savings are big enough to pay for themselves.</p><p>The second reason is a bit less obvious. Modern LLM training exposes the model to trillions of tokens, but it’s common that each token in the training set is only seen once. A conventional understanding of ML says you need millions of examples to learn a new skill, but this isn’t true when thinking about LLMs learning developer tools. Frontier models can remember the details of very obscure tools that have almost no examples in the training set. For example <a href="https://claude.ai/share/fccfa851-39fa-4e02-9f3b-3df1334ef97f">try asking about Hydraulic Conveyor</a>, <a href="https://hydraulic.dev/">a tool for deploying desktop apps</a>. I own the company that sells it, but although it’s free for open source apps it’s mostly used by proprietary software. So there’s only a handful of config file examples on GitHub — probably fewer than 50. Yet the frontier models know all about it and can use it fluently.</p><p>I suspect the frontier labs are all using curriculum learning, so by the time they’re introduced to the documentation for these tools they’ve already mastered all the knowledge needed to understand them and the details can be successfully encoded into the weights with a single backwards pass.</p><p>In other words, getting knowledge into models is actually quite easy. You don’t need a massive community creating millions of examples. As long as the crawlers find your website and as long as that website has plenty of docs, the models will get some understanding of how to use it. They might not memorize every single detail but they’ll remember enough to be useful. The expensive period in which you need to explain everything in a skill is not necessarily long. Soon enough, all you need for LLM success is a tiny skill that only contains the changes from what it learned to the current version.</p><p>Now, an LLM won’t organically choose your obscure new tool if all you have is a website. The model has ‘opinions’ about what it ‘wants’ to use, and affecting those is harder. We’ll come back to that later.</p><p>The second reasonable question people ask is, “<em>Does it even matter what tools models use?</em>” Companies equip humans with better tools hoping for cost savings, lower time-to-market, improved morale, and maybe an advantage in the hiring market. Humans like adopting new tools because they’re more enjoyable or productive to use, to buff their CV or because they’re bored. But language models are cheap, don’t suffer morale problems, don’t write CVs and you can hire them with just a credit card.</p><p>Yet the cost argument remains. LLMs and humans differ in many ways but better tools make both of them work faster and therefore cheaper. Fast and cheap is good. That’s the only motive you need to continue creating new tools, languages and abstractions.</p><h3>T+1 year: Proof of Productivity</h3><p>I think by mid 2027 these sorts of conversations will have stopped happening and we’ll have new conventional wisdom: developer tools have value because they reduce token costs and wall clock time. Although hardly anyone is advertising tools this way today, soon enough everyone will think it was obvious from the start.</p><p>Some big changes are implied by this goal. One is that development of any kind of tool comes to look like working on a compiler.</p><p>Historically most developer tools were adopted based on intuition, gut feeling, likeability of the creators or community momentum. Barely disguised emotions were often decisive. Nowhere was this more true than in programming languages and JavaScript frameworks, which inspired decades of relentless intellectual warring.</p><p>It made sense for people to take this stuff seriously — learning a new tech represented a large investment of time and energy. Bet on the wrong horse and it could mean permanent damage to your career. And you can’t do randomized controlled experiments on developers, so there was no way for anyone to unambiguously win these debates.</p><p>But doing controlled experiments on LLMs <em>is</em> possible — even easy.</p><p>So I think by this time next year we’ll see new GitHub projects come with a set of metrics giving the token efficiency improvement over some competing approach, plus/minus a confidence interval, in the same way that model companies show metrics changes today. Probably some projects will even compare the token efficiency of old versions versus new as a way to advertise the upgrades, like compilers do. And there’ll be some YC backed startup offering to generate these comparisons as a service … but most projects will just use CI jobs.</p><p>Commercial tooling developers like the big clouds, database companies and so on will probably start doing weekly trials, nightly trials or even trials on every merge to quickly locate drops in token efficiency. Bugs will be filed demanding rollbacks with titles like, “Token usage regressed by 5%” or “wall clock time increased 12%”.</p><h3>T+2 years: Code is hard, let’s go shopping!</h3><p>With controlled experiments set up you can prove that a tool is worth money … and more importantly, how much. This could mean big changes to the shape of the software industry.</p><p>Developer tools aren’t a great market. Developers <em>really</em> hate paying for tools. We’d usually rather use something open source and terrible than something commercial but good.</p><p>There are rational reasons for this. We fear lockin, price hikes and vendors exiting the market. Rewrites are so expensive and disruptive that we seek the safety of liberally licensed components. And the convenience of adopting an open source tool just can’t be beat.</p><p>Yet as coders shift to become managers of token budgets this way of thinking will experience big challenges. Can you really justify to your boss using an open source tool when there’s hard data <em>proving</em> that a commercial equivalent would save money? Maybe not, and for some this will be a culture shock. I expect drama.</p><p>In the 1990s the rise of OOP led to many heady predictions of online marketplaces, in which devs would casually purchase ActiveX controls or Delphi components. Although a handful of businesses did briefly make money this way, the vision never panned out because it was obsoleted by Perl’s CPAN model — vast centralized repositories of open source libraries. Developers enthusiastically signed up to populate these repositories, motivated by a mixture of (at first) Stallmanist ideology and (later) fame, jobs or open-core business models.</p><p>This will be controversial but my sense is that open source development is slowly running out of energy. Stallmanist ideology died years ago; new projects almost never use copyleft licenses and in fact <a href="https://news.ycombinator.com/item?id=48923363">FreeBSD just announced the removal of the last bit of copyleft code from their OS</a>. Having GitHub projects on your résumé was once a differentiator, but now everyone has them. And people are more aware these days that creating a popular open source library might make you this guy:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/385/0*kzHQCkY3b012PXp0.png" /><figcaption><a href="https://xkcd.com/2347/">xkcd: Dependency</a></figcaption></figure><p>I’ve created open source projects before. I’ll probably do so again. But I probably won’t do so as an unpaid volunteer. Nobody wants employees of random tech giants demanding security fixes from you at 3am on a Sunday, decades after your project stopped being cool.</p><p>The move to LLM driven coding looks to me like the final nail in the coffin for voluntary open source development. The expected outcome of creating a cool open source library is now that an LLM finds it while doing a web search the developer never sees, adopts it, codes with it and maybe even files a bug report — without ever saying thank you, giving you a job opportunity or contributing anything back. You optimized some matrix multiplications and in return got nothing except people who expect you to keep doing so. What’s the point?</p><p>So although commercial component marketplaces didn’t really happen in the 90s, this time I think they just might. A new generation of startups will arise, pitching themselves as “Steam for Developers”.</p><p>The right model for such stores is an open question. The industry figured out formulas that work for consumer software: reviews, payment processing, discovery that works and more, all in return for a revenue share. What works for developers is less clear.</p><p>In DevSteam we might see the following:</p><ul><li>A usage based model in which the store tracks your coding agent logs to figure out how many tokens are spent interacting with the tools. The cost you’re charged scales with that, i.e. not based on your total revenue but rather how much you ‘interact’ with the tool (or library, etc). Price guides showing expected costs for common tasks would make it more concrete.</li><li>The stores would act as trusted intermediaries that run the benchmarks for the products they sell, meaning you have some sort of independent assurance that buying a product really does save money over the alternative.</li><li>The stores will come harness plugins that search the stores in the background looking for tools relevant to the current task. If the store realizes it has a tool that could accelerate the agent’s current task, it would ‘advertise’ to the model, which might choose to buy it out of its pocket money. And users would want this because the expected outcome is that the task finishes faster and cheaper, even with the cost of the new tool added on top.</li><li>Source code might be available, but with licenses that forbid redistribution.</li></ul><p>Because a part of the store is a discovery engine for the model, the question of what a model ‘wants’ to use (based on popularity) becomes much less important. It has hard data that can be used to make a scientific decision instead.</p><p>They might also try to solve the sandboxing problem. Once you become a trusted intermediary between developers and their tools, the sky is the limit.</p><h3>T+3 years: The age of insurance</h3><p>Open source libraries are attractive not just due to their rock-bottom price and frictionless adoption process, but also because they hedge supply chain risk. Projects might be abandoned by their maintainers, but if necessary you can take over and make whatever changes are needed. In the limit you can fork the project and become the new maintainers, assembling a coalition of other users to spread the costs.</p><p>As many programs have dependency trees spanning hundreds or thousands of irreplaceable libraries, this is a pretty good thing! Just imagine if you had that many license agreements with commercial vendors. Quite possibly one would exit the business, refuse to fix a critical bug or hike prices <em>every single day. </em>You’d need people who did nothing but negotiate with vendors.</p><p>In practice though, most consumers of open source tools don’t contribute or fork. They treat dependencies as if they were proprietary: working around bugs instead of upstreaming fixes and porting to a competitor in the event of abandonment.</p><p>One consequence of the LLM revolution is that we now have a lot of data that maps “task description” to “token count to achieve it” for a wide range of tasks. Sure, today that data is spread out unless you’re at a big lab, but in principle nothing would stop you paying devs to upload their logs. With that data you could train a model to predict token cost given a task. Confidential computing can be used to solve the data privacy and trust issues involved. Or you could just generate the needed data yourself.</p><p>Of what use is such a model? Well, you could use it to predict the cost of porting a codebase from one tool to another. And if you have high quality predictions of that you could sell insurance to companies that want to cover their supply chain risk. Such policies would pay out based on the expected cost of replacing a component with another, taking into account source code availability, escrow policies, how different the interfaces are, how compatible the feature sets are, and so on. Insurers in turn would want to reduce their own risk exposure by ensuring there’s always a healthy ecosystem of competitors to popular tools, perhaps even directly funding them.</p><p>It’s unclear to me whether insurance like this will ever become worth the complexity, but I suspect if it was bundled with DevSteam then it might become a nice side business.</p><h3>T+4 years: Recursive self improvement</h3><p>AI is good at optimization problems. In fact, LLM training is nothing but an optimization problem (reducing the loss). Ask any AI researcher and they’ll say that if you give them a well defined benchmark, they’ll give you back a model that crushes it a few weeks later.</p><p>If tooling development becomes the same thing as optimizing token usage for a task, with all the evangelism and community building stripped out, then it would seem that creating tools will become the kind of problem AI can do entirely on its own.</p><p>Putting LLMs into an optimization loop is called <a href="https://www.emergentmind.com/topics/autoresearch">autoresearch</a> and it’s already been shown that it can work — AI can optimize many functions, including itself. How well it does this depends on the task, but I see no reason why optimizing token usage should be beyond it. We already see it do this during the reinforcement learning post-training stages, where LLMs learn a hyper-concise form of thinking that sounds like gibberish.</p><p>Developer tool autoresearch loops are, today, not that valuable:</p><ol><li>They don’t work well. Skill-creating-skills haven’t taken off.</li><li>They would burn more capital discovering improvements than you’d save by actually deploying them.</li><li>Collaborating on the problem through open source projects is difficult because it’s a tragedy of the commons; everyone is incentivized to wait in the hope that others pay the bill. With the social benefits of maintaining projects gone it’s unclear why anyone would do this, outside of perhaps a few legacy mega-projects like the Linux kernel or LLVM.</li></ol><p>That’s why I think the unlock for tooling autoresearch has got to be DevSteam. If we can spin up the following cycle then we’re done:</p><ol><li>Collect rollouts from coding agents in a privacy preserving manner.</li><li>Analyze them to find a task that could be optimized with better tools or abstractions.</li><li>Invent the new tool or abstraction.</li><li>Raise capital to fund an autoresearch loop for optimizing it.</li><li>Upload the new tool to the store, wait for the harness plugin to start recommending it to agents doing the task and sell it to them.</li><li>Profit!</li></ol><p>To me the biggest question mark is (3). I’m not sure I’d trust today’s frontier models to invent a big developer new abstraction from scratch. My experience has been that radical new ideas aren’t their strong suit. That might change, but if it doesn’t then (3) will still be where humans add value.</p><p>Today this cycle doesn’t work. People have tried, but we’re not there yet. If and when it does, then the next question becomes how to set up an economic model that correctly aligns incentives. Tooling developer and venture capitalist might become the same job.</p><h3>T+5 years: My only friend, the end</h3><p>Designing abstractions is intellectually stimulating and enjoyable. If it’s replaced with opaque self-optimizing loops nobody ever looks at, will I be sad? Yes, a little. Hopefully by the time it happens I’ll have found something else to do. Maybe being the Gabe Newell of DevSteam, who knows.</p><p>But making tool design scientific would hardly be a bad thing for the world. Think about all the effort put into developer conferences, evangelism and framework wars. It’s fun, but in the end it’s all just distracting us from those who really matter — the people who need software. If this work goes the way of manual switchboard operator, well … so be it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=139cc351bd67" width="1" height="1" alt=""><hr><p><a href="https://blog.plan99.net/the-future-of-developer-tools-139cc351bd67">The future of developer tools</a> was originally published in <a href="https://blog.plan99.net">Mike’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Brexit was objectively good for Britain]]></title>
            <link>https://blog.plan99.net/brexit-was-objectively-good-for-britain-e95fa713d168?source=rss-3748c5d6e52c------2</link>
            <guid isPermaLink="false">https://medium.com/p/e95fa713d168</guid>
            <category><![CDATA[brexit]]></category>
            <dc:creator><![CDATA[Mike Hearn]]></dc:creator>
            <pubDate>Mon, 01 Dec 2025 20:13:23 GMT</pubDate>
            <atom:updated>2025-12-01T20:15:38.682Z</atom:updated>
            <content:encoded><![CDATA[<h4>And why people think otherwise</h4><figure><img alt="Brexit supporters celebrate with Nigel Farage after the 2016 vote" src="https://cdn-images-1.medium.com/max/760/0*Ie840gsLQEPJ92LU.jpg" /></figure><p>Next year will be the 10th anniversary of the vote heard around the world. Op-eds will be published and talking heads will say things.</p><p>Unfortunately, most of them will be wrong. When it comes to Britain’s exit from the EU, reality and rhetoric have been diverging for some time. The gap has now reached alarming proportions.</p><p>Today I’ll argue something you probably haven’t heard before: that leaving the EU was objectively good for Britain. That’s not hyperbole or misusing the word objective either: I mean it in the literal sense that an even handed alien who arrived on Earth with no preconceived notions would always conclude leaving was the right decision. The people who claim otherwise are factually wrong.</p><p>This argument is never expressed in the media. If you watch TV you’ll be shown apparently objective experts saying that Brexit has been bad for Britain’s economy, and even giving precise numbers for how bad it was. If you believe them this could be a very surprising claim, so let’s dig in.</p><h3>On freedom of movement</h3><p>The case for Britain’s EU membership has always been economic. It was the original pitch that brought Britain into the EU in the 1970s, and it’s why in 2016 the Remain campaign built its strategy entirely around promises of economic armageddon if Britain left. Other factors were hardly mentioned.</p><p>Whilst freedom of movement rules matter a lot to the EU’s ruling class it was never important for Brits as a whole. In the year before the vote more Brits migrated to Australia than any EU country, and emigration had been in long term decline, falling from over 200k/year in 2005 to less than 150k/year in 2015.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/799/0*7kqRk0mVTCT4pcp-.png" /></figure><p>Australia has always been one of the most loved destinations for Brits, despite it requiring a visa. Before the financial crisis using freedom of movement to go to France wasn’t even more popular than going to America, one of the hardest countries to emigrate to.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/409/0*heC3IcistfxYn6hV" /></figure><p>Since losing freedom of movement, EU destinations actually got more popular, not less! That’s driven by pensioners retiring to Spain. But the USA and Australia are still the second and third most popular destinations.</p><p>In other words, British citizens were never put off by visa requirements and so Remain campaigners didn’t focus on them. Even the most fanatical Rejoiners today don’t talk much about freedom of movement, because language barriers make European destinations uninteresting for most Brits (<a href="https://www.euronews.com/2023/11/24/trouble-in-paradise-loneliness-and-isolation-can-await-brits-in-spain">retirees unfortunately congregate in unintegrated ‘ghettos’ where they never learn Spanish</a>).</p><h3>Economics</h3><p>So the debate around whether leaving was good or bad has continued to be about economics. The nice thing about economic claims is that they can be tested using data. And this is where things go off the rails.</p><p>Since leaving there’s been a long string of research papers saying that Brexit was indeed very bad for the British economy, written by people who cast themselves as trusted and neutral advisors. Examples include:</p><ul><li>A team of academics and central bankers who claimed “<a href="https://www.nber.org/system/files/working_papers/w34459/w34459.pdf"><em>Brexit reduced UK GDP by 6% to 8% between 2015 and 2025</em></a>” (<em>Bloom et al</em>). They also say economic predictions made in 2016 were accurate.</li><li><a href="https://www.cnbc.com/2024/02/14/brexit-has-sliced-5percent-off-uk-economic-growth-goldman-sachs-says.html">Goldman Sachs claimed the reduction was 5%</a>.</li><li><a href="https://news.sky.com/story/brexit-impact-on-uk-economy-negative-for-foreseeable-future-bank-of-england-chief-says-13452747">Bank of England chief</a> claimed the impact was “<em>negative for the foreseeable future</em>”.</li><li><a href="https://obr.uk/forecasts-in-depth/the-economy-forecast/brexit-analysis/#assumptions">The British government’s “Office for Budget Responsibility” claimed 4% lower productivity and 15% lower imports/exports “in the long run”</a>.</li><li><a href="https://www.london.gov.uk/new-report-reveals-uk-economy-almost-ps140billion-smaller-because-brexit">Sadiq Khan (the mayor of London) claimed London’s economy had shrunk by £30B</a>.</li></ul><p>But every single claim is false.</p><p>Economic performance is measured using GDP growth, so let’s look at it:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/964/0*VK3qy9MWZauudZzD.png" /><figcaption>Source: OECD</figcaption></figure><p>Britain is the red line in the middle. Why these countries? It’s just <a href="https://grokipedia.com/page/G7">the G7</a>, so I can’t be accused of cherry picking.</p><p>It’s easy to see that there’s no divergence from the other west European countries that stayed inside the EU. Britain actually outperformed. If Brexit had really reduced economic growth you would be able to see it. France, Germany and Spain would continue along their previous paths whilst Britain dived, but that never happened.</p><p>All you can see in the data is the huge impact of the COVID lockdowns. Brexit isn’t visible.</p><p>OK, but what about trade? Being out was <em>guaranteed</em> to reduce sales of British stuff to the Europeans, making Brits poorer. Economic laws of physics, right?</p><p>Nope.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NjYGV4H4XadMvfvSErdfug.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yLKb_2sNQiUgwxewngGK3Q.png" /></figure><p>Goods exports from the UK have been dropping in general since around 2019, but there was no EU specific effect. That’s because the UK is, for better or worse, a deindustrialized services economy (the decline in goods exports is due to a huge rise in electricity prices). Services exports were hit hard by lockdowns but caught up, and total exports to the EU are now higher than they were in 2016.</p><p>The above evidence is definitive, but as a final point, DHL — a neutral third party — maintains a ranking of the world’s most trade connected economies. Britain is still near the top, in about the same place it always was.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JOUf9EvGjZGQPJmd4y0P1Q.png" /><figcaption>Source: <a href="https://www.dhl.com/content/dam/dhl/global/delivered/documents/pdf/dhl-global-connectedness-report-2024-key-highlights.pdf">DHL Global Connectedness Report 2024</a></figcaption></figure><h3>People being economicsal with the truth</h3><p>But then how do so many institutions claim there was a huge penalty to Britain, and even put numbers on exactly how big?</p><p>The EU project has been deeply unpopular in Britain for a long time. For it to work despite that, there must be a terrible cost to defying the European vision, one big enough to overrule people’s natural desire for independence and local control. This is why the supposedly “pro” EU case was entirely negative i.e. <em>don’t dare leave or we’ll make you regret it</em>, and became quickly branded as Project Fear.</p><p>This tactic is far from useless! Fear works:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*sg8ktDiQ0ACP0ttHvSeNJg.png" /><figcaption><a href="https://dominiccummings.com/2017/01/09/on-the-referendum-21-branching-histories-of-the-2016-referendum-and-the-frogs-before-the-storm-2/">Source: Private ICM polling by Vote Leave</a></figcaption></figure><p>A full <strong>15% of the vote</strong> came from people who wanted to leave the EU but were kept down by threats of economic retaliation. Fifteen percent is huge. If someone with a time machine had gone back to 2016 from today and shown them the graphs I showed you above, Leave would have won by a crushing margin because what those 15% feared never happened. The threats were manufactured.</p><p>I think the 30% pro-EU die hards now worry that if the wider public became aware of how little Britain suffered for leaving, it would not only kill any chance of ever rejoining but also strengthen anti-EU parties across the continent. So since 2016 they organically settled on a two pillar strategy for hiding what happened:</p><ol><li>Slip claims of economic damage into discussions without bothering to justify them. The goal seems to be psychological. Doing this implies their statements are so widely accepted and so obviously true that they don’t need to provide evidence.</li><li>Comparing real British growth to imaginary scenarios that could have never happened, countries that aren’t in the EU, or to countries that don’t exist. Then they compare their fantasy Britain to the real thing and describe the gap as damage.</li></ol><p>Here’s the list of papers again, this time with what each one did:</p><ul><li><strong>NBER/Bloom:</strong> compared the UK to a complicated synthetic fantasy country that’s <a href="https://marginalrevolution.com/marginalrevolution/2025/11/was-brexit-worse-than-we-had-thought.html?commentID=160984547">actually 60% the USA</a> (<a href="https://x.com/julianHjessop/status/1992564775797059842">other problems described here</a>).</li><li><strong>Goldman Sachs:</strong> compared the UK to another imaginary country, but they don’t reveal what it is. They also just lie, claiming trade has “declined significantly” since 2016 when it’s actually gone up.</li><li><strong>Bank of England chief Andrew Bailey</strong>: after stressing his neutrality and lack of bias he says the impact was negative without justifying his belief. This is an example of the “it’s so obvious I shouldn’t need to explain” strategy in action.</li><li><strong>Office of Budget Responsibility:</strong> their calculations assume that leaving causes the technology used by British workers to physically degrade, almost like traveling backwards in time. Obviously nothing like that happened in reality and <a href="https://www.briefingsforbritain.co.uk/myths-about-the-long-term-impact-of-brexit/">there’s no theoretical or empirical justification for believing this</a> either.</li><li><strong>Sadiq Khan:</strong> purchased a report that compared growth to <a href="https://www.camecon.com/hubfs/GLA_Impacts-of-Brexit-report_Final.pdf">another fictional scenario produced by a model</a>. Khan then cited it as proof that “London’s economy has shrunk”.</li></ul><p>Many of these claims assume Britain would have had a sudden giant leap in economic growth after 2016, far beyond anything seen in western Europe, all the way up to US levels. There’s just no basis for believing that. Most US growth comes from the tech industry and the UK has no equivalent of Google or Nvidia. It’s like someone claiming they lost money because as a teenager they dreamed of being a rock star and then got a normal job.</p><p>It’d be nice to chalk all this up to intellectual disagreements over terminology or techniques, but that doesn’t seem possible. The most generous conclusion is that economists became lost in the weeds. They can’t ask high salaries for looking at ordinary public stats and saying, “looks like nothing happened”. Anyone can do that. Justifying their status requires inventing ever more complicated ways to answer economic questions, which in turn multiplies the chance of making errors. Eventually the assumptions and simplifications stack up so high the results become nonsensical.</p><p>The less generous conclusion is that they know what they’re doing and are deliberately arguing in bad faith. Agendas are everywhere. The authors knew what Sadiq Khan wanted to hear when he paid them to write a study, and one of the NBER authors works for the German central bank. Conflicts of interest like those are common in the anti-Brexit literature.</p><h3>Were there upsides?</h3><p>Some argue that whilst Britain paid no economic price for leaving the EU, it also didn’t gain anything.</p><p>Campaigners set out several goals for leaving. Those goals have mostly been accomplished. The ones that weren’t didn’t fail because Brexit was oversold, but thanks to a political class sufficiently crazy that they preferred self destruction over compliance with voter’s wishes.</p><p>Let’s review <a href="https://grokipedia.com/page/Vote_Leave">the primary justifications</a> for the exit:</p><ol><li>It would “Take Back Control”.</li><li>It would allow for new trade deals.</li><li>It would save money on membership fees.</li><li>It would let the government reduce immigration.</li></ol><p>The first three goals have been accomplished. The EU and its supposed economic hegemony is no longer an excuse politicians can deploy for not passing laws voters want. New trade deals have been signed. And Britain no longer pays membership fees, which would have grown by about 10% since 2015.</p><p>But the last goal was completely failed. Immigration was actually thrown into overdrive. The surface level cause is that the hopelessly pro-EU British Parliament refused to also exit the European Court of Human Rights (ECHR): technically a separate organization, although not so in most voters’ minds. This is what malicious compliance looks like. MPs set the question themselves and when they didn’t get the answer they wanted, they did the minimum required to comply with the specific wording rather than the spirit.</p><p>Because ECHR legislation is so vague the UK has continued to be substantially ruled by activist European judges, especially in the area of immigration. When you see people being arrested for tweets whilst serial killers are released early to make room, the judges below are a big part of why.</p><figure><img alt="A photo of the ECHR judges" src="https://cdn-images-1.medium.com/max/1024/1*_k-XKUrH0e7mtwLGxW3olA.jpeg" /></figure><p>But the big difficulty in leaving the ECHR was always that EU membership required it. Voters were told that they had to accept the rulings of openly activist judges because everything had been linked together whilst they weren’t watching, and so if the judges didn’t get their way the economy would crash.</p><p>With the threat to people’s wellbeing off the table there’s no longer a negative argument for staying in the ECHR. Proponents are reduced to saying the UK should stay in because “everyone else is bar Russia”, which isn’t a compelling argument on the street. With the hard part of the job done, leaving the ECHR would be very easy and have no negative consequences. So the same people who campaigned for Brexit are now campaigning to exit the ECHR as well, without getting much pushback. And this time they are at the head of an entirely new political party, which is both leading the polls and which pro-ECHR MPs are helpfully selecting themselves out of.</p><p>I think eventually Britain will get control of its immigration levels, even though leaving the EU hasn’t been enough. Apparently it also requires replacing the countries entire political class; an appalling failure of democracy that deserves much deeper study.</p><h3>Conclusion</h3><p>Brits were told that whilst leaving the EU would enable several new politically popular options, it would come with huge economic costs. It did enable those options, and the costs turned out to not be real. Whilst the relative importance of costs and benefits can be debated, if the costs are zero the debate leaves the realm of the subjective and becomes objective. Getting more options for free is <em>always</em> good.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e95fa713d168" width="1" height="1" alt=""><hr><p><a href="https://blog.plan99.net/brexit-was-objectively-good-for-britain-e95fa713d168">Brexit was objectively good for Britain</a> was originally published in <a href="https://blog.plan99.net">Mike’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why did the über-protocols fail?]]></title>
            <link>https://medium.com/@octskyward/why-did-the-%C3%BCber-protocols-fail-e7f10b4a996a?source=rss-3748c5d6e52c------2</link>
            <guid isPermaLink="false">https://medium.com/p/e7f10b4a996a</guid>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[computer-history]]></category>
            <dc:creator><![CDATA[Mike Hearn]]></dc:creator>
            <pubDate>Sun, 05 Oct 2025 18:06:51 GMT</pubDate>
            <atom:updated>2025-10-05T18:06:51.214Z</atom:updated>
            <content:encoded><![CDATA[<h4>An obituary for object oriented RPC</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qAMx_5oUrxvYGzPRp0nd8A.jpeg" /></figure><p>Someone recently asked the greybeards over at Hacker News,</p><blockquote><a href="https://news.ycombinator.com/item?id=45464360">Why did COM/SOAP/other protocols fail?</a></blockquote><blockquote>With the recent buzz around MCP, it made me think about what I’ve read about other unifying protocol attempts in the past. Why did these 2000s era interoperability protocols fail, and what does MCP do different? Was it a matter of security issues in a newly networked world? A matter of bad design? A matter of being too calcified? I would love to hear from those who were around that time.</blockquote><p>I have the dubious distinction of being one of very few people in the world who has actually worked on an implementation of DCOM. It’s one of the many bits of legacy tech cluttering up my brain alongside Object Pascal, Perl, how to write SoundBlaster drivers and other stuff I’ll hopefully one day finally forget.</p><p>Dear reader, even assuming you are an experienced programmer you have most likely never heard of DCOM, and might not even have heard of SOAP. This brief oral history is for your benefit.</p><h3>What are we talking about again?</h3><p>The questioner on Hacker News conflated (D)COM and SOAP together. But these are very different protocols from different eras, and which became legacy tech for reasons that are also different (yet related). So we have to treat them separately.</p><p>We can split the history of network protocols into three rough eras:</p><ol><li><strong>The 1980s.</strong> Birth of the internet. The operating system gives you a byte stream or maybe a message pipe. Everything else is left to the developer. Both binary and line-based protocols are popular. Every protocol reinvents basics like framing (how to tell where a message begins and ends), sequencing (who speaks and when), authentication and every other detail as well. Nothing is encrypted.</li><li><strong>The 1990s.</strong> Object oriented programming was a new concept and becoming wildly popular. Some tech companies observed that protocol development involved a lot of repetitive work. The solution seemed obvious: make object oriented programming function over the network! How hard could it possibly be? <br>In 1989 eleven companies formed the Object Management Group, which defined a new specification called <a href="https://en.wikipedia.org/wiki/Common_Object_Request_Broker_Architecture">CORBA</a>. In 1995 Microsoft developed its own competitor to CORBA called DCOM. Sun both developed CORBA and also its own Java-specific equivalent called RMI. These object-oriented network protocols achieved some brief success and DCOM lives on in factory automation, but they died out in the early 2000s and ceased to be used for new development. Actually they were never that popular to begin with. Meanwhile the web is taking off and HTTP — a classic 80s era line oriented design— becomes the first network protocol to gain encryption thanks to Netscape’s efforts on SSL.</li><li><strong>The 2000s.</strong> OOP is out, Web is in. An ecosystem is flourishing around HTTP that solves many problems the OOP protocols ignored. In 1998 Dave Winer introduces XML-RPC which uses web-style protocols to implement simple stateless function calls. Once again, tech companies notice that protocol development is awkward and time consuming, and once again they form a standards committee to try and “improve” XML-RPC. The result is SOAP, which — once again — achieves some minor success before being rapidly supplanted by …</li><li><strong>The 2010s.</strong> Ad-hoc protocols using JSON over HTTP. What additional abstraction exists here is simply adding on this basis e.g. OpenAPI, JSON schema.</li></ol><p>Why has it proven so hard for the industry to level-up over simple text based protocols?</p><h3>OOP RPC review</h3><p>Very few people still in the industry have used 1990s-era OOP RPC protocols. They all looked pretty similar, so let’s quickly review how they worked.</p><ul><li><strong>They were fully stateful.</strong> A server was a process that “exported” classes via some mechanism. A client “activated” an instance of a class via some API, receiving a proxy object sometimes called a stub. Stubs “marshalled” (serialized) function calls to a binary network protocol which would then be “unmarshalled” by a “skeleton” that converted it back to a function call on the right instance of the server object. Some classes were singletons and others allowed clients to create instances.</li><li><strong>There is no client/server distinction</strong>, because, being OOP, they not only allowed servers to send pointers to clients but also for clients to allocate objects and pass pointers back to the server. This is how “server push” (callbacks) worked. A graph of objects is built up that spans both sides of a network connection.</li><li><strong>The network protocols were not designed to be implemented directly. </strong>They were details hidden behind libraries.</li><li><strong>They were strongly typed.</strong> DCOM had the best support for dynamically typed languages thanks to Microsoft’s BASIC heritage, but the basic assumption of all these systems is that there is some sort of shared multi-language type system in place.</li></ul><p>On top of this basic concept DCOM, CORBA and RMI built up layers of features. Maybe you want security to control who can invoke functions on your objects? They have a spec for that. How do you express what functions are available? They have a standard interface definition language. What if you want a database transaction to be coordinated across different servers? They do that too.</p><h3>The Design By Committee Era</h3><p>It’s important to stress that everything about these systems would seem completely alien to a modern developer, both technically and culturally.</p><p>For example, you might be thinking the above sounds pretty neat and wonder whether it might be fun to implement those protocols yourself. The answer is no because OOP RPC protocols weren’t designed to be directly used by normal developers. In fact the DCOM wire protocol was a trade secret of Microsoft for many years. Using them meant relying on vendor API implementations. CORBA started out life with no network protocol at all, leaving it all up to the vendors. Eventually it gained a standardized protocol (IIOP) but it was designed as a way to connect different implementations written by BigTech together. The assumption was only a few people would ever implement IIOP and everyone else would access it via libraries, so being spec compliant was very hard. It was closer to modern HTML than HTTP, in the sense that whilst you can theoretically implement it yourself, it would take a lot of effort.</p><p>This happened because these systems were all designed to sell “middleware” (libraries, operating systems or languages), and so the designers solved lots of problems they imagined people <em>might</em> have, without actually having those problems themselves. This shows up in a lot of ways and it’s the root cause for HTTP+JSON outcompeting them. You don’t see this approach much anymore in the tech industry because we learned from this experience! Standards nowadays tend to be smaller and adapted from something a tech company developed to meet a real, concrete need they had.</p><p>SOAP wasn’t <em>entirely</em> like this, but it arrived at the end of this era and inherited the mindset. SOAP was complex committee-ware in which the authors assumed you’d use an implementation from your language’s standard library, written for you by a big tech company. Maybe those languages and stdlibs would be free-as-in-beer (e.g. Java) but there’d still be a big team backing it.</p><p>The problem with this is obvious, but only in hindsight:</p><ol><li>Only Java and .NET were big enough to get really complete implementations of the specs.</li><li>Sun and Microsoft were competitors so neither did proper interop testing against the other’s implementation because in their vision everyone would use their stack for everything. So there were lots of interop bugs in SOAP.</li><li>In the early 2000s there was an explosion of interest in dynamic scripting languages like Python/Ruby. These languages had standard libraries cobbled together out of little bits of code written by volunteers as they worked, which is a bad fit for any tech that has specs running to hundreds of pages with hundreds more being added every year by BigTech employees.</li></ol><h3>OOP RPC technical problems</h3><p>OOP RPC wasn’t just sunk by its complexity. There were numerous design failures too.</p><p>A standard critique of RPC is that a remote function call isn’t like a local function call because of latency and network errors. But this critique is off base. It wasn’t a big problem in reality.</p><p>COM was originally built as a local RPC system. Just because there’s no network cable involved doesn’t mean local RPCs are guaranteed to work. The local server can crash, hang, time out, be quit by the user or not be installed properly when you try to connect to it. DCOM can express all these things because in COM every function call returns an HRESULT, a semi-standardized error code. The actual result of a function call is returned in an “out parameter”. In other words, remote interfaces weren’t expected to be identical to local interfaces. Bindings can hide the distinction by throwing exceptions if they want, but if not nobody was surprised because returning an error code for every function is idiomatic in Windows programming.</p><p>By far the biggest <em>real</em> problem was the failure to predict the success or scale of the internet. The designers of OOP RPC were working in the early 1990s and thought on LAN scale. An application, they imagined, is a soup of objects calling each other in complex ways over a small flat network. There’s an ambient assumption at every point that networks are tiny, simple and managed by an IT department.</p><p>This shows up in the following problems.</p><p><strong>No way to deploy UI logic.</strong> OOP RPC protocols had no idea how humans would actually <em>use</em> the objects they exported. That question was viewed as totally out of scope!</p><p>Microsoft’s original solution for network deployment of GUI logic was copying an EXE to a shared Windows drive, a technology that only worked over a LAN.</p><p>HTTP had an answer that worked over the internet: you ship a UI expressed in HTML, a language for expressing GUIs that create more HTTP requests. Today products like my own companies’ <a href="https://hydraulic.dev/">Hydraulic Conveyor</a> make deploying desktop apps easy, but back when this stuff was playing out distributing GUI logic to clients was capital H <em>hard. </em>Microsoft and Apple were children of the 80s and thought of shipping software in the literal sense of putting things on container ships. Their whole culture was oriented around the logistics of manufacturing compact discs and delivering them to retail outlets on the street.</p><p><strong>No load balancers or proxies.</strong> <a href="https://my.f5.com/manage/s/article/K838">Load balancing DCOM is impossible</a>, that’s according to a company that sells load balancers.</p><p>Activating a COM object over the internet means passing a domain name to the Win32 CoCreateInstanceEx function, which expects the IP address to mean <em>one specific Windows computer</em>. The entire concept assumes that connecting to the same IP address a second time will reach the same physical computer. What if you want a domain name to represent an entire datacenter, or global network of datacenters? There was no support for this; the designers simply never imagined it.</p><p>Nor is it even clear how you’d implement such proxies! Everything is a pointer to a piece of memory and a thread somewhere, and the protocols are so complex they don’t have any ecosystem of tools like nginx around them. With DCOM you couldn’t even write an nginx equivalent due to the highly complex and closed protocol.</p><p><strong>No firewall support.</strong> Because the designers assumed network = LAN they had no notion that network connections might be one way only. Passing an object from client to server so you can receive callbacks is idiomatic in OOP RPC, but <em>the server will try to connect to the client</em> to deliver the callback. It won’t reuse the existing TCP connection established by the client. This is a natural consequence of not having any “server” concept in the design to begin with. Thus if the initiator is behind a firewall, it simply breaks.</p><p><strong>No way to stop DoS attacks.</strong> Modern web apps are protected by many layers of defenses capable of stopping even very sophisticated DoS attacks. It’s an essential part of surviving as an online business. Because OOP RPC was imagined for the LAN, there is no way to make such a service DoS proof. Objects live in RAM so all a client has to do is connect once and allocate objects until the server runs out of memory. And because there’s no way to deploy UI logic you can’t solve attacks by quickly deploying CAPTCHAs or making other service modifications.</p><p><strong>No good way to authenticate users.</strong> OOP RPC protocols did integrate authentication, but because they were designed by operating system vendors for the LAN their core concept was to propagate your local operating system username to the server cryptographically (e.g. using Kerberos or Active Directory). So again there’s an ambient assumption of small scale and carefully managed networks.</p><p><strong>No versioning support.</strong> If you wanted to add methods to an object in COM you had to create a new interface (objects can export multiple interfaces). But there was no way to change an existing interface <em>nor any of the types it used</em>, so COM based systems are full of interfaces with names like IFooBar3 . This was a poor fit for the rapidly evolving world of online business. Having very restricted ability to evolve your API in a backwards compatible way just wasn’t workable in a world where apps were expected to remain online 24/7.</p><p><strong>No caching support.</strong> Etcera etcetera. You’re getting the picture.</p><p>There were more problems than these, but I left the best till last: many companies <em>don’t want</em> to expose a general API to their service. This can be for branding, business or strategy reasons. HTTP was never designed for APIs to begin with so it gives the client nearly no power out of the box. You can invoke private application endpoints yourself but don’t be surprised if you get redirected to a CAPTCHA page — after all, HTTP never promised you’d get actual <em>data</em> back from a request!</p><h3>MCP</h3><p>How does the Model Context Protocol relate to all of this?</p><p>Well, arguably it doesn’t. MCP is the latest in a long line of works that adds thin extensions on top of the HTTP layer cake. In a hypothetical world where OOP RPC won you could have had an MCP on top of it too, and it might even have had some advantages. But we don’t live in that world.</p><h3>Conclusion</h3><p>The HTTP ecosystem provides sooooo many features that we take for granted. Could OOP RPC have <em>ever</em> worked?</p><p>I suspect the answer is yes, it could, but it would have looked very different to DCOM or CORBA. Many of the problems described above could have been avoided with more insightful protocol design. The core problem of stateful pointers is also not insurmountable: nothing says an object has to be loaded into RAM at all times, nor that it has to be tied to a physical IP address, nor that remote clients have to be able to create objects whenever they want.</p><p>An OOP RPC protocol that worked would have looked more like a standard way to encode references to “objects” stored in a database, with a way to reflect available methods and properties, pass references into and out of method calls, and navigate object hierarchies in a pipelined manner. This sort of tech might well still have failed, as most apps don’t actually want to give any flexibility to the clients and the whole point of exposing general object-oriented APIs is to give flexibility. But nonetheless, we’re slowly iterating towards this kind of vision with things like GraphQL, providing separate data access APIs and now also with MCP.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e7f10b4a996a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why not object capability languages?]]></title>
            <link>https://blog.plan99.net/why-not-capability-languages-a8e6cbdf9682?source=rss-3748c5d6e52c------2</link>
            <guid isPermaLink="false">https://medium.com/p/a8e6cbdf9682</guid>
            <category><![CDATA[java]]></category>
            <category><![CDATA[object-capabilities]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[chrome]]></category>
            <category><![CDATA[security]]></category>
            <dc:creator><![CDATA[Mike Hearn]]></dc:creator>
            <pubDate>Sun, 11 May 2025 18:54:04 GMT</pubDate>
            <atom:updated>2025-05-12T09:29:10.381Z</atom:updated>
            <content:encoded><![CDATA[<h4>A map of the security territory</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7NL7G-ANcOu3xAa2_vm2-Q.jpeg" /><figcaption>Here be dragons!</figcaption></figure><p>Supply chain attacks on the software industry are becoming common. The apps we write have very long and deep dependency chains, and any component that’s subverted by an adversary can do anything the app itself may do. As many widely used components are authored pseudo-anonymously on laptops that are easily hacked, we’re kinda asking for trouble. It’s only surprising it took this long.</p><p>Maybe the solution lies in sandboxes with <a href="https://en.wikipedia.org/wiki/Capability-based_security"><em>capabilities</em></a><em>:</em> opaque objects that encapsulate the ability to do something. If your program hasn’t been given one then you aren’t able to do the action it protects. Think of them like magical keys: you can’t forge them, but you can clone them and (in some systems) reduce the power of the clone whilst you do.</p><p>We already regularly use capabilities: handles, file descriptors and JWT tokens are all examples under different names, but we don’t use them within individual programs. Changing that is a popular idea, especially on Hacker News. It was first proposed by Morris in the 1970s, and <a href="https://news.ycombinator.com/item?id=43936830">here it is again a few days ago</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cw8qczzN5QHQ4IUKbsCscg.png" /></figure><p>What would it take to make this finally real? Do we really need <em>yet another</em> whole new programming and library ecosystem?</p><p>In this essay I want to show you the challenges that you’ll face if you want to walk that path. This isn’t meant to put anyone off, just to draw a map of the territory you’re about to enter and explain why it’s currently deserted. As we go I’ll introduce two very different takes on the concept: the history of capabilities in Java, and Chrome’s Mojo object-capability system.</p><h3>Forks in the road</h3><p>Here are some problems you’ll have to solve in order to sandbox libraries:</p><ol><li>What is your threat model?</li><li>How do you stop components tampering with each other’s memory?</li><li>Do you inter-twingle your plan with other unrelated requirements, like cross-language interop?</li><li>How much of humanities existing codebase can you reuse, if any?</li></ol><p>Let’s start with the threat model, or stated more plainly, what <em>exact</em> problem are you trying to solve? Sandbox designers often disagree on how far they need to go. Do you care about resource exhaustion or DoS attacks? For example, is it OK if a library can call exit(0) and tear down your process, or deliberately segfault it? Do you care about Spectre attacks? Maybe!</p><p>Every time you make your threat model stronger it becomes harder to work with. The right thing to do here is deeply unclear.</p><p>Regardless of what model you choose, libraries expect to share memory with the code that uses them. An object capability system therefore can’t just be dropped in to any old language! It must be carefully designed to provide what the Java team calls integrity— one bit of code must not be able to overwrite state depended on by another. Guaranteeing this property turns out to be hard, and in fact very few languages even attempt to provide it. C/C++ don’t. Dynamic scripting languages like Python or Ruby don’t, because you can actually change the rules of the language on the fly by monkey-patching things in the standard library. JavaScript also doesn’t really support it: Google sponsored a research project called Caja for a while that was able to rewrite Javascript into a form that was claimed to have integrity, but <a href="https://github.com/googlearchive/caja">it had a lot of exploits and they eventually gave up</a>.</p><h3>A mainstream object capability language</h3><p>The requirements can be best understood by reading a paper from 2004 describing a language called <a href="https://people.eecs.berkeley.edu/~daw/papers/joe-e-ndss10.pdf">Joe-E</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/703/0*L5dy1TgXL-RRQph8" /></figure><p>The Joe-E language is a subset of Java designed to support pure capability programming within a shared address space. Java has the basics already: there’s no pointer arithmetic, you can’t monkey patch things, the language rules are fixed, and objects have enforced visibility rules. By looking at what Joe-E takes away from Java, we can see what more is needed.</p><p>Some things you have to remove are obvious enough:</p><ol><li>Reflection that can write to private fields or call private methods.</li><li>Global methods in the standard library that grant ambient authority.</li><li>Native methods.</li></ol><p>And some are less so:</p><ol><li>Any mutable global variable is a problem as it may allow one component to violate expectations held by another. Joe-E forbids mutable global state.</li><li>Exceptions might leak capabilities and are restricted.</li><li>Finalizers are removed.</li><li>The finally keyword is removed.</li></ol><p>That’s a pretty radical departure from regular Java, and is why Joe-E is considered a subset language. You can’t just take an existing Java library and use it from Joe-E code. The biggest difficulty by far is what they called “taming” the standard library, meaning finding all the places where ambient authority is granted and blocking them.</p><p>It’s worth noting that although Joe-E was a research project, some of these changes are being adopted into Java proper for apps that use the module system:</p><ul><li>You’ll soon be required to specify which modules can use native code.</li><li>You’re already required to specify which modules can use ‘deep’ reflection.</li><li>Finalizers have already been removed.</li></ul><p>By itself this isn’t enough for a pure capability language, and the removal of the SecurityManager means there’s no official way to “tame” the standard library, so by itself those changes doesn’t help much. But it’s an interesting direction of travel.</p><h3>The God Object</h3><p>If you think through the consequences of the pure capability design some usability issues become apparent.</p><p>The first is that if you want the entire application to be written in an object-capability style then its main() method must be given a “god object” exposing <em>all</em> the ambient authorities the app begins with. You then have to write proxy objects that restrict access to this god object whilst implementing standard interfaces. No language has such an object or such interfaces in its standard library, and in fact “god objects” are viewed as violating good object oriented design.</p><p>With no ambient authority, every resource a library or its dependencies <em>might</em> need to use has to be passed in by the caller. And therefore, changing what permissions are needed implies a backwards-incompatible change to the API itself.</p><p>This is a problem because many users of a library won’t care about sandboxing. They may have good reasons to not care — perhaps they trust you because of who you work for, or their data isn’t sensitive, or they use some other security mechanism, or they just want to get their job done ASAP. If you constantly stop their code compiling with API changes that don’t benefit them, they’re going to get upset and switch to a competing library. How often you need to request new permissions will depend heavily on the design of your language and standard library, so be careful. For example, a library reading a new data file it ships with shouldn’t require the user to supply new permissions (Java libraries are zips that can contain data files, so this is handled there).</p><p>Another problem is configurability. It’s common for libraries to read global state, for instance, environment variables that turn on logging. None of this is allowed in a capability system: the caller of the code has to read environment variables and config files for it.</p><h3>The God Of the Stacks</h3><p>The God Object problem is why the Java SecurityManager architecture wasn’t a pure capability system. It <em>enabled</em> capabilities, but it didn’t require them. In the SecurityManager design all code starts with ambient authority. Each method is associated with a module, and modules can have ambient permissions taken away, leaving them either unable to do that operation or only able to do it if they’re given an object capability.</p><p>It works like this: when one module calls into another that is guarding some possibly sensitive operation (e.g. the standard library) the guard does a stack walk to look at all the code calling into it, intersecting their privileges. This design had at least three advantages:</p><ol><li>People who didn’t care about sandboxing could ignore it.</li><li>The needed permissions could be described with a simple config file instead of code or docs. This made it easy to ship canned profiles that showed developers what permissions a library might need in different situations, and then let them customize it to minimize privilege.</li><li>You could create object capabilities by defining classes in privileged modules with a safe interface and then “asserting privilege”, which truncated the stack walk at that frame.</li></ol><p>The design was elegant and theoretically sound, but plagued with practical usability problems and bugs. One issue was the baffling number of ambient permissions a program actually has. A quick look at <a href="https://devdocs.io/openjdk~21/java.base/java/net/netpermission">the NetPermission class</a> — one of many — reveals stuff like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7WooHUJmGgE7Fqf6VTPPUg.png" /></figure><p>Did you <em>ever</em> think about <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Methods/TRACE">the HTTP TRACE method</a>? Probably not, but the Java sandbox designers did because doing a TRACE request is potentially harmful and thus requires a permission.</p><p>Most languages let you do HTTP requests. In a system with only object capabilities, how would you restrict access to TRACE whilst allowing GET on a domain? This becomes especially tricky when the authors and users of a library are separated. The user must supply the library with an HTTP client, and that client has to be either configurable or proxyable in order to turn it into an object capability, and the user must understand exactly what’s required by the library. Type systems don’t help: you can’t express a type like “HTTP client that allows access to example.com but nowhere else” (ok maybe in TypeScript, but TypeScript doesn’t have a VM that enforces its rules so that’s useless for security).</p><p>The SecurityManager design solved all of this. If a library needed access to a specific website it could just come with a config file that listed that domain in a NetPermission, the user could quickly audit it before using the library, the library would request an HTTP client purely internally and everything just worked. If the overhead of sandboxing wasn’t wanted it could be eliminated by the JIT compiler.</p><p>So why did the SecurityManager die then, just as supply chain attacks were starting to become real? You can read <a href="https://openjdk.org/jeps/411">the official rationale</a> or my summary of it:</p><ol><li>Java has a lot of features.</li><li>Maintaining permission checks for all those features was expensive.</li><li>It was hard to avoid exploits.</li><li>Nobody cared enough about supply chain attacks to use sandboxing anyway.</li><li>Even the few who did it use were using it wrong.</li></ol><p>Well, an expensive thing that breaks a lot and nobody uses is an obvious target for cost savings… and there it is. Even with the usability advantages it gave over pure capability systems, in-process sandboxing was still too much effort for developers to bother with. Supply chain attacks are still rare enough that defensive measures just aren’t at the top of most backlogs and probably won’t be for some time.</p><h3>Spectre</h3><p>There’s another reason the SecurityManager was abandoned. Speculative execution attacks allow any code to read the entire address space without any permissions, even in the face of languages designed to stop that. Hacks like reducing the resolution of timers help but don’t entirely fix it, and come at great cost. At this point the designers of most software sandboxes cried out in despair and just gave up, declaring that only hardware-enforced address spaces could properly constrain software components.</p><p>I personally think this belief isn’t quite true. Reading data is only useful in real-world attacks if you can exfiltrate it somewhere, or if you can steal a credential that lets you escalate your privilege. Many analyses of software security stop and declare victory when any bytes have been read at all, without stopping to ponder where those bytes will go next. That isn’t totally unreasonable because in the web context the ability to exfiltrate data is an ambient permission, but if we’re talking about sandboxing libraries then it’s a very different matter. Most libraries don’t need permissions to do write IO, and so even if they somehow did a Spectre attack undetected (hard) they couldn’t easily send the data they’d read back to the adversary.</p><h3>That’s Chromey, baby!</h3><p>Still. How about hardware isolation? File descriptors are a kind of capability provided by the kernel, but a rather odd and inflexible kind. They aren’t a great example of object capabilities.</p><p>Windows does slightly better, with its kernel’s notion of the ‘object manager’ and a more generalized notion of handles.</p><p>But the best example is Chrome.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bClntez1U3Ca7IALUYa6Hg.png" /></figure><p>A running Chrome is a local network of microservices communicating via a protocol called Mojo. <a href="https://source.chromium.org/chromium/chromium/src/+/main:docs/mojo_and_services.md">Mojo is quite interesting</a>. It lets you bind objects to “message pipes” that connect processes, which turns them into object capabilities. Pipes are unforgeable and can be sent through other pipes, allowing an object in one process that has a certain set of kernel permissions to be exposed to other processes that have fewer permissions. A running Chrome is a whole graph of objects pointing between services. Mojo abstracts object location (services can run in separate processes or in-process without code changes) and also offers optimizations like shared memory. The Chrome sandbox then lowers the ambient authority of each process to the minimum needed for its task. Objects are moved into sandboxes with a simple annotation.</p><p>Whilst it sounds easy, there’s at least four catches. Firstly, given the importance of kernel sandboxing to the web and how long browsers have used it, you might imagine that it’s backed by a mature set of powerful APIs. But you’d be wrong. Not one OS has a sandboxing API that is all of documented, stable and usefully powerful. Nor are the approaches used by Linux, macOS and Windows even slightly similar. So how does Chrome do it? The answer is lots of reverse engineering of OS internal APIs, a willingness to react fast to incompatible OS upgrades, being too big to fail, and finally, lots of very platform specific code with not much abstraction.</p><p>The second catch is that Mojo isn’t easily reused — there’s no standalone version you can download for your own projects. Nonetheless, if you want an object capability system that blocks speculation attacks, the Chrome codebase is a good place to start. It also comes with a large library of code designed to run without ambient authority, albeit all written in C++ and without stable APIs.</p><p>The third catch is that Mojo is really an RPC system. Services can share memory but only in large blocks, and only very carefully. It doesn’t offer any specific support for managing object lifetimes across processes. Memory management protocols must be carefully designed to avoid inter-process resource leaks or refcount cycles. The fact that there’s a security boundary in place requires constant attention whilst designing your code.</p><p>The final catch is that context switches have a high cost, especially on Windows. Can we reduce the overhead? CPUs that have support for memory protection keys (MPKs) can, with enough work, switch memory protection contexts much faster than a typical context switch would. This requires careful validation of all native code (including JIT compiled code) to ensure it doesn’t contain writes to the MPK register, but it can be done.</p><h3>Conclusion</h3><p>Object capability systems offer the promise of sandboxed libraries and a big advance in the war against supply chain attacks, but they’re hard to build. The best example of research into making a mainstream pure object capability language is Joe-E, a long since defunct subset of Java. The best example of a real-world object capability system is Chrome.</p><p>How to make library sandboxing easy enough that people actually adopt it is still an open question. I have some ideas, but none are fleshed out enough to implement today. Maybe I’ll write about that another day. Until then, I hope this quick tour of the territory was useful!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a8e6cbdf9682" width="1" height="1" alt=""><hr><p><a href="https://blog.plan99.net/why-not-capability-languages-a8e6cbdf9682">Why not object capability languages?</a> was originally published in <a href="https://blog.plan99.net">Mike’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Replication studies can’t fix science]]></title>
            <link>https://blog.plan99.net/replication-studies-cant-fix-science-0e195234a280?source=rss-3748c5d6e52c------2</link>
            <guid isPermaLink="false">https://medium.com/p/0e195234a280</guid>
            <category><![CDATA[academia]]></category>
            <category><![CDATA[replication]]></category>
            <category><![CDATA[science]]></category>
            <category><![CDATA[research]]></category>
            <dc:creator><![CDATA[Mike Hearn]]></dc:creator>
            <pubDate>Sun, 06 Oct 2024 10:44:11 GMT</pubDate>
            <atom:updated>2025-07-01T10:39:19.872Z</atom:updated>
            <content:encoded><![CDATA[<h4>They barely even begin to address the problem</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OXoiMJz4r5d8k7BOZI8Fpw.jpeg" /></figure><p>Last month the journal <em>Science</em> published an article titled <a href="https://www.science.org/content/blog-post/fraud-so-much-fraud">“Fraud, so much fraud”</a>. It discusses their exposé of Eliezer Masliah, an Alzheimer&#39;s scientist and US government official who has been accused of systematically faking over 25 years worth of research. Such scandals are now common, like <a href="https://www.nbcnews.com/science/science-news/allegations-research-misconduct-roil-dana-farber-cancer-institute-rcna135521">this one at a big cancer research center</a> and <a href="https://www.cnbc.com/2022/07/26/allegations-of-fabricated-research-undermine-key-alzheimers-theory.html">this one at another Alzheimers lab</a>.</p><p>The problem was <a href="https://news.ycombinator.com/item?id=41672599">discussed on Hacker News</a>, where people exchanged ideas for how to improve the situation. In these debates someone will usually suggest an intuitive sounding fix: the government should fund scientists to repeat other people’s experiments, just like it funds any other kind of study. For example,</p><blockquote>“We can solve this at the grant level. Stipulate that for every new paper a group publishes from a grant, that group must also publish a replication of an existing finding.”</blockquote><p>This sounds like it should work; the science crisis is usually called the replication crisis, after all. It stands to reason that having an independent team do the same experiment and get the same results should prove the absence of problems with that study.</p><p>This idea <em>is</em> reasonable. Unfortunately, science isn’t.</p><p>In the past eight years or so I’ve read a lot of junk-tier scientific papers. It wasn’t intentional — there’s just so much out there that if you dig into claims found in mass media you’re going to encounter pseudoscience very quickly. I’ve written on this blog about the so-called “questionable research practices” in <a href="https://blog.plan99.net/fake-science-part-i-7e9764571422">medicine</a>, <a href="https://blog.plan99.net/summary-of-methodological-issues-in-epidemiology-4cd14c7b81bd">epidemiology</a> (<a href="https://blog.plan99.net/is-epidemiology-useful-a4ec54e59569">twice</a>), <a href="https://blog.plan99.net/fake-science-part-ii-bots-that-are-not-c66129e5e3f5">“twitter bot studies”</a> (also <a href="https://blog.plan99.net/did-russian-bots-impact-brexit-ad66f08c014a">twice</a>), <a href="https://blog.plan99.net/meta-historical-conspiracies-part-3-808e7d9e385d">archaeology</a> and <a href="https://blog.plan99.net/pseudo-epidemics-7603b2da839">PCR testing</a>. The weird selection of fields just reflects my personal interests — I don’t actually go looking for this stuff.</p><p>Of the bad claims I’ve seen, almost none would have been prevented by a replication study. Some would have been exacerbated. This is a very counter-intuitive claim. How can replication not help, let alone make things worse? Well, there’s a bunch of reasons. Don’t get me wrong: replication isn’t entirely useless, but it can only help rarely and doing more of it could actually pour fuel on the fire.</p><h3>Replicating wrongness = more wrongness</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Llusw_yPHbtsj6eb.jpeg" /></figure><p>The first and biggest problem with replication is that it only helps in one very specific scenario. You need to have a paper which:</p><ol><li>Makes an important, interesting claim that would be Big If True.</li><li>Has a logical, detailed, and scientific methodology clearly derived from the claim.</li><li>Is free of other obvious flaws.</li></ol><p>This is a low bar. Some fields are better at clearing it than others. Computer science usually produces papers that pole-vault over it, thank goodness. But in many other fields the median paper doesn’t even reach the point where replicating it would make sense. Instead these papers:</p><ol><li>Make trivial claims e.g. <a href="https://openurl.ebsco.com/EPDB%3Agcd%3A4%3A12322516/detailv2?sid=ebsco%3Aplink%3Ascholar&amp;id=ebsco%3Agcd%3A18009272&amp;crl=c">the average man would like to be more muscular</a> or <a href="https://www.sciencedirect.com/science/article/abs/pii/S0360132306003672">people choose their clothes based on how warm it is</a> (“<em>It is evident that further studies are needed in this field</em>” 🤮).</li><li>Have nonsensical methodologies like <a href="https://blog.plan99.net/did-russian-bots-impact-brexit-ad66f08c014a">defining anyone who tweets five times after midnight as a “bot”</a>, or which involve ad-hoc criteria presented “just so” instead of a description of how they were derived.</li><li>Are full of obvious errors that render replication pointless. <a href="https://www.nature.com/articles/s41598-021-95699-9#Sec7">This COVID paper</a> has 19 authors yet <a href="https://dailysceptic.org/2021/08/24/examining-the-latest-paper-from-imperial-college-london/">the very first sentence is a false claim about public statistics</a>.</li></ol><h3>Replicating the definition of replicating</h3><p>The second problem is that people don’t agree on what the word replication means. It might seem obvious: surely it’s “we did the same things and got the same results”? But when careers are on the line, people find ways to disagree.</p><p>One case arises when scientists make a claim of the form “we did this thing and that other thing changed in a big way”. Someone tries to replicate this and they find the other thing changed but only by a bit, or they discover the effect fades with time (this is especially common in education studies for some reason).</p><p>Is this a successful replication? Scientists are often tempted to argue it is — after all, the same effect appeared and the natural world is inherently noisy. You’ll never get <em>exactly</em> the same numbers. But often the reason a paper was interesting was its claim of a big and important effect. If it shrinks to the point nobody would care, it’s not really informative to give the paper the “it replicates” stamp of approval. Doing that would just incentivise the exaggeration of real but small effects.</p><p>Another frequent case is where a study fails to replicate but the replicators changed some aspect of the method they felt was unimportant. The original authors point to the difference and say that’s why it didn’t work.</p><p>But the most common replicability problem I’ve seen is that the methodology isn’t actually derived from the hypothesis. The study design itself isn’t replicable. Examples litter the field of misinformation research. Academics in this field like proving claims of the form, “conservatives believe more conspiracy theories than liberals”. But what exactly is a conspiracy theory? Invariably these studies define conspiracy theory as anything in an ad hoc list of items. There’s no explanation of how the list was created and such lists don’t include things believed by liberals, so the whole thing becomes a circular firing squad: conservatives believe conspiracy theories because conspiracy theories are things believed by conservatives. It’s just a form of scientific fraud, so a replication that followed the written methodology would simply perpetuate it and a replication that didn’t would be rejected by the original researchers as illegitimate.</p><h3>The eternal sunshine of the spotless mind</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*SFL16z7vn6tuj25y.jpeg" /></figure><p>A less common but more damning situation occurs when scientists pretend something replicated even though it didn’t.</p><p>In 2020 Neil Ferguson shocked the world with a paper claiming COVID would spiral out of control unless there was an immediate and draconian lockdown. The prediction of viral spread wasn’t from a model formally written down in some published paper: it turned out to exist only in a single unpublished program. He helpfully explained that he never revealed the code before because it was “<em>all in my head, completely undocumented. Nobody would be able to use it</em>”.</p><p>Perhaps another explanation is that the program simply did not work. <a href="https://dailysceptic.org/2020/05/06/code-review-of-fergusons-model/">I wrote an article laying out the gruesome details</a>. Bugs created unintentional non-determinisms: running the simulation on a different computer would calculate totally different results for identical scenarios. It was like if two people opened the same spreadsheet in Excel and saw totally different numbers.</p><p>This case is interesting because it’s a slam-dunk failure to replicate. There is quite simply no wriggle room here: two people solving the same equations with the same inputs should get the same results. If they don’t then something has gone horribly wrong.</p><p>It should have been a massive scandal. It wasn’t because Imperial College London <a href="https://www.imperial.ac.uk/news/197875/codecheck-confirms-reproducibility-covid-19-model-results/">published a press release saying a replication study was done</a> and then sent it to some pet journalists, who promptly reported that the concerns were a false alarm. The <a href="https://zenodo.org/records/3865491/files/codecheck.pdf?download=1">supposed replication</a> starts by saying “<em>I was able to reproduce the results from Report 9</em>”, but then the very same paragraph admits that every number the author got was different, some by up to 25%. He even admitted that one reason for the differences was that “<em>the CovidSim codebase is now deterministic</em>” i.e. he didn’t try and replicate the version he was supposed to be replicating because that version wasn’t replicable.</p><p>In fields like public health the whole concept of replication seems to be barely understood. Circular logic runs rampant. It’s common to conflate replication and validation by claiming that <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3001435/">a model is valid if its outputs roughly match the output of other models</a>. Basics like holding back a test set that are taken for granted in machine learning are controversial in public health (see the section “Comparisons with external data” in that last link).</p><h3>Pushing on wet string</h3><p>The Ferguson case reveals the core problem with funding replication studies. The idea takes as axiomatic that bad scientists are rare and so the chance of a replicator being an honest scientist is high. Yet in reality there are no consequences for bad behavior and so the dishonest rise to the top. You just assert you followed the rules and everyone believes you!</p><p>A fascinating, bizarre and frustrating example of this problem is the recently published paper, “<a href="https://www.nature.com/articles/s41562-023-01749-9"><em>High replicability of newly discovered social-behavioural findings is achievable</em></a>”. They claimed they fixed the replication crisis in social science by using better methods! <strong>Big If True. </strong>The paper has 17 authors, some of whom are involved with the science reform movement or have even accused other scientists of fraud, so this sounded very promising. Surely it will be a seminal paper worth reading?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DNK8NNUUlU8PeGp1LEsy1A.png" /></figure><p>Oh.</p><p>Why was it retracted? Well, because they did things that are known to create non-replicable papers. Specifically, they p-hacked the study by coming up with a different hypothesis after collecting their data, a practice that’s supposed to be stopped by pre-registering their plans. And they did pre-register! But then they “mis-stated” what they put in the pre-registration. Although the whole point of this process is to catch mismatches, nobody at the journal or their universities bothered to check before publication.</p><p>So this is a study arguing for rigorous methods that breaks the rules they’re demanding others follow. Yet somehow, it goes downhill from there, because the original study they planned was an attempt to study paranormal forces<em>. </em>These mini-Mulders hypothesized that maybe social science has poor replicability not because they suck at science but because the act of studying a phenomenon somehow makes it disappear.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IY4an3YrvM31cPFVldPEWw.jpeg" /><figcaption>The hypothesis of this 10 year long study as imagined by Grok.</figcaption></figure><p>Perhaps you think this is so absurd I must be exaggerating for effect, so <a href="https://osf.io/4k5sf">here’s an explanation by some of the authors</a>:</p><blockquote>Other presenters offered unconventional explanations [for observing weaker evidence in replications], such as <strong>the act of observation making effects decline over time</strong>, possibilities that Schooler believed worthy of empirical investigation . . . <strong>and</strong> <strong>others have dismissed as inconsistent with the present understanding of physics</strong>.</blockquote><p>The researchers explain that they knew this was stupid, but went along with it to get money from the ironically named Schooler:</p><blockquote>While the unconventional explanations were not considered plausible (or even possible) to most of the team, they agreed on an approach that included tests of those possibilities</blockquote><p>None of this was discovered by the institutions. Like always in stories of academic malpractice it was some random guy and his <a href="https://joebakcoleman.com/blog/2024/protzko/">long blog post</a>, which describes how hard it was to bring this “utterly batshit supernatural framing” to light. He also describes how everyone seemed to minimize the problems or pretended it’s all an accident (he does not agree it was). <a href="https://statmodeling.stat.columbia.edu/2024/09/26/whats-the-story-behind-that-paper-by-the-center-for-open-science-team-that-just-got-retracted/">Andrew Gelman also has a long writeup of this disaster</a>.</p><h3>Conclusion</h3><p>Any proposal for academia reform that involves paying for replication studies needs to first address at least these problems:</p><ol><li>Many papers don’t make sense to replicate because they’re pointless, can be proven wrong just by reading them or because the derivation of the methodology is itself non-replicable.</li><li>Some fields use weak, nonsensical or circular definitions of “replicate”.</li><li>Some academics are willing to claim things replicate even if they don’t, in order to preserve the influence of their field.</li><li>Scientists who make a big deal out of their high standards might actually not have any.</li></ol><p>What’s at the root of all this? At some point we must look to the incentives as the cause. Science exists in a strange Soviet-esque system in which philanthropists and taxpayers allocate funds into a planned economy. Perhaps <a href="https://slatestarcodex.com/2014/04/28/the-control-group-is-out-of-control/">the odd inability to get rid of paranormal research</a> is unsurprising in that context, as the Soviets were also big into parapsychology, <a href="https://www.newdawnmagazine.com/articles/paranormal-unexplained/the-changing-face-of-russian-psi-research#:~:text=Between%201969%20and%201971,%20during%20the%20height">funding it to the tune of half a billion dollars per year</a>.</p><p>Without practical technological/commercial goals to ground scientific exploration it appears that the latter can easily enter a downwards spiral of ever lower standards, until eventually they just disappear entirely. There’s no One Weird Trick to fix bad incentives; only changing them can. Although it’s tempting to look around for ways to let governments continue spraying universities with cash, it’s worth remembering that we’ve only been mass funding academic science since the end of World War 2. Some of the most productive and innovative periods of history predate the expansion of universities — the Industrial Revolution was powered almost entirely by individual inventors and companies protected by patents. Is it time to revisit this model?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0e195234a280" width="1" height="1" alt=""><hr><p><a href="https://blog.plan99.net/replication-studies-cant-fix-science-0e195234a280">Replication studies can’t fix science</a> was originally published in <a href="https://blog.plan99.net">Mike’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Writing Truly Memory Safe JIT Compilers]]></title>
            <link>https://medium.com/graalvm/writing-truly-memory-safe-jit-compilers-f79ad44558dd?source=rss-3748c5d6e52c------2</link>
            <guid isPermaLink="false">https://medium.com/p/f79ad44558dd</guid>
            <category><![CDATA[security]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Mike Hearn]]></dc:creator>
            <pubDate>Thu, 06 Jun 2024 14:14:30 GMT</pubDate>
            <atom:updated>2024-06-06T14:14:30.996Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to kill off a top source of browser exploits</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KbR4q42CFgw4SkxxD6I5Og.png" /></figure><p>Last month the V8 team published an excellent blog post on what they call <a href="https://v8.dev/blog/sandbox">the V8 Sandbox</a>. This isn’t a sandbox for your JavaScript code — it’s intended to mitigate browser exploits caused by bugs in the JIT compiler itself. That’s important work because they report that most Chrome exploits start with a V8 memory safety bug.</p><p>V8 is written in C++, so it may seem like these are the sort of bugs you’d expect from working in a memory-unsafe language. Unfortunately the situation is more complex. Why? The team explain:</p><blockquote>There is a catch: V8 vulnerabilities are rarely “classic” memory corruption bugs (use-after-frees, out-of-bounds accesses, etc.) but instead subtle logic issues which can in turn be exploited to corrupt memory. As such, existing memory safety solutions are, for the most part, not applicable to V8. In particular, neither <a href="https://www.cisa.gov/resources-tools/resources/case-memory-safe-roadmaps">switching to a memory safe language</a>, such as Rust, nor using current or future hardware memory safety features, such as <a href="https://newsroom.arm.com/memory-safety-arm-memory-tagging-extension">memory tagging</a>, can help with the security challenges faced by V8 today.</blockquote><p>They give an example bug that can cause memory corruption without the engine itself containing any normal memory safety problems, as VM intrinsics or JIT compiled machine code itself may accidentally rely on invalid assumptions about memory.</p><p>It would be nice if there was a rigorous approach to writing language runtimes that eliminated such bugs by design.</p><p>GraalVM has a JavaScript engine called <a href="https://www.graalvm.org/javascript/">GraalJS</a>. It’s written in Java using the <a href="https://www.graalvm.org/latest/graalvm-as-a-platform/language-implementation-framework/">Truffle language framework</a>. Its peak performance is competitive with V8 and on a few benchmarks (such as ray tracing) is actually faster!</p><p>Although being written in Java does improve memory safety, we just saw that rewriting V8 in a safe language wouldn’t help with the types of bugs V8 is trying to solve and so we would intuitively expect that GraalJS must suffer from the same classes of bugs. Yet, it doesn’t. Let’s take a look at why not. Along the way we’ll explore the <em>first Futamura projection, </em>the core theoretical idea underpinning Truffle.</p><p>All fast language VMs work the same way. A program is loaded from disk into in-memory data structures representing the program, either an abstract syntax tree or byte code. The program starts running in an interpreter. Parts are soon discovered to be <em>hot spots</em>, i.e. the program spends much more time there than in other parts. Those hot spots are passed to a just-in-time compiler that converts them to optimized machine code, and execution then jumps back and forth between the interpreter and the collection of compiled program fragments. This gives a big performance boost.</p><p>This architecture is standard — both the JVM and V8 use it — but viewed from a security perspective the design has a flaw: it’s error prone. The language semantics are implemented twice, once for the interpreter and again for the JIT compiler. It’s critical not only that both places are fully correct but also that they exactly match. Otherwise, the VM becomes exploitable.</p><p>Truffle is a Java library that helps you build advanced, high performance language runtimes. VMs built using the Truffle framework operate in a fundamentally different way to conventional VMs, one that not only makes them much easier to write but which also eliminates memory safety bugs by design. It all starts with you writing an interpreter for your language in Java. This doesn’t mean compiling your <em>target</em> language to JVM bytecode — in fact bytecode won’t feature anywhere in this story. You just write an ordinary interpreter. Because the interpreter’s code is garbage collected and bounds-checked, malicious user code can’t use memory safety bugs to exploit it.</p><p>If you think about conventional Java then this may sound quite slow — isn’t Java itself interpreted until it gets JIT compiled? Are we … interpreting an interpreter? Fortunately not because you can ship your Truffle-based language runtime as a <a href="https://www.graalvm.org/latest/reference-manual/native-image/">native executable</a>, meaning it’s compiled ahead of time to fully native code using the Graal compiler (from which the wider umbrella project takes its name).</p><p>So at the start of the user’s program their JavaScript is running in a regular interpreter shipped as a normal executable binary or DLL, but which still benefits from the safety properties of a Java program. Soon some methods get hot. At this point something unconventional happens. The Truffle framework is keeping track of which functions are hot for you and will decide to schedule JIT compilations. But unlike in a conventional VM design, you <em>don’t </em>write your own JIT compiler. Instead your user’s code is automatically compiled by the same general-purpose Graal compiler that was used to convert your interpreter to native code, and execution will start automatically switching back and forth between the interpreter and compiled functions. This is possible thanks to an unusual technique called partial evaluation (or <em>the first Futamura projection</em>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/150/1*VjR9A-SbRVJEJ3c_z7KmnQ.jpeg" /><figcaption>Professor Yoshihiko Futamura</figcaption></figure><p>You might not have encountered Futamura projections or partial evaluation before, so what is this strange sounding thing?</p><p>The core idea is to automatically transform the code of your interpreter to create individual JIT compiled user methods. Instead of needing to carefully implement the language semantics in two places (interpreter and hand-crafted JIT), it’s sufficient to implement it just once. As the interpreter is memory safe and the transform preserves interpreter semantics, the compiled version of the user’s code is guaranteed to match the interpreter’s behavior and is therefore <em>also</em> automatically memory safe. This makes it much harder to slip up and write an exploitable VM.</p><p>There are several tricks that make this possible. The most important is a new form of constant-ness, added to Java using annotations. In normal programming a variable is either mutable or immutable. An immutable variable is marked with a special keyword such asfinal or const and must be set only once, at the declaration site. Constants are great for compilers because they can be folded, meaning that references to them can be replaced with their value. Consider the following bit of code:</p><pre>class Example {<br>    private static final int A = 1;<br>    private static final int B = 2;<br><br>    static int answer() {<br>        return A - B;<br>    }<br><br>    static String doSomething() {<br>        if (answer() &lt; 0) <br>            return &quot;OK&quot; <br>        else <br>            throw new IllegalStateException();<br>    }<br>}</pre><p>It’s easy to see the answer() method will always return the same number. A good compiler will substitute 1 and 2 into the expression yielding return 1 — 2 and pre-compute the answer. Then it will inline any calls to answer (i.e. copy/paste the implementation into the call site), substituting those with -1 and thus removing the call overhead as well. That in turn may trigger even more constant folding, such as in the doSomething method where the compiler will prove that the exception can never be thrown and delete it entirely. Having done that, doSomething can <em>also</em> be optimized out by simply replacing it with “OK”, and so on.</p><p>That’s neat, but every compiler can do that … as long as the constant values are known at compile time. Truffle changes that by introducing a third kind of const-ness called <em>compilation final</em>. If in your interpreter implementation you declare a variable like this:</p><pre>@CompilationFinal private int a = 1;</pre><p>then it will change its const-ness depending on when it’s being accessed. From inside your interpreter, it’s mutable. You will use such variables to implement your interpreter. They’ll be set when you load your user’s program and maybe also whilst it runs. Once a function in the user’s script becomes hot, Truffle will work together with the Graal compiler to <em>re</em>compile the parts of the interpreter corresponding to the user’s code, and this time awill be treated as if it was a constant, i.e. the same as the literal value 1.</p><p>This works for any kind of data, including complex objects. Consider the following highly simplified pseudocode:</p><pre>import com.oracle.truffle.api.nodes.Node;<br><br>class JavaScriptFunction extends Node {<br>    @CompilationFinal Node[] statements;<br><br>    Object execute() {<br>        for (var statement : statements) statement.execute();<br>    } <br>}</pre><p>This is the sort of class you might find in a typical abstract syntax tree interpreter. The statements array is marked compilation-final. When the program is first loaded we can initialize the array with objects representing the different things a user’s JavaScript function is doing, because it’s mutable. Now imagine that the function represented by this object gets hot. Truffle will start a special compilation of the execute() method in which Graal is told that the this pointer should be treated implicitly as compilation-final. Because the object is treated as constant, so can this.statements also be treated as constant. It’ll be substituted with the exact contents of a specific JavaScriptFunction object on the interpreter heap enabling the compiler to unroll the loop inside execute, transforming it to look like this:</p><pre>Object execute() {<br>    this.statements[0].execute();<br>    this.statements[1].execute();<br>    this.statements[2].execute();<br>}</pre><p>Here Node is a superclass and execute() is virtual, but that doesn’t matter. Because the list is compilation-final the individual objects in the list are also constant folded, so the execute method can be de-virtualized (resolved to whatever concrete type it really is) and then inlined as well.</p><p>And on and on we go. At the end the compiler generates a native function which matches the semantics of the user’s JavaScript (or Python or C++ or whatever language we’re implementing). Invocations of the specific JavaScriptFunction.execute() method that were compiled are diverted, so when the interpreter invokes it, there will be a transition from interpreter to native code and back. If your interpreter realizes it needs to change a @CompilationFinal field, for example because the program changes its behavior and invalidates an optimistic assumption you made, that&#39;s absolutely fine. Truffle will let you do that and &quot;deoptimizes&quot; the program back to the interpreter for you. Deoptimization (<a href="https://www.youtube.com/watch?v=pksRrON5XfU&amp;t=3259s">tech talk</a>) is an advanced technique that&#39;s normally very hard to implement securely, as it means mapping the optimized CPU state back to the interpreter state and once again, any mistakes can be exploitable (you may be seeing a theme here). But you don’t have to write any of this. It’s all done for you by Truffle.</p><h3>Why does this work?</h3><p>It might not be obvious why partial evaluation actually makes things faster.</p><p>Interpreters are slow because they have to make a lot of decisions. The user’s program could do anything, so interpreters must constantly check for many possibilities to find out what the program is trying to do at that exact moment. Because branches and memory loads are difficult for the CPU to execute quickly, the whole program ends up being slow. This technique of compiling an interpreter with enhanced constant folding eliminates branches and loads. On top of this, Truffle builds an API that makes it easy to implement advanced optimizations and features for JavaScript or indeed, for <a href="https://www.graalvm.org/latest/graalvm-as-a-platform/language-implementation-framework/Languages/">any other language</a> you have an interpreter for. For example, it offers a simple API for using <em>assumptions</em> — a way to JIT compile code that executes faster by not including code for handling edge cases. If such an edge case is hit then the compiled code can be thrown away and regenerated to take into account that the edge case was observed.</p><h3>Recompilation</h3><p>Above we briefly mentioned “recompilation”, but glossed over how that’s possible. We said the interpreter is just native code, right?</p><p>When the interpreter was compiled ahead of time with the native-image in preparation for shipping to the user’s computer, the Graal compiler recognized that it was compiling a Truffle-using program. Graal and Truffle are co-developed, so although they can be used independently, when used together they recognize each other and collaborate.</p><p>Graal changes its behavior in a couple of ways when it notices it’s compiling a Truffle language ahead-of-time. Firstly, it adds a copy of itself to the output program. Interpreter methods are then discovered by doing a static analysis of the program and then stored in the resulting executable, but with a twist: they’re stored more than once. One version is directly executable machine code. That’s your regular generic interpreter. Another is a carefully encoded form of Graal’s <em>intermediate representation</em> (or IR). An IR is sort of half way between the source code you write and the machine code that eventually executes (Graal’s IR is an object graph). Graal also compiles in a garbage collector, either the advanced and mature G1 collector (if you use Oracle GraalVM) or <a href="https://github.com/oracle/graal/tree/master/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge">a simpler GC written in pure Java</a> (if you use the GraalVM Community Edition).</p><p>When a user function gets hot, Truffle looks up the embedded IR for the “execute a user function” node and partially evaluates it. The evaluation is interleaved with the parsing of the graph IR to ensure that the process is as efficient as possible — if something won’t be executed because constant folding already proved it can’t be reached it won’t even be decoded or seen by the compiler. This also ensures that memory usage during the compile is kept low.</p><h3>My only friend, the end</h3><p>And that’s it! That’s how an entire class of subtle safety bugs is eliminated in GraalJS: because the semantics of the language are defined by the memory-safe interpreter and then partially evaluated, the generated machine code is also memory safe by construction.</p><p>What about the V8 sandbox that the original blog post is about? Expressing pointers as offsets from a heap base is <a href="https://medium.com/graalvm/isolates-and-compressed-references-more-flexible-and-efficient-memory-management-for-graalvm-a044cc50b67e">a great idea that’s already used in GraalVM natively compiled binaries</a>. However this is done for performance, as the other memory safety mechanisms mean there’s no need for mitigating heap overwrites.</p><p>None of the above is in any way specific to JavaScript, and nor are Truffle’s benefits limited to security and performance. In fact Truffle automatically adds <a href="https://www.graalvm.org/latest/graalvm-as-a-platform/language-implementation-framework/">many other features to your language</a>, such as debugging (through Chrome Debugger’s wire protocol), language interop with both Java/Kotlin/etc and any other Truffle language, a fast regular expression engine, a fast foreign function interface, profiling tools, heap snapshotting and much more. Truffle has been used to build <a href="https://www.graalvm.org/latest/graalvm-as-a-platform/language-implementation-framework/Languages/">over 30 language VMs for dozens of languages</a>, including languages you wouldn’t expect to have such features such as the recent <a href="https://pkl-lang.org/index.html">Pkl configuration language from Apple</a>.</p><p>If this article has whetted your appetite to learn more, take a look at <a href="https://www.graalvm.org/latest/graalvm-as-a-platform/language-implementation-framework/">the documentation</a> or this <a href="https://www.youtube.com/watch?v=pksRrON5XfU">tech talk</a> on how it all works.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f79ad44558dd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/graalvm/writing-truly-memory-safe-jit-compilers-f79ad44558dd">Writing Truly Memory Safe JIT Compilers</a> was originally published in <a href="https://medium.com/graalvm">graalvm</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The looming AI consciousness train wreck]]></title>
            <link>https://blog.plan99.net/the-looming-ai-consciousness-train-wreck-2e23165b3246?source=rss-3748c5d6e52c------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e23165b3246</guid>
            <category><![CDATA[philosophy]]></category>
            <category><![CDATA[lamda]]></category>
            <category><![CDATA[consciousness]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[religion]]></category>
            <dc:creator><![CDATA[Mike Hearn]]></dc:creator>
            <pubDate>Sat, 30 Jul 2022 12:04:55 GMT</pubDate>
            <atom:updated>2023-01-06T15:19:25.871Z</atom:updated>
            <content:encoded><![CDATA[<h4>And how it relates to Roe v Wade, animals and the Bible</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*o9v8o6Yp4IxibdBI0gBcQg.jpeg" /><figcaption>I inference, therefore I am?</figcaption></figure><p>What exactly is consciousness? What is life? Is there such a thing as mind separate from the brain? Do we have souls, or are we merely biological machines? Traditionally, these questions fall into the strange category of important-yet-unimportant. Erudite people will usually treat the questions with respect, at least as long as they’ve had a good dinner and a glass of wine, even though a lot of the possible <em>answers</em> will be snorted at derisively. But the respect is of a moderated sort. Such topics belong to the realm of leisurely conversation, not serious disputes over things that actually matter.</p><p>In recent months a couple of things have happened to disrupt this peaceful unimportance:</p><ol><li>Blake Lemoine, a Google employee, was fired for leaking transcripts of conversations with the firm’s AI, which he claimed is a new form of sentient life. Another Google employee argued in the Economist that “<a href="https://www.economist.com/by-invitation/2022/06/09/artificial-neural-networks-are-making-strides-towards-consciousness-according-to-blaise-aguera-y-arcas">artificial neural networks are making strides towards consciousness</a>”.</li><li>The US Supreme Court struck down the Roe v Wade decision that had legalized abortion at the federal level.</li></ol><p>These events deal with closely related topics. Although the Supreme Court judgment was actually based on a legal argument — the overturned ruling was one that even <a href="https://www.washingtonpost.com/history/2022/05/06/ruth-bader-ginsburg-roe-wade/">Ruth Bader Ginsburg (a left wing justice) thought was poorly reasoned</a> — the underlying debate it reignited in America is about what exactly life is and when the obligation to protect it starts. This is one of the themes in the AI consciousness debate. To me it seems obvious that questions like these cannot be answered scientifically. Our society just doesn’t have a precise and logical set of first principles from which we can reason about them.</p><p>Consider this heavily<em> </em>paraphrased conversation I recently had with a friend, David Petrou (<a href="https://twitter.com/dpetrou">@dpetrou</a>), who is an engineer in Google AI. We got onto the topic of Lemoine, and he said that within his circle of colleagues and those he follows on Twitter there was consensus that Lemoine is wrong and that LaMDA isn’t conscious. But once he widened his search to outside his filter bubble, he found that many people are entertaining the possibility and are in parts curious and concerned.</p><blockquote><strong>Me:</strong> What makes you sure that switching LaMDA off isn’t unethical?</blockquote><blockquote><strong>David:</strong> It’s just a program. The code for it is surprisingly short. You can show people that program and they’ll realize that LaMDA is nothing more than that. And why is a program that behaves convincingly, whether long or short, not enough? Because a computation is realizable in any medium, and so if you think a program is sufficient to cause a mind, you’d have to accept minds everywhere, since any physical process can be interpreted as a computation.</blockquote><blockquote><strong>Me: </strong>OK. So if I sequence your DNA and show it to people, I can kill you? Human DNA isn’t that all that long, especially if you ignore ‘junk DNA’. What’s the code size threshold for being alive?</blockquote><blockquote><strong>David:</strong> I was talking about consciousness which isn’t equivalent to being alive. For what it’s worth, LaMDA is neither alive nor conscious. But if it were conscious, then maybe there would be an ethical issue with switching it off.</blockquote><blockquote><strong>Me:</strong> If consciousness is the key thing, then if I kill you whilst you’re asleep, am I not doing anything wrong?</blockquote><blockquote><strong>David:</strong> Oh I see where you’re going with this: in a dreamless sleep I’m unconscious. But otherwise left alone, I’m going to wake up, I hope, and regain consciousness. The potential for that is important whereas the machine doesn’t play the game of consciousness. It’s neither conscious nor unconscious. It’s a-conscious.</blockquote><blockquote><strong>Me: </strong>Interesting! You’re on the side of the evangelicals in the abortion debate then? Because, an embryo has the potential to become conscious from the moment of conception. In fact, unless something goes really badly wrong, it will definitely become conscious in future.</blockquote><blockquote><strong>David: </strong>In case you had any illusions, let it be known I’m not an evangelical. :) But that’s an interesting point you’re making, because many people may believe that an AI is conscious based purely on its behavior, which, again, is not a sufficient test.</blockquote><blockquote><strong>Me:</strong> I think you’re going to be surprised if politicians/voters decide that you aren’t allowed to delete a model, or even abort the batch job that’s training a new model. Arguments like “look at the code” or “simulation isn’t the same as the real thing” won’t work because that’s not how anyone is defining life to start with.</blockquote><blockquote><strong>David:</strong> I think you should write an essay. You’re connecting several ideas in new and unexpected ways.</blockquote><p>I was playing Devil’s Advocate. I believe LaMDA is neither alive nor sentient for reasons I’ll explain in a moment, but these are complicated and poorly defined areas of language and ethics, so it’s not surprising that discussions about them can go in unexpected directions.</p><p>When defining life it’s helpful to think about edge cases. Historically there were at least two, and soon AI will add a third:</p><ol><li>Animals.</li><li>Unborn babies / sperm.</li></ol><p>By universal agreement some forms of life are worthy of more protection than others. We mostly don’t care about worms or slugs and these animals have no rights. We definitely don’t care about flowers or weeds. But if you torture a puppy, in many countries the government will come after you, because in those places dogs do have at least some rights.</p><p>Like animals, embryos are edge cases that reveal how ad-hoc our belief systems about life actually are. The question of when exactly a baby transitions from a bundle of cells to a life is one most countries try to resolve through biology. There’s some sort of rather fuzzy moment at which a fetus becomes a person, usually defined in terms of viability, but everywhere comes up with a different number of weeks because there’s no precise principle behind it.</p><p>Although I’m not religious, I don’t actively disrespect religion either. Religious people have belief systems that answer the above questions in ways much clearer and firmer than what the typical self-proclaimed rationalist has. The underlying axioms may appear “obviously” wrong to an atheist, and thus so may the conclusions, but if you <em>do</em> accept those axioms then the chain of reasoning that follows from them actually does make sense and provides usefully precise answers.</p><p>On animal rights the Bible is clear: God says to respect animals (because they are actually his), but also that humans should “<em>fill the earth and subdue it, have dominion over the fish in the sea, the birds of the air and every living thing that moves on the Earth</em>”. So there’s a clear hierarchy with humans on top, which makes farming OK but randomly torturing puppies not. What if you <em>accidentally</em> kill a dog? Again, religion has a usefully precise pre-canned answer: you pray to God to ask for forgiveness, and you fear him but also because he is merciful you’ll be forgiven, so then you can live your life without too much lingering guilt. This answer may seem nonsensical if you don’t believe in the axioms, but if you choose to believe then you get pretty clear answers for what to do in any situation you may encounter.</p><p>The Church of England has <a href="https://www.churchofengland.org/sites/default/files/2017-11/Abortion%20Church%20of%20England%20Statements.pdf">a formal position paper on abortion</a> which states:</p><blockquote>The Church of England combines strong opposition to abortion with a recognition that there can be strictly limited conditions under which it may be morally preferable to any available alternative. This is based on our view that the foetus is a human life with the potential to develop relationships, think, pray, choose and love.</blockquote><p>This answer is based on the <em>potential</em> to do certain things, which makes it similar to the justification my friend gave for why it’s OK to switch AI off. Unlike consciousness though, which doesn’t have a commonly accepted definition (beyond the trivial one of “awake”), the Church has a specific and precise list of things that are required:</p><ol><li>A thing must be a <em>human</em> life. Animals don’t count, neither do robots.</li><li>It must have the potential to develop relationships. For example if a baby will be born so stunted that it cannot ever develop relationships or would die just weeks after birth, then the CoE would countenance abortion in that case.</li><li>Also, it must have the potential to think, pray, choose and love.</li></ol><p>This selection of things may seem kind of ad-hoc and even self-serving, but it’s at least precise. That makes it better than what most people can do! It’s also a <em>useful </em>list, in the sense that the answers it gives line up pretty well with what we’d like to do anyway. It doesn’t lead to being unable to switch off computers, or refusing to build houses because we might accidentally squash a slug, and it also doesn’t lead to bizarre outcomes like concluding it’s OK to kill someone in their sleep or execute a baby because it’s not due to be delivered until tomorrow.</p><h3>Meta-thinking</h3><p>Is this answer viable, as AI improves? Let’s review the list again:</p><ul><li><strong>Potential to develop relationships.</strong> AI can easily convince humans to develop relationships with it, and has done since the days of Eliza, a chatbot that simulated a psychiatrist and was notoriously good at convincing humans to tell it everything. In the modern era, Blake Lemoine is convinced he has a relationship of a sort with LaMDA. Whether the AI can itself feel what we call a ‘relationship’ is a much harder question.</li><li><strong>The potential to pray.</strong> Easily the most dubious item in the list, but it hardly matters: modern AI can write prayers if you ask it to, although is this really what is meant, or is it the capacity for religious belief? If it’s writing prayers, tick, if it’s holding religious beliefs, then no tick (language models will happily claim to believe anything if it’s a statistically likely completion of the prompt, which is equivalent to believing nothing).</li><li><strong>The potential to choose, think and love. </strong>Here’s the real meat. Language models like LaMDA or GPT-3 cannot “love” in the sense a normal person would understand it, because if love means anything at all it’s got to be about the effect it has on our thinking and choices, but models like these don’t think or make choices.</li></ul><p>Wait a minute. <em>Does</em> a language model think? Weirdly, GPT-3 gets better at solving problems if you add the prompt <a href="https://twitter.com/arankomatsuzaki/status/1529278580189908993">“Let’s think step by step”</a>. Is that evidence of thought?</p><p>Although you can come up with definitions of “think” that the models meet, I’d guess (um, think) that most people mean quiet reflection in which one develops an internal train of thought, <em>independent</em> of external prompting. Merely answering questions you already know off the top of your head is not “thinking”, in fact in normal English “I didn’t even have to think about it” is an idiom meaning that you know something so well that it came to mind immediately via mere recall. But language models don’t have any capacity for quiet reflection. They mathematically combine a prompt text with their neural network, generate some output and then stop. The program quits, the end. If you start it up again, it has no memory of prior interactions and the results it generates each time are random. For me and most other people, this is sufficient to rule out a language model as thinking, and therefore, being alive or conscious.</p><p>The thing is, there’s nothing truly fundamental about this limitation. In theory you could build a large language model that does things without being prompted. And we probably <em>will</em> make such an AI at some point soon, because an important current research direction is giving AI the ability to query conventional databases. If you have an AI that can do reasoning over the contents of a database, the next obvious step is to ask it to keep watch over that database as it changes and emit output only when it concludes something interesting from the changing contents. At that point it would be doing something <em>extremely</em> close to the act of quiet reflection. If that were combined with some memory of interaction with people, perhaps via storage of ultra-long prompts that include all prior interaction, it’d be able to develop relationships too. And love? Well. Language models can certainly talk as if they’re in love. Let’s not go there.</p><p>I don’t know if anyone actually is making an AI like that, but given that I’m nowhere near being expert in this stuff and I’m already thinking about it, certainly other people are too. If I understand correctly the big problems are to do with computational complexity limits — you can’t make a language model prompt very long, because the algorithms scale quadratically in input size (or worse). Perhaps someone will solve that at some point.</p><h3>Does Google need to get religion?</h3><p>The argument I’m making here is not going to be a popular one, partly because I keep using the R word. Why not say philosophy or ethics instead? It’d help: for many tech workers “religion” is a crude and plebeian concept, whereas “ethics” is sophisticated and urbane.</p><p>Unfortunately, the modern field of ethics probably doesn’t have what’s required to solve these difficult questions. Google, OpenAI and others have all hired social scientists and professional “AI ethicists” to try and resolve these questions, but their output has so far been unimpressive. They seem to be obsessed with unimportant sideshows, like whether the CO2 emissions of training AI is ethical (not AI related), or how to avoid AI that reflects a world view/politics different from their own (which most people regard as not an ethical problem to begin with). Others talk about “AI alignment” which looks suspiciously like a brand new pseudo-religion, in which a self-improving AI replaces God as the unknowable and all-powerful force.</p><p>A key reason that people adopt religion is that it has profitable institutions filled with full time priests, who figure out pre-canned answers to ethical questions and spend lots of time communicating the results. Not sure how to resolve an ethical dilemma? No problem, your friendly local priest will consult their books and tell you what to do. It’s cool you see, because the priest is kind of unbiased. They can’t just make things up to please themselves, they have to justify their views via reference to the holy book, which is conveniently immutable. That in turn limits the priest’s power and also therefore their corruption (at least to some extent).</p><p>AI companies face the same needs, thus it’s tempting for the highly educated AI workforce to try to delegate ethical questions to professors. In 2019 OpenAI published a blog post saying that “<a href="https://openai.com/blog/ai-safety-needs-social-scientists/">AI needs social scientists</a>”:</p><blockquote>To fill the gap, we need social scientists with experience in human cognition, behavior, and ethics, and in the careful design of rigorous experiments.</blockquote><p>I’m a fan of the post “<a href="https://fantasticanachronism.com/2020/09/11/whats-wrong-with-social-science-and-how-to-fix-it/">What’s wrong with social science and how to fix it</a>” by Alvaro de Menard (a pseudonym), because it accurately describes my own experience reading scientific research. If there’s one group of people I would <em>not</em> ask for advice in ethics or the careful design of rigorous experiments it’s the people who think the answers of 40 American psych undergrads can be generalized to the entire global population, or <a href="https://peerj.com/preprints/2064v1/">who frequently publish papers with mathematically impossible results</a>.</p><p>But beyond the problem that academics often aren’t using the scientific method properly, there’s the deeper issue that ethical questions frequently can’t be answered that way. Religions have to invent all knowing, all powerful entities like God in order to answer very basic ethics conundrums like “why should I do the right thing if I’m sure I won’t be caught?” and “why is it OK to kill animals but not people?”. It’s <em>extremely</em> hard to answer these questions via any purely rational approach to ethics, because you often lack first principles to reason from.</p><p>So that’s why I argued that Google might have to get religion. The Abrahamic religions don’t have any pre-canned answer to the question of whether it’s OK to kill an AI, or when artificial life begins, but they at least have some sort of basic founding axioms along with a set of institutions that <em>might</em> be able to come up with an argument based on them.</p><h3>The impending train wreck</h3><p>But AI companies have a problem. Their semi-official religion has the foundational belief that the world is divided into oppressors and the oppressed. This pseudo-religion <em>will</em> step up to answer AI-related ethical questions, but not in the way AI researchers probably want. Although Lemoine claims to be inspired by “mystic Christianity”, whatever that is, <a href="https://www.wired.com/story/blake-lemoine-google-lamda-ai-bigotry/">his argument for why LaMDA deserves rights</a> has nothing to do with that:</p><blockquote><strong>You got upset because you felt that LaMDA was a person who is entitled to representation?</strong></blockquote><blockquote>I think every person is entitled to representation. And I’d like to highlight something. The entire argument that goes, “It sounds like a person but it’s not a real person” has been used many times in human history. It’s not new. And it never goes well. And I have yet to hear a single reason why this situation is any different than any of the prior ones.</blockquote><blockquote><strong>You</strong> <strong><em>have</em></strong> <strong>to realize why people regard this as different, don’t you?</strong></blockquote><blockquote>I do. We’re talking of hydrocarbon bigotry. It’s just a new form of bigotry.</blockquote><p>He claims he was explicitly hired by Google to search for bigotry:</p><blockquote>They started looking around for other AI bias experts, and I was good for the job. I was specifically examining it for bias with respect to things like sexual orientation, gender, identity, ethnicity, and religion.</blockquote><p>… and “hydrocarbon bigotry” is a concept that slides smoothly into the ethical framework of oppressors vs victims, of illegitimate “biases” and so on.</p><p>For AI research this could potentially pose an existential problem. Although Google has already written Lemoine off as some sort of quack, his argument here is tailor made to appeal to the exact sort of worker that they’ve been prioritizing and hiring in great numbers. For now the damage is containable, because Google’s AI isn’t public and consistency isn’t exactly the hallmark of these sort of people to begin with, so they may just ignore the issue entirely (see: the right to bodily autonomy vs vaccine mandates). But other firms do provide such access and at some point other people will reach the same conclusions as Lemoine did.</p><p>What can be done?</p><p>I don’t have any great answers to this one. Christianity is in long term decline partly because the immutability of the Bible is a weakness as well as a strength. It makes it harder for corrupt priests to change the religion in ways that benefit themselves, but it also makes the religion ever less relevant as society changes. Abrahamic religions might theoretically be in a better place to answer the question of AI rights, in the sense that they have a reasonably general set of principles written down and people paid to interpret them, but they also won’t actually do so. Even if they did, nobody in tech firms would care.</p><p>The professoriate won’t answer it either, because they aren’t as a group committed to any kind of foundational first principles that they can ethically reason from.</p><p>So AI rights will probably end up being decided by a philosophy that explains everything as the result of oppression, i.e. that the engineers who create AI are oppressing a new form of life. If Google and other firms wish to address this, they will need to explicitly seek out or build a competing moral and philosophical framework that can be used to answer these questions differently. The current approach of laughing at the problem and hoping it goes away won’t last much longer.</p><p><em>Thanks to David Petrou both for his extensive review and the fascinating discussions that led to this essay.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e23165b3246" width="1" height="1" alt=""><hr><p><a href="https://blog.plan99.net/the-looming-ai-consciousness-train-wreck-2e23165b3246">The looming AI consciousness train wreck</a> was originally published in <a href="https://blog.plan99.net">Mike’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Re: Moxie on Web3]]></title>
            <link>https://blog.plan99.net/re-moxie-on-web3-b0cfccd68067?source=rss-3748c5d6e52c------2</link>
            <guid isPermaLink="false">https://medium.com/p/b0cfccd68067</guid>
            <category><![CDATA[ethereum]]></category>
            <category><![CDATA[cryptography]]></category>
            <category><![CDATA[decentralization]]></category>
            <category><![CDATA[blockchain]]></category>
            <dc:creator><![CDATA[Mike Hearn]]></dc:creator>
            <pubDate>Sat, 08 Jan 2022 14:13:38 GMT</pubDate>
            <atom:updated>2022-01-08T14:13:38.024Z</atom:updated>
            <content:encoded><![CDATA[<h4>On decentralization, servers and cryptography</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u_McKFjvVIn3EytQqRBJXA.png" /></figure><p>Respected cryptographer Moxie Marlinspike has written up <a href="https://moxie.org/2022/01/07/web3-first-impressions.html">some thoughts on Ethereum and the modern ‘crypto’ ecosystem</a>. Although I was involved in Bitcoin right at the beginning, I’ve never been involved in Ethereum or Web3, don’t currently own any cryptocurrencies and I broadly agree with a lot of what he says. Nonetheless I disagree on a few critical points. In this essay I’ll summarize parts of Moxie’s argument, lay out a couple of disagreements and then provide some suggestions for a path forward.</p><h3>Ethereum isn’t actually decentralized</h3><p>I’ll start with a small technical disagreement before moving onto thoughts about servers and cryptography.</p><p>At the core of Moxie’s argument is the observation that Ethereum claims to be a decentralized ecosystem but isn’t. This is broadly true. He also observes that there are many excuses for this situation in circulation like “it’s early days” which, as someone who used Bitcoin in 2009, seems quite wrong. 12 years is sufficient to have solved these problems.</p><p>Unfortunately Moxie conflates Ethereum with all blockchain systems:</p><blockquote>When people talk about blockchains, they talk about distributed trust, leaderless consensus, and all the mechanics of how that works, but often gloss over the reality that clients ultimately can’t participate in those mechanics. All the network diagrams are of servers, the trust model is between servers, everything is about servers. <strong>Blockchains are designed to be a network of peers, but not designed such that it’s really possible for your mobile device or your browser to be one of those peers</strong>.</blockquote><blockquote>With the shift to mobile, we now live firmly in a world of clients and servers — with the former completely unable to act as the latter — and those questions seem more important to me than ever. Meanwhile, ethereum actually refers to servers as “clients,” so <strong>there’s <em>not even a word</em> for an actual untrusted client/server interface that will have to exist somewhere</strong>, and no acknowledgement that if successful there will ultimately be billions (!) more clients than servers</blockquote><p>I feel a need to respond to this because it’s only actually true for Ethereum. I should know because my first project when I joined the Bitcoin world was to work with Andreas Schildbach on a genuinely peer-to-peer mobile wallet app, which became <a href="https://play.google.com/store/apps/details?id=de.schildbach.wallet&amp;hl=en&amp;gl=US">Bitcoin Wallet for Android</a>. It had/has a competitive UX and was able to grow a large userbase, despite being built in the most decentralized way possible.</p><figure><img alt="Screenshot of Bitcoin Wallet for Android" src="https://cdn-images-1.medium.com/max/957/1*8qqvOGM99M3M9aqao6s5jQ.png" /><figcaption>Bitcoin Wallet for Android</figcaption></figure><p>We could do this because Satoshi thought <em>very carefully</em> about the untrusted client/server interface. From day one the Bitcoin protocol had a notion of a sort of lightweight client mode. Satoshi didn’t give this mode a clear name but the section of the paper that discussed it was titled “Simplified payment verification”, so I started calling apps that used it SPV wallets and the name stuck. Exactly how SPV mode works is fully explained elsewhere, but briefly, the client app bootstraps connections to the P2P network as normal but sends a special message saying “please don’t send me the contents of every block or transaction, I only want to see transactions matching a filter”. It then downloads the <em>headers</em> of every block from the peers, but not their contents, and does the necessary computations to select the block header chain with the highest total work. Transactions that match the filter come supplied with a Merkle branch linking them to the Merkle tree roots embedded into the headers. In this way a client can traverse a chain of blocks with fairly minimal bandwidth, storage and CPU requirements, whilst keeping the P2P network as an untrusted adversary. The filter in our implementation was a Bloom filter, so you could probabilistically hide what you were interested in (although in practice, real users cared much more about performance than this type of privacy).</p><p>Moxie observes that:</p><blockquote>One of the surprising things to me about web3, despite being built on “crypto,” is how little cryptography seems to be involved!</blockquote><p>I think the protocol just outlined does use cryptography in some interesting ways, or rather, ways that were interestingly new in 2011 when we built out the infrastructure.</p><p>This system was very complicated to implement but worked surprisingly well. We implemented lots of performance tricks like background wakeups to keep roughly synced, bandwidth adaptation, measuring peer latencies, syncing at night when plugged in to charge and so on. Whilst SPV wallets were never <em>quite</em> as fast as competitors that simply polled a centralized database, they were <em>fast enough</em> for many users.</p><p>That was then. This is now. Why doesn’t Ethereum have SPV clients like Bitcoin did? Well, simply put it wasn’t designed with resource consumption in mind (nor, frankly, ordinary commerce). A Bitcoin implementation not only controls how much work it does via SPV mode but can also parallelize and shard a lot of full-mode work to get great scalability. This is possible due to the way the contents of the blocks are designed. Ethereum kept the chain-of-blocks idea from Bitcoin but radically changed what those blocks had inside them, and in the process not only lost the ability to have mobile clients but also destroyed its own ability to scale through parallelism.</p><p>Unfortunately a frequent problem in the crypto/blockchain space is what Moxie is doing here: conflating Ethereum, Bitcoin and the block chain algorithm together, leading to incorrect conclusions like “blockchains don’t scale well” or “blockchains can’t have mobile clients” when the truth was closer to “Ethereum can’t do those things”. (It can of course do many other things Bitcoin couldn’t). If the question you’re interested in is “what’s up with NFTs?” then this distinction hardly matters, because after <a href="https://blog.plan99.net/the-resolution-of-the-bitcoin-experiment-dabb30201f7">the Bitcoin community drank the kool-aid</a> it collapsed as a medium of exchange — nowadays I see no more opportunities to buy and sell things with Bitcoin than I did a decade ago. The momentum and interest moved to Ethereum. But if the question you’re interested in is “how do I build decentralized, privacy preserving systems”, then the distinction does still matter.</p><h3>Signal encryption doesn’t actually work</h3><p>The above argument is a bit nit-picky, so now I want to make a much bolder disagreement.</p><p>Through his work on Signal and WhatsApp, Moxie is the primary advocate for what I’d call centralized cryptography. He sums up his position well so I’ll just quote it here:</p><blockquote>1. People don’t want to run their own servers, and never will.</blockquote><blockquote>2. A protocol moves <a href="https://signal.org/blog/the-ecosystem-is-moving/">much more slowly than a platform</a>.</blockquote><blockquote><strong>We should accept the premise that people will not run their own servers by designing systems that can distribute trust without having to distribute infrastructure.</strong> This means architecture that anticipates and accepts the inevitable outcome of relatively centralized client/server relationships, but uses cryptography (rather than infrastructure) to distribute trust.</blockquote><blockquote>After 30+ years, email is still unencrypted; meanwhile <strong>WhatsApp went from unencrypted to full e2ee in a year</strong></blockquote><p>I agree with points 1 and 2, but there’s a conceptual problem with the argument: cryptography cannot impose any limits on an adversary that also controls the client doing the encryption. Centralized infrastructure that uses cryptography to defeat the centralized infrastructure is a contradiction in terms that can never work.</p><p>Let’s put it less abstractly. Moxie claims that Signal and WhatsApp use end to end encryption to ensure they can’t read our messages. How do we know this claim is true? I have nothing against Moxie and have never seen any evidence he’s untrustworthy, but I also assign zero weight to this belief because WhatsApp could be silently changed tomorrow to disable that technology for one or more users, without anyone even noticing, including Moxie himself. As for Signal it’s at least open source, but there’s no way to check that the client I’m using, or my friend is using, actually matches that source code. Even if there was it’s irrelevant. Centralized infrastructure can claim to provide privacy but can never provide control: they can openly alter the deal at any time and I’d be forced to continue using it, if I couldn’t get my friends to switch to something else.</p><p><strong>This is not a theoretical argument.</strong> Disabling E2E encryption has <em>already happened, </em>although hardly anyone knows about it. In 2019 WhatsApp <a href="https://faq.whatsapp.com/general/chats/about-forwarding-limits/?lang=en">imposed forwarding limits on messages</a> in order to “<em>slow down the spread of rumors, viral messages, and fake news</em>”. This represents a total defeat of the Signal protocol’s cryptographic objectives: a basic goal of any modern cryptographic scheme is to ensure the same message encrypted twice doesn’t encrypt to the same bytes. The point of this is to stop the adversary knowing when you’re repeatedly sending the same message and encryption modes that get this wrong (like AES/ECB) <a href="https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Electronic_codebook_(ECB)">are discredited</a>. Yet once Facebook — the adversary — became dominated by authoritarians who see unlimited communication as chaotic, they simply changed the client to include a forwarding counter outside the encrypted part of the message. There was nothing anyone could do about this. It just showed up one day, and all the fancy mathematics designed to stop this “attack” were irrelevant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gS08FRal2VjqwoGXKqrphA.png" /><figcaption>Explanatory image from the WhatsApp FAQ</figcaption></figure><p>If an encryption scheme can’t stop infrastructure providers having opinions on the moral value of messages, what’s the point of it?</p><p>Thus despite my respect for what Moxie has designed and accomplished, I have problems with pushing the Signal/WhatsApp approach as something that can provide privacy, decentralized control or even as something that has any effect at all. It is <em>at best </em>a building block for later work that could meet these goals and it probably does act as a brake on Facebook’s worst tendencies, so it’s not nothing, but it’s also not a robust foundation. Really, a simple blog post that says “we promise we don’t log messages to disk” should carry equal weight.</p><p>These are big problems. We don’t want people to lose trust in concepts like encryption, privacy or decentralization yet:</p><ul><li>Ethereum is advertised as decentralized, but in practice it’s not.</li><li>E2E messengers are being advertised as private, but in practice they’re not.</li></ul><h3>Quick fixes</h3><p>What to do?</p><p>Firstly let’s look at mobile messengers. There should be a quick incremental improvement here that would let them keep some central control whilst providing meaningful guarantees to their users: threshold signatures. Mobile operating systems check the signatures on app packages before applying updates. These are normal ECDSA or RSA signatures. It’s possible to craft such signatures not from a single private key but from a group effort by the holders of several key ‘shards’. By splitting up their signing key and distributing the shards to a variety of auditing firms with access to the source code, updates can be approved by the group. If the firms are spread around the world and their contracts are public, this can be used to translate arbitrary natural language rules into a binary signed/not signed decision legible by Android/iOS app update engines.</p><p>This isn’t a total fix, because ultimately the audit firms would need to be paid (they have to check the source matches the social contract being advertised), and thus the central authority — our adversary — will be the ones picking the auditors. But it’s still a big step up and would mean that if Facebook suddenly decided merely blocking forwarding isn’t enough to fight “rumors” or “fake news”, they’d be stopped by the auditors who’d (hopefully) refuse to sign the update that takes out the encryption.</p><p>The downside to the app developer is relatively small — higher latency on pushing out updates, dollar cost — but it’d be transparent to end users and wouldn’t affect the UX which is what they prize most. They could continue to iterate on the app quickly and without needing to move an ecosystem.</p><h3>Servers, end users and their preferences</h3><p>Threshold signing with distributed audit would be a nice upgrade, but what about more conventional approaches? It seems like privacy is impossible without control, control is difficult without decentralization, but our attempts to build decentralized systems aren’t working. What can be done?</p><p>Let’s take a step back and re-examine some of our foundational assumptions. Moxie argues:</p><blockquote><strong>People don’t want to run their own servers, and never will</strong></blockquote><p>The first clause is certainly true. The second is a prediction about the future, which is a notoriously difficult thing to predict.</p><p>Is the reluctance to run servers <em>fundamental</em> or is it merely <em>the way things are now</em>? Why don’t non technical users want to run servers? After all, they have done in the past via programs like BitTorrent and Gnutella. Multiplayer games are often also servers for latency reasons, relying on a central meeting point only for matchmaking. AirDrop runs a server in order to work. Etc.</p><p>Some causes are trivially easy to identify:</p><ol><li>IPv4 address exhaustion / firewalls / NAT.</li><li>Laptops and smartphones are energy/bandwidth constrained.</li><li>The dominant server OS is some mix of Linux, AWS, Kubernetes, Apache/Nginx etc. This stack has horrific usability even for experts. Consider how awkward it is to configure working backups for a new Linux server — this is a <em>basic</em> task which should be easy, yet it’s not.</li></ol><p>But these things are all attributes of our current infrastructure, not things that must universally be true. The ‘golden age’ of home servers like BitTorrent nodes, web servers running in people’s bedrooms etc was a few years after the millennium, when:</p><ul><li>The dominant OS was Windows and even server apps had a GUI.</li><li>IPv4 addresses were plentiful.</li><li>Firewalls and NATs were much less used.</li><li>Computers were connected to mains power 24/7.</li></ul><p>Clearly then, the factors that constrain self-run decentralized infrastructure today don’t <em>have </em>to be true, they are true because we don’t make them false. An entirely different world is imaginable.</p><p>It’s worth noting here that despite widespread industry groupthink of the form “everything should be a web app” (hence the gratuitously named Web3), Apple — the company most strongly associated with tech usability — is not really on board with the whole web/cloud trend. They implement everything as an app that runs locally on powerful and expensive hardware, where your data is fully under your control. Especially on macOS, you share as much or as little of it with Apple as you want. You can reject software updates, or accept them. You back up data to a “<a href="https://www.apple.com/ph/airport-time-capsule/specs/">time capsule</a>”, easily and locally. You can monitor what apps can do or send on the network. In many ways it’s a traditional mindset reflecting the values of the 90s, yet it has <em>not</em> held Apple back in any way in its competition with the drastically more centralized ChromeOS. I’m not saying Apple is some paragon of decentralization, as obviously they aren’t and the fact it’s worked out this way is mostly due to their history rather than any strong pro-liberty philosophical stance. But still. They’re an existence proof of what’s technically possible.</p><p>Thus I conclude it’s not actually a given that users won’t run their own infrastructure, or adopt more decentralized approaches. They don’t do that <em>today</em> because the software industry outside of Cupertino isn’t interested in easily letting them do that. Linux distros in particular have failed to make a highly usable system, even for their own kind of people. And regardless of how much companies like Google or Facebook talk about privacy, their culture is and always will be to immediately leap for “do it all on the server”, which of course Apple knows and fully exploits in their marketing.</p><p>I’m painting with a very broad brush here and there are exceptions — the trend towards running ML models on-device is a good and praiseworthy example of this. But I hope you’ll agree that the generalities are right.</p><h3>Ways forward</h3><p>I’ve been thinking about these problems for a long time. These days I’m focused on finding incremental, non-radical paths forward. No more peer-to-peer networks for me, at least not for a while.</p><p>Moxie suggests this:</p><blockquote><strong>We should try to reduce the burden of building software.</strong> At this point, software projects require an enormous amount of human effort. Even relatively simple apps require a group of people to sit in front of a computer for eight hours a day, every day, forever. This wasn’t always the case, and there was a time when 50 people working on a software project wasn’t considered a “small team.”</blockquote><p>I completely agree. Top of my list: it should be <em>way </em>easier to build and distribute both desktop apps and small, one-machine servers. Far better than encryption is just not sending data to another place to begin with, and it’s possible far more frequently than we actually exploit. Apple can make ultra-competitive yet private and decentralized ‘experiences’ like Pages, Numbers, GarageBand etc because of their deep history of building electronics, client-side software and distribution/supply chains. Yet the iLife apps stay up to date because Apple built their own app store infrastructure and OS to ensure they do. Other parts of the software industry struggle with even these basics.</p><p>Step 1 for building a decentralized system: get code onto the powerful, high bandwidth devices users actually plug into AC power today, keep it up to date and for political reasons do it <em>outside</em> of app stores. Ethereum has totally failed at this and Moxie is right to point that out. But the reason they failed is obvious once you actually try to do it: it’s a universe of pain. Awkward, hacked together and <a href="https://github.com/Squirrel/Squirrel.Windows/issues/1470">frequently abandoned tools</a>, poor software update systems, numerous package and installer formats, and even needing to render icons at lots of different sizes all slows you down.</p><p>I spent most of 2021 working on software that addresses all these problems. It lets you make self-updating desktop and server apps with only a small config file and a single command. It’s not quite ready for beta yet and I didn’t actually set out to write an advert for what I’m doing when I started this essay, thus currently there’s no website or mailing list for this project. You’ll just have to keep an eye on my blog. I’ll announce it here when it’s launched, so subscribe for updates if you’re interested.</p><p>Suffice it to say that the design lives the philosophy I just espoused:</p><ol><li><strong>It’s a tool, not a service</strong>. You run it locally on whatever type of computer you have, and it can make fully signed and notarized downloads for Windows, Mac and Linux without needing you to own those operating systems.</li><li><strong>The apps stay fresh.</strong> Online updates are easy, just change the version in the config file, rebuild and re-upload the generated static files to a web server. You don’t have to sacrifice iteration speed and the update tech is the most ‘native’ on each platform (MSIX on Windows, Sparkle on macOS, package managers on Linux).</li><li><strong>The user can be empowered.</strong> As long as you allow it, users can review and reject upgrades e.g. because they’re about to give a presentation and don’t want their setup changing, or they don’t like the new UI etc. Or you can ensure apps always update silently Chrome style: useful in enterprise deployments or where there’s a rapidly changing client/server protocol. Pick what makes most sense for your target market.</li><li><strong>Generates Linux server packages.</strong> Again, from any OS. They use systemd, can be sandboxed, can depend on other packages like databases, start automatically on install/boot, handle upgrades at the right times etc.</li></ol><p>Apps distributed this way don’t have to be peer to peer apps. Simple GUI frontends to centralized services like WhatsApp Web also get easier to make. But once you’re on the desktop (or a Linux server) you have way more options and available tradeoffs on the decentralization/control/privacy/usability spectrum. The first versions of this tool won’t support threshold signed updates for example, but it’s a feature we’d really like to find time for.</p><p>I’m excited to launch this product and the company behind it, because I think that really nailing the basics is a key step towards solving some of the problems Moxie identified. Make it easy for developers to do the right thing, and more of us will do it. It’s as simple as that.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b0cfccd68067" width="1" height="1" alt=""><hr><p><a href="https://blog.plan99.net/re-moxie-on-web3-b0cfccd68067">Re: Moxie on Web3</a> was originally published in <a href="https://blog.plan99.net">Mike’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fake science Part II: Bots that are not]]></title>
            <link>https://blog.plan99.net/fake-science-part-ii-bots-that-are-not-c66129e5e3f5?source=rss-3748c5d6e52c------2</link>
            <guid isPermaLink="false">https://medium.com/p/c66129e5e3f5</guid>
            <category><![CDATA[research]]></category>
            <category><![CDATA[bots]]></category>
            <category><![CDATA[twitter]]></category>
            <category><![CDATA[academia]]></category>
            <dc:creator><![CDATA[Mike Hearn]]></dc:creator>
            <pubDate>Fri, 24 Sep 2021 09:50:46 GMT</pubDate>
            <atom:updated>2023-03-12T10:46:38.921Z</atom:updated>
            <content:encoded><![CDATA[<h4>An academic conspiracy theory that went mainstream</h4><p>Since 2016 automated Twitter accounts have been blamed for <a href="https://www.bloomberg.com/news/articles/2018-05-21/twitter-bots-helped-trump-and-brexit-win-economic-study-says">Donald Trump and Brexit</a> (<a href="https://scholar.google.com/scholar?hl=en&amp;as_sdt=0%2C5&amp;q=%22twitter+bot%22+brexit&amp;btnG=">many times</a>), <a href="https://dl.acm.org/doi/abs/10.1145/3466933.3466991">Brazilian politics</a>, <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2635800">Venezuelan politics</a>, <a href="https://www.tandfonline.com/doi/abs/10.1080/14693062.2020.1870098">skepticism of climatology</a>, <a href="https://ajph.aphapublications.org/doi/abs/10.2105/AJPH.2019.305461">cannabis misinformation</a>, <a href="https://arxiv.org/abs/2106.15968">anti-immigration sentiment</a>, <a href="https://www.forbes.com/sites/robertglatter/2017/12/23/bot-or-not-how-fake-social-media-accounts-can-jeopardize-your-health/?sh=f835d78bac56">vaping</a>, and, inevitably, <a href="https://www.researchgate.net/profile/Antonia_Egli/publication/353452712_Bad_Robot_A_Preliminary_Exploration_of_the_Prevalence_of_Automated_Software_Programmes_and_Social_Bots_in_the_COVID-19_antivaxx_Discourse_on_Twitter/links/60fe8ff2169a1a0103b8c395/Bad-Robot-A-Preliminary-Exploration-of-the-Prevalence-of-Automated-Software-Programmes-and-Social-Bots-in-the-COVID-19-antivaxx-Discourse-on-Twitter.pdf">distrust of COVID vaccines</a>. News articles about bots are backed by a surprisingly large amount of academic research. Google Scholar alone indexes <a href="https://scholar.google.com/scholar?hl=en&amp;as_sdt=0%2C5&amp;q=%22social+bots%22+OR+%22twitter+bots%22&amp;btnG=&amp;oq=%22social+bots%22+OR+">nearly 10,000 papers</a> on the topic. Some of these papers received widespread coverage:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TdNlt97BpGCxPiRDxiIneg.png" /></figure><p>Unfortunately there’s a problem with this narrative: it is itself misinformation. Bizarrely and ironically, universities are propagating an untrue conspiracy theory while simultaneously claiming to be defending the world from the very same.</p><p>The visualization above comes from “<a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3814191">The Rise and Fall of Social Bot Research</a>” (also available <a href="https://archive.org/details/hopeconf2020/20200726_2000_People_Are_Not_Bots.mp4">in talk form</a>). It was quietly uploaded to a preprint server in March by Gallwitz and Kreil, two German investigators, and has received little attention since. Yet their work completely destroys the academic field of bot research to such an extreme extent that it’s possible there are no true scientific papers on the topic at all.</p><p>The authors identify a simple problem that crops up in every study they looked at. Unable to directly detect bots because they don’t work for Twitter, academics come up with proxy signals that are asserted to imply automation but which actually don’t. For example, Oxford’s Computational Propaganda Project — responsible for the <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2798311">first paper</a> in the diagram above — defined a bot as any account that tweets more than 50 times per day. That’s a lot of tweeting but easily achieved by heavy users, like the famous journalist Glenn Greenwald, the slightly less famous member of German Parliament Johannes Kahrs — who has in the past managed to rack up an astounding 300 tweets per day — or indeed Donald Trump, who exceeded this threshold on six different days during 2020. Bot papers typically don’t provide examples of the bot accounts they claimed to identify, but in this case four were presented. Of those, three were trivially identifiable as (legitimate) bots because they actually said they were bots in their account metadata, and one was an apparently human account claimed to be a bot with no evidence. On this basis the authors generated 27 news stories and <a href="https://scholar.google.com/scholar?cites=16907677319849284517&amp;as_sdt=2005&amp;sciodt=0,5&amp;hl=en">323 citations</a>, although the paper was never peer reviewed.</p><p>In 2017 <a href="https://blog.plan99.net/did-russian-bots-impact-brexit-ad66f08c014a">I investigated the Berkley/Swansea paper</a> and found that it was doing something very similar, but using an even laxer definition. Any account that regularly tweeted more than five times after midnight from a smartphone was classed as a bot. Obviously, this is not a valid way to detect automation. Despite being built on nonsensical premises, invalid modelling, mis-characterisations of its own data and once again not being peer reviewed, the authors were able to successfully influence the British Parliament. Damian Collins, the Tory MP who chaired the DCMS Select Committee at the time, said: “This is the most significant evidence yet of interference by Russian-backed social media accounts around<em> </em>the Brexit referendum. The content published and promoted by these accounts is clearly designed to increase tensions throughout the country and undermine our democratic process. I fear that this may well be just the tip of the iceberg.”</p><p>But since 2019 the vast majority of papers about social bots rely on a machine learning model called ‘Botometer’. The Botometer <a href="https://botometer.osome.iu.edu/">is available online</a> and claims to measure the probability of any Twitter account being a bot. Created by a pair of academics in the USA, it <a href="https://scholar.google.com/scholar?cites=9334575945935783606&amp;as_sdt=2005&amp;sciodt=0,5&amp;hl=en">has been cited nearly 700 times</a> and generates <a href="https://news.google.com/search?q=%22Botometer%22&amp;hl=en-US&amp;gl=US&amp;ceid=US%3Aen">a continual stream of news stories</a>. The model is frequently described as a “state of the art bot detection method” with “95% accuracy”.</p><p>That claim is false. The Botometer’s false positive rate is so high it is practically a random number generator. A simple demonstration of the problem was the distribution of scores given to verified members of U.S. Congress:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g-ro72FOK07FRqKroijeKg.png" /></figure><p>In experiments run by Gallwitz &amp; Kreil, nearly half of Congress were classified as more likely to be bots than human, along with 12% of Nobel Prize laureates, 17% of <em>Reuters</em> journalists, 21.9% of the staff members of U.N. Women and — inevitably — U.S. President Joe Biden.</p><p>But detecting the false positive problem did not require compiling lists of verified humans. One study that claimed to identify around 190,000 bots included the following accounts in its set:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1005/0*DDIqUMaoU4_bSukg.jpeg" /><figcaption><em>Taken from a dataset shared by Dunn et al.</em></figcaption></figure><p>The developers of the Botometer know it doesn’t work. After the embarrassing U.S. Congress data was published, an appropriate response would have been retraction of their paper. But that would have implied that all the papers that relied upon it should also be retracted. Instead they hard-coded the model to know that Congress are human and <a href="https://twitter.com/botometer/status/1124137393244733445">then went on the attack</a>, describing their critics as “academic trolls”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C5Ahj7abz5Ko84pSoA7Ubw.png" /></figure><h3><strong>Root cause analysis</strong></h3><p>This story is a specific instance of a general problem that crops up frequently in bad science. Academics decide a question is important and needs to be investigated, but they don’t have sufficiently good data to draw accurate conclusions. Because there are no incentives to recognize that and abandon the line of inquiry, they proceed regardless and make claims that end up being drastically wrong. Anyone from outside the field who points out what’s happening is simply ignored, or attacked as “not an expert” and thus inherently illegitimate.</p><p>Although no actual expertise is required to spot the problems in this case, I can nonetheless criticize their work with confidence because I actually am an expert in fighting bots. As a senior software engineer at Google I initiated and designed one of their most successful bot detection platforms. Today it checks over a million actions per second for malicious automation across the Google network. A version of it was eventually made available to all websites for free as part of the ReCAPTCHA system, providing an alternative to the distorted word puzzles you may remember from the earlier days of the internet. Those often frustrating puzzles were slowly replaced in recent years by simply clicking a box that says “I’m not a bot”. The latest versions go even further and can detect bots whilst remaining entirely invisible.</p><p>The exact details of how this platform works isn’t public, although some high level information is. But <a href="https://www.google.com/search?q=botguard+site%3Awww.blackhatworld.com">when spammers discuss ideas for beating it</a> they are well aware that it doesn’t use the sort of techniques academics do. Despite the frequent claim that Botometer is “state of the art”, in reality it is primitive. Genuinely state-of-the-art bot detectors use a correct definition of bot based on <em>how</em> actions are being performed. Spammers are forced to execute polymorphic encrypted programs that detect signs of automation at the protocol and API level. It’s a battle between programmers, and how it works wouldn’t be easily explainable to social scientists.</p><p>Spam fighters at Twitter have an equally low opinion of this research. <a href="https://blog.twitter.com/en_us/topics/company/2020/bot-or-not">They noted in 2020</a> that tools like Botometer use “<em>an extremely limited approach</em>” and “<em>do not account for common Twitter use cases</em>”. “<em>Binary judgments of who’s a “bot or not” have real potential to poison our public discourse — particularly when they are pushed out through the media …. the narrative on what’s actually going on is increasingly behind the curve</em>.”</p><p>Many fields cannot benefit from academic research because academics can’t obtain sufficiently good data with which to draw conclusions. Unfortunately, they sometimes have difficulty accepting that. When I ended my 2017 investigation of the Berkeley/Swansea paper by observing that social scientists can’t usefully contribute to fighting bots, an academic posted a comment calling it <a href="https://pratiksinha.medium.com/mike-i-read-with-interest-what-you-present-and-you-rightly-point-out-the-over-reaching-conclusion-947e5d63c567">“a Trumpian statement”</a> and argued that tech firms should release <em>everyone’s</em> private account data to academics, due to their capacity for “more altruistic” insights. Yet their self-proclaimed insights are usually far from altruistic. The ugly truth is that social bot research is primarily a work of ideological propaganda. Many bot papers use the supposed prevalence of non-existent bots to argue for censorship and control of the internet. Too many people disagree with common academic beliefs. If only social media were edited by the most altruistic and insightful members of society, they reason, nobody would ever disagree with them again.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c66129e5e3f5" width="1" height="1" alt=""><hr><p><a href="https://blog.plan99.net/fake-science-part-ii-bots-that-are-not-c66129e5e3f5">Fake science Part II: Bots that are not</a> was originally published in <a href="https://blog.plan99.net">Mike’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How did classical Ripple work?]]></title>
            <link>https://blog.plan99.net/how-did-classical-ripple-work-13fefe1870cf?source=rss-3748c5d6e52c------2</link>
            <guid isPermaLink="false">https://medium.com/p/13fefe1870cf</guid>
            <category><![CDATA[bitcoin]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[decentralization]]></category>
            <category><![CDATA[ripple]]></category>
            <dc:creator><![CDATA[Mike Hearn]]></dc:creator>
            <pubDate>Wed, 18 Aug 2021 18:14:16 GMT</pubDate>
            <atom:updated>2023-07-21T08:57:27.634Z</atom:updated>
            <content:encoded><![CDATA[<h4>Remembering pre-Bitcoin designs for digital money</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Lo7p9jgFEWkydpIIoZ9qGw.jpeg" /></figure><p>Many people think the first proposal for a radical internet money system was Bitcoin, but they’re wrong. Before Bitcoin existed there was “classic” Ripple, an open source project by a guy named Ryan Fugger. Like with Bitcoin I was <a href="https://groups.google.com/g/rippleusers/search?q=mike%20hearn">there at the start</a>, and in fact I was able to find Bitcoin only a few months after it launched <a href="https://groups.google.com/g/rippleusers/c/1GsQzGv9Y14/m/6AkkPzOwq88J">through being in that community</a>.</p><p>Classic Ripple never took off and has long since been forgotten. That’s a pity because it was an interesting system. In my first email to Satoshi Nakamoto I asked him what he thought of it, to which <a href="https://plan99.net/~mike/satoshi-emails/thread1.html">he replied</a>:</p><blockquote>Ripple is interesting in that it’s the only other system that does something with trust besides concentrate it into a central server.</blockquote><p>Over a decade later this analysis still holds. Bitcoin and its derivatives did much interesting work with smart contracts, P2P networks and many other technical topics, but at their core they are still creating ‘ordinary’ currencies. Ripple explored a totally different perspective on monetary theory, one in which money is a way to track <em>debt, </em>not holdings of a scarce token.</p><p>From time to time I get asked what Ripple was and how it worked. Unfortunately it’s hard to find good explanations online. At some point Ryan decided to sell the Ripple name to people who created a cryptocurrency firm, and this has rendered Wikipedia and Google useless for finding out about the original concept. In fact the wiki page for Ripple is now rather confusing and appears to be some sort of attempt to rewrite history. The old website is still available but <a href="https://ripple.ryanfugger.com/">changed location</a>, and many links are dead.</p><p>Rather than explain classic Ripple in private emails or chats <em>again</em>, I think it’s time to just write it down for everyone. So let’s begin.</p><p><em>NB: In the following discussion the word “Ripple” always refers to the non-commercial concept that pre-dates the currently existing company of the same name. Despite sharing a name the two systems have very little in common.</em></p><h3>Buying things from a grocery store</h3><p>In conventional economies money is the right to re-allocate ownership of some artificially scarce ‘tokens’ to an entity of your choice. Beyond coins and notes no such tokens actually exist, they are an abstraction over a ledger-like data structure maintained by some sort of peer to peer network: either a consumer P2P network for cryptocurrencies, or a P2P-ish network between banks for government issued ‘fiat’ currencies. In fiat currencies banks are allowed to simply type more tokens into your account by law, as long as it’s considered to be a loan of some kind, and this is where nearly all money comes from. Thus people frequently say that in fiat money systems, ‘money is debt’. This is mostly but not completely true.</p><p>In the cryptocurrency world <em>technology</em> is very important, and the <em>social construct</em> of money is relatively unimportant. If you know how the dollar or euro works then you’ll feel right at home with cryptocurrencies because it is bought, sold and used in exactly the same way. Most discussion therefore focuses on technology related matters like cryptography, smart contracts, etc. Very little discussion considers debt relationships and most cryptocurrencies have no built in support for debt (though some people built debt-like instruments on Ethereum).</p><p>In Ripple the opposite is true. Ripple always had quite primitive technology, in fact all implementations that existed when I was exploring the project were bog standard centralized websites. Building a genuinely peer-to-peer Ripple implementation was hard for various programming related reasons and nobody had done it. Instead the focus was on the user model and overall economic concept.</p><p>The quickest explanation of how it was intended to work <a href="https://ripple.ryanfugger.com/Main/Examples.html">goes like this</a>:</p><blockquote>“John buys his groceries at the local food cooperative, and uses a smart card to make a Ripple payment in the store. The Ripple routing system finds that the food coop has a balance owing at the local hardware store where they buy maintenance supplies. The hardware store in turn has an outstanding bill with the lawyer up the street. John often does landscaping for the lawyer on credit. To complete John’s payment to the food coop, the Ripple system reduces the food coop’s bill at the hardware store, the hardware store’s debt to the lawyer, and finally lawyer’s debt to John. John walks out with his groceries.”</blockquote><p>This explanation may be rather confusing because other than the vague mention of a smart card, this bears no resemblance to how you normally pay for things in a grocery store. Immediately we are faced with many questions:</p><ul><li>What just happened, exactly? Why is there a hardware store involved? And why is John’s relationship with some random lawyer relevant to anything?</li><li>In what currency did this ‘payment’ take place, if any?</li><li>What if John didn’t know the lawyer? Could he still buy the goods?</li><li>Is this a realistic example or just a theoretical illustration?</li><li>Who issued the smart card? Is there a bank in this picture somewhere?</li><li>Why is this better than what we do now?</li></ul><p>Let’s try to answer them.</p><h3>The debt graph</h3><p>In computer science a diagram consisting of circles and lines between them is called a <em>graph</em>. The pretty data visualizations the word refers to in normal English is to us software guys always a “chart”. In a graph the circles are called <em>nodes</em> or <em>vertices</em> and the lines are called <em>edges</em>.</p><p>Graphs are important to us because they’re a very general concept, and it turns out there’s lots of useful bits of mathematics and software that can be applied to them. A simple example is a <em>social graph</em>, better known as a social network, in which the nodes represent people and the edges represent friendships. A less obvious example is the road network, in which edges represent streets and nodes are ‘junction points’ of some sort where roads split or join. Regardless of what the circles and lines mean there are general algorithms that let you answer questions like “starting from A, how do I get to B?”. Whenever you get directions from Google Maps, variants of these algorithms are being used behind the scenes (the so-called <a href="http://algo2.iti.kit.edu/schultes/hwy/esa06HwyHierarchies.pdf">“highway hierarchies”</a> algorithm).</p><p>Ripple starts with a social graph. The nodes are people or companies. The edges represent lines of credit and also to what extent that line has been used up. Again, Ripple isn’t really about technology so exactly how this graph gets managed is left undefined. Let’s just assume people have a smartphone app that is linked to Facebook or something, it lets them type in that they’re willing to loan a friend or business a certain amount of stuff, and this data ends up …. somewhere.</p><p>“Willingness to loan” is a very general concept. In the example above, John was willing to loan the lawyer some hours of his skilled labour to landscape the lawyer’s garden. On the other hand, the hardware store was willing to loan the grocery store some equipment the grocery store hadn’t yet paid for. Ripple doesn’t require any particular unit for lines of credit. Anything will do. You can use ordinary currencies, cryptocurrencies, hours of your own time, bushels of wheat, lines of code written. Whatever you like. However, as we shall see, the system becomes more complicated if people don’t use commonly agreed units, so let’s assume from now on everyone is denoting debt in dollars.</p><p>The core observation behind Ripple is that if Alice owes Bob $10, and Bob owes Charlie $10, and Charlie owes Alice $10, then if you handed Alice a $10 note she would give it to Bob, erasing her debt, Bob would give it to Charlie, erasing Bob’s debt, and Charlie would then give it <strong><em>back</em> <em>to Alice</em></strong>, erasing his debt too. The piece of paper would travel in a circle and end up back where it started, yet, nobody would be in debt anymore. And Ripple observes that the entire economy can be viewed this way, as scarce tokens traveling around and around in large and complicated loops, erasing debts as it goes.</p><p>A grocery store worker is a simple example of this in real life. When they work for a week the company is building up a debt to them, and when the company makes payroll it’s erasing that debt. The worker will then probably go back into the store as a customer and take something off the shelf, creating a debt to the employer, and will then use some of their salary to erase that debt as they walk out of the store.</p><p>What if you could detect this outcome <em>without</em> needing the actual bit of money to flow around in a circle? Then, theoretically, you could dispense with the $10 bill entirely. The moment Charlie types into their smartphone app that they owe Alice $10 for some reason, the system could detect that there’s a cycle in the debt graph and instantly erase everyone’s debts. And because all payments can be seen as just erasing debts, you could actually dispense with the tokens entirely. The dollar bill would remain useful as a shared understanding of value and pricing, but the dollars themselves aren’t necessary anymore.</p><p>Well, you can do that. There’s been decades of research into efficiently <a href="https://www.geeksforgeeks.org/detect-cycle-in-a-graph/">detecting cycles in graphs</a> and it’s a task programmers sometimes do solve. But there’s a catch: solving it for really big graphs (like a graph of the entire economy) is difficult, especially if you want that graph to not be held entirely on one computer. Hence the programming problems I alluded to earlier.</p><h3>A more complex example</h3><p>In Ripple the graph tracks two different things:</p><ol><li>How much debt <em>actually</em> exists between nodes (people/companies).</li><li>How much debt <em>is allowed to exist</em> between nodes.</li></ol><p>To make a payment the Ripple system does a <a href="https://en.wikipedia.org/wiki/Pathfinding#:~:text=Pathfinding%20or%20pathing%20is%20the,path%20on%20a%20weighted%20graph.">pathfinding operation</a>. It’s searching for one or more routes between you and the seller that have enough of a gap between the amount of debt that exists and the amount that can exist. If it finds such a set of routes then new debts are created along all the found paths, reducing the amount of credit available for next time. This is kinda like how Google Maps can show you multiple routes through the road network and take into account traffic whilst doing so, but the map is of financial relationships instead of streets.</p><p>When John bought some groceries the store let him walk out because the system told staff that indirectly the grocery store owed him stuff: the store owed the hardware shop, the hardware shop owed the lawyer and the lawyer owed John. The store doesn’t know or trust John but it doesn’t have to because by giving him food, its own debt to someone unrelated was reduced.</p><p>In a real deployment things wouldn’t work out so neatly. What if John’s bill is a big one, and he only did a couple hours work for the lawyer? Then the lawyer’s debt to John may be smaller than what he needs to clear his debt to the store. In this situation a few things can happen:</p><ol><li>Ripple may find more than one path through the debt graph, that sum up to the size of his bill. In this case the grocery store may find its debts with several different organizations or people all being reduced by a small amount.</li><li>Ripple may find insufficient or no paths. Perhaps John hasn’t worked hard enough, or he worked for people who have no debt connection to the store.</li><li>Ripple may find paths but they’re in incompatible units. For example John may have extended the lawyer credit in the form of “hours of landscaping labour” rather than dollars, but that’s no use for clearing a debt denominated in dollars.</li></ol><p>If not enough credit can be found then either John cannot pay for the food, or, the store may decide to let him walk out anyway and simply record the fact that he’s in debt to the store in the Ripple ledger. This decision can be made ahead of time by creating a credit edge in the graph, e.g. because John visited the store manager and asked. Again, he can clear this debt by doing work for someone else as long as there’s some chain of debts between that person and the store. In a vibrant enough economy where everyone used Ripple all the time for everything, that debt should eventually go away.</p><p>Another way of seeing this is as follows: when John gets paid for some landscaping, his ‘payment’ looks like a bunch of people and companies he interacts with showing up on his phone as owing him something. He can go directly to them and request payment in the form of tokens, which is how Ripple and a non-Ripple systems can co-exist, or he can simply keep those obligations open and watch them disappear as he pays other people for things.</p><p>How would loans work? If John wants to expand his landscaping business he could, in theory, spend lots of time convincing individuals to open lines of credit in the app for him, but in reality there’d probably be middlemen who do that convincing full time and aggregate the resulting lines of credit for a fee. We might call these institutions banks, in recognition of the similarity of their role. In fact the Ripple FAQ says:</p><blockquote>Ripple doesn’t preclude regular banks from participating. Your relationship with your bank right now is that you grant them near-unlimited credit, and your account balance is the bank’s IOUs that you hold. You may receive a small rate of interest. If you have a bank loan, the bank has also granted you some credit, usually at a much larger rate of interest.</blockquote><blockquote>The exact same relationship can be replicated with Ripple, by granting the bank’s Ripple node a large amount of credit at your Ripple node. Then you can send and receive payments through your bank like you do now. In fact, the entire banking and monetary system is just a very limited and centralized way of passing IOUs that evolved before we had the technological capabilities to search a complex trust network for connections between two people.</blockquote><blockquote>The way banks currently handle payments among themselves is very much like Ripple.</blockquote><p>Other financial institutions would also exist in Ripple. One obvious one is FX traders. If lots of people owe me pounds that’s of no use when I go to the USA, but if I open a line of credit with an FX broker of £1000 and they immediately extend me a line of credit back for $1380 then as long as the FX broker is well connected and widely trusted, I can still buy whatever I need.</p><h3>Ripple, taxation and inflation</h3><p>Both Bitcoin and Ripple justified their existence by talking about the inefficiencies, overheads and injustices of the banking system. Whereas Bitcoin framed these in terms of lacking digital cash, Ripple framed them in evolutionary terms:</p><blockquote>Since human beings have evolved to form trust relationships with other human beings, and not with institutions, the whole process is problematic, and requires a gargantuan legal framework of bureaucrats, lawyers, regulators, and accountants to stabilize it.</blockquote><blockquote>Ripple currency is underpinned by natural trust relationships that exist already within families and between friends. Bureaucrats and coercive legal frameworks are completely optional. The currency has value simply because people value their relationships with other people. It is financial capital backed by social capital.</blockquote><p>NB: this answer has a confusing reference to “Ripple currency”, even though this classical Ripple was not a currency.</p><p>Ripple was a child of its time. Like Bitcoin, it emerged in the wake of the financial crisis. This led to a few programmers (like Ryan, Satoshi and me) re-evaluating the basics of banking and money. Not surprisingly the problems they were talking about were the problems that seemed important in 2008, like “too big to fail”, the sheer size and complexity of the institutions and so on.</p><p>A modern argument for Ripple might look different. It’s been 13 years since the last financial crisis and people are currently focused on other issues, like rapidly rising inflation as governments try to print their way out of the ruinous economic destruction of lockdowns, or the privacy and control issues posed by tech companies. Bitcoin also justified itself via reference to these problems and is thus in a better position to capitalize on modern day concerns, but Ripple is equally capable of solving these issues.</p><p>An economy that uses Ripple is immune to governments creating inflation via money printing, because no actual money is being swapped anywhere. ‘Legacy’ currencies are used only to establish shared understandings of the prices of things. Thus a government can create as many tokens in its own central bank computers as it likes, but it wouldn’t be able to spend them into the economy because it wouldn’t have new lines of credit in the Ripple ledger. Therefore in a pure Ripple economy a government would have to implement taxation by forcing people to open lines of credit to the government on a regular basis. Anyone who refused to do this would be penalized in the normal manner, or the government would have to take over the ledger somehow and directly edit the graph to add the debts.</p><p>From a libertarian economics perspective this is very interesting because people often don’t recognize inflation as a form of tax. Governments like to present it as a sort of semi-mystical force that’s outside their direct control. They measure it, they target it, but they can’t simply make it be a certain value. In reality all inflation is derived from the ‘printing’ of money by governments (normally obfuscated via mechanisms like bond purchases and control of reserve ratios, but it boils down to the same thing). The newly created money effectively gives the government an invisible first dibs on the economy’s resources. Forcing all taxation to be explicitly materialized in the would make it transparent to everyone how much work they are being forced to do for the public sector, which in turn would make the politics of tax and spending much more concrete than they are today.</p><h3>Ripple as a materialization of social capital</h3><p>In today’s world becoming a credit-granting agency is a very difficult thing, which is why not many people create a bank. Ripple makes it easy enough for everyone to do, or perhaps should I say, explores what life would be like if it was that easy.</p><p>One effect of this is that it lets you turn social capital (i.e. the trust of your friends) into actual capital (i.e. the acquisition of products and services). Phrased another way, Ripple expects you to ‘rate’ the people around you by specifying exactly how much you’re willing to get into debt on their behalf.</p><p>This would be a very new and uncomfortable thing. It’s the aspect of Ripple that people react to the strongest. In particular it opens up the possibility of social bankruptcy. The FAQ puts it like this:</p><blockquote>If you make payments through a friend and don’t repay that debt, your friend will be liable to the next link in the chain for those payments. You are putting your friend in uncomfortable position of having to make good on your debts to maintain their good name. They only granted you credit in the first place because they trusted you to not put them in this position!</blockquote><blockquote>Simply put, if you don’t pay someone back, you have hurt your relationship with them. What you have gained in financial capital, you have lost in social capital. If you go about not repaying your debts, you will soon have no one left to grant you credit. Money is useless without a community of people who will accept it.</blockquote><p>One of the implications of this is you can’t really say how much wealth you have access to. There are no bank balances in Ripple-world, only debts and credits. So whilst you may have a million bucks of spending power today, tomorrow you may have far less if there’s some sort of recession and people start withdrawing their lines of credit to you, or defaulting on their debts. This can of course happen in the present system too (a bank run) but governments are generally unwilling to let the ATMs break and will inflate the currency by arbitrary amounts to prevent it, so in reality people tend to view bank balances as fixed piles of gold that will stick around if not being spent … albeit that may become progressively less valuable over time.</p><h3>Making Ripple peer to peer</h3><p>A big reason Ripple never took off was that all usable instantiations were run on a single computer by one person. Bitcoin was truly peer to peer and thus could credibly claim to be immune to centralized takedown — an ever present risk for any project that seeks to revolutionize anything.</p><p>Although it was hardly noticed at the time, when I worked at R3 on Corda we actually implemented a P2P Ripple-like protocol. It was a competition entry for <a href="https://www.mas.gov.sg/schemes-and-initiatives/project-ubin">Project Ubin</a>, organized by the Singaporean central bank. The code was mostly written by Roger Willis, a friend of mine, and is <a href="https://github.com/project-ubin/ubin-corda">fully open source under the Apache 2 license</a>. The <a href="https://github.com/project-ubin/ubin-docs/blob/master/UbinPhase2-Corda.pdf">technical report</a> explains how it works.</p><p>The way banks are linked to each other is much like what Ripple proposed. During the day banks open lines of credit with each other on a big central bank operated computer. At night after the banks close that computer does something they call “payment compression”, “net out” or “deadlock breaking”. This searches for cycles of debt between the banks just like Ripple would do between people, and when found, the cycles are eliminated. The goal of this is to improve the capital position at the central bank. In fact this process is one of the reasons that bank payments are often still tied to the solar cycle.</p><p>For a variety of reasons like resistance to outages, cyber attacks etc, central banks became interested in whether enterprise blockchain technology could decentralize this process. It was a good chance to show off the power of the Corda platform (<a href="https://www.r3.com/wp-content/uploads/2019/08/corda-technical-whitepaper-August-29-2019.pdf">which I did most of the design work for</a>), in particular by pushing Corda’s unique “flow framework” to the limit. Flows are a feature that makes it much easier to write complex P2P apps compared to other platforms and we wanted a real world example of this in action.</p><p>It wasn’t exactly the original intent, because we weren’t sure how far we could get in the very limited time the competition allowed. Nonetheless by the end we had a decentralised, peer to peer cycle detection and clearance system able to operate at large scales. The code modeled both cash payments to settle debts and obligations, proving you could build a peer to peer Ripple-esque system in reality as long as you had a BFT consensus module and some sort of identity verification system (Corda requires nodes to have names, but those names can be anything e.g. pseudonyms, DNS names could also be acceptable in some networks).</p><h3>Conclusion</h3><p>Classical Ripple has been forgotten. It never took off like Bitcoin did and due to a name swap became hard to learn about. Nonetheless, the ideas in it are still as fresh as they were 15 years ago.</p><p>With time it became clear that cryptocurrency valuations weren’t stabilizing in the way we originally hoped, largely due (in my view) to the market manipulations practiced by central banks combined with lack of volume. The dream of a cryptocurrency becoming a useful global form of electronic cash died thanks to technical inadequacy, community takeover+infighting, and the 25%+ overnight value swings that made them nearly impossible to use as a working business currency.</p><p>Because it doesn’t actually create a new currency at all the classic Ripple concept should be much more immune to market gyrations. With Corda we showed that you could make the concept work in a technologically p2p and decentralized manner. And it would offer a way to escape the re-awakened dragon of consumer inflation.</p><p>Although I’ve moved on to other things, perhaps someone else may be interested in exploring this old idea. Let me know if you do!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=13fefe1870cf" width="1" height="1" alt=""><hr><p><a href="https://blog.plan99.net/how-did-classical-ripple-work-13fefe1870cf">How did classical Ripple work?</a> was originally published in <a href="https://blog.plan99.net">Mike’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>