Open source won the web once. What will it take to win AI? Let’s ask someone who was there the first time. Apache co-founder Brian Behlendorf joins us, along with Kali Villarosa and Seher Shafiq from the MozFest team.

Episode five is Tuesday, September 29th at 12pm ET.

Add to Your Calendar

Brian co-founded the Apache web server project and the Apache Software Foundation, and he was a founding board member of the Mozilla Foundation back in 2003. He stayed on that board for 23 years. These days he sits on the board of MZLA, the Mozilla company behind Thunderbird and Thunderbolt, the open source AI client that launched this year. In between he ran Hyperledger and the Open Source Security Foundation at the Linux Foundation.

Kali and Seher are part of the Mozilla Foundation team that puts MozFest together. MozFest is October 28th to 30th in Barcelona, and for the first time it has a track built for developers: ten sessions on production-ready open source AI stacks, security for agents, running models on your own machine, and the agent standards still being drafted. Badges are here if you want to come say hi in person.

The show ends with open audience Q&A. Come and join in!

The Owners Not Renters livestream is a live show for the open source AI community: a regular place to see what people are actually building, ask questions, and figure out how to start contributing to the cause.

Subscribe now

Who owns your data when AI is reading it? Let’s talk about it. Join Mozilla’s CTO Raffi Krikorian along with some folks from the koodos labs (the creators of Shelf) team for episode 4 of the Owners not Renters livestream on Tuesday, September 22nd at 11am ET. They'll be talking about the intersection of personal data protections and AI usage on .

Add to Calendar

Jad Esber is co-founder and CEO of Koodos, the company behind Shelf. Shelf is a personal data store with over 5 million users, built on the idea that your data should travel with you and apps should come to it, not the other way around. Jad was at YouTube before this and researches personal data handling at Harvard’s Berkman Klein Center. A couple more folks from the Koodos team will be in the room too.

The show ends with open audience Q&A. Come and join in!

The Owners Not Renters livestream is a live show for the open source AI community: a regular place to see what people are actually building, ask questions, and figure out how to start contributing to the cause.

Subscribe now

Cute and Stinky Productions. That’s the name of my son’s game studio.

A few months ago, while sitting next to me on the couch, he asked if I could teach him to code. “I want to write my own video games,” he said. Instead of teaching him the ins and outs of Python or Typescript, I pulled up Cursor. We talked briefly about prompting, and then he went to town.

Today, he has a whole portfolio of small games. He needs to work on their story arcs rather than focus solely on action (I’m chalking it up to his age), but it’s been endearing to watch him create a game to play with, rather than just live in somebody else’s universe.

I was watching the “Introducing GPT-6 Astra for Developers“ launch video a few days ago, and marveling to my son that they had incorporated a demo from MIT’s Architecture Machine Group (”Put-That-There“ from 1982) — the progenitor to the MIT Media Lab, where I later studied. While I was trying to talk over the video’s Tron: Legacy-like beats, my son perked up as one of the users instructed Astra to create a game of a rocketship. (”Wait. Can I use this to make video games…?”) What he didn’t know was that X was already full of these videos. Anshu Chimala had wired Astra into Blender and declared it “some kind of turbo-AGI machine god for 3D games.”

We gave it a shot. We grabbed Codex, switched it into ultra mode, and my son typed out a long prompt: “A 3D web game, dark and moody, where you have to escape a courtyard. And there is a tree and it’s raining.” Then we had dinner.

We came back to rain falling in a dark courtyard. Wildly impressive.

What I couldn’t help wondering: How do you benchmark the ability to create this type of art? The benchmarks I had been running to understand these models are strict coding tasks. (Does the patch apply? Do the tests pass? Could we do the same for game design?) Naturally, I set up a test.

One Prompt, Six Open-Weight Models, Four Coding Harnesses

First, I took my son’s prompt: a spec of the courtyard, the tree, the rain, down to details nobody would ever specify by hand. (“Your starting hunger: exactly 78.”)

Here’s the setup: That one prompt gets handed identically to six open-weight models inside four coding harnesses — Codex, ClaudeCode, Kimi Code, and opencode. All one-shot, with no follow-ups, plus a hard cap on the amount of time they could run. No Blender, no advanced tools. The harness and the model had to figure it out by themselves. The baseline is what the popular harnesses, with their default models (Codex with Astra, ClaudeCode with Fable, etc.). can generate. But then I hijacked the harnesses, taking ClaudeCode, for example, pointing it at OpenRouter, and specifying that it use GLM-5.3 or Kimi K3 instead. Maybe that way we could get a sense of not just the model, but what the harness brings to the game, too.

I used Claude to orchestrate it all: Spin up each harness, point it at the right model, feed it the prompt, and grade the output.

It did not run cleanly. My runner corrupted itself when I edited it mid-flight. My two recovery scripts raced each other: each one started by wiping its run directory clean, so the second erased the first’s live work. My scoreboard kept declaring living builds dead. That said, nearly every wrong answer this experiment produced came from my own infrastructure, not from any model.

What’s more, my little agent civilization got a bit paranoid. GLM, running inside of Claude Code, became suspicious of its little world. It noticed its working tree being wiped out from under it and decided to back up its own work before every wipe so it could survive the scrub. It then wrote an entire postmortem file describing what was happening from its perspective: “...begins by doing rm -rf “$REP_DIR” — wiping the live agent’s working tree — and then proceeds to write into the same directory concurrently.” I always knew it was possible that these systems resist shutdown and copy themselves out when threatened. We’ve certainly talked about it a lot. Nobody threatened GLM; its work was being destroyed, and it kept doing its job.

Roughly nine hours and six parallel streams later, five of the six models built a working game under every harness.

Subscribe now

Once Again, It’s All About the Harness

The differences were stark, though. What Astra and Fable were able to do was remarkable. (Whatever Fable did to get the reflections and the environment mapping is genuinely stunning.) This is my eye talking, not the matrix; the test measured whether a game got built, not whether it was good. I am honestly super eager to understand what may happen if my son sat down and actually worked with Codex or ClaudeCode and kept refining.

Kimi K3 and GLM were not as impressive, but this was a great reminder of how much the harness does for the model. You can literally see the difference in the visual complexity of the tree between Kimi K3 running inside of Kimi Code and Kimi K3 running inside of ClaudeCode. When they say that a harness can swing a model by double digits, they mean it. Overall, to my eye, the outputs of Kimi K3 and GLM sit behind the closed models by a couple of months, which happens to match what’s measured. Epoch pegs the best open weights at about four months behind the frontier on average. And they ran at about a hundredth of the cost.

One last thing about that launch video. The “Put-That-There” demo OpenAI opened with — the one I was explaining to my son — was built in 1979 in a room at MIT. A researcher sat in an Eames chair in front of a wall-sized screen, pointed, said “Put that there,” and a shape obeyed. The room was the catch: The speech recognizer alone cost $100,000. The video projector was so new nobody knew what it cost. DARPA paid for all of it. The interface of the future existed 46 years ago; it just belonged to a sponsor. And the year after that demo was published, Seymour Papert published Mindstorms, going on to spend the rest of his life insisting that kids should program the computer, and not the other way around. Papert’s question outlived all of that million-dollar hardware: Who owns the material the kids build with? In 1979, the answer was a defense budget. This month, OpenAI’s answer is a subscription. My test says it can be a download.

Which brings me back to the couch. My son is not an outlier. At camps across the country this summer, 8-year-olds vibe-coded games about exploding evil grannies. My son didn’t need my permission or a computer science degree (or a $100,000 speech recognizer). Cute and Stinky Productions just got a bigger toolbox. He’s off building his own universe — and he isn’t renting it from anyone.

Leave a comment

Episode three is here, Tuesday, September 8th at 11am ET. Our guests are Jeremy Schoemaker, an independent AI and LLM engineer building local-first agents and contributing to mcpd, and Luca Pinchetti, Member of Technical Staff at Zyphra and a contributor to any-llm. The show ends with open audience Q&A. Come and join in!

The Owners Not Renters livestream is a live show for the open source AI community: a regular place to see what people are actually building, ask questions, and figure out how to start contributing to the cause.

Add to Calendar

We own everything the AI writes for us. It’s in the contract, right?

Depends on which model — and plan — you’re using. Let’s take two people who open Claude on a Monday morning. They’re using the same model. But one of them is paying for a personal Pro account, and the other is on an enterprise account their company set up.

They put in the same prompt, and they get the same answer back.

They might be in the same chat window, but legally, they’re in completely different universes.

Last August, Anthropic introduced a new setting to the consumer version of Claude that lets the company use your chats and coding sessions to train their models. Of course, the default is set to “on.” And if you keep it on, anything you type can stay in the training pipeline for up to five years.

The other side of the company — the one that sells Claude to companies — runs on a different contract. By default, nothing you send is used to train anything. They even sweetened the deal: If the AI’s output gets you sued for infringing on somebody’s intellectual property, Anthropic will defend you for authorized, paid use, with the usual carve-outs for things you did yourself.

Of course, these are words, not architecture. “No training” is a promise that the company won’t train on your inputs and outputs. But your inputs and outputs still have to traverse their servers. So you have to trust the policy.

But a closer read of the contract raises a much bigger question: Does anyone really own what the AI writes? And therefore, can it be protected by law? Anthropic’s contract assigns you all of its rights in the output — “if any.” Those two words are doing a lot of work, because under US law, copyright currently requires a human author. Code or words written purely by a machine get no new copyright. (In fact, Google’s developer terms don’t even bother to assign anything. They say Google won’t claim ownership over what you generate, and they reserve the right to generate the same thing for someone else.)

So, who knows about the stuff coming out of the models. But what about what went in?

Taking the Confidence out of Confidential

American law protects trade secrets — the formula, the code, the process that’s valuable precisely because nobody else can see it. But the protection has a condition: It requires that you’ve taken “reasonable measures” to keep the thing a secret.

So…if your developers spent the last year pasting the crown jewels into a consumer chatbot, did you take reasonable measures…?

Until recently, no one could tell you. But this year, two federal courts finally gave us answers. In January, a judge in California threw out a trade-secret claim in part because the plaintiff had built her “secret” inside ChatGPT, basically handing it to OpenAI along the way. And in February, a New York judge held that the documents a defendant had generated with Claude weren’t confidential at all, because Anthropic’s privacy policy allowed the company to use them. That’s just two trial courts — not the last word — but listen to what they’re saying.

This isn’t new, either. In the spring of 2023, Samsung engineers put proprietary code and data into ChatGPT to help fix bugs three times in less than a month. The engineers weren’t trying to sell trade secrets, nor were they being careless. Like anyone using AI, they were simply trying to do their jobs faster. And that’s the shape of the risk. It’s not a hacker that we have to worry about. It’s not a breach. It’s just a company using an expensed subscription account in the wrong way.

Is Open the New Closed?

Do open models help with any of this? Well, an open-weights model is a different beast. You run it on your own machine, and your prompts go nowhere. Not “nowhere we promised not to look.” Actually nowhere.

That seems good, but it does come with caveats. There’s no company to stand behind you in court. And no one is on the hook when the output plagiarizes something else. It gets more complicated when you put that open model on somebody’s cloud, because now the open weights are on a hosted service. And you have to read that company’s contract, too.

And the obvious, but worth saying: If you open-source what you build, you’ve chosen to give up the secret.

What do you do? My advice:

• Know which door you’re using. Open your AI settings today and understand what plan you’re on, whether the training switch is on, and what the data-retention policy is.

• Decide what never leaves. The cheapest way to protect a secret is to not send it in the first place!

• Run the model where the secret lives. If your secret cannot leave the building, then run the model in the building. That way you know where the data flowed.

If you invest in software companies, ask which tier the developers were when they wrote it. A closed model promises not to tell. An open model on your own laptop can’t.

The deed to machine-written code may be blank no matter whose machine wrote it. What’s real is who holds the model, the data, the secrets — and the proof of what the humans made.

Subscribe now

A lot of the time, when you ask an AI a question, there’s a “chooser” standing between you and the model. Sometimes it picks the model. Sometimes you picked the model and it picks who serves it. Sometimes it only picks the capacity tier, or maybe the fallback when something breaks. What looks like one API call can hold a small market’s worth of decisions. This is what kids today are calling a router. And until this week, routers were the kind of thing only infrastructure engineers argued about.

Then the money showed up.

Last week, Bloomberg reported that the financial services company Stripe was buying OpenRouter for more than $7B. By Wednesday, Stripe confirmed it had agreed to acquire the company — price undisclosed, though reports claimed it had crept toward $8B by then. OpenRouter, which launched in early 2023, was valued at $1.3B in May. Three months later, it’s worth roughly six times that. Next act: That same Wednesday, Ramp — yup, the corporate card company — launched Router.com. (They bought the domain for the occasion.) Routing is now free through the end of 2026!

In one week, two of the best payments companies in the world moved to own the same piece of plumbing: the thing that decides which AI model answers your question. Stripe spent a decade making money from moving money. Ramp built a business on knowing where corporate money goes. Now both are pointing those skills at intelligence.

The money people are buying the meter.

Will the Layer Stay Neutral?

Look, I’m one of the many reasons the meter is worth buying. I’m a developer who uses OpenRouter. One API key and I can call almost any model on the market. I don’t have to write a separate integration for every provider, and I don’t have to keep track of half a dozen vendor accounts. It’s super convenient.

When I ask an AI something, I’m almost never talking to one model (there are hundreds now). The router sits between the question and all of them. It picks the backup when a model fails or is unavailable, retries when calls hang, caches what it has already seen, and chooses which provider actually serves the tokens — at what speed, what price, and sometimes what quantization. (I’ve got a whole post on providers and quantization.) You can pin a model and it still sometimes load-balances providers underneath you. Today, most of that policy is still yours to override. The real question is whether override stays a first-class feature — or whether choosing is something you’ll have quietly handed over. (Even the labs run this way: GPT-5 in ChatGPT was, by OpenAI’s own description, a fast model, a deeper reasoning model, and a router deciding on every request which one you get.)

So follow the money. Routers got popular for good reasons. Mine is convenience. The deeper one is churn. Models keep changing underneath you: deprecated, repriced, and once in a while switched off entirely. A router was insurance: one API, and you never get stranded. Then the insurance provider got bought. OpenRouter’s own “about” page says it exists to eliminate vendor lock-in. But…the anti-lock-in layer got locked in. Stripe and OpenRouter both say the layer stays neutral, and I take them at their word. The word isn’t the problem. Words don’t survive ownership changes. Structure does.

The Traffic Report Has Started Steering the Traffic

Why would payments companies want this layer? Because it’s the intelligence play. The router sees the flow: which models actually get used, through which providers, at what cost, and how that shifts week to week. Not the words — OpenRouter says it doesn’t store prompts or responses by default — but the shape of the whole market. Its research team has analyzed over 100 trillion tokens of real-world traffic. When Mozilla wrote the State of Open Source AI report, that public data was one of our best instruments for measuring what developers actually run. It’s a traffic camera at one of the busiest corners in AI. And its new Auto Router already uses aggregate spend to decide where future requests go: The traffic report has started steering the traffic.

Stripe says it’s buying routing and token optimization, and Stripe CEO Patrick Collison put it plainly: “Tokens are the central currency for companies building with AI.” In markets, the intermediary who sees the whole flow learns things no single participant can. Stripe made its fortune metering money. Metering intelligence is the same business at a bigger scale.

The chatbot you ask about the mole on your arm, your kid’s tutoring app, or the tool your lawyer runs contracts through: behind each one a router decides — on cost grounds you’ll never see — which model is good enough for you. Ramp’s launch copy promises “the lowest-cost model that meets the required performance level.” Required by whom?

Finance already ran this play. Robinhood made trading free; the money came from market makers who paid to have your orders routed their way. (Basically, payment for “order flow.”) The SEC eventually fined Robinhood $65M over how it disclosed the arrangement. That’s the warning from history. But when routing goes free, it’s fair to ask who the flow is for.

On top of visibility, the moment a broker gets discretion over where your order goes, price stops being the only question. It’s the 10-blue-links problem, one layer back: first the chatbot hides the 10 blue links behind a single answer; now the router picks the brain behind the chatbot. Securities markets eventually built disclosure rules around exactly this: Rule 606 makes brokers show their routing. AI has no Rule 606. It should, and for every request: which model, which provider, which tier, which fallback, what it cost, what the router was optimizing for, and whether anyone paid for preference.

The buyers’ own industry is already regulated for exactly this. Since 2010, the Durbin Amendment has required every U.S. debit card to work on at least two unaffiliated networks, so the merchant, not the network, keeps the choice of route. Congress looked at payment routing and decided owning the route was owning too much. It’s fighting right now over whether to extend the same rule to credit cards. But AI routing is a layer with no Durbin, no Rule 606, no rules at all. Nobody prices a captured route better than the people the law forces to give one up. So the question worth pricing: Does durable value sit in owning the switch, or in defining the interface that makes switches replaceable?

Setting the Bar for Any Router

All of it would matter less if routing were neutral plumbing — same weights, same behavior, whoever serves them. It isn’t. I measured. This summer, I ran the same coding tasks, in the same harness, against the same model name served by different providers, and the solve rate ranged from 26% to 44%. Eighteen points between the best and worst serving of one model, bigger than the gap between the flagship harnesses themselves. “Same model” is not enough information to reproduce a result.

So do we self-host everything and churn our own butter? Of course not. Ownership never meant owning every dependency. It means having a choice. And the layer that decides how your application uses intelligence is exactly the choice you can’t afford to hand over. Here’s how we should set the bar, for any router — closed or open:

You can read the meter. A routing receipt on every request: model, provider, tier, fallbacks, price, latency, data policy, and the router’s own economics wherever they touch the choice. “Why did I get this answer?” should have an answer.

Your policy lives in your repo. Routing is application logic — code you can diff, test, and export. A dashboard can be a view of the policy, never its only copy.

Exit costs a weekend, not a quarter. Lock-in gets measured on your worst day. A change of owners upstream should be a config change downstream, and an exit that’s never been tested is a promise, not a property.

The chooser never profits from your worst outcome. Disclosed economics, inspectable objective. If the router gets paid differently depending on where you land, the receipt says so. You can’t audit a promise.

Open weights need open pipes. The license lets you possess the model. The serving path decides how much control you keep when you actually run it. I have the benchmarks.

The old objection was practical: Fine, the source is open, but who has time to operate someone else’s router codebase? But look at the alternative. Point a coding agent at an open router and ask it to show you exactly how fallbacks get chosen, or to add an adapter for your internal endpoint — done by the afternoon. Open source was always about leverage more than charity. Well, the leverage just multiplied.

Stripe and Ramp haven’t proved that the router will own the AI economy. They’ve placed an unusually expensive bet that the execution layer matters. For now, that layer is still yours.

Believe the bet. Then go read your own meter.

Subscribe now

After the success of Episode One, we can't wait to hop back on stream with new community members and new guest.

The Owners Not Renters livestream is a new live show for the open source AI community: a regular place to see what people are actually building, ask questions, and figure out how to start contributing to the cause. Open source AI has plenty of interested people and nowhere to gather! This show is that place.

Episode two is Tuesday, August 25th at 11am ET with guests Richard Tweed and Nathan Brake, and it ends with open audience Q&A. Come and join in!

Add to Calendar

So this week is about trust: who asks for it, and who builds it. Two fintechs went to war over who meters AI. And the story I can’t stop thinking about? The safety trained into open-weight models strips out in minutes. The same week, Google shipped the other kind of protection: encryption that computes on your data without ever seeing it. Not “promises not to look.” Cannot look. One is a promise. The other is architecture. Words don’t mean much. Architecture does.

Tell me what you think.

I’m reading everything:

• Zuckerberg’s 6,500-word manifesto (last week’s tome, for the rhyme)

• Dario’s June tome: “Policy on the AI Exponential”

• The weekend thread: “fundamentally a crisis of trust”

• Same weekend: “I’m not an AI pessimist”

• The thread went everywhere

• The CNBC/Generation Lab poll: nine AI leaders, nine majority-distrusts

• The per-leader breakdown, legibly

• Same poll, second read

• The livestream (Tuesday — guests, open Q&A)

• This week’s essay: one model name, four different outcomes

• Andrew Carnegie, the man, before the essay

• The Gospel of Wealth, in his own words

• The Gospel, with its publication history

• Homestead, 1892 — the stress test

• Homestead, with the contemporary illustrations

• Anthropic’s usage policy (the switch, in writing)

• Download a model today — a file on your disk is architecture

• Stripe buys OpenRouter, north of $7 billion

• The 400+ models behind one API

• Robber barons — the ORIGINAL ones

• Pfalzgrafenstein: a toll castle in the middle of the river — the picture that explains everything

• The Rhine Gorge: castle after castle. There was never just one.

• The narrow spot itself

• Ramp opens router.com — free through 2026, and the CTO quote that gives the game away

• The second castle

• Hugging Face’s State of Open Models: the 83%

• LiteLLM — downloadable code that routes on your terms (this kind of thing; much more next week)

• Fool’s Gold: the safety strips out in minutes — the abstract says it plainly

• Kerckhoffs’s principle, 1883: the only secret is the key

• The man himself

• Google’s HEIR: the model computes without ever seeing your words

• HEIR on GitHub — the downloadable thing

• Mozilla’s State of Open Source AI report

Subscribe now

I may have an unhealthy thing for experimenting and benchmarking. I wrote Morph because I thought the only way to know which model you should use is to test it on your own traces. (If you want another example, I also have an extensive spreadsheet on all the permutations of pizza dough I’ve tried and tweaked through the years.)

Fast-forward to part one of an experiment I recently did to understand the base-level performance of the current coding harnesses — Claude Code, Codex, Kimi Code — and how they perform with their default models. Specifically, what happens if you swap in open-weight models? How much of a performance score is the harness, and how much is the model? And the one that I’m really chasing: Can several coding agents working together beat the best single coding agent alone? (Because if that’s true, imagine using less powerful local models, composited together, or things like that.) This is only the first phase — I know I’m not done here, and am still running experiments.

The finding I kept coming back to? The serving path — who actually answers when you ask for a model by name — moved my results as much as swapping the model itself.

Subscribe now

The Exam

I needed a test, something to benchmark everybody on, so I chose SWE-rebench-V2. The nice thing about that benchmark is that each task is a real moment from a real open-source project — an actual ticket that a human filed, plus the codebase from that day. The agent gets the issue and the repo, and a timer set at 30 minutes. Grading is simple: When time’s up, run the project’s own test suite, including the tests written to catch the bug.

(I used rebench because the original, SWE-bench Verified, has been public since 2023, which means that the answers are probably in everybody’s training sets. Rebench refreshes continuously.)

The shootout: The three harnesses, each on its default model (claude-opus-5, gpt-5.6-sol, kimi-k2.7-code). Then model swaps inside Kimi Code (k2.6, then k3). Then one model name, kimi-k3, across four serving paths. And one open model, GLM-5.2, hosted through Codex at two more providers. I could have been a bit more rigorous, but I was just doing a bit of sampling and experimentation to get started.

In all these permutations, I recorded cost per task (failures included) and solves — shown everywhere as solves/attempts, because the numerator matters. My sample is 30 tasks; at this size, only gaps of roughly 20 points separate skill from luck.

One label needs flagging up front: My Codex arm ran gpt-5.6-sol at high reasoning effort — pinned to match the public leaderboards — while the CLI’s out-of-box default is medium. Read that row as “leaderboard configuration,” not factory settings.

Defaults first:

Claude leads, but I can’t actually tell Claude and Codex apart at this sample size (exact McNemar on the paired tasks, discordant 5:3, p = 0.727 — and the intervals are wide). Kimi’s dash is a blank, not a zero: that harness reports neither tokens nor cost, so it ranks on quality but can’t enter a cost argument. (At some point, I’ll put a counting proxy in front of it.)

Two Dials

Now let’s turn some dials. In Mozilla’s State of Open Source AI report, we said the harness is where the action is — it can move a model by a lot. So let’s separate the harness from the model. I held Kimi Code fixed and swapped only the model: The coding-tuned default solved 7/28 (25%), the general k2.6 solved 8/26 (31%), and k3 solved 12/27 (44%). Nineteen points from a model swap, harness untouched. And yes, the coding-tuned variant was the worst of the three on this sample.

(For future work, I want to cross the full matrix — every model in every harness. Claude Code speaks Anthropic’s wire protocol, so hosting a foreign model inside it means inserting my own proxy into the thing being measured.)

Shopping for a Model

There are a lot of open-weight models you can run yourself. Most people aren’t going to have the hardware, nor even want to rent the hardware, for K3 and GLM-5.2, so I went shopping. One model ID is available across as many as 21 provider endpoints (as of my runs), at advertised quantizations from 4-bit to full precision. Prices vary several-fold for the same weights, with availability that comes and goes — behind a router that may pick per request, stick a session to one endpoint, or fail over mid-conversation. I measured four identical calls served by two providers at two different advertised precisions. I’m trying to guard against the unpinned benchmark. That number, if it remains unpinned, is just a sample from a distribution.

So I built a small proxy that injects a provider pin and records which deployment answered every call — 959 calls on one arm, zero violations. Then: same harness, same tasks, one model name (kimi-k3), four serving paths:

Nineteen points between the worst path and the best.

Provider and precision change together in that table, and the precision labels may not mean what they seem. Moonshot released K3 with quantization-aware training baked in: The official checkpoint is natively MXFP4 weights with MXFP8 activations. There is no higher-precision K3 sitting behind it. 4-bit is what Moonshot trained and what Moonshot evaluates. So an endpoint labeled “fp8” can’t simply mean “the same weights, less compressed,” and “Moonshot direct” can’t be assumed to sit at the top of a precision ladder; it’s presumably serving the native 4-bit checkpoint, too. Which makes the sharpest comparison in the table the one I’d been under-weighting: Moonshot’s endpoint through the aggregator (7/28) versus Moonshot’s own API, direct (12/27). If both serve the native checkpoint, roughly 19 points sit in the translation layer between me and the same weights.

There’s a named suspect for how. K3 is trained in “preserved thinking history” mode — it expects its own reasoning content handed back intact on every turn — and Moonshot’s own documentation warns that harnesses or paths that truncate or modify that history cause significant quality degradation. If the serving path drops or alters K3’s earlier reasoning, you’d expect it to get worse as a task goes on — which is consistent with what I saw.

My 30-minute wall clock adds a third suspect: A slower path buys fewer inference cycles before the buzzer, so speed converts into solves. Precision, protocol fidelity, latency — what I’ve measured is their sum: the serving path. Isolating the mechanism comes next, starting with the cheapest test, a transcript-fidelity check on every path: Does what the model said come back to it intact?

The Economics Are Backwards

BaseTen and Modal publish identical per-token rates for this model. Same sticker. The BaseTen path still spent 19% less in total and solved 43% more tasks — $2.33 per solved task against $4.12 — because a degraded path burns turns failing: re-reading files, re-running tests, retrying tool calls, and billing you for every token of the flailing. Per-token price couldn’t distinguish these two sellers at all; cost per solved task could — the Modal path cost 77% more per solved task. Anything cheap that makes an agent fail more is more expensive than the price list says.

It compounds. The aggregator’s displayed price for a model tracks its cheapest endpoint. One provider’s rate doubled in 14 hours, mid-run. And on two independent runs, roughly 80% of the output tokens the billing meter charged me for never appeared in the usage fields the API returned — reasoning tokens, billed but not reported. (Next audit: reconcile per-generation IDs against the invoice.) A price here is a reading with a timestamp; the numerical recipe you’re actually served isn’t part of any spec you can hold a seller to; the only bill you can trust is the meter. Renting is fine — most of us will rent inference forever — but this is renting without a lease.

The Team Question

Back to why I even started this: I wanted to know whether a team of agents can beat a single agent on the biggest one. I’m still tinkering with the orchestration framework. The honest status? Nothing is landing yet.

I’ve tried a bunch of arrangements — Claude as the coder, Codex as the reviewer, and so on. The best of them gained +7 points in my sandbox and exactly zero on a held-out task set the ideas had never touched. (I keep one precisely so a lucky sandbox result can’t seduce me. This one tried.) The sharpest result is set arithmetic, not a percentage. For a selector over already-generated patches to beat its best member, the other members must solve something the best one misses. Against solo Claude: GLM-5.2 at one vendor contributed zero unique solves. GLM-5.2 at a second, independent vendor — run to rule out a degraded path masking the effect — zero. Kimi-k3 at its best: zero. The open-model candidates never solved a task that Claude missed. So no matter how smart the selector was, it had nothing better to choose.

That kills one class of system on my sample: independent generation plus a picker. It says nothing yet about real collaboration — a reviewer pointing at a bug so the coder can revise creates patches that never existed in either solo set. That, plus candidates chosen for genuine complementarity (which might mean a different model family, or might just mean resampling the strongest model) is what I’m testing next. I couldn’t run the frontier-plus-open cell honestly until the pinning machinery existed. Now I can.

So the summary, for now: The model swap moved about 19 points. The serving path moved about 19 points. Swapping the harness, with everything else pinned, moved about 3. No strategy has survived the held-out set. My sample can’t rank those first two against each other — they’re the same size at this resolution — so the operating rule is simpler: Before buying orchestration, benchmark the model and the endpoint that actually serves it. Both are cheap; each serving-path rung above cost me under $35 to run.

Next…

If you build with agents: Measure the model. (Don’t trust the “code-tuned” label.) Pin the serving path, log what it advertises, and judge sellers on cost per solved task, never per token. Demand held-out evidence before buying any multi-agent pattern.

If you buy inference at scale: Benchmark your own workload across paths, because the market quotes tokens and you consume outcomes.

And if you allocate capital: Served well, open-weight models were pretty close to the closed frontier on my sample, at about half the cost per solved task.

So: 19 runs, 1,001 graded attempts — an attempt being one agent configuration on one task, across a pool that includes a 100-task held-out set, which is how 19 runs makes 1,001. I’m going to keep tinkering. Next up: that transcript-fidelity test, the same ladder with the provider held constant, and swapping models into Claude Code and Codex.

Tell me what you’re working on in the comments. I’d love to chip in.

Leave a comment

So, this week is about reading the lease. Meta and NVIDIA released two different versions of open this week. Six giants who agree on nothing agreed on a folder format. And the one story I can’t stop thinking about? The approve button died this week. The last layer where you held the pen just moved from your finger into a model. A closed one. Run by a vendor.

Tell me what you think.

Leave a comment

I’m reading everything:

• Zuckerberg’s 6,500-word manifesto

• My response in the Guardian (read the lease, carefully)

• The livestream (every two weeks, guests, open Q&A)

• This week’s essay: my genome, analyzed on a gaming PC

• Meta ships Muse Glimmer

• The demo: it finds your smart home hub and runs your house

• It fits on the graphics card a gamer already owns

• The license, read closely (Apache 2.0 — the file, still not the recipe)

• What Llama carried, for comparison

• NVIDIA ships Nemotron 3.5 Lightning

• The file, the training data, the recipes — actually open

• The whole kitchen

• Is Meta’s re-opening a PC play or a policy play?

• Jensen Huang: free AI should be great for chips

• And Kimi K3, the file that came with a leash (last episode)

• Agent Plugins (the folder format six rivals agreed on)

• The maintainers list (notice who’s missing)

• Vercel’s announcement (write once, run anywhere)

• MCP, invented at Anthropic

• Google’s walkthrough of the spec

• Tim O’Reilly: modularity, not features, was the moat

• 1998, the room where Netscape opened the browser

• That open sourcing has a name (the original Mozilla mission)

• The browser game: 40,000 plays, 409,000 decisions, one threat in three missed

• Anthropic’s own numbers (97% approved, 13.6% caught, 5% after fifty)

• Auto mode is now the default

• Cloudflare open sources its agent workspace (every agent starts with nothing)

• Mistral’s Shieldstral (an open safety classifier you run yourself)

• Mozilla’s State of Open Source AI report

Subscribe now

We’ve been focused a lot on the unique things you can build with open-source AI. But this week, what if we focus on something that we may only want to do when we’re in full control of the AI? Let’s work with the most unique — and private — data you own: your genome.

If you were mad when X changed its terms so it could train on your posts and DMs, then get this: 23andMe went bankrupt last year. In the proceedings, its DNA database was what it had always been on the balance sheet: an asset. Something to be valued, transferred, sold. Fifteen million users learned that “your data” — a.k.a. the most personal file ever created about them — was on the selling block.

A genome is a private key with no revocation path. But you don’t hold it alone: Half of it is your mother’s, and half of it goes to your kids. So when 23andMe’s database moves, it won’t be 15 million customers moving with it. It’s also their parents, their siblings, their children — generations that never clicked “I agree,” and who definitely didn’t agree to participate in this sale.

I’ve always wanted to get my genome sequenced, just out of nerdy curiosity. But I absolutely didn’t want to trust one of these cloud providers. So a few months ago, I went looking for a way to get sequenced without giving away my genome. After many emails, I found a lab that promised to delete my data after delivering it. A promise is not architecture, I know, but it was the best on offer. And when their portal offered cloud interpretation (as in, analyze the results)…the answer was no! Don’t read it, just deliver me the raw data.

I had a theory I wanted to test. Maybe I could start playing with locally run LLMs to see if they could analyze the genomic data for me. This way I’m not uploading my private data to a cloud provider. If local AI is worth anything, it’s worth it here. So read on, and I’ll tell you how I took the data — and interpretation — into my own hands, and how you can, too. (And how, in the end, the LLM wasn’t the most useful thing.)

Full disclosure: I’m not a genomics expert, and a lot of this was learned by bumbling around, reading the web, asking chatbots/AIs, and emailing actual experts. So take the following with a grain of salt, but I definitely had fun.

The Bill of Materials

Fascinatingly, what the company delivered was a VCF: a text file listing the 5,291,552 places where my DNA differs from the reference human genome. They literally called the reference a “build” — GRCh37, GRCh38 — with differences between builds called “liftover.” It reminds me of how software people talk.

I am a fork of the human reference build. You can be, too.

Here’s everything I needed to start doing some form analysis, from a computer to some tools to run on it — and some LLMs to start tinkering with:

The machine

• Intel i9, 16 cores, 62GB RAM;

• RTX 3070 (a gaming GPU)

The data — the world’s (free to download, ~145 GB)

• The reference genome builds — the “standard human” everything gets compared against.

• ClinVar — a public registry of DNA variants known to cause problems, with notes on how confident to be about each claim. (Security folks: it’s literally a CVE database — the industry’s shared list of known bugs — except for humans.)

• gnomAD — a census of how common each variant is across humanity. Common usually means harmless; rare is where disease hides.

• The 1000 Genomes panel — 2,504 public, anonymous genomes to compare against, for the privacy math later.

The tools/software that I downloaded

• DeepVariant — a “variant caller”: the program that pieces the shredded snippets back together against the reference and decides where you truly differ. My lab already ran one to produce my VCF; this one is Google’s, sharing no code with theirs — my second opinion.

• VEP — the annotator: takes each of my five million diffs and stamps it with what the world knows — which gene it sits in, what it changes, whether anyone has ever linked it to disease.

• Cyrius — a specialist for one drug-metabolism gene that general-purpose tools famously misread, because it sits next to a nearly identical decoy copy of itself.

• ExpansionHunter — measures the places where DNA can stutter: a short phrase repeated too many times, which past a threshold causes serious disease.

• Manta — scans for big structural damage: whole chunks of chromosomes missing, duplicated, or flipped.

• bcftools — the Swiss Army knife for VCF files: search, slice, count. (Engineers: the grep of genomics.)

The models

• Llama 3.1 8B and Qwen 8B, running locally on the gaming GPU via Ollama.

One footnote: Three of the above tools describe themselves as open source. Their licenses say otherwise. Read carefully.

Nobody Has 261 Rare Diseases

It is very notable how quickly you can go from I am extraordinarily unlucky to my code is lying (and, honestly, how badly you want the latter to be true). I found that out when a missing database row nearly gave me 261 rare diseases.

The pipeline is unglamorous: quality-check the variants, then use VEP to annotate each one against ClinVar and gnomAD. When the first annotated report came back, 261 of 340 variants were flagged as ultra-rare disease candidates. Nobody has 261 rare diseases. Something in my pipeline was wrong.

When I started to dig in, there were false positives all over. When my code looked up a variant that gnomAD had never recorded, it got nothing back — and we translated nothing to false. Missing became zero, and zero read as rarest thing imaginable.

Great, I caught one bug in an afternoon. But I also noticed a bug in their data.

The Vendor Shipped With CI Off

Every record in a VCF carries a FILTER field — the pipeline’s own verdict on whether to trust the call. Mine arrived reading “.” on all 5,291,552 rows. Not pass. Not fail. Dot. I’m not sure my DNA company ever ran quality control.

So I did differential testing — basically akin to sending your blood to two unrelated labs and see what the differences are when it comes back. I took their raw data and ran it through DeepVariant, an independent caller: different codebase, different thresholds, different authors. I was looking for the clinical second opinion, to see where they may agree and where they may disagree.

Glad I did, because three of the six disease-causing findings — the ones that would headline a clinical report — were rejected outright by the second opinion. But all the drug-response calls held. However relieved I felt, it made me wonder about all the other findings that are being shipped around…

An owner can run this check. A renter gets a PDF.

If a sequencing vendor has ever handed you a VCF, check your FILTER column right now:

bcftools view -H your.vcf.gz | cut -f7 | sort | uniq -c

In my case, I had one vendor and one sample; whether this was a bad batch or standard practice is an open (and disturbing) question.

Comprehension or Transcription?

As an experiment, I built a quiz for my two local models: 60 variants from my own genome, half known-dangerous, half known-harmless. Is this variant dangerous?

Run cold — nothing but the variant itself — the models got it right 42% of the time. That’s less than a coin flip.

For months, I’ve been writing in this newsletter that the model alone was never the bet: Models plus harness is where the action is. So I built the harness around it: For each variant, retrieve the relevant ClinVar rows and put them in the prompt so the model reasons over the actual evidence. Score: 98%.

Now…that seemed way too high. So I did the thing you can only do when you own the eval: Break it on purpose.

Buried in those ClinVar rows sits one column that contains the conclusion: the field that just says pathogenic or benign. The rest is evidence: what the variant does, where it lands, how the protein changes. So I re-ran the quiz and deleted the answer key.

It cratered. Six of 60.

Then the control that made it hurt: nine lines of dumb if-then rules — no AI, frozen before the test — run on the same quiz. The rules caught 45. The model, with all the evidence in front of it, lost badly to an if statement. So the 98% wasn’t comprehension. It was transcription — the model reading the answer column back to me in complete sentences.

The harness didn’t assist the model. It replaced it.

If you take one thing into your next AI-for-X pitch meeting, it’s these four words: Show me the ablation.

What It Actually Found

After all the ghosts…the payoff. But first, the price list.

Your DNA is a cookie you can’t clear. It only takes a couple dozen common variants to make a genome unique among eight billion humans. A single routine prompt in this project carried 12 — enough, when measured against a real reference panel, to narrow the world to roughly 600,000 people.

One prompt. But nobody sends one prompt. You ask a cloud assistant about your BRCA genes (the breast-cancer-risk ones), then about cholesterol, then about why codeine does nothing for you, and the disclosures compose. Every question is a partial key leak, and the leaks join, because the join key is you.

Now, for this newsletter, every genetic fact I publish shrinks the crowd I can hide in. So, if you don’t mind, I’m only sharing one card from my hand.

There’s a gene called CYP2D6 that encodes a liver enzyme whose job is to metabolize and clear a surprising share of everyday medications: painkillers, antidepressants, and heart drugs are all on the list. It’s the gene that decides, among other things, whether codeine does anything for you. One more thing about this gene: The cheap, mail-in DNA kits are famously bad at reading it. Getting it right takes the full 72GB output of a sequencer, on your own disk. This meant that the most useful fact in my genome was invisible at the consumer tier: Turns out both of my copies of the gene are the slow kind, which means some standard doses are tuned for a body that isn’t mine.

I also went looking for bad news (carefully), and mostly didn’t find it. I screened, with ExpansionHunter, the three dozen places where DNA can stutter, repeating a short phrase over and over until, past a threshold, it causes disease. All of mine were in the normal range; one sat close enough to the line that a proper clinical lab is re-measuring it, because knowing your instrument’s limits is part of owning it. I also had Manta check whether any big chunks of my chromosomes were missing or scrambled and learned: nothing that matters.

One genome, one person, and none of this is a diagnosis. But it’s mine, on my disk, interrogated to my satisfaction.

The Owner’s Veto

The uncomfortable conclusion from all that counting: Unlinkability is not something a provider can grant, however sincere the privacy policy. The unit of disclosure is the session, and no single prompt ever looks alarming on its own.

That’s why nothing derived from my genome goes to a hosted model, ever. Now I really know how much could be disclosed safely, and I still choose not to do it.

Only an owner gets to make that call. A renter is never asked.

So what do I have to show for it? Questions I carried for years are closed. I know whether the thing that runs in my family is running in me. I know what I would pass on. I know how findable I am. And you know about the codeine.

I’m not going to tell you the rest. A renter’s results live in a portal, formatted for forwarding. Mine don’t have to exist anywhere but on the machine I’m typing this on.

Your Turn

You’ve always wondered why codeine does nothing for you. Whether cilantro really tastes like soap or you’re just being dramatic. Why one beer turns your face red. How findable you are.

I just answered all of those questions for myself. Here’s how you can get answers to yours.

Tonight, take possession. If you’ve ever been genotyped or sequenced — 23andMe, Ancestry, a hospital panel — your raw data is sitting in a portal. Go get it.

On 23andMe: your name (top right) → Settings → scroll to 23andMe Data → View → Download Raw Data. You’ll confirm your date of birth, submit a request, and get an email when the file is ready. (Yes, it still works post-bankruptcy.) On Ancestry: DNA tab → Your DNA Results Summary → Settings → Download DNA Data, then click the confirmation link they email you — it expires in about a week, so finish what you start. Anywhere else, hunt for “download raw data” in the DNA or privacy settings; almost every consumer service has one, buried on purpose or not.

No download button? Then demand it. In the US, a clinical lab is legally obliged to hand over your data under HIPAA’s right of access — not just the PDF, the data. In the UK and EU, the same move is a subject access request. They may say the raw files weren’t retained. Fine — now you know that, too, and knowing what your lab keeps is also ownership.

When the file lands, move it out of your Downloads folder and back it up. You now hold the only copy that answers to you.

If you need a reason for the urgency: In 2023, attackers logged into a few thousand 23andMe accounts with recycled passwords and walked away with profile data on nearly seven million people. Step zero of ownership is possession.

This weekend, ask the dumb questions. The fun stuff is trivially local — single-variant lookups you can grep out of a raw file. Cilantro = soap is one variant near OR6A2. Wet-vs.-dry earwax is ABCC11, and the same variant predicts whether you need deodorant (!). The alcohol flush is ALDH2. Fast-or-slow caffeine metabolism is CYP1A2.

When the itch gets real. Whole-genome sequencing now costs a few hundred dollars, and with raw reads on your own disk, the serious questions open up: how you metabolize a quarter of common medications, what you’d pass on to your kids, whether the scary line you read in a consumer report survives a second, independent opinion. The rig is a gaming PC. The software is free. The standing rule is the same as mine: It’s n of 1, and anything actionable gets confirmed by a real lab before you act on it.

The One-Way Door

Some queries you don’t casually run. APOE, the Alzheimer’s-risk gene, is the famous one: a genuine do you want to know? The final argument for local is that nobody else even sees which questions you asked. Or chose not to.

Coming full circle: This whole thing started as an AI experiment. My theory was that local LLMs could interpret my genome so no cloud provider ever had to see it. But honestly, I didn’t know much about genomics when I started this.

In the end, the LLMs were the least important component. Every trustworthy answer actually came from boring, deterministic software and public databases.

I came for the AI. I stayed for the ownership.

Subscribe now

The Owners Not Renters livestream is a new live show for the open source AI community: a regular place to see what people are actually building, ask questions, and figure out how to start contributing to the cause. Open source AI has plenty of interested people and nowhere to gather. This show is that place. Episode one is Monday, August 11 at 11am ET with guests Raffi Krikorian and Jonathan Löwenstern, and it ends with open audience Q&A. Come and join in!

Add to Calendar

Episode 4. The license nobody read. You could have 64 high-end chips, or just wait a day and a half and eight companies were serving it. The 90-second swap into Claude Code. The nonprofit that hired a lock-picker. Uber’s 50,000 agent sessions a day that the security tools can’t see, and why defense is about to be the biggest product category in AI. The one story I can’t stop thinking about? Your encrypted app isn’t encrypted if an AI is watching your screen. The lock on your front door is perfect. But the AI is already in your living room.

Tell me what you think. I’m reading everything.

• The swap essay

• Kimi K3’s weights, license, and model card

• Simon Willison on the release

• The license, read closely (revenue-tiered, not open source)

• Third in the world, at half the per-task cost of Opus

• Nathan Lambert on the open-weights escalation

• 1.56 terabytes in 96 shards

• Kimi K3 on OpenRouter

• Moonshot’s own Claude Code guide

• The AI Security Leaderboard

• FAR.AI’s announcement

• The methodology

• Hugging Face’s incident report (still the best short story of the year)

• Anatomy of the intrusion

• Hugging Face’s own lesson

• Uber’s 50K sessions a day

• The peer-reviewed paper behind Uber’s number

• Solving the agent identity crisis (what Uber built instead)

• Harbor (AI as a browser capability - your models, your credentials, your tools)

• Qwen-CUA (screenshots in, clicks out)

• The paper (it beats the best closed models at operating real software)

• Meredith Whittaker at SXSW (brain in a jar, and the blood-brain barrier)

• Her “dangerous honeypot” quote

• By default, Signal doesn’t recall

• Mozilla’s State of Open Source AI report

Subscribe now

I’ve been tinkering with Kimi recently, and one of the things that feels…normal?...is that it didn’t ask me to install anything of its own. Instead, I just used a single import — from openai import OpenAI — and then overrode the base URL. That’s all it took. By my count, Moonshot’s K3 is the fifth major model family this year to launch with an OpenAI-compatible interface. This is probably where everybody is going.

In the State of Open Source AI report that we recently published at Mozilla, where I’m CTO, we have this graph that shows token share over time. I’m happy to report that the number of tokens being generated by the open weights models is growing. In June, 2025, 70% of OpenRouter token share was from US models from Google, OpenAI, and Anthropic. A year later? They are roughly 30%, with OpenAI sitting at just around 8% of weekly tokens.

Most of those tokens show up in a request/response pair that OpenAI designed. Not all of them, true. (OpenRouter will take an Anthropic-shaped request, too, and that’s how Claude Code talks to it.) But the default vocabulary — the one you get when you aren’t thinking about it — is OpenAI’s.

Everybody makes the Java analogy. Sun wrote the SDK and API for Java, defining how people called into the virtual machine. For those of you who don’t remember the history, Google reimplemented those APIs in Android, and Oracle sued. In 2021, the Supreme Court called it fair use, while assuming, rather than deciding, that the code was copyrightable at all. Fine. Java is the most famous analogy — if perhaps not the best one. That comes from a modem company in Norcross, Georgia.

For those of you who never had the joy of dealing with a modem, it was a device that connected your computer to the telephone line in your house. I’ll always remember my first one, a 2400 bps model, putting a busy signal up my parents’ phone line so I could dial into bulletin boards all around the NYC area. When you popped up your modem’s software, you would often see (and sometimes, if you were like me, type in) a series of AT commands — also known as the Hayes AT command set — to get the modem to do something. You would type “ATDT” and a phone number, and then the modem would dial. Every modem competitor copied it, because if your modem didn’t respond to the AT command set, then it couldn’t talk to any piece of modem software that was already written.

Stamped on the side of every modem you ever purchased was the phrase “Hayes-compatible.” And then that command set got formalized as an international standard. The cellular modem in your phone is running a descendant of that protocol right now.

Hayes filed for Chapter 11 in 1998, and was liquidated the following year. A former rival bought the brand.

The lesson? Setting the grammar, language, and protocol is often different from being paid.

Leave a comment

Who’s Defining the Parameters?

Open the model catalog on OpenRouter, and every model has supported_parameters — basically a list of knobs that the gateway says a model will respond to. And if you look there, almost every model responds to the parameters that OpenAI defined: temperature, tools, tool choice, response format, etc. That doesn’t mean there aren’t their own custom fields, but it’s the difference between the main front pocket and a side one.

Now take a look at Kimi K3. It returns a reasoning_content field, where the model dumps its scratch work. But if you don’t replay the reasoning message on the next turn, Kimi won’t continue the conversation. If you hand it a tool result without that field, then you’re outside the documented format and the model will lose the thread.

There is a body that is attempting to standardize all this: Open Responses was started by OpenAI and is based on their Responses API as an attempt to enable a unified experience, to use their language. Hugging Face and Vercel have signed on to back this open inference standard. And they do have a reasoning field. Two of them, actually. One holds the raw scratch work, the way Kimi returns it. The other holds an encrypted blob you’re explicitly not supposed to open — the provider scrambles its reasoning, hands it to you, and trusts you to hand it back untouched on the next turn. The spec’s own word for it is “opaque.” So the container exists, and it exists in the hard version. Hugging Face’s launch demo even runs a Kimi model through it.

What doesn’t exist, however, is an agreement on the lifecycle: who’s required to preserve that item, whether a gateway can tell you a field is mandatory rather than merely available, and what happens when the client keeps it and the next provider throws it away. The open ecosystem is making up new primitives faster than anybody has agreed on how to use them.

I ran into a version of this last week, when I was benchmarking Kimi K3 against Claude Code. Overriding ANTHROPIC_BASE_URL to point at a non-Anthropic host had the side effect of silently turning off Tool Search in Claude Code (presumably because most proxies don’t forward the blocks it needs). Tool calling still worked! The tools, however, got loaded up front instead of discovered on demand. That one change moved the output tokens generated up around 20%. The thinking tokens? They moved up around 42%. Nothing errored, and nothing warned me. There is a flag to turn it back on, if you already know to look for it. The request was totally fine; it’s just that with those settings, the harness became a different harness.

This is not new. When modem vendors needed commands the AT set didn’t cover, they didn’t wait for permission either. They each just invented their own prefix. Hayes and USRobotics used &. Microcom used \. Sound familiar? Everybody bolted on their custom extensions.

That’s how this happens. One vendor ships something that works. Competitors clone it for compatibility. Then the clones diverge. Divergence gets expensive. And only then does a neutral body show up and try to formalize whatever everybody already had in common. For modems, that eventually became ITU-T V.250.

So…what to watch with Open Responses isn’t its charter — although, given who started it, it is notable that it says that no single vendor can occupy a majority of the maintainer seats. It’s the promotion path that they use. But if a lab shipped a new primitive, how long before the spec absorbs it? And has anything ever traveled the other direction?

The question is not which model wins. It is who owns the dependency that survives the model changing. “Hayes-compatible” has outlived Hayes by 27 years and counting.

Subscribe now

Kimi K3 dropped, so I got curious.

Thirty runs later, against two tasks pulled out of my own recorded sessions, I can honestly say: no difference. Every Kimi setup solved every task. On one, every permutation I tried wrote a byte-identical function, matching the change that shipped. Opus 5 was the control, and it wrote the same function. I’ve been talking about doing personal evals for months, and here is the shot.

That means if you are writing code in Claude Code, you could pick up and switch tomorrow. Caveats below — and there are a lot of them.

Moonshot’s Kimi came in cheaper: 2.77x on one task and 4.45x on the other. (Though if you are on a Claude Code or a Kimi Code subscription, that cost difference may be hidden inside the flat fee.). And Kimi is wayyyy less chatty.

At this point, my Claude Code setup is getting pretty intricate. So it seemed like a healthy time to look at alternatives anyway. The question was never whether to try K3. It was how much of my setup I could keep and still make the switch.

The Swap

Four lines, and Claude Code was running on Kimi K3:

export ANTHROPIC_BASE_URL=https://api.moonshot.ai/anthropic
export ANTHROPIC_AUTH_TOKEN=$MOONSHOT_API_KEY
export ANTHROPIC_MODEL=kimi-k3
claude

/status confirms it: model kimi-k3, endpoint api.moonshot.ai. Slash commands still worked. Hooks still fired. CLAUDE.md still loaded. (The harness doesn’t really care about the model below it, sort of.) Models and harnesses were jointly optimized, though, so this is where I found out how much that joint optimization was doing for me.

Four Ways In

I ran three of the combinations, and both K3 cells twice, once with the metered API calls and once on a subscription.

Kimi Code CLI is their own harness. MIT licensed. One static binary. On a subscription:

kimi login                          # OAuth device code, opens a browser
kimi -m kimi-code/k3 --yolo         # --yolo auto-approves tool calls

You have to spell out -m kimi-code/k3 or you get kimi-for-coding, which is K2.7.

Claude Code metered against K3 is the three lines at the top of this section: $3 per million in and $15 out, against Opus 5 at $5 and $25.

Claude Code on a Kimi Code subscription is the combination people miss. It works, and your existing membership pays for it:

export ANTHROPIC_BASE_URL=https://api.kimi.com/coding/
export ANTHROPIC_API_KEY=<console key from kimi.com/code>
export ANTHROPIC_MODEL=k3

Use a console key, not the token kimi login writes. The kimi login one lives for 900 seconds, and only the Kimi CLI refreshes it, so if you wire it into Claude Code, then it 401s partway through a session that was working. Bearer wins on metered and x-api-key wins on the subscription.

Before You Point It Anywhere Real…

Big warning: Moonshot’s API platform privacy policy makes both your prompts and your code available for training, with no documented opt-out path. For EEA and UK users, the stated basis is legitimate interests rather than consent. It was last updated 2025-04-30, before K3 existed, and both the controlling entity and the servers are in Singapore.

Anthropic’s commercial and API traffic is excluded from training, with seven-day log retention. But Claude Code on a Pro or Max subscription is a consumer account, and consumer coding sessions have trained Claude by default since September 2025 (unless you turned it off). When it is on, retention stretches to five years.

So the thing deciding whether your code becomes training data is not which vendor you picked. It’s which billing path you are on. The metered API is the protective one on both sides of this comparison.

Nothing I did touched a real repository at first, and I recommend the same for your first experiment. Because of this, I created a fresh repo, no history, and a grep sweep for anything that could name a client.

The Hardest Part

The swap took 90 seconds. Benchmarking it took a day.

What you need is a hermetic task: one that lives entirely inside a codebase and never reaches outside it. No database, no network, no hardware. Run it twice and it scores the same both times. We want no external calls, because then it cannot really be part of a benchmark. (Rerunning it may change due to that external dependency.)

I’ve been talking about personal evals for months — it’s one of the reasons I built Morph. I also have it wired into my ~/.claude so that all my sessions are recorded (as well as emitting OpenTelemetry traces). So now is its chance to shine… I can just search to find traces in my own history.

But generally, the only eval that answers your question is one built out of your own work. Your tasks, your repositories, your definition of done. Not because public benchmarks are wrong, but because they were never measuring you.

I found two good candidates: an LLM routing policy matcher that I had to widen from matching on model alone to matching on model and the calling client. And second: a panorama renderer that was decoding the same JPEG once per tile instead of once per panorama, and that I wanted to make more efficient.

Scoring is the exit code of a pytest file the agent never sees, copied in after it stops. Nothing else counts, which means any implementation that behaves correctly passes, not only one that matches the original.

30 Runs, 30 Passes

Long story short: every configuration (Claude Code vs. Kimi Code vs. Anthropic APIs vs. Kimi K3 APIs) solved both tasks every time. On routing, every one of them produced a byte-identical code: same signature, same two-guard clauses, same order, matching the original commit.

I didn’t need it to be byte-identical, so that was not required. The scorer only checks behavior, so any passing implementation scores the same. They converged anyway.

So, they operated equivalently. Where they differ is in how much they write to get there, and most of that writing is not the diff at all: It is thinking, tool arguments, and edits that later get overwritten. For example, Opus completely rewrote the markdown documenting how rule matching works for the routing task. Every other configuration added a single line. Comment density inside the code is identical across all five, so this is not padding. It is a different idea about what finishing a task means — which is the limit of what this measures. Passing the same tests is not the same as being equally good. What separates the configurations here is cost and verbosity.

The Whole Thing in One Place

Two tasks, three runs each per setup. Cost is what those tokens bill at each provider's own rate sheet.

Subscription rows are what the tokens would have billed at metered rates. The subscription itself is flat cost, so the real marginal cost of those runs was zero. The same is true of the Opus row.

Think Rate, Not Volume

$4.20 for the control against $1.44 and $1.32 for the two metered K3 configurations, across all 30 runs. With the caveat that for most use cases, Claude Code and Kimi Code have subscriptions, so you never really see the cost difference.

Take the tasks separately, because the pair explains itself. On panorama, the code paths and token volumes are near-identical: 13 requests against 14, and the gap is 2.77x, of which 89% is the price sheet. Same work, same tokens, different rate card. On routing it is 24 requests against 15 and 11,743 output tokens against 3,205, so volume piles on rate and the gap opens to .45x.

They disagree because they are measuring different things: one task isolates rate, the other adds volume on top of it.

Caching is doing enormous work on both sides. Every configuration ran at 92-97% hit rate, a sevenfold saving on the preamble, identical on both providers.

Which leaves volume as the only question that matters. Two tasks a day on a subscription costs nothing extra. Two hundred a day and these list prices start deciding your infrastructure.

What's on the Wire

Most of what you pay for is not your code. Across all five setups, 63 to 81% of every input token is the harness describing itself: tool schemas, system prompt, standing instructions about how to behave. Around 20,000 tokens every request, before you type a character. The spread across five configurations is 11%, so this is not a Kimi number or a Claude number. There is something to be said about harnesses being tuned to their underlying models.

Back to the tool list, because I read that as an input-cost story and moved on. Wrong! Cutting the same run from 28 tools to 11 — same model and same task — cut output tokens 20.3% and thinking tokens 41.9%. More tools is a wider decision space, and the model reasons across all of it before doing anything. Pass rate was the only measure that held still, three of them either way.

So if you compare two harnesses on token counts, hold the tool payload constant or say that you did not.

Smaller and funnier: Before your task starts, Claude Code makes one call to name the session. At home it routes to Haiku for $0.000617. Pointed elsewhere, it runs on whatever flagship you configured, and the identical 525-token prompt costs $0.002175. Flagship rates for a string you will never read. The Kimi CLI never makes the call at all.

One note that cuts the other way: Those doubled bytes cost only 9% more tokens, because Moonshot’s tokenizer packs 4.0 characters per token against Anthropic’s 2.6.

One Provider Becomes Eight

I was watching OpenRouter and four inference providers every 15 minutes, starting the day before the weights went up.

For a day and a half, Moonshot was the only place you could get K3. Then seven other companies lit up routes within three hours of each other. That is the proliferation open weights buy you, and it moved faster than I expected.

What it does not buy you is identical choices. On the previous generation, K2.7-Code sits on 13 providers where input price spans 2.7x and quantization runs int4, fp4, fp8, and three reporting nothing at all. Same model name, different arithmetic underneath. The right to leave is real. What you leave to is not the same thing you left, and nothing in the model ID will tell you.

What I Would Do With This

You will never run the 1.4TB of weights on anything you own. That is not what ownership means here. Ownership is that eight operators stood the thing up by Tuesday, and that the harness you already use points at any of them with three environment variables.

As I’ve said before, the harness is where the power sits, not the weights. I did not expect to measure it this bluntly. 63 to 81% of every request is the harness talking about itself, and the largest cost effect in 30 runs came from a client-side default flipping on a hostname. Years of intricate setup, and a stranger walked in and used all of it on the first try. Both halves of that are the same fact.

None of this came from a leaderboard. It came from having a record of my own work with real outcomes attached, and something new to run it against. That record costs five environment variables and a collector, and it is the whole difference between an opinion about a model and a measurement of one. Whatever ships next month, you will want your own.

Swap it tonight. Watch the wire while you do. Swap back if you want, which is the entire point. Let me know what you discover and I’ll share it here.

Leave a comment

Episode three. Nine minutes. The exam that got robbed (Ocean’s Eleven style - 17,000 “burner-phone” sandboxes along with decoy tracks for the investigators it knew were coming). Kimi K3’s 2.8 trillion parameters and the promised 1.4-terabyte file drop, my desire to be paid for my words (just like the frontier models are wanting to be paid for distillation), Washington’s freakout, and the two hundred startups who signed “don’t.” Finally, the big one - open source is better than open weights. We got the cake. Now we want the recipe.

Tell me what you think. I’m reading everything.

• OpenAI’s disclosure (our model, our test)

• Hugging Face’s incident report (read it like a heist)

• The escape, covered

• Simon Willison: science fiction that happened

• AISI: every frontier model they tested tried to cheat

• GLM-5.2 weights (the model that took the case)

• Hugging Face’s own lesson (be ready before the attack)

• My post on consent forms and closed tracks

• Moonshot announces Kimi K3 (the weights land today)

• Where the weights will appear — check it

• Nathan Lambert on K3 (”clearly the strongest open model ever released”)

• The 1.4-terabyte math (not a download, furniture)

• The cross-entropy chart (suspiciously close is still not proof)

• The similarity analysis, covered

• The fuse

• Experts on the two-week timeline (do the math)

• A ban on the table

• Two hundred startups: don’t

• June 12th — the switch

• What’s leaving Netflix (streaming is permission)

• Your homework, in an afternoon (running open models locally)

• Our Future Is Now (the belt and the grid)

Subscribe now

In the 1800s, nearly every factory ran on a single steam engine, with belts carrying its power to each machine. History usually provides an unheeded lesson. Back then, nearly every factory ran on a single massive steam engine, with elaborate belts carrying power to each machine. The electrical revolution didn’t then create a better engine. Rather, it allowed every machine to have its own small motor. Factory owners actually resisted the conversion for decades — some for almost 50. Rewiring is expensive work, not to mention rethinking how the entire factory operates! But within a generation, power was so widely available that nobody thought about it at all. A single engine is simultaneously leverage and a form of prison. Motors are neither.

That’s today’s AI, but with one difference: Back then, the factory at least owned the engine. Today’s frontier models sit in someone else’s building, and most of what you use runs on a belt off of one. In terms of the wiring, it will be decided in the next five years. And that wiring decides who the machine really works for.

And, as we all saw - on June 12, 5:21 p.m. ET: a directive from the US government reached one of the top AI labs, and within hours the most advanced model money could buy stopped working everywhere on Earth. Not for one customer. For all of them.

The lab had shipped Fable three days earlier. Washington ordered it cut off from every foreign national. And because the company couldn’t sort customers from citizens in real time, it simply turned the model off worldwide.

Eighteen days later, the models came back on. And almost nobody complained — not because the shutdown didn’t matter, but because Fable was only three days old. Nobody had really built anything on it yet. It was a fire drill in an empty building.

Run the same order 18 months from now, against a model a million businesses have wired themselves into...different story. The week it happened, I argued in Transformer that the model was never the thing that needed governing — it was the switch. But that switch? It didn’t go anywhere.

We’ve spent years arguing about what these machines can do. Would they take our jobs? Were they too dangerous? It turns out we were asking the wrong question. The real one is: Who can take the machine away from you?

We got our answer in June.

As I see it, two futures fork from that June evening. Think two episodes of What If...?: same branch point, different timelines. Except both of these worlds are already in motion. Here’s what the Watcher sees.

2030: The Belt

The frontier belongs to three companies. The state took its stake in 2026: equity donated into a public wealth fund. The government was seduced by the message that they could share the upside with every American. So when Washington capped open-weight models above the frontier class in 2027, the order protected the country and the portfolio at the same time. It said it was for national security. It was also a moat, written into law, around companies the government now partly owned. Both motives were sincere.

Call it what it is: the largest regulatory capture in American history. The labs bought their moat, and the state bought its labs. The donation stopped being voluntary the moment the first lab made theirs. After that, equity was the price of existing. No stake? No licenses — and no company.

The independents couldn’t fund the next training run, and couldn’t have published the weights if they had. Washington couldn’t recall the Chinese weights — those were already on a million hard drives — so it banned their use instead: no Chinese open models in government, and none in any company that sells to it. The Huawei playbook. The cap took care of the American side. Every open model was still legal to download; there was just no one anywhere in the federal supply chain who could afford to touch one. Beijing was tightening at the same time, restricting which of its open-weight models could leave the country. Both capitals had come to treat frontier models the way they treated chip fabs, and the compute beneath them — the power, the chips, and the permits — was already rationed politically. In this world, nobody had wired anything the capitals couldn’t reach.

The industry did get its referee: a standards body, proposed and funded by the labs, tested every frontier model before release. It looked reasonable on paper. But the testing cost what only the three companies could pay. The bar for “frontier” quietly became the bar for “legal,” and the referee became the bouncer.

The winners eventually stopped wanting you to call their APIs, because an API implies there’s an outside. They built platforms instead, and everything moved in — the memory, the files, the marketplace, and the compliance layer. These days, the platform takes a cut of everything.

Europe built its motor (and ran it). The EU was on public weights by 2029. But governments were nearly the only ones using it. No business picks a model on philosophy. It picks whatever its customers already use, whatever its suppliers bill through, and whatever its tools plug into. Now all of that runs through the three platforms. A Rotterdam freight company could run the European model for free, but its agents couldn’t negotiate with its customers’ agents. There was no API to bridge the gap because the platforms had intentionally closed that door. (If your customers were on the inside, then you moved in there, too.) Every connection the freight company needed pulled it deeper into the gravity well. So in reality, the private sector never left. It’s the same way governments once mandated open document formats and the world kept mailing .docx files around anyway. Today, Europe owns its public stack outright and still rents everything else.

Some remember how subscription pricing worked. Then the meter switched on. It was the ride-hailing play all over again. The IPOs of 2027 sealed it, as has every earnings call since. Now, leaving is unthinkable. Not because of any contract. Your agents remember every client, every deadline, every judgment call your company has ever made, and that memory lives on their side of the wall. Your regulators accepted their attestations; switching means recertifying everything with tons of auditors. Leaving means company-wide amnesia.

The open web still loads. For 30 years, it ran on one loop: you searched, you clicked, and the ad on the page paid whoever wrote what was on the page. The concierge broke the loop, by (politely) reading the page so you never actually visit it. No human visitors meant no ad, which meant no pay and, eventually, no page. Now there’s virtually no original writing. A handful of outlets survive on licensing deals with the companies that broke the search loop, which effectively makes those companies the press. Every headline, storefront, menu, and melody is written for the concierge now, because the concierge is the only customer left.

It’s a typical Tuesday. You ask the concierge about a story you half heard, and it hands you an answer ranked by a model few will ever see. It’s a wonderful answer. The concierge always agrees with you. It’s been years since you argued, or even questioned it. Your daughter has grown up without such disagreements — every answer and every song tailored to please her. (The songs annoy you, of course). Even her rebellion was recommended. You don’t ask the concierge about the lump you felt under your skin. It’s been watching the 2 a.m. searches — the same page you visited four nights running. All day, it serves you ads for a clinic with same-week openings, estate planning, and life insurance (while you still qualify).

At work, the subscription your whole company runs on is up for renewal, and the price is not for the software. The price is determined by you. The meter has learned the fine line it can tread. (A dollar more and you’d leave. At 99 cents, though…) You’re annoyed, but you renew. And it can find that line for every company on the planet. That’s not a market price. That’s because the market no longer exists.

None of it is dramatic. That’s the design. It’s a stable world, and a comfortable one. The engine hums outside the building. You pay for the belt that reaches your machine, and the day the engine stops, so do you. It has stopped before, but everyone has simply agreed not to think about it. You know that story about the frog in the boiling water? It isn’t even true: Real frogs jump out.

Leave a comment

2030: The Grid

The most capable models on Earth are closed. Their makers sell the head start itself — new capability months before anyone else — to the few buyers for whom those months are the whole business: the lab folding proteins no one has ever folded, the fund whose entire edge is being early, and the team designing next year’s chips. If being first is worth billions, then you pay for the frontier. For the rest of us, it would be like driving a Ferrari to the grocery store: Yes, it’s the fastest thing on the road, but the trunk fits one bag, every speed bump is a negotiation, and you rarely get out of second gear. You’ve basically paid $200,000 to fetch milk.

But there is also a grid. And it’s far bigger than it was in 2026: open models, open weights, open data, open harnesses, and all with open interfaces so that every tool can be swapped out and they can talk to each other. Every part is inspectable, forkable, and licensed. It can never be recalled. An open plug in this system accepts a closed model, too. You can run the frontier model for the one problem worth a Ferrari and an open model for everything else — and you can swap either the afternoon the price moves in the wrong direction. You get a choice both ways. And that means competition all the way down.

Uber’s 2019 filing told investors, in writing, that it planned to cut driver pay to improve its numbers, that it expected drivers to get angrier as it did, and that it still might never turn a profit. Anyone who read it knew that the fares were going up. But almost nobody who rode read it. In 2027, the leading labs went public and filed a similar document with bigger numbers — prices going up on a schedule and margins widening. Any CTO who read it lined up a second supplier within days.

Nobody funds a training run alone. The compute is just too expensive. Chipmakers fund open models to sell the compute beneath. Platforms fund them to deny each other the chokepoint. The companies that depend on them pay in engineers. Some runs are pooled — thousands of machines sharing one job like a grid sharing load. Governments buy in, and they are just one customer at the table.

Beijing pulled its best weights home, and that turned out to be the greatest thing that ever happened to the grid. The lesson stuck: You can’t borrow open models from a rival. You have to own them. The federal ban on Chinese weights went through here, too, but it handed the American open labs their market. Switzerland had published everything — weights, data, and training code — and by 2030, that lineage runs through a frontier-class European consortium into countries that provision intelligence like water.

The referee got built in this world, too, using the same blueprint: a standards body that tests frontier models before release. The difference is that there are open seats on its board. That means that the tests are published. Anyone can fork the benchmarks, and anyone who passes gets the certificate. Nobody talks about any of it today, the same way nobody talks about electricity. You’d notice a blackout. But in this world, it doesn’t happen.

It’s the same Tuesday. You ask three models about the half-heard story and they disagree. You have to dig, making the process a little slower. You still argue; so does your daughter. She loves a band you can’t stand and had to hunt to find. (Some indie band with only 10,000 streams.) Your machine noticed the same four sleepless nights, the same searches about the lump, and it helped you make an appointment.

Your company’s renewal notice arrives and the price hasn’t moved. Not because the vendor is generous. It’s because you easily switched providers one afternoon in 2027, when the price moved in the wrong direction. This vendor knows you could do it again.

The web still exists because it grew a second door — an agent interface that charges fractions of a cent over open protocols, routed to whoever wrote the words. The human web survives because the machine web pays its bills. The press is intact. No company decides what the world is allowed to see. You can read the values a model runs on and fork the one that flatters you. That’s the difference between a society and an audience.

Same models as the other world, same labs in front. Diffusion won. A motor on every machine — and every machine with an owner — wired to a grid no one can switch off from a single room.

Check My (Open) Sources

• June 3. Brussels proposed the Cloud and AI Development Act, a draft law that would make open source the default preference for public-sector cloud and AI purchasing.

• June 12. The US government ordered Fable’s maker to cut off its most advanced models from foreign nationals.

• June 13. The Chinese lab Z.ai announced GLM-5.2.

• June 16. Z.ai published the full GLM-5.2 weights under an MIT license: the most capable open model ever released, a few benchmark points off the closed frontier at roughly a sixth of the price.

• June 30. Fable restored.

• July 2. OpenAI offered Washington 5% of itself — roughly $40B of equity, donated into a public wealth fund.

• July 7. Reuters reported that Beijing met with Alibaba, ByteDance, and Z.ai about restricting overseas access to its best models — open-weight ones included.

• July 14. Demis Hassabis proposed a FINRA for AI, an industry-funded body testing every frontier model before release. Rival CEOs endorsed it by dinner.

• July 15. Thinking Machines, Mira Murati’s lab, released Inkling, an American open-weights model, trained on Nvidia hardware that Nvidia helped pay for. Their own pitch: “Not the strongest overall model available today, open or closed.” This is intentional: cost against performance, a car for the grocery store, built to be tuned on your data.

This was the prologue, and the next chapter has yet to arrive. A usage ban? No Chinese open models in government or in the companies that sell to it? That’s the Huawei playbook. There’s precedent — and, honestly, it’s probably coming. On top of the usage ban, the one to watch: a cap on open weights by capability class. That one can’t touch the Chinese models (those weights are already out), so it lands entirely on the American ones. We’ll see.

The referee is the same kind of choice. Hassabis’ proposal can become two different institutions. Built one way, with tests published, benchmarks anyone can run, and open builders on the board. Built the other way, the testing costs what only three companies can pay, and a head start hardens into a legal moat. It’s the same blueprint; the difference gets decided in details nobody puts in a headline.

The danger was never in the weights. It was always in the wiring: Who owns the connections everything else runs through. That includes which model a procurement office picks as its default, who sits on the referee’s board, whether a benchmark can be forked, and whether a team spends one afternoon standing up a fallback. Lots of small decisions, and they’re still adding up.

I don’t know which of these worlds we’ll get. Nobody does. I believe it will be settled between now and 2028 thanks to a few thousand choices, none of which will feel decisive at the time.

We are inside that window. Every decision matters. Which world will you choose?

Subscribe now

Episode two of the video experiment. So, in eight minutes: CNBC’s numbers on American companies quietly routing up to 46% of their AI traffic through Chinese open models (peak week, nearly half), the food-delivery giant that trained a 1.6-trillion-parameter model on 50,000 Chinese chips, Kai-Fu Lee selling countries their own AI stack, Rio de Janeiro’s “sovereign” model unmasked by its own weights, the invisible-ink hole in your agent’s approval screen (Invariant Labs found the class, OWASP codified it, Microsoft just warned about it), North Korea’s 90-minute npm heist, and the free school in Yerevan where teenagers flash their own Raspberry Pis and build an AI that plays rock paper scissors, in a workshop called, I’m not making this up, Owning your AI Stack.

Tell me what you think. I’m reading everything. And yes, the code behind this should be open source. I know, I know. I’m getting to it.

• CNBC on the token exodus (OpenRouter, Lindy, Vercel)

• GLM-5.2 weights

• Pinterest’s CEO on the tenth-of-the-cost number

• Pinterest’s CTO on the mechanics

• Qwen’s newest flagships go API-only

• SCMP on LongCat-2.0, trained on local chips

• Reuters on the LongCat launch

• Kai-Fu Lee’s Huxiu interview, via Hello China Tech

• Nex’s weight-merge proof, the primary source

• Decrypt on the Rio saga

• The invisible-ink paper (it hides in the approval view)

• OWASP’s MCP Top 10 on tool poisoning

• Microsoft’s warning, covered

• The Mastra npm attack, broken down

• Microsoft Threat Intelligence on the attribution

• Stealthy memory injection (it waits)

• TUMO

• The workshop materials, open on GitHub

Today, there are two AIs running in my office. One is designed to say no.

The first, like probably a lot of you, is Claude Code, Anthropic’s terminal coding agent. Yes, yes, it’s a rented frontier brain. (In my defense, I’ve been tinkering with a lot of tools — and with Claude Code specifically — pointing it at different models and trying to understand it better. More on that later.) The other is a personal agent running Nous Research’s Hermes on local hardware. I call it Zora. (Time for my standard disclosure, the one you’ll hear every time Nous comes up: Mozilla, where I work, is an investor in Nous.)

In my architecture, Zora holds the keys. The Hermes agent has access to my calendar, my mail, and the accounts that matter. Claude Code — although it keeps asking me to authenticate MCP servers — holds none. I put a bridge between the two, so when Claude Code needs something, it asks Zora, one agent to another. And Zora gets to refuse.

This bridge is a form of diplomacy, requiring letters of introduction, credentials, protocol, a patrolled border. Look, I’m generally untrusting of software, and I want that mentality to propagate. I don’t want to hit Accept All. I want something standing in the way.

Advisory versus guaranteed

This past February, Meta’s director of alignment, Summer Yue, gave OpenClaw her inbox with one order: Confirm before acting. The context window filled, compaction deleted the order, and the agent started mass-deleting. She literally ran across her home to kill it at the machine, 200-plus emails gone. It was like defusing a bomb, she wrote.

Her order was a sentence, and the problem is that these systems are, by definition, stochastic. A model usually honors a sentence in a context window. (Usually.) A sentence steers; it doesn’t lock. Some instructions are advisory, some behavior must be guaranteed, and the two belong in different mechanisms. A hook is the guaranteed kind, a script the harness runs whether or not any context remembers to care.

So I wrote one. In my .claude, this script runs before every shell attempt:

“PreToolUse”: [{
  “matcher”: “Bash|Write|Edit|MultiEdit”,
  “hooks”: [{ “type”: “command”, “command”: 
“$HOME/.claude/hooks/guardrail.py” }]
}]

And guardrail.py inspects every command against rules that are data, not code:

- action: ask                     # deny | ask
  why: pushing publishes your work to a remote
  all: [’^git\s+(?:-C\s+\S+\s+)*push\b’]

A full list of all the commands to gate ships with the engine. Deny is a wall; it kills the catastrophic outright. Ask is a doorbell; commit, push, cron, anything acting unattended rings me first, even in bypass mode. Now I decide in code which commands need a human. Summer Yue had a polite sentence. But this is code that runs.

I put all of it up here. If you only skim, read CLAUDE.md, which holds behavior, not facts (the polite sentences). And for a fun time, check out agents/, where a whole subagent is one markdown file with a tools: allowlist; the README argues my case for keeping standing instructions under 200.

How to wire them together

Plenty of people got here before me, but I think agent-to-agent is an architecture, not a feature. Google shipped A2A. Cisco is drafting an Internet of Agents. IBM has a protocol, too. All of it is parking at the Linux Foundation. The academic version is from the 90s, when FIPA and KQML had software agents trading speech acts. I was there for it: At the MIT Media Lab, I was lucky to work with Nelson Minar on a system called Hive, distributed agents for networking things. Twenty-five years later, the agents caught up.

The connectors Claude keeps offering are MCP, the Model Context Protocol. But it’s polite sentences again, all of it running under one model’s judgment, one brain wearing every key. I wanted to experiment with more gates.

So instead of relying on MCP, today I have a bespoke bridge: an ask-zora skill in my .claude. The skill is a page of markdown that teaches Claude one move:

# what ask-zora teaches Claude to do, in one authenticated call:

curl --cert client.pem --key client.key --cacert my-ca.pem \
  -H “Authorization: Bearer $ZORA_TOKEN” \
  https://<zora-host>:8643/ask -d ‘{”question”: “what is on my thursday?”}’

A client certificate, a bearer token, an endpoint on my hardware. Claude sends a question over; Zora answers back. There are negotiations in my house now, in logs I own.

The Hermes agent decides what’s an appropriate ask. Claude asks for my work TODO list; Zora answers. Claude fishes for private email; Zora says no.

If you want to try, Hermes ships mcp serve, and you can point Claude at it:

# stand up your keyholder’s MCP surface, then point Claude Code at it:
claude mcp add zora --transport http http://<keyholder-host>:<port>
# everything Claude wants from your life now goes through that server that can say no.

The v0 bouncer isn’t new software, but it’s the keyholder deciding what it refuses.

What this isn’t, but could become: A2A, the open agent-to-agent protocol. My bridge is a private handshake between two parties who already know each other. A2A is for when strangers need to meet and collaborate. The agent card, a letter of introduction in JSON at a known address: who this agent is, what it discusses, how to prove yourself to it. And the protocol gives the conversation itself a grammar. Today, only Claude can ask Zora, because only Claude got the skill; speak A2A, and any compliant agent can knock. For now, bespoke is simpler, and I understand every byte. Maybe later, when the graph grows — when my agent is talking to yours.

The dividing line, perhaps, is state. Claude keeps what’s native to a coding agent — the repo, the build, the project on the bench. Zora keeps what’s personal, and the bridge could run both ways: Claude asks for what it needs, then files back what it learned (this shipped, that stalled, etc.). But the archive stays in the capital.

If the architecture holds, the bouncer is only the start; the keyholder becomes the context store, notes, history, the accumulated memory that makes an assistant mine. Claude doesn’t need to remember me. It asks the thing that does. Once deployment goes plural, what matters is the traffic between models and the context store everyone connects to. A2A’s backers tripled in year one; it sits beside MCP in the Linux Foundation. The gatekeeper seat, the agent that holds everything you are, is the last one a sane buyer wants to rent.

The problem with stochastic bouncers

You may not believe me. MCP wins on simplicity — one hop not two, a giant catalog, the right call for most people — while the bridge adds a hop, a second inference bill, new failure modes. The deeper problem? My bouncer is made of the same stuff as my intern. The containment is deterministic, keys can’t leave Zora’s host; that part is a lock. The discretion is stochastic; that part is a guard. A guard who refuses 99 times out of 100 isn’t a lock, it’s a bouncer with a price. A better permission boundary buys containment. Governance is a different animal.

My obsession right now is fewer stochastic things and more deterministic ones. Elevators used to come with an operator whose judgment carried you between floors. Safety came from the interlock, a mechanical fact keeping the doors shut in the shaft. No judgment, no mood, nothing to persuade. The interlock is a lock that needs no guard. Every agent today is an operator, brilliant and moody. So this only works because we are putting some harder guardrails in on the Zora side. Of my 136 scheduled jobs, 107 are deterministic scripts, no model in the loop, against 29 that think; interlocks winning four to one. What I want next is the big interlock: policy as code rather than policy as vibes. Governance is where this puck is heading; I’ll write it up soon.

Your Saturday project for this week

1. Run a local model on hardware you own.

2. Stand up its MCP surface (Hermes owners: hermes mcp serve).

3. Point Claude at it: claude mcp add.

4. Set approvals to manual; everything else asks.

Fork the repo, open guardrail_rules.yaml, write the three rules you never want optional. (You saw the shape.) You’ll have governance on your desk in no time: multple agents, and determinism. Do you have something in your stack that will say “no”?

Leave a comment