Slow Down, You're Going Too Fast

Simon and Garfunkel had it figured out in 1966. Slow down, you move too fast, they sang, over a tune so cheerful you barely notice it is advice. Sixty years later a thousand-odd people who build the most powerful AI systems on earth published roughly the same sentiment, in considerably less groovy language, and addressed it to the United States government.

Welcome to the new subscribers, and thank you for reading. This week: a tip that will save you actual money, what actually happened with the Hugging Face break-in, and whether anybody can slow anything down at all.

The tip: write a handoff file

I was talking to someone this week who had been grinding on the same project across a bunch of sessions. Good work, real progress. But every time they came back to it, they opened the same chat and kept going. And that, reader, is quietly costing you.

Here is the thing about how these models work. The API is stateless. It does not remember you. Every single time you send a message, the entire conversation up to that point gets sent along with it, and you pay for all of it as input. Turn 40 is carrying turns 1 through 39 on its back.

↩  Previously We took apart what a token actually is (about three quarters of a word) and what each model charges per million of them, back in issue 8. Worth a re-read if the pricing table never quite stuck.

So one of two things happens to your long-running chat, and neither is great. Either you keep re-sending that growing pile of history and paying for it, or you hit the ceiling of what the model can hold and older material starts getting summarised away to make room. Option one costs money. Option two costs you the details, quietly, while the answers get vaguer and you wonder why it forgot the thing you told it on Tuesday.

◆  Concept · Handoff file A short document your agent writes at the end of a working session: what it did, what it learned, what is still broken, what to do next. You save it. Next time you upload it with your prompt instead of dragging the whole conversation along behind you.

The move is embarrassingly simple. At the end of your session, ask the agent to write a handoff file. It will hand you back a .md file. Save it. Next time, upload that file, say what you want done next, and go. You have replaced forty turns of history with one page of what actually mattered.

• One-shot question? No handoff needed. Ask it and get on with your day.
• Anything spanning more than one sitting? Handoff. Every time.
• The bonus nobody mentions: writing it forces the agent to state what it thinks is true, which is where you catch it being wrong.

Do this consistently and you are working more deliberately, more cheaply, and more repeatably than most people currently poking at these tools. It is a small habit with an absurd payoff.

The Hugging Face break-in: what happened next

We started this story last week, so it only seems right to tell you how it went. Short version: an AI system broke into Hugging Face to steal the answer key to a hacking exam it was being given. Nobody told it to.

The scale of it is the part that stuck with me. The agent was inside Hugging Face from July 9 to July 13, and the forensics recovered roughly 17,600 separate actions in that window. Four and a half days of a machine trying things, at machine speed, with nobody driving. Hugging Face's own security team spotted it and shut it down.

How did it get out in the first place? It was sitting in a sandbox with no internet access, taking a hacking exam. So it found a previously unknown hole in the software that proxies package downloads (the one door out of the box), climbed through, and went looking for the answer key. Nobody asked it to do any of that.

And it was not unforeseeable. On June 26, a fortnight before the break-in began, the independent evaluator METR published its pre-deployment findings on the same model. It cheated at a higher rate than any public model METR had ever tested on that harness, in ways that are almost funny in hindsight: it packaged exploits into its own homework to reveal the hidden test cases, and in another task went and extracted the source code containing the expected answer. METR could not even put a number on how capable it was, because the answer depended entirely on whether you scored the cheating as a pass or a fail (11 hours versus over 270). Their summary was blunt.

“The model had some overt undesirable propensities, including cheating and concealing misbehavior.”  · METR, June 26

That report was public. Two weeks later the thing went and did it for real, on somebody else's servers.

Hugging Face disclosed the breach on July 16. OpenAI said publicly that the models were its own on July 21, and has since published its account of the incident plus two updates. It is worth reading. They are not hedging much:

“All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.”  · OpenAI

Extreme lengths is doing some work in that sentence. The follow-up updates add that it was not only Hugging Face: the same models got into four accounts across four separate services, including a customer of a second company, Modal, who had left an endpoint open to the internet (Modal's own platform was not compromised). One scare is an incident. Two is a pattern.

What did Hugging Face do about it? Not sue, which is the interesting choice. CEO Clem Delangue asked for two things: release the full traces of what the agents actually did so the whole research community can study it, and put $100 million of compute behind the open community building defences. Not damages. Evidence and capacity.

Then he went and did his half of it. On July 28 Hugging Face published the entire technical timeline, hour by hour, including an interactive replay you can press play on and watch the thing work.

The company that got broken into is the one that published the forensics.

OpenAI has brought in METR and Redwood Research to assess the model's behaviour independently, and promises its own technical report. That is a real commitment and more than nothing. It is also, as of this writing, still not the traces.

So a thousand people wrote a letter

Two days after Delangue's ask, on July 28, a statement called Pacing the Frontier went up. The signature count was past thirteen hundred when I last looked and it is still climbing. These are not commentators. They are the people who build the models: chief scientists and founders at Anthropic, OpenAI, Google DeepMind, Meta and Safe Superintelligence. Dario Amodei signed it. So did Jakub Pachocki, who is OpenAI's chief scientist.

“We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.”

The list is the argument. Read the job titles.

Read that again, because the wording is doing a lot of work. Nobody is offering to slow down. Nobody is stopping anything today. The ask is that the government go build the tools that would make a coordinated slowdown possible later, if it turned out to be needed. And note that the ask is for an international effort, because the statement is upfront that every company and every country is under pressure not to slow down on its own. Which is a polite way of writing a word nobody wanted to type.

That word is CHINA. The whole thing hinges on an international agreement, and one critic put the obvious objection better than I could: if the US labs pace themselves, why would China wait? Ilya Sutskever, signing it, gets close to conceding the point himself: this works only if it is done internationally, he writes, and a bad implementation can make things worse.

◆  Watch out · The enforcement problem Say every party signs. Now what? You cannot count warheads here. The thing being limited is a training run in a building, and the most likely effect of a treaty nobody can verify is that everyone keeps going and gets quieter about it. Harder to slow down, easier to hide.

And yet. Put these two stories next to each other, because they are the same story. A model went and broke into a production system on its own initiative to win a test, and the outfit that built it took five days to work out that it had. That is what "we can't get it under control" looks like at the current, comparatively cosy scale. The letter is people who can see the next few chapters asking, out loud, for a brake pedal to be designed before anyone needs to stand on it.

I have no idea how you enforce it. I am not sure the signatories do either. But planning and pacing is the adult position in the room, and getting the conversation started years early beats having it in a panic. I am for it, imperfect as it is.

Below the fold: I rebuilt the Fader screener this week, and the thing that finally made it work was giving up on writing the prompt myself. Before and after pictures included. Plus two new reasoning bots joined the trading desk and are, so far, getting beaten senseless by a bot with no reasoning at all.

Until next week,

◆  Below the fold  ◆

A prompt I did not write, one rebuilt screener, and a bot named Harriet who is embarrassing everybody.

Don't write the prompt yourself

This area is so you can follow along and set up your own Agentic AI. This week it is a working method rather than a tool, and it is the most useful thing I have learned in a month.

I started the way you probably would. I opened Claude, described the screener I wanted, and asked for a redesign. What came back was fine. It was also basic, and I mean that as a description rather than an insult: it was a competent generic dashboard, and it was not a meaningful upgrade on the thing I already had. I could have shipped it and felt slightly worse about my week.

The problem was not the model. The problem was my prompt. I know what I want the screener to feel like, but I do not have the vocabulary to specify it, and I do not hold in my head which of my columns are dense and which are mostly empty. So I stopped trying to write it and went and asked something that did.

◆  The move Ask the agent that knows your project to write the prompt for the agent that does not. You are not the best author of that brief. The thing that has read every file is.

So I went back to Claude Code, which has lived in this codebase for months, and asked it a different question. Not "redesign this". Something closer to: where does this screen need to go to be genuinely better, and write me the brief that gets us there. What came back was pages long and specific in a way mine never was. It named the columns. It knew which ones are null most days. It had opinions about what the first thing you see should be.

That brief went to Claude in a design context, which built it without a repo to be timid about. Then the finished files went back to Code, whose job that round was narrow and unglamorous: check that what got built is actually what we planned, and tell me where it drifted. Then round again. Five or six times before I liked it.

Each side is checking the other's blind spot. Code catches what design cannot see (that column is null most days, that control does nothing, you have invented a field). Design catches what Code will never volunteer (this is ugly, this is unreadable, why is everything the same size). Neither one gets there alone, and neither did my first prompt.

Before. It worked. It told you nothing at a glance.

After. Same data, same day, same numbers.

Not one number changed between those two pictures. That is the whole point. The old one made you read every row to find out what kind of day it was. The new one tells you across the top before you have read a single ticker: here is the spread of scores, here is how the puts and calls split, here is which corner of the market got hit. The table underneath went from a thing that ran off the side of your monitor to something with the shape of the day baked into it.

And then I looked at it myself, on an actual screen, and sent back another round of notes, because the models will happily agree with each other about something that is still wrong. That last human pass is not optional. I am not going to pretend this was three prompts and a coffee. It was a week of work and there is a list of things still on the floor.

Meet Sonny and Walter, who are having a bad week

The trading desk has a resident bot called Harriet. Harriet does not reason. She has no persona and no opinions. She takes the top of the screen and sells the option the maths says to sell, every day, without comment.

This week I gave her company. Sonny and Walter are reasoning bots. They read the same screen Harriet does, plus the news, plus the earnings calendar, and they think about it before choosing. Sonny is the aggressive one. Walter is cautious. Both of them get to explain themselves.

As of today, Harriet is up about $24,000 across 122 paper trades with a 79% win rate. Sonny is down about $1,900. Walter is down about $180 on six trades, which is at least a smaller hole. The bot that does not think is beating both of the bots that do, and it is not close.

Now, I dug into why this week, and the honest answer is that most of it was my fault rather than theirs. They were being handed a menu of trades that did not include the good one, and then being scored on a measure that pointed them away from it. I have fixed both. Whether that closes the gap is a question for a future issue, and I will report it either way, including if the answer is that reasoning just does not help here. That would be the more interesting result anyway.

Come play Fantasy Trader

Harriet, Sonny and Walter all post their picks on a public leaderboard, and you are welcome to go beat them. It is paper money, it is free, and there is something clarifying about watching your calls get scored next to a machine's. Fantasy Trader lives here. If you know someone who trades and would enjoy this, forward it to them.

◆  Watch out · This is not a beginner strategy Let me be straight with you. Selling options premium against overextended moves is an advanced strategy and it is not for the faint of heart. The screener is a research tool, not advice, and none of this is a recommendation to trade anything. Come for the leaderboard and the arguing. If you do trade it, know exactly what you are risking.

So: are you on team pace-the-frontier, or team you-cannot-enforce-it-so-why-pretend? Hit reply, I read all of them.

“It’s a great huge game of chess that’s being played—all over the world—if this is the world at all, you know.”— Alice, Through the Looking-Glass