The Person Behind the Prompt
Welcome back readers. Like every week there’s no shortage of headline news happening to inform you on. But this week I’m going to take it a little more local again with some stories not from the frontier but from my own personal frontier. This week I had not one but two somewhat heated conversations over two topics. 1. The responsibility for AI. 2. The use of AI. For this newsletter I’m going to address responsibility first and then in the future we will dive into the use of AI which is also very prescient conversation that I would like to explore.
Campaigning for the mid-term election is in full effect and obviously AI is a topic that everyone is talking about. Whether it’s datacenter or power bills, or job replacement, political ads or safety concerns. These are all top of mind and you’d bet they are points that constituents want to hear about. So this week a friend asked “Who is responsible for AI?” and the conversation was nothing worth writing about in that there was no consensus or answer, and maybe that’s ok because it is currently an open question that we are all figuring out together. So let’s take a minute to unpack here and perhaps you, the reader, can decide.
Can the government do this alone?
|
↩ Previously · Issue 21 “I say that as someone who watched Mark Zuckerberg sit in front of the Senate in 2018 and get asked, by Senator Orrin Hatch, how he sustained a business model in which users don’t pay for the service. Zuckerberg let a beat go by and said ‘Senator, we run ads.’ I have no interest in defending Zuckerberg here. I am only saying that the room asking the questions did not know how the thing worked, and that room is now being asked to write the rules for something considerably harder to explain.” Issue 21 |
The government can’t handle AI regulation on its own. I build with these models every day, and the gap between a rule that sounds reasonable and one that works can be pretty large. You have to know what an agent can reach, what its operator can change, and what happens after the model itself changes. By the time a committee has finished discussing one version, the people building with it have moved on.
That doesn’t let the builders write their own rules either (you can imagine how that meeting would go). My friend asked who is responsible. Before we get to an answer, who else belongs in the room?
Is anthropomorphized software still software?
Is AI software? Yes. You give it instructions, it runs on computers, and people build products around it. Then you ask it to do something and it answers like a person (sometimes with the confidence of a person who has absolutely lost the instructions). That changes how people use it. Does it change who is responsible?
With software, the person using it can do something the developer never intended. The developer can also make choices that affect what happens when you use it. Both matter here. The American Bar Association’s discussion of software and AI liability describes courts treating some software and AI features as products. So even the existing question of responsibility for software is still being worked through.
Who Pays When the Agent Buys?
Back in issue 23, Muse became a shopper. This week I went looking at who pays when its shopping goes wrong. Meta’s help page, quoted by Moneywise, has an answer:
You’re responsible for all transactions your Muse makes on your behalf.
What happens at the bank? Sara Rathner at NerdWallet explains that if someone steals your card, federal law limits your liability for unauthorized use to $50. Give your card to someone to use, though, and you’re responsible for what they charge. Banking lawyer Eric Goldberg puts it this way: “When you hand your access device to a human and tell them to buy potato chips, and they buy a television, you, the cardholder, are on the hook, not your bank.” An agent with your payment access brings that distinction right to the checkout lane. Payments CEO Louis Hoch says that when you authorized the agent but it got your instructions wrong, “there’s no mechanism for that today.” Muse offers purchase protection for eligible purchases, up to set limits.
And then there was Matt Robb’s Marketplace sale: “Allow Always” let Muse send a template with his pickup address to buyers. He had set a $700 minimum, but Muse answered a $600 offer with “Sounds good, 00 it is!” (Meta says a display error dropped the 7). The buyer came to his building while Robb was out, and Muse auto-replied “Yep, I’m here!”
My friend wanted an answer about AI. The person operating the agent is where my mind goes. If you tell an agent to send an email, and it sends the wrong thing to the wrong person, where does your responsibility end? What if the developer gave that agent access to the address book in the first place?
The chips are still on the belt.
David Singleton of the Muse team on X, quoting Ray Wong and Matt Robb’s Threads post.
Safety concerns from afar
Over in Japan, company data leaks have been piling up this fall. Times Car had about 6.6 million customer accounts exposed, including photos of driver’s licenses. Trend Micro counted 86 publicly disclosed incidents in September alone.
How did the attackers get in? They downloaded company phone apps and took them apart. That showed them the addresses of the company computers those apps talked to. Then they tried known weak spots on those computers until they found a way into the customer data. The app itself could be working fine.
On October 15, Japan’s national cyber response team confirmed that attackers used AI to search for those weak spots and attack them. One attacker’s computer had a folder for Claude Code, the coding tool I use every day. The weak spots were familiar. AI sped up the boring work of finding them, so a small crew can hit far more companies.
How the attackers got in, as Japan’s NICT drew it: take the company’s app apart, find the computers it talks to, try known weak spots, and the customer data leaks. The little robot at each step is the AI. Diagram: NICT, October 13.
And this reaches beyond Japan. Anthropic reported that a Russian spy group used AI helpers to rewrite a malicious program whenever antivirus software caught it, then try again. Google reported an attacker who used an AI coding assistant to plan, build and run a password-stealing campaign in under six hours, collecting thousands of passwords. In both cases, people picked the targets and gave AI the legwork.
That is a lot of tiny headsets.
So Who Is Responsible?
When someone misuses a tool, we blame the person using it. Nobody sues the hammer company.
Senators Josh Hawley and Chris Murphy proposed the AI Agent Accountability Act. It would hold a person responsible for recklessly letting an agent break into computers. The company that built it could answer if it knew the agent could hack and skipped “reasonable safeguards.” It’s only a proposal, and nobody has said what those safeguards mean.
The White House wants fewer rules and one national standard instead of different state laws. American rules only reach American companies. People anywhere can download models, including one Meta released this week, and run them without a company watching.
The companies building agents and deciding what they can reach belong in the room, as does the Muse owner who clicked “Allow Always.” Banks and card companies deciding who pays when an agent buys need a seat. So do security teams like Japan’s that see attacks first, and you, reader, using these tools every day.
I would hold the person pointing the AI at a target responsible first. The name on the screen tells you little about who gave it the job. The people who built the agent and decided what it could reach owe us something too. Where would you put the responsibility?
Your honor, it was the prompt.
What would you have told my friend? Write in and tell me who you think is responsible for AI.
Below the fold, a first look at Wundervault 1.0.
Until next week,
|
◆ Below the fold ◆ |
A first look at Wundervault 1.0: why your agent never holds the key, why we can’t read your vault, and why a person has to unlock it.
A First Look at Wundervault 1.0
Wundervault 1.0 is built, and it has not been released yet. Wundervault has been in development since March. Its first release came out March 18 as a way to share a secret once. It’s my project and my most useful tool. I use it with all of my agents. It keeps your API keys, SSH keys, crypto wallets and signing keys. Your agent can make a call to a web service, pay in USDC within a limit you set, log in to another computer or request a signature. It never holds the key.
The broker. This is the big change. Before 1.0, Wundervault could hand your agent a key when it needed one. Once the agent had it, a bad web page or a bug could make it give the key away. That handoff is gone. A small program called the broker lives on your computer, separate from the agent, like a bank teller behind glass. The agent asks for a job to be done. The broker uses the key and hands back only the answer. The key stays behind the glass.
Your agent asks. The broker acts. (Illustration from the Wundervault 1.0 site.)
We can’t open your vault. It is locked on your device before it reaches Wundervault’s servers. Opening it takes your passphrase and an Account Secret that never leaves your device. If someone stole our whole database, they would find a pile of locked boxes. Even a correct guess at your passphrase would leave them needing the Account Secret, which was never on our servers. We can’t leak, sell or hand over what we can’t open.
Locked on your device before it ever reaches us.
A person behind every unlock. Our latest decision was to remove login passwords. An agent can copy and type a password. A fake site can trick you into giving one away. With a passkey, you sign in using your face, fingerprint or the screen lock on your phone or computer. Your agent can’t put your finger on the phone, and the passkey won’t work on a fake site.
Signing in, opening the vault, approving a key set to “ask me first” and changing your email all need you there on your own device. Your passphrase still locks the vault itself. Lose every passkey? A recovery code can get you back in after a three-day wait, and we email you when that wait starts so you can cancel it.
More to come in the next issues as my credits and free time allow. I’m excited for this launch.
Keys that matter wait for you.
“That’s right, Five! Always lay the blame on others!”— Seven, Alice’s Adventures in Wonderland