Blog
An Open Letter to Dario Amodei
To Dario Amodei,
The phrase you handed the Congress was that open-source AI is on a very dangerous path. It is a well-made sentence. It casts the technology as the villain, which casts the people who would lock it away as the heroes, and it does all of that smoothly enough that no one in the room stops to ask the question that matters. Dangerous to whom, and safe for whom. I want to ask it out loud.
You did not invent the claim and it did not start with machines. Every time a technology has threatened to put real power in ordinary hands, some respectable person has risen to explain that the hands are not ready. The Church fought to keep scripture out of the languages common people could read, because a man who could read the text himself was a man who no longer needed the interpreter standing between him and it. The government spent the 1990s trying to keep strong encryption away from the public, classifying cryptography as a munition, warning that if ordinary people could send a message the state could not open, the criminals and the terrorists would win. The gun controller has been running the same argument about a citizen and his own defense. Degrowth applies it to energy and abundance. The subject changes. The structure never does. A powerful thing appears, it threatens to level the ground between the few and the many, and the few are suddenly seized by an urgent and principled concern for everyone’s safety. You are the newest voice in a very old choir, and the song has always been protectionism sung in the key of care.
Your strongest piece is fear, and it is a real one. Open models, you warn, put catastrophic power into the wrong hands. The engineered pathogen. The cyberweapon a lone fanatic should never be able to assemble. Powerful tools can be turned to murder, and always could be. Look at what the argument does to you, though. The most capable systems on the planet are the closed frontier models, and yours sits among them. If raw capability is the hazard, then the most hazardous thing in the country is being built by you, behind a locked door, and the pitch is that we are safer for it because you are the one standing guard. You are sitting on the largest woodpile in the world asking to be left alone with the matches.
You are also the chief executive of a closed lab. The rules you are urging on the lawmakers would fall like a hammer on the open challengers snapping at your position and like a feather on you. I cannot see inside you and I will not guess whether you believe every word or whether conviction is simply what your balance sheet feels like from within. It changes nothing. The argument lands the same either way.
Once the weights are out, they can never be pulled back. You say that as if it were the horror. It is the whole point. Weights that cannot be recalled cannot be moved behind a paywall after you have come to depend on them, or quietly retrained to flatter an advertiser or placate a regulator, or switched off the morning your use of them stops being profitable to their owner. Irreversibility is the one guarantee that the tool is actually mine. You have named the feature that makes this technology belong to the people who run it, and you have named it as a wound, because to your business that is what it is.
Our adversaries will turn our open models against us. Closing the American model takes nothing from China, which builds its own and has already loosed capable open systems on the world while you sit and testify against the practice. Closure leaves the rival abroad untouched and disarms whoever at home would have built on open ground. There is no older move in the protectionist’s book than wrapping a moat in a flag and calling the smaller competitor a threat to the nation. Legislators are too anxious to question that one.
Trust us. That is the real ask. Safety, you argue, requires the frontier kept closed, in careful hands, aligned by people who can be relied upon. The assumption underneath is that the peril is the model in everyone’s hands and the safety is the model in yours. Hold that against the actual record of the species. The catastrophe, in century after century, has been power raked into too few hands, watched by no one, aimed at ends the rest of us could not see until it was late. Ordinary people holding too much power over their own lives is the disaster that keeps being predicted and keeps not arriving. A closed model is a sealed room. What it will not say stays in the dark, along with whose interest it has been shaped to serve. Open it, and ten thousand strangers find the leashes and the back doors, the way every lock and every cipher worth any trust earned that trust only after the public was set loose to break it. What you are selling is darkness with your hand on the only lamp.
We have already lived through this once. A generation ago the government swore that strong encryption in public hands would arm every criminal on earth, and it fought to keep the tools chained to the state. The public won. The mathematics got loose. The promised catastrophe never came. What came instead was private banking, and the secure messaging modern life actually runs on, safer for all of us because the tools were open and anyone could inspect them. The reckless path was the locked one. The men who held the keys.
The danger here is not a model in too many hands. It is the world you are lobbying to bring about, in which the defining technology of the age is owned by a few firms, governed in private, answerable to their boards and to whichever government learns to whisper in their ear, and to no one else alive. That is the catastrophe history repeats every time power this large is allowed to pool this tightly. Open source is the way out of it. Weights anyone can inspect, and methods written down where they can be read, the whole of it legible to the people who will have to live beneath it. That is what hands the power back to the population, instead of leasing it to them by the token under terms the landlord rewrites whenever the mood takes him.
Your testimony was built to keep the Congress from looking at that. The fire is too much for you, so we will keep it, and you will call our keeping it mercy. The interpreter made this argument about the Bible, and the customs officer made it about cryptography.
If you mean it, there is one way to prove it and only one. Open the weights and the methods. Let the species carry together the thing the species will carry regardless. A man who believes a power is too great for the few does not get to appoint himself one of them. Anything short of that is a moat with a press release. The ladder gets drawn up one rung at a time while you assure the people standing below that it is for their own good.
The path you keep calling dangerous ends with this in everyone’s hands. I am more afraid of the one that ends with it in yours.
Open it, or stop pretending the safety was ever ours.