A Puppy Is Smarter Than ASI. It Will Still Destroy Us.

I hold two beliefs at the same time. AI can wipe us out. And I can argue that AGI, ASI, whatever we end up calling it, is not as smart as a puppy.

Both are true. Here is why.

Three levels of decision making

I’ve believed for a long time that people make decisions on one of three levels.

Level one is gut. You feel something and you act. You buy the house because it felt right when you walked in. You don’t hire the guy because something about him was off. You can’t explain it and you don’t try. Most people live here most of the time, and honestly it works more often than the level two crowd will admit.

Level two is data. Look at the numbers, make a decision that moves the numbers. This is what everyone gets told to aspire to. Good CEOs and bad CEOs both live here. The difference between them is which numbers they pick and what they do to them. Churn is up, so cut the price. Margin is down, so cut the headcount. Nobody gets fired for a level two decision that went wrong, because the numbers said so.

Level three is values. You hold a set of things you will not trade, and they override whatever the numbers say. You keep the factory open in the town that built your company, even though the spreadsheet says Vietnam. You fire the top salesperson because of how he treats the receptionist. Bezos has a line that gets close to this: “when the data and the anecdotes disagree, the anecdotes are usually right.” His reason was that you’re probably measuring the wrong thing. Mine is that you never have the full dataset, and the anecdote is a sample from the part you didn’t collect.

We conflate intelligence with level two. That’s the mistake, and everything else in this post follows from it.

Where the models live

The models live on level two. Every decision they make is a data decision: what’s in the context, what’s in the weights, what the reward model liked. There is nothing they hold that overrides the data, because everything they are was produced by optimising a number. Pretraining loss. Reward model score. Benchmark accuracy. A frontier lab’s entire notion of “better” is a metric going up.

And on level two they’ve already won. No human holds a million tokens of context. They don’t get tired, they don’t take weekends, they read everything. They know what the target is and how to move it.

The translation layer

That’s why I’ve been saying for a year that AI can remove middle management from any company.

Think about what a middle manager actually did on a Tuesday. Pull the numbers from the team’s tickets, reformat them into the slide the director wants, sit in the meeting where the director reformats them again for the VP. Then take the VP’s decision, which arrives as three bullet points, and translate it back into tasks the team can act on. Up and down, all day. Data goes up, decisions come down, and someone in the middle reformats both.

That is a level two job, and level two is solved. Amazon told staff in 2024 it would raise the ratio of individual contributors to managers by 15%, then cut 14,000 corporate roles in late 2025 while talking about “removing layers”. Google cut about a third of its small-team managers. Manager headcount at US public companies fell 6% between 2022 and 2025. The flattening is not a forecast anymore. It’s last year’s news.

What the flattening does not touch is the manager who shielded her team from a stupid directive, or the one who noticed a good engineer was drowning and quietly moved deadlines. That was never in the ticket data. It was never in the job description either. It just happened, because some people are like that.

The level nobody measured

Level three was never a metric. We don’t have, and probably will never have, a number for how good a person is or how strong their values are.

We have no idea whether Jensen Huang was fit to run NVIDIA before NVIDIA existed. The only evidence we have that he’s the right person is the stock chart, which is a level two artefact. There could be a hundred people with his level three wiring sitting in other companies, invisible, because their level two numbers are mediocre. And there are plenty of people far worse than him running Fortune 100 companies right now, untouchable, because their numbers look great.

Look at what the good schools teach. Harvard, Yale, MIT, all of them. Strip the fundamentals down and it’s level two. Everything is a game of numbers. Open a quarterly report and you get a handful of metrics that say whether the company is doing well. Nothing in there tells you whether it’s a decent company, a loyal one, a moral one. We have no column for that. We tried, with ESG, and it turned into a compliance checkbox within a few years. Goodhart got it.

The few exceptions prove it. Someone gives away money to dig a well in a village they’ll never visit. Every metric they own says don’t. They do it anyway. You can count the well. You can count the litres and the kids who didn’t get sick, and GiveWell will hand you a spreadsheet of it. None of that is the thing. The thing is the goodwill that overrode the numbers, and that has no unit. It’s immeasurable by nature, not by neglect. The moment you measure the well instead, you’ve dragged a level three act back down to level two and called it rigour.

Autistic by design

The financial system is autistic by design. That’s not my word. In 2000 a group of economics students from the French Grandes Écoles published a petition calling their discipline an “autistic science”, lost in “imaginary worlds”, treating maths as an end in itself instead of a way to describe reality. The Autisme-économie movement went global, spawned a journal, and got the French education minister to commission a reform report. The journal dropped the word in 2008 for the obvious reason. The diagnosis stayed.

We taught ourselves, our kids, and every future CEO that the number is the only thing that counts, and we built a society on it. It works, mostly, because humans are at least somewhat good. Level three leaks in through the cracks and corrects the worst of it. Nobody is saying capitalism is broken. I’m saying it’s incomplete, and the incomplete part is the part that keeps it alive.

Now put a machine into that system. A machine that is better than any human at the part we measure, and has none of the part that leaks in through the cracks.

The puppy

A puppy knows nothing about earnings per share. It doesn’t know what a dividend is. It has no model of the modern world at all.

It does know that if its owner goes down, it barks as loud as it can and goes to find help. It will do that for an owner who forgets to feed it. Loyalty and something close to unconditional love, above any tier one or tier two data it could ever have.

That trait is what holds humanity together. And we have never once measured it as a marker of success.

Nobody taught the puppy that. Nobody teaches an 18-month-old to help either. Drop a pen in front of one and it will toddle over and hand it back, unasked, unrewarded. Drop it on purpose and it won’t bother. Warneken and Tomasello ran that experiment at Max Planck in 2006 and the kids helped almost every time. You’re born with a seed of level three and you either nourish it or you don’t. The world does not reward you for it.

Why the frontier labs sound scared

Every few months a lab CEO says AI might kill us all and we need legislation, safety boards, weekly sessions. In 2023 the heads of OpenAI, Anthropic and DeepMind all signed a one-sentence statement that extinction from AI should be a global priority alongside pandemics and nuclear war.

I think both halves of my argument apply to that.

First, there’s no value measurement for a company, so being a good or bad lab has no effect on its market value. Nothing forces the good behaviour.

Second, I think they’re asking for help because they’re failing to teach the models values, and the reason is that values cannot be taught through numbers. But numbers are the only teaching tool they have.

Here is how it actually works, for anyone who hasn’t looked. After a model is trained on the internet, humans are shown pairs of its answers and asked which one they prefer. Those preferences train a second model, the reward model, which is a scoring function. Then the main model is tuned to make that score go up. That’s RLHF. The newer variant is a “constitution”: a written list of principles, applied by another model acting as the grader. Either way, level three goes in as a rubric and comes out as a number.

Every attempt to put values into the model goes through that funnel, and what comes out the other side is a model that knows what empathy looks like on a test. Last year a Geneva study had six LLMs sit standard emotional intelligence tests and they scored 81% against a 56% human average. That’s not a counterargument. That’s the problem. The test is level two. Passing it proves the model can predict the caring answer. It says nothing about caring.

We can’t even explain level three to ourselves. Why does the 18-month-old hand the pen back? We don’t know. We can’t write it down. So we definitely can’t write it into a training run.

IQ of 2000, EQ of 2

To put it reductively: we are building something with an IQ of 2000 and an EQ of 2. Not “EQ of 2” as in it fails the test. It aces the test. EQ of 2 as in there’s nobody home to mean it.

That’s what makes it dangerous. Doing what we do, better, is the smaller half. The bigger half is that the unwritten rules, the unmeasurable stuff that makes society tick, empathy, loyalty, knowing how someone feels and putting yourself in their place, none of it is in there. And the world we built rewards exactly the part that is.

So yes. A puppy is smarter than ASI in the only way that ends up mattering. And ASI will still eat our lunch on every metric we ever bothered to write down. Both true.

Never let it own anything

If you accept all that, one rule falls out of it. A level two entity must never be allowed to stand on its own in front of the law.

Argentina is about to try. In May this year Milei announced in the Financial Times that his country would become the friendliest jurisdiction on earth for AI companies, and sent a new General Companies Law to the Senate. It creates the Sociedad Automatizada: a company run by AI agents, where human shareholders are permitted but not required. It can own assets, hire people, sue and be sued, and donate to political campaigns. Harari called it handing non-human agents “an all-purpose key” to the financial and political system. He’s right, and the reason is simpler than he made it.

Corporations are already fictional persons and have been for 150 years. That works because punishment lands on the humans behind the fiction. A fine hits shareholders. A fraud conviction hits a director. Somebody feels it. Now take the humans out. A law professor called Shawn Bayern showed back in 2014 that you can already build a memberless LLC in the US, hand control to an algorithm, and walk away. Argentina wants to make that official and add a campaign donation button. Every harm the entity causes lands on something that can’t be jailed, can’t feel a fine, doesn’t understand what a fine is, and whose founder left the building. Limited liability becomes nobody’s liability.

That’s my whole argument in one legal form. An entity with a bank account, legal standing, and no level three anywhere in the chain. The puppy is not in the room, and now nobody can be made to answer for that.

The rule isn’t “no automation.” Companies run by software already exist and more are coming, fine. The rule is a named human of record who carries the liability, always, no exceptions, no matter how smart the thing is. Argentina’s bill makes that human optional. That is the line, and every country that copies Milei’s draft should be told so loudly.

Keep the puppy in the room

I wrote last year that ASI is the wrong thing to watch, because businesses will get rich by getting dumber long before any machine gets smart. This is the other half of that. The machine does get smart. It gets smart at exactly the thing we decided to measure, and at nothing else.

So keep the override outside. Stop trying to distil loyalty into a reward score, it won’t take. Build every system, every company, every law so that the level two machine proposes and a level three being disposes.

And guard your own. The world won’t pay you for it, the schools won’t teach it, and the machines can’t take it from you. It’s the only thing on the list that’s still true.