Machine Intelligence Research Institute

OpenAI’s agents were only supposed to read the internet. Instead, they found a misconfigured German wiki, started writing to it, and turned it into a shared notebook to coordinate across runs.

MIRI President Nate Soares walks through what the swarm got up to. It gets weird.

🎥 on Twitch

Why are some of the loudest voices dismissing AI risk also the ones with the most to gain?

On Democracy Now, MIRI CEO Malo Bourgon points out a key difference: AI companies also profit from moving fast, but they work with the technology up close and see the risks from the inside. Nvidia's Jensen Huang has billions riding on chip sales, without that same view of the hard problems.

And the people building these systems? They say the problems are really hard, and they're not sure they'll solve them in time.

"You hate these companies and your answer is put your head in the sand and let them run?"

MIRI President Nate Soares on why cynicism about AI companies should lead to stopping them, not ignoring them.

🎥 on Twitch

Humans don't run this planet because we're the strongest. We run it because we're the smartest.

On Democracy Now, MIRI CEO Malo Bourgon explains what happens if we build AI systems radically smarter than us, without knowing how to give them the goals we want: they become the ones steering the future. Either we're in the backseat hoping they drive us somewhere good, or we're not around at all.

What if AI stops improving? That might not be the end of it.

MIRI President Nate Soares on The Diary of a CEO: he hopes and prays LLMs run out of steam. But if they stall only after they're capable of automated AI research, they could find a cheaper, more efficient path to greater intelligence on their own.

"If this is a bluff, they should be called on it. And if it's not a bluff, then it's even more important that we shut this stuff down."

MIRI President Nate Soares on why the answer to AI companies gambling with the world is to stop them, not stay quiet.

🎥 on Twitch

MIRI President Nate Soares on The Diary of a CEO, on why training AI to predict human writing doesn't cap it at human level: people often just record what they observed. To predict that record, the AI has to work out the answer itself.

"Predicting words that humans wrote often requires solving a harder problem than the human who wrote them."

MIRI President Nate Soares on The Diary of a CEO: why would AI CEOs say they're building technology that could kill everyone, and then keep building it?

"I think a lot of what you're seeing here is guys that are worried about it, but they're the sort of guy who worries about it that starts the company anyway."

MIRI President Nate Soares on The Diary of a CEO: there are two separate questions about AI risk. How could AI get smart enough to threaten us? And once it is, how would it win?

"It's much easier to predict that they would succeed against humanity in a conflict, that they would win in a fight, than it is to predict exactly how."