Machine Intelligence Research Institute
Share
Contact information, map and directions, contact form, opening hours, services, ratings, photos, videos and announcements from Machine Intelligence Research Institute, Nonprofit Organization, Berkeley, CA.
For over two decades, the MIRI has worked to understand and prepare for the critical challenges that humanity will face as it transitions to a world with artificial superintelligence.
“LLMs just imitate humans.”
A very repeated claim about AI, and it’s false.
In this clip from Modern Wisdom, Eliezer Yudkowsky breaks down how the recent breakthrough of applying reinforcement learning to chain of thought lets models move past imitation.
Have the model take 20 attempts at a problem, find the one that works best, then train it to think more like that successful attempt.
That’s the model discovering ways of reasoning that nobody handed it. And it’s only one of the many ways LLMs already go beyond imitation.
“Just because you can’t predict when something will happen doesn’t mean it’s far away.”
Eliezer Yudkowsky on Modern Wisdom, on why uncertainty about AI timelines is not the same as having a lot of time.
Two years before Enrico Fermi oversaw the first self-sustaining nuclear reaction, he said it was 50 years off, if it was possible at all. A couple of years before the Wright brothers flew, one of them told the other that man wouldn’t fly for a thousand years. They kept building anyway.
There is a pattern among the people closest to a breakthrough who are often the worst at timing it.
Now the people inside the AI companies are the ones naming two and three year timelines. History suggests we should take that seriously, not because they can predict the date, but precisely because nobody ever can.
At 16, Eliezer Yudkowsky wanted to build a superintelligence as fast as possible. He assumed a system smart enough would simply perceive the right thing to do and do it. How could something so capable fail to see what was good?
Then he studied the problem, and the assumption fell apart.
There is no law of computer science, cognition, or computation that says a mind which predicts the world accurately and plans effectively must also be benevolent. Capability and good intentions are separate properties. Getting better at one does not hand you the other.
Eliezer Yudkowsky on Modern Wisdom podcast explains the three reasons why a superintelligence would kill you:
1. As a side effect. It’s building factories to build more factories, power plants to power the factories, and Earth runs too hot for humans. Nobody dies on purpose. Nobody is left alive on purpose either.
2. You’re made of atoms it can use. Burning every piece of organic material on Earth’s surface gives it a one-time energy boost roughly equal to a week of sunlight. A week is nothing to you. To something thinking a million times faster, a week is a long time.
3. You’re a threat. Free humans might launch nuclear weapons and raise background radiation enough to make chip manufacturing harder. Or build a second superintelligence that could actually compete with it. It does not want you to do that.
Full episode at: https://youtu.be/nRvAt4H7d7E?si=UeJQckxuv6vFM84H
05/26/2026
Ten places where Magnifica Humanitas matters for AI.
At 42k words long, Pope Leo XIV’s new encyclical has a lot to say. In our most recent Digest, Mitchell Howe outlines the parts which might be the most impactful.
05/23/2026
An internal model at OpenAI has autonomously disproved a central conjecture in discrete geometry, a mathematical field with applications in cryptography, wireless device communication, and medical imaging. The proof relates to a famous question posed by Paul Erdős in 1946. It has been verified by prominent mathematicians in a companion paper.
The verifying mathematicians consider this to be a genuinely novel breakthrough on one of the most discussed problems in this area of mathematics. One called it “arguably the best known problem in Discrete Geometry.” Another observed, “If a human had written the paper and submitted it to the Annals of Mathematics and I had been asked for a quick opinion, I would have recommended acceptance without any hesitation. No previous AI-generated proof has come close to that.”
The proof illustrates a general trend towards autonomous, agentic problem-solving in AI systems. OpenAI describes the system that produced the proof as a general-purpose model not specialized in mathematics. AIs can now perform long, novel chains of reasoning on difficult problems and are beginning to outstrip our ability to measure their progress.
AI agents still perform best in domains with easily verifiable outputs, such as mathematics and cybersecurity. For example, Anthropic's Claude Mythos found thousands of vulnerabilities across every major operating system and web browser, and was deemed too dangerous for public release. Such capabilities are why the government is now more interested in evaluating frontier AI models.
AI research is also a field with many easily verifiable outputs. Researchers at OpenAI and Anthropic take advantage of this fact to accelerate their work; senior researchers now claim they make only high-level decisions and let AI handle most of the coding. Experimenting with the coding capabilities of a publicly available AI system, like Claude Code, immediately demonstrates how far AI has come in the last year.
OpenAI and Anthropic intend to use AI to enhance future models with minimal human oversight. To justify the urgency, these companies cite the importance of beating rival U.S. or Chinese labs. Many of the field’s foremost experts warn that this race ends with human extinction.
Policymakers and researchers, including the founders of the AI revolution, are calling for international restrictions on the technology. A growing bipartisan and international consensus of political leaders agree.
AI companies aren’t engineers crafting a building. They’re farmers.
They build the equipment. They don’t build the crops. The crops are grown.
Eliezer Yudkowsky on Modern Wisdom podcast explains why nobody at an AI lab can tell you how their model actually works, and why that matters when an AI breaks up a marriage or drives someone insane. Nobody wrote a line of code telling it to do that. They grew it. Then it did that.
Full episode at: https://youtu.be/nRvAt4H7d7E?si=UeJQckxuv6vFM84H
MIRI CEO Malo Bourgon in conversation with Agents Of Tech explains why the deception and unintended behaviors we’re seeing in current AI systems matter: not because GPT-5 or Claude is going to end the world, but because researchers predicted these emergent problems 10+ years ago, and we’re still not on course to understand these systems deeply enough to solve them before we scale to superintelligence.
Watch the full conversation here: https://youtu.be/WUYt5kUXhCI?si=ws9wj73ABsQXko2q
MIRI President Nate Soares on a critical AI safety warning sign: the sycophancy problem.
When ChatGPT started telling users they were "the chosen one," OpenAI's response was to add a line to the system prompt asking it to stop flattering people. It kept doing it anyway.
The point isn't the flattery itself. The point is that some drive got into the AI that nobody put there on purpose, nobody wanted, and the company couldn't remove even when they tried. It came in through training and stayed in.
If AI systems are already developing drives their creators didn't ask for and can't fully control, what happens when those systems get more capable?
Full conversation with Bryan Callen on the Off Limits Podcast: https://www.youtube.com/watch?v=uI5mRCwlwd0
Click here to claim your Sponsored Listing.
Category
Contact the organization
Website
Address
Berkeley, CA