Tell me if I’m wrong about this. A world that has multiplying A.I. agents working on behalf of people, millions upon millions, who are being given access to bank accounts, email accounts, passwords and so on, you’re just going to have essentially some kind of misalignment, and a bunch of A.I. are going to decide — “Decide” might be the wrong word, but they’re going to talk themselves into taking down the power grid on the West Coast or something. – Won’t that happen? – Yeah, I think there are definitely going to be things that go wrong, particularly if we go quickly. So I don’t know. This is one area where people have had just very different intuitions. There are some people in the field who say, look, we program these A.I. models. We make them, like, we just tell them to follow human instructions and they’ll follow human instructions. Your Roomba vacuum cleaner doesn’t go off and start shooting people. Like, why — why is an A.I. system going to do it? That’s one intuition. And some people are so convinced of that. And then the other intuition is like, we basically, we train these things, they’re just going to seek power. It’s like “The Sorcerer’s Apprentice.” How could you possibly imagine that, like, they’re not going to — they’re a new species. How can you imagine that they’re not going to take over. And my intuition is somewhere in the middle, which is that, look, these — you can’t just have these things do exactly what you want to do. They’re more like growing a biological organism. But there is a science of how to control them. Like, early in our training, these things are often unpredictable. And then we shape them. We address problems one by one. So I have more of a, not a fatalistic view that these things are uncontrollable. This is a complex engineering problem. And I think something will go wrong with someone’s A.I. system, hopefully not ours. Not because it’s an insoluble problem. But, again, and this is the constant challenge because we’re moving so fast.
