Maarten Boudry | Will Humanity Be Subjugated by Superintelligent AIs?

Some people are worried that if we ever create superintelligent AIs, they might turn against us—trying to subjugate humanity, wrest control, and grab resources, much like living creatures shaped by evolution. Dan Hendrycks from the Center for AI Safety has argued that AI systems are already undergoing a form of natural selection, facing ruthless market competition in the current AI race. Will this endow them with the instinctive drives for self-preservation and dominance typical of evolved creatures? In this talk, I push back against this evolutionary doom scenario, using the framework of “Darwinian spaces” by Peter Godfrey-Smith. A better analogy for AI evolution might be the domestication of animals. Just as humans have bred dogs to be friendly and obedient, we might shape AIs in similar ways, selecting for desirable traits like helpfulness and non-aggression. Even in a highly competitive AI race, AIs are unlikely to become selfish or power-hungry. That said, we do agree with the AI doomers on one point: if we allow AIs to “go feral” and be subjected to truly blind evolution—like wild animals competing in nature—that could become very dangerous.