This talk presents an innovative approach in AI research aimed at the development of generally-intelligent, alignable agents. Inspired by kin-selection theory, we use reward sharing to create dynamic, socially complex environments. We believe that training agents in these environments may lead to both general intelligence and cooperative behaviors.

Our approach centers on creating adaptable agents capable of handling unforeseen scenarios, beyond their training environments. This adaptability is achieved through a self-generating learning curriculum, which challenges agents without overwhelming them. A key aspect of our methodology is the introduction of a social dimension, where agents interact with and adapt to the actions of others. This mirrors real-world social complexities and shifts the learning focus from the simulated environment to the minds of other agents.
This setup encourages a diverse range of strategies, fostering continuous learning and adaptation. It explores the full spectrum between cooperation and competition, avoiding the pitfalls of stagnant overly adversarial behaviors.

The end goal is to create AI that not only exhibits advanced intelligence but also allows for intra-agent cooperation, a vital step towards AI alignment.