This update will tell you how in the pastĀ month, we have been fulfilling our mission of advancing revolutionary technologies by rewarding research excellence, raising awarenessĀ againstĀ recklessness, and creating a community to promote beneficial uses and reduce risks associated with these technologies.
We brought the top experts together around coordinating progress inĀ Artificial General IntelligenceĀ Safety
After the encouraging response to last year’s Artificial General Intelligence (AGI)Ā strategy meeting onĀ Timelines & Policy (white paper), this year Foresight Institute organized another one-day AGI strategy meeting around the Effective Altruism Global Conference, June 8-10, to gather important AGI safety organizations.
In addition to examining strategies for impact, this meetingĀ served as point of contact to coordinate efforts with other AI safety organizations.Ā
Given the deserved attention this topic has recently received by Good AI, FHI, FLI, OpenPhil, and other organizations (via grants, prizes, and opening positions), we dedicated this yearās strategy meeting to discuss AI/AGI Coordination, especially amongst Great Powers.
Topics addressed:
Stay put, white paper is coming soon!
The participants of this meeting truly made it a proficient venture into coordination. Heartfelt thank you:
Olga Afanasjeva – Good AI
Ā Stuart Armstrong – Future of Humanity Institute
Ā Seth Baum – Global Catastrophic Risk Institute
Ā Haydn Belfield – Centre for the Study of Existential Risk
Ā Rob Bensinger – Machine Intelligence Research Institute
Ā Malo Bourgon – Machine Intelligence Research Institute
Ā Niel Bowerman – 80,000 Hours
Ā Ryan Braley – Lightbend
Ā Tom Brown – Google Brain
Ā Samo Burja – Bismarck Analysis
Ā Ryan Carey – Ought, Foresight Fellow
Ā Betsy Cooper – Center for Long-Term Cybersecurity
Ā Owen Cotton-Barratt – Future of Humanity Institute
Ā Miron Cuperman – Base Zero
Ā Jessica Cussins – Future of Life Institute, Center for Long-Term Cybersecurity
Ā Jeffrey Ding – Future of Humanity Institute
Ā Allison Duettmann – Foresight Institute
Ā Peter Eckersley – Electronic Frontier Foundation
Ā Kevin Fischer – Crypto Lotus
Ā Carrick Flynn – Future of Humanity Institute
Ā Benjamin Garfinkel – Future of Humanity Institute
Ā Melody Guan – StanfordĀ
Ā Geoffrey Irving – OpenAI
Ā De Kai – Hong Kong University of Science & Technology
Ā Alex Kotran – AI Initiative, Harvard Kennedy School
Ā Victoria Krakovna – DeepMind
Ā Janos Kramar – DeepMind
Ā Tony Lai – Legal.io
Jade Leung – Future of Humanity Institute
Ā Matthew Liston – ConsenSys
Ā Terah Lyons – Partnership on AI
Ā Matthijs Maas – FLI, Ā Global Catastrophic Risk Institute
Ā Richard Mallah – Future of Life Institute
Ā Fiona Mangan – Justice and Security in Transitions
Ā Tegan McCaslin – AI Impacts
Ā Joe McReynolds – China Security Studies Fellow, Jamestown Foundation
Ā Eric Michaud – Rift Recon
Ā Mark Miller – Foresight Institute, Ā Agoric
Ā Ali Mosleh – John Garrick Institute for the Risk Sciences
Ā Mark Nitzberg – Center for Human Compatible AIĀ
Ā Jim OāNeill – Mithril Capital
Ā Catherine Olsson – Google Brain
Ā Michael Page – OpenAI
Ā Christine Peterson – Foresight Institute
Ā Peter Scheyer – Foresight Fellow
Ā Carl Shulman – Future of Humanity Institute
Ā Tanya Singh – Future of Humanity Institute
Ā Jaan Tallinn – Future of Life Institute, Center for the Study of Existential Risk
Ā Alyssa Vance – Apprente
Ā Michael Webb – StanfordĀ
Ā Qiang Xiao – School of Information UC Berkeley, China Digital Times
Ā Mimee Xu – UnifyID
Ā Roman Yampolskiy – University of Louisville,Ā Foresight Fellow
We had a fantastic time probing the minds of David Eagleman and Arvind Gupta on the future of Neurotech and Biotech
Representing IndieBio, David Eagleman and Arvind Gupta joined Foresight agents Allison Duettmann and Lou Viquerat on a missionĀ to explore the paths forward for human biology.
The evening had two practical purposes: a.Ā laying out a roadmap of biotech and neurotech in order to b.Ā establish an ethical frameworkĀ to guide the development of those technologies in the very near future. All participants were encouraged to inform the discussion, and our audience questions proved to be on point! Watch both salonĀ videos and enjoy this engaging and participative discussion.
Part A:Ā Roadmap for Human Biology.
Part B: Ethics of Future Human Biology
Our salons aim at helping our civilization endure through a healthy critique of our current paradigms.
The monthly salon series in 2018 focuses on strengthening civilizationĀ āan approach we first discuss as strategy for AI safety in our paper on Decentralized Approaches to Reducing Existential Risks.Ā Since itĀ is a mouthfull, we divided the topic up into manageable pieces.
Topics we discussed to date (all salonĀ videos available in this YouTube playlist):
We cannot wait to read “Artificial Intelligence Safety and Security”
Foresight FellowĀ R. Yampolskiy‘s book with contributions from Eric Drexler, Nick Bostrom, Max Tegmark, Bill Joy, Ray Kurzwell, Ellezer Yudkowsky, Ian Goodfellow, David Brin, Kevin Warwick, Edward Frenkel, Samy Benglo, and many more.Ā
Roman V. Yampolskiy, 2018 Fellow in Artificial Intelligence Safety & Security isĀ publishing a book on AI. We recommend you pre-order now, lest you forget!
Features
Ā
Summary
“The history of robotics and artificial intelligence in many ways is also the history of humanityās attempts to control such technologies. From the Golem of Prague to the military robots of modernity, the debate continues as to what degree of independence such entities should have and how to make sure that they do not turn on us, its inventors. Numerous recent advancements in all aspects of research, development and deployment of intelligent systems are well publicized but safety and security issues related to AI are rarely addressed. This book is proposed to mitigate this fundamental problem. It is comprised of chapters from leading AI Safety researchers addressing different aspects of the AI control problem as it relates to the development of safe and secure artificial intelligence. The book is the first edited volume dedicated to addressing challenges of constructing safe and secure advanced machine intelligence.”
The chapters vary in length and technical content from broad interest opinion essays to highly formalized algorithmic approaches to specific problems. All chapters are self-contained and could be read in any order or skipped without a loss of comprehension.
About the Author:Ā Roman V. Yampolskiy, Foresight Fellow in Artificial Intelligence Safety & Security
Ā
Roman is a Tenured Associate Professor in the department of Computer Engineering and Computer Science at the Speed School of Engineering, University of Louisville. He is the founding and current director of the Cyber Security Lab and an author of many books including Artificial Superintelligence: a Futuristic Approach. Roman holds a PhD degree from the Department of Computer Science and Engineering at the University at Buffalo. He was a recipient of a four year NSF (National Science Foundation) IGERT (Integrative Graduate Education and Research Traineeship) fellowship.
And we wanted to share with you Ryan Carey’s last research
Check out this paper from Ryan Carey, 2018 Fellow in Artificial Intelligence & Safe Machine Learning. Incorrigibility in the CIRL frameworkĀ discusses whether you can make redirectable agents via various mechanisms.
Abstract
A value learning system has incentives to follow shutdown instructions, assuming the shutdown instruction provides information (in the technical sense) about which actions lead to valuable outcomes. However, this assumption is not robust to model mis-specification (e.g., in the case of programmer errors). We demonstrate this by presenting some Supervised POMDP scenarios in which errors in the parameterized reward function remove the incentive to follow shutdown commands. These difficulties parallel those discussed by Soares et al. 2015 in their paper on corrigibility. We argue that it is important to consider systems that follow shutdown commands under some weaker set of assumptions (e.g., that one small verified module is correctly implemented; as opposed to an entire prior probability distribution and/or parameterized reward function). We discuss some difficulties with simple ways to attempt to attain these sorts of guarantees in a value learning framework.
Ryan Carey, Foresight Fellow in Artificial Intelligence & Safe Machine Learning
Ryan is a research contractor at Ought Inc. His current work focused on aggregating answers from safe question-answering systems. Previously, he has worked on the task ofĀ predicting slow human judgments. In his past work at the Machine Intelligence Research Institute, he touched on issues such as how systems ought to behave if they haveĀ bugs in their code, and how systems ought to learn and explore ifĀ they occasionally encounter catastrophes.
Donate today for a better tomorrow
Your support enables us to push for better futures on three frontiers:
1. Advocating for neglected risks arising from technologies
2. Selectively advancing beneficial technologies
3. Fostering ongoing debate on to decide which risks to advocate for and which beneficial technologies to advance
Because we value accountability, here you will find the list of our 2017 achievements at one glance ā and how we could continue each project in 2018 with your support.
If you would like toĀ keep up with Foresight Instituteās eventsĀ and publications,Ā follow us onĀ on Twitter, on LinkedIn, befriend us onĀ Facebook, or subscribe to our YouTube channel.
Thank you,
Foresight InstituteĀ
