AI Safety: Neuro/Security/Cryptography/Multipolar Approaches

This grant program seeks to support projects working to make progress on three areas we consider underexplored when it comes to AI Safety.

1. Neurotechnology, Whole Brain Emulation and lo-fi Uploading for AI safety

Recently many people have updated their AGI timelines towards earlier development, raising safety concerns. That has led some people to consider whether neurotechnology, in particular WBE development (or lo-fi approaches to uploading which may be more cost-effective) could be significantly sped up, producing a differential technology development re-ordering that might lessen the risk of unaligned AGI by the presence of aligned software intelligence.

This includes exploring ideas such as:

WBE as a potential technology that may generate software intelligence which is human-aligned simply by being based directly on human brains
Lo-fi approaches to uploading (e.g. extensive lifetime video of an organism that could be used to train a model of that organism without referring to biological brain data)
Other neuroscience and neurotech approaches to AI safety (e.g. BCI development for AI safety)
Other concrete approaches in this area
General scoping/mapping opportunities in this area, especially from a differential technology development perspective, or exploring why this area may not be impactful

For further information, consider skimming this workshop on the same topic, or reading some of the following – Whole Brain Emulations, A Hybrid Approach to the human-AI safety problem, Digital People Could Make AI Safer, Brain-like AGI., and BCIs and AI safety.

2. Security, Cryptography, and Auxiliary Approaches for Infosec and AI Security

Explore the potential benefits of cryptography and security technologies in securing AI systems. This includes:

Computer security to help with AI infosecurity or approaches for scaling up security techniques to potentially apply to more advanced AI systems
Cryptographic and auxiliary techniques for building coordination/governance architectures across different AI(-building) entities
Privacy-preserving verification/evaluation techniques
Other concrete approaches in this area
General scoping/mapping opportunities in this area, especially from a differential technology development perspective, or exploring why this area may not be impactful

For further information, consider our workshop on the topic; or look in to some of the following, AI infosec: first strikes, zero-day markets, hardware supply chains, AI safety and the security mindset: user interface design, red-teams, formal verification, Boundaries-based security and AI safety approaches, Specific cryptographic and auxiliary approaches to consider for AI, Infosec Considerations for AI and the Long-term Future, Defend Against Cyber Threats, and Gaming the Future.

3. Safe and Beneficial Multipolar AI Scenarios

Explore the potential of safe multipolar AI scenarios, such as:

For further information, consider Encultured AI, Open Source Game Theory, Incomplete Contracts and AI Alignment, Paretropian Goal Alignment, and Comprehensive AI Services.

Why are you launching this program?

Given that short AGI timelines are a possibility, we received $1M funding per year for supporting underexplored approaches specific to this area.

Who can apply?

Funding may be distributed to fund independent researchers, regrant to existing organizations or organizations to be incubated, or to hire/contract in-house researchers through Foresight Institute (limited research H1b support available if relocation to US desired). In order to support underexplored work, we may give priority to non-US applications, however we also welcome US applications.

When can you apply?

This is a rolling application, and will be open for one year until the budget is depleted. We will aim to get back to you on the decision of your proposal within 8 weeks. If you require a faster turnaround, please let us know in the application form and we will aim to get back to you faster.

How do I apply?

Please fill in the form at the top of this page to apply for funding. If we decide to move forward with your application, we will invite you for a brief interview, in which you are also welcome to ask any questions you may have. Please don’t take longer than 3 hours to fill in this application.

How do you evaluate projects?

Projects will be evaluated by a mix of Foresight staff and external advisors. We aim to focus on projects that have a chance of being successful within short AI timelines. Rather than funding many projects with the potential of making a small difference in the long-run, we may be more inclined to fund projects that are high-risk high-reward, in the sense that they are more speculative but would make a big difference if successful. Generally, we are interested in proposals for scoping/mapping opportunities in this area, especially from a differential technology development perspective.

Are you a potential funder/advisor?

If you are a funder interested in supporting work in the areas above, or if you are someone who has a substantial track record in these areas and would like to be considered to be an advisor, please email [email protected].

Do you have any further questions or feedback to share?

Contact [email protected].

Tax Responsibility:

The tax implications of receiving funds from this grant may vary depending on your jurisdiction. We strongly recommend that you consult a tax professional to understand your specific obligations.

Gaming the Future: The Book!

AI Safety:

Neuro/Security/Cryptography/Multipolar Approaches

ADMINISTERED BY