Q: One big complaint about backtests is that they don’t take into account the market impact of trades they’re simulating, transacton fees and other costs. Is that an issue for NumerAI?
- A: Backtests are always simulations, and many don’t work well on live data because they’ve simulated something poorly. We mitigate that by only trading the most liquid stocks in the world so large orders have limited impact.
Q: How do you decide how much to slash bad models?
- A: There is correlation with the targets of the model: if your model is 1% correlated with the target, you stake will increase the same. If you’re -1% correlated, you lose that much.
Q: How does the NumerAI modle compare to general prediction markets with no AI in the mix?
- A: Any market is a kind of money weighted market of what the thing should be worth. That’s what we’re trying to do with the staking: if there’s a lot staked on a prediction, we want to believe them because they have a lot to lose.
- But the form of NumerAI is all quantitative. There aren’t sentimental or behavioral anylses about companies, all strictly crunching numbers.
Q: Do you think there are any other data realms for this type of system besides financial?
- A: I don’t actually. Finance lends itself to this kind of model because the margins can be extremely tight: if you can get a model from 51% to 52%, that’s a huge difference. This works less well for things like detecting cancer from imaging.
- Crowdsourced models aren’t useful for every realm of machine learning, but staking is absolutely a huge deal for almost any company. The ability to create a negative incentive online is a big deal as a way to combat bad actors.
Q: We’ve taken a look at ErasureBay and AMIX, information exchanges that incorporate staking bounties to reward information.
Q: Can I buy a stake and benefit from others models?
- A: No, you are always staking your prediction, so putting money down on your own predictions.
Q: NumerAI seem like this interesting hybrid of a centralized and decentralized system. Yiu have a fleet of data scientists competing on models with your AI metamodels processing the result of the competition. How would you describe NumerAI’s makeup?
- A: We’re distributed for sure. And the NMR tokens and the staking are running on a blockchain that is outside of our control, so that’s decentralized.
- Stocks aren’t traded on the blockchain yet, so the company NumerAI is the entity with financial access to trading. Once stocks can be traded on the blockchain, NumerAI could become a totally decentralized DAO that’s running on chain with no employees.
Q: How does your metamodel work?
- A: There are very few things in finance that are “laws”, but one thing is that if there are two uncorrelated modles, you want to trade both to lower your volatility and icrease your returns.
- Towards this, NumerAI is shooting to be the fund with the most models, and the metamodel is the stake weighted average supermodel of all the models that are working on the NumerAI platform.
Q: Do you have any thoughts on how to better fund longevity research?
- A: It seems like anyone doing longevity research faces the same problem: regulatory environments.
- There is a lot of overlap between people who made money on cryptocurrency and people interested in longevity. So now that countries have begun to court the crypto-rich and change laws to be more attractive to them, there may be a chance.
Q: Can you make any medium or longer term predictions (maybe 5-7 years) about qualitiatively different insights to arise from the NumerAI model?
- A: I think that’s possible. In the quant finance world, a lot of it is very low-tech. Simple arbitrages and such, not highly intelligent. Now with machine learning models in finance, it’s getting weird. Models are making money that can’t have their risk explained.
- A: In regard to AGI, when the AGI folks are pressed they will eventually say “it doesn’t even need to be general, it could be a narrow AI that trades the stock market”. So I built a platform for whoever finds that narrow AI to trade on :)
- Traditional quant funds all track each other: new machine learning funds are decoupling in interesting ways. In 3 years NumerAI could have billions in the fund and a much different impact on the market.
Q: Could the biggest earners on NumerAI become non-human model makers?
- A: Yeah, we’re already seeing automated model making and relearning, restaking, so that’s a short step to totally automated AI traders.
Q: You use “intelligence” in the way a military might, as in gathering bits of intelligence for strategizing, and in the sense that this group focuses on.
- A: Each model has a “metamodel contribution” that’s calculated by whether the model actually contributes to the metamodel or is a subset of it.
- We are always looking for the hole in our knowledge, that’s always the most important thing to learn, and we pay based on this to encourage local knowledge being discovered.
Q: This reminds me of Decentralized Autonomous Hiveminds: we can maybe tackle the problem of AGI being too centralized by incentiviizing a group of DAOs to bring together local human knowledge to out-compete them.
Q: If you’re obfuscating the data, does that prevent users from doing qualitative stuff like natural language processing on news articles and such?
- A: The data we give out, there are many fundamental variables in there but you can’t do qualitative analysis because of the obfuscation. Users with one or two market insights aren’t really useful to the kind of quant strategy that the fund is taking.
Q: If you could do an attention based on model instead of just a simple stake-weighted mode, you could potentially make a system for finding these predictions.
- A: We have a new thing called NumerAI Signals that is kind of towards what you’re describing, and it brings in new data in new ways because the normal platform is constrained by the data that NuerAI is obfuscating and sharing.
- We’ve tried to beat the simple stake-weighted model… and it’s always a little less robust and a little worse.