Two years ago we commented on the success of “citizen scientists” playing an online game in outperforming the best available computerized design algorithms in designing RNA molecules to fold into predetermined structures. A news article appearing last month in Science, written by John Bohannon and discussing an Open Access paper (“Principles for Predicting RNA Secondary Structure Design Difficulty“) just published in the Journal of Molecular Biology, makes clear that the progress has continued “For RNA paper based on a computer game, authorship creates an identity crisis“:
… Today’s paper shows how far the effort has come. Among the game’s thousands of RNA design “puzzles,” there seem to be a small set that are particularly difficult. Among the most challenging structural features to figure out is symmetry, where an RNA strand folds into two or more identically shaped loops. The Eterna game includes an interface for players to propose hypotheses about how particular RNA structures will or will not fold into particular shapes. Those were distilled into a set of “designability” rules. The question was: Do only human designers struggle with thorny design problems, or do computer simulations tussle too?
The answer is that the computers struggled just as much as the people. Researchers report that three of the best existing computer algorithms, running on a supercomputer at Stanford, struggled to solve the very same RNA design problems as the humans. The result shows that the human “designability” rules do indeed correspond to problems that are hard not just for human brains but also for computers, the team reports today in the Journal of Molecular Biology. In fact, the hardest puzzles that could be solved by experienced Eterna players were unsolvable by the computer even after days of crunching. And to help improve the algorithms, computer scientists now have a set of benchmarks—the Eterna100—to gauge the design difficulty of RNA structures.
“This paper is a significant contribution,” says Jane Richardson, a biophysicist at Duke University in Durham, North Carolina. “The new design principles seem very understandable now that they’ve been described, and attention to them should indeed help automated design.” She notes that the game itself has become a useful tool for scientists. “One of our lab’s former students is an avid Eterna player who has also gotten insights from it for his own current research projects.”…
It is an interesting observation in the developing story of human vs. machine that despite computer dominance in games like Chess, and more recently Go, that in a domain in which the rules are still not all known, large groups of amateurs playing a game can outperform computers and teach new design rules to professional scientists. No doubt the computers will eventually triumph, but for now a long road with many opportunities for collaboration seems to lie ahead.
—James Lewis, PhD
