Summary
We work on how nature collectively computes solutions to problems & how these computations are refined in evolutionary and learning time.
We explore these ideas at all levels of biological organization—from societies of cells to animal societies to markets to machine-human hybrid societies.
Presenters
David Krakauer, Santa Fe Institute
David’s research focuses on the evolutionary history of information processing mechanisms in biology and culture. This includes genetic, neural, linguistic and cultural mechanisms. The research spans multiple levels of organization, seeking analogous patterns and principles in genetics, cell biology, microbiology and in organismal behavior and society…
Presentation: Collective Computing: Learning from Nature
- There are two styles of reasoning. Although they are not orthogonal, they tend to reflect important differences embodied by human beings.
- Logical thinking: what we can understand.
- Human behavior, which we often can neither understand nor control.
- These are illustrated in observations made by Heroditus and Newton.
- Newton’s Principia was so difficult to understand that many people had to wait for an English translation of a French work by Voltaire and Emile du Chatelet that explains it.
The Enlightenment impulse was to apply Newtonian physics both to human beings and to society as a whole. This is illustrated by a work by Julien Offray de La Mettrie.
- Rather than worship a divinity, we began to worship a divinity that represented Reason. We sublimated our needs for Gods into a Demi-God of rationality.
- Saint Just, a friend of Robespierre, explained how the manifestation of the mechanical worldview manifests in society through law.
- In this period, there was this connected belief that through reason we could transcend superstition. This didn’t work out very well. Mechanical thinking led to the horrors of The Terror and the guillotine.
- Much of current thinking is a continuation of the Englightenment idea that human behavior is so complex, that the madness of the multitudes is so uncontrollable, that the best that we can do is to replace minds with guillotines, in this case, software.
- The idea that human cognition is fundamentally limited in ways that need not apply to machines suggests that if we just outsource decision-making, for example, legal decision-making, everything would be much better.
- But there is an alternative mode of thinking. It is illustrated by Maeterlink’s The life of the bees, which explores the power of collectives–as contrasted with what one gets with a mechanistic structure.
This sensibility is present in the modern world among those who are interested in how we manage semi-autonomous collectives that are noisy and unreliable but that in the aggregate produce useful functions.
- The remainder of the talk focuses on the work the C4 group has done in analyzing the history of successfully functioning evolved collectives.
- Let’s start with a hierarchy of collectives. We study the mappings between scales. In particular, we study the question of how elements at each level come together to create an ensemble at the next higher level
- How do cells make brains?
- How do brains make organisms?
- How do organisms make societies?
- A number of principles have been found that contribute to this process–as shown in blue on the slide. These principles often apply at multiple levels and need not be specific to any one particular upward mapping.
- Following is a very quick tour of the various levels.
It is possible to build transistor-like elements from chemical constituents.
If one adds feedback to these systems one can develop memories.
This allows us to understand how biological cells can do computing.
Out of that work, we discovered that noise is critical to the functioning of these systems. The following slide summarizes that finding.
- In collective digital stochastic dynamical systems (and everything in the natural world fits that description) it is essential to understand the role noise plays in such collective dynamics.
- We shouldn’t think of these systems in classical terms in which noise is to be minimized. The classical, i.e., noiseless, limit is the privilege of engineering, not the reality of the natural world.
- The next level up from molecules is that of cells and brains.
The great mysteries of the brain are the following.
- How noisy elements allow us to do things like group theory and category theory. These abstractions are so clean and pure, with no noise. Deduction seems so deterministic. Yet the elements that are doing those calculations are a mess. How does this all work?
- We live in a world of 8 billion people. There seems to be no way to coordinate humans at that scale. Yet brains have about an order of magnitude more neurons, which somehow collaborate and cooperate to produce thought. How does that work?
In an experiment, monkeys were given the task of deciding, on average, whether dots on a screen were moving right or left.
The following examples, animated in the video, the probabilities indicate the probability that any dot will move right.
- Top left: neither right nor left is favored.
- Top right: motion is more to the left.
- Bottom left: motion is even more to the left.
- Bottom right: motion is clearly to the right.
Brain probes help us understand what’s happening in the monkey’s brain.
The primary conclusion is that there are two distinct phases.
- The brain cells are independently sampling the world.
- The brain cell together produce a correlated decision.
These two phases correspond to a Sparse-independent phase and a Dense-Correlated phase. We call this the Dual-Coding Principle. It is found in quite a few animal species
This conflicts with the notion that agents are always autonomous. They are more autonomous during the first phase and less autonomous during the second.
Moving on to the mapping from individuals to societies.
- We studied two species of monkeys: spider monkeys and pig-tailed monkeys.
- These species diverged 35 million years ago. (For perspective, humans and chimps diverged only 6 million years ago.)
- Observed the fights that occurred.
We can record these fights and then extract an “enhanced” Bayes net–a Bayes net that includes logical elements.
From that, we can construct a strategic circuit of the entire society. Each vertex is an individual, and each arrow is an interaction between individuals.
We can use these circuits to predict future behavior. For example, we can predict the duration of fights, and when, if we remove someone will the fight stop.
We can do the same thing within a cooperative regime of spider monkeys and predict future behavior, e.g., how the groups behave if it’s raining or dry, who consorts with whom, etc.
- We can use these social circuits to predict how a society will evolve and what are the dominant causal vertices, or sets of causal vertices.
- We can actually think of a society as performing a stochastic computation to answer questions such as: When should I fight? Or With whom should I cooperate?
- At the human level, I’ve been studying a community of highly competitive Rubik’s Cube players.
- This slide shows how much better the community as a whole has become over time. The community solves Rubik’s cube both sighted (left) and blindfolded (right).
- In the following graph, each vertex is a cube configuration. (The number is the size of the cube in one dimension; ‘B’ means blindfolded. The cube images don’t reflect their linear sizes. All images are of 3x3x3 cubes.) An edge indicates that a competitor has broken a record in both cubes.
- For example, the three edges from vertex 6 (upper left) to vertex 4 (lower center) mean that three different people (or perhaps one person on three different occasions?) broke the record for both of these cubes — presumably at three different times as the records become increasingly challenging.
- The graphs on the right in the following slide show the learning rates for solving the cubes of various linear dimensions–top graph sighted; bottom graph blindfolded. In both cases, the learning rate decreases as the linear dimension of the cubes increases–except for 4x4x4 cubes in the blindfolded case. This apparent anomaly can be predicted by the structure of the competitive network.
- The community of solvers has discovered which competitions maximize learning.
- In particular, the social/information organizations in which we live determine, at least in part, individual learning rates. Schools have similar effects, which unfortunately, we do not exploit.
- The systems we have been examining have the following properties, which tend to be very different from engineered systems.
Although the following slide is an exaggeration, there is much to learn by studying what nature has created.
Q&A
Can we conclude that the brain is Bayesian?
- Bayesian statistics is a very useful tool, but the brain is much more complex and rich that a simple Bayesian perspective implies.
Does the brain operate on a winner-takes-all in which the model that is most consistent wins out?
- There is a divide in neuroscience between an ensemble approach and a grandmother-neuron approach. There is a bit of both. We go back-and-forth between them.
The use of noise sounds a lot like simulated annealing.
- That’s a useful metaphor, but natural systems are more complex. Natural systems oten they operate with critical points. Although this seems like bad design, it turns out to be important.
How do Rubik’s-cube solvers share their algorithms?
- Blindfolded solvers use ~5 algorithms. SIghted solvers use ~70 algorithms. As in blindfolded chess, blindfolded solvers tend to construct high-level representations rather than storing all the details of a cube’s configuration. As is well known, when a blindfolded expert is presented with a configuration that would not normally occur, they tend to be confused and don’t know how to proceed. In karate, this is known as a drunken-master strategy. The community is very generous in sharing their techniques, e.g., in blogs.
What about neuromorphic computing?
- Natural systems typically have components that compete with each other. This tends to produce robustness, if not speed.
Have systems been built that use these ideas?
- Genetic algorithms, convolutional neural nets, immune systems for cyber-security. We haven’t figured out how to build computing systems that use energy at the level that human beings use.
Any more to say about the immune system as a model for cyber-security systems.
- Sorry. Don’t know enough about those systems. WRT problem solving in general, humans tend to think in terms of gestalts and higher level representations.
Can you talk about Go as a way to build our understanding of evolution as a game.
- There is so much more to evolution than incremental fitness improvements, which produce more offspring. If that were true, there would be no complex life. There are too many valleys to cross using that strategy. Evolution is much more like Go than like Tic-tac-toe. The value of Go is that it makes the gestalt level important. That’s how you describe the game and play it.
What’s next in your work?
- Will continue to work on natural distributed intelligent systems. Over the past 6 months have been working on bringing out a (fat) book on what SFI can contribute to the response to COVID.
Summary by Russ Abbott.
