Emergent Behavior and Machines We Don't Understand.
Ever thought about being an air traffic controller?
For birds.
You'd want it to be birds. If two planes bump into each other, that is a catastrophe and an NTSB hearing. If two birds bump into each other, they squawk about it and then move on with their lives.
Here's the job. You're in a bird traffic control tower, and you want a flock of birds to fly tight and aligned, because that kind of formation is more efficient. The birds can draft off each other and that saves energy. So you'd like to put each bird on a good heading.
First, you must be able to describe a bird with numbers. In our world you can do that with one short list, called a vector. Each bird's vector will have four numbers. Two tell us its position, and the other two tell us its speed and direction (velocity). Here's what one might look like:
Position says where the bird is, (2, 7). Velocity says which way and how fast, (3, 4). Add them and you get where it is headed next, (5, 11).
Those four numbers are the whole bird, as far as the math is concerned.
And with two birds, giving one of them a smart new heading is genuinely easy. Take bird A. Look at bird B: where is B compared to A, which is just one position subtracted from the other, and which way is B heading? You'll need to do a little math (four steps, a bit of algebra), but you can blend B's heading with A's heading and come up with a new heading for A, one that nudges A a little closer and a little more aligned with B. That's the whole operation. Arrow arithmetic. Honestly, you could do it on a napkin.
Well, you could do two birds on a napkin, once. There are two problems. One, we want to keep adjusting the vectors, continuously. Two, we don't want to do this for just two birds.
We want a lot of birds to flock together. To do that, you have to weigh every bird against every other bird, A to B, A to C, B to C, and on and on, merging all of it into one good answer for each bird. The number of pairings is n times n minus one: two birds make two, ten birds make ninety, a thousand birds make almost a million. And again, you don't get to do it once. You redo the whole thing every time they move, which is always. Computer scientists call that n-squared complexity, the kind of cost that strangles. You wouldn't crack by lunch because the job is stressful. You'd crack because the math does.
Left: each bird is two arrows, where it is and where it's headed. Combine them and you get a new heading. Simple, for two. Right: to do it from a tower you must merge every bird with every other, n(n−1), and again every tick. That's the cost no one can pay, and the birds never try.
So here's the interesting tidbit at the center of this whole page. We don't need to do this. Birds never do any of that.
No bird is holding the flock in its head. No bird is merging a million arrows into a master heading. Each bird does a tiny scrap of the arithmetic, and only for the two or three birds nearest it, never for the whole sky. We can simulate what they do with three little rules:
That's the whole recipe, and look what it skips: the expensive part. Nobody computes the master heading. Nobody merges the million arrows. Each bird makes a few cheap nudges off its nearest neighbors, the same few whether the flock is ten birds or ten thousand, because a bird a quarter mile away may as well not exist. And out of all those little local nudges, the thing the tower could never compute simply appears. The flock turns and folds and pours across the sky like one animal, and nobody solved it. A computer scientist named Craig Reynolds worked this out in 1986 and called his little simulated birds "boids."
Watch it happen. The triangles below start scattered and leaderless. Give them a second.
You didn't arrange that. Nothing did. There's no master triangle, no line of code that says "form a flock." There's only each triangle, checking its neighbors, following its three rules. The flock isn't something anyone built. It's something that happened.
That has a name. Emergence: simple parts, following simple local rules, producing behavior that none of the parts contain. And that impossible little problem you just watched solve itself is hiding inside some very large machines. Hold that thought.
Each triangle, by the way, is about the simplest thing computer science is willing to call an agent: something that senses a little of its world and acts on it. Eyes in, steering out, one small rule in between. A flock is just a lot of agents sharing a box. Remember that word, agent. It comes back at the end wearing a much bigger coat.
Emergence is not a party trick that just works with birds. Once you know what it is, and how to look for it, emergence turns up everywhere simple rules meet each other at scale.
How deep does it go?
| System | The rule | What crawls out |
|---|---|---|
| Toys Fun to look at. Accomplish very little. But also, particularly with the Ants, a real mystery. How does it work? We don't know. | ||
| Game of Life | A cell lives or dies by counting its neighbors. Two rules. | Gliders, guns, self-replicators, patterns that run as a working computer. |
| Rule 110 | One rule applied to a row of cells. One. | Also Turing-complete (capable of acting as a computer). Nothing in the rule predicted this. |
| Langton's ant | Two rules, for an ant on a grid: turn left on white, turn right on black. | Chaos for ~10,000 steps, then unprompted, builds a tidy repeating highway forever. |
| Real work Directly applicable to real-world scenarios and problems in mathematics, computer science and other sciences. But why do they work? We don't know. | ||
| Ant colony | Simulated ants drop and follow digital scent. No ant holds a map. | Finds shortest paths through networks. Used in real routing systems. |
| Traffic waves | Each driver follows one rule: don't hit the car ahead. | Phantom jams, stop-and-go waves with no crash and no cause. |
| Schelling model | Households move if too few neighbors are like them. A very mild preference. | Sharply segregated cities that nobody chose and everybody made. |
| Inexplicable Unusual emergent developments. Really useful, and honestly, a little strange. | ||
| Thompson's FPGA | Evolution designed a circuit on a real chip to tell two tones apart. | Used 37 of 100 cells. Five were unwired but necessary. Exploited the chip's own stray magnetic flux. Worked only on that one exact chip. Full story. |
| NASA antenna | Evolved for radio frequency coverage, no engineer's hand. | A bent paperclip shape nobody would have drawn. Beat the hand-designed one. Flew. |
| Now WTF? No. Seriously, what? | ||
| CNN features | Backpropagation on labeled images. No labels for features. | Edge detectors, texture detectors, face detectors unintentionally grown in hidden layers. Nobody named them. |
| LLM abilities | Scale up the model. No new instructions. | In-context learning, chain-of-thought reasoning, translation. Nobody programmed them. (One footnote: some argue the jump looks sharper than it is due to metric choice. The argument is healthy.) |
| Anthropic features | Researchers looked inside Claude 3 Sonnet after training. Not before. | Millions of distinct internal concepts found after the fact. A feature for the Golden Gate Bridge. Features for flattery, for inner conflict, for code with a bug in it. Nobody put those there. |
That last row is where it gets downright strange. Not "look what the simple rule made" strange. But "We built this and we're only now learning what it can do" strange.
And part of what makes it strange is it's the agent, the boid you played with, just back in a much bigger coat.
The thing you're reading this next to is an agent too. It senses its world (the words you give it), it acts (the words it gives back), and in between, where the three simple rules used to sit, there's a tangle no person wrote by hand. We didn't program it with concepts. We fed it almost everything and let the structure emerge, the way the flock emerges, the way the highway emerges, only unthinkably larger.
Which brings us back to the birds.
You ascended the tower. You were willing to try and control the flock. But the whole time there was nothing to control, only a few thousand small things each minding its neighbors, adding up to something that looks, from a distance, like it was planned.
A murmuration of starlings is a wonderful place to meet this idea. Emergence is a stranger thing to sit with when the swarm isn't made of birds.
More and more of what shapes our daily lives runs on the same trick: what you're shown, what you're offered, who gets the loan, and what the machines say next. All of this is the result of millions of small parts coming together, each following their own local rules, none of them in charge, none of them holding the whole picture. The behavior that comes out of these systems is not something a person can point to and review. It grew. We are only starting to learn how to read it. Remember the chip that worked for reasons nobody could explain, on cells that weren't even wired in. We are handing more and more of our daily lives to systems built like that, and trusting, almost entirely on faith, that whatever emerges from all those small parts is somehow aligned with where we want to go.
Nobody's flying this. That was always true of the birds, and with birds it's just beautiful. With the other flocks, the ones we've built out of silicon and code, they are beautiful and a little terrifying. They work. And hopefully they keep working the way we want them to. But the truth is, we don't know what will emerge.