AriakiIn early September, a fruit fly's brain playing Doom blew up on Twitter.
Then came Minecraft, Super Mario 64, racing games, The King of Fighters, and the Chrome dinosaur game. The official Google AI account joined in. All of them ran on the same dataset: MaleCNS v1.0. It contains the complete central nervous system of one male fruit fly: 166,700 neurons and 310 million synaptic contacts. The data came out in June. The paper appeared in Cell on September 3.
I went through the repository documentation for fourteen projects. Everyone was asking the same question: is wiring shaped by biological evolution better than random wiring?
Nobody had finished the control experiment.
The idea had certainly occurred to them. FlyDoom had already implemented degree-preserving rewiring, but had no completed results table. Flyhard listed it under “E08” in its future plans. Connectome Fighter included it in the experimental protocol. Its status page still said that an advantage from biological structure had yet to be demonstrated.
I ran it. Two games, ten paired comparisons, three layers of criteria. A tie.
For the numbers, skip to section 3. For the two times my own probes fooled me—and how absurd the second mistake was—go to section 5.
1. Fourteen projects and one shared gap
First, what are these projects actually doing? Their designs are strikingly similar. Take a connectome, the map of connections between neurons, and use its wiring as a fixed network. Put a hand-written sensory encoder in front to turn game observations into neural inputs. Put a hand-written action decoder behind it to turn neural activity into button presses. Leave the biological wiring in the middle unchanged.
Then ask whether that wiring is doing anything useful.
There is only one way to answer the question posed here: shuffle the wiring, hold everything else fixed, and run it again. But the shuffle has to preserve the right things. Each neuron must keep its in-degree and out-degree: the number of connections it receives and the number it sends. Only the destinations change. This is degree-preserving rewiring. Without that constraint, a win for the biological network remains ambiguous. Did its topology help, or did a few heavily connected neurons account for the result?
I read the README and methods documents in all fourteen repositories. Grouped by whether they had completed this control, they looked like this:
Code implemented, results absent: FlyDoom. The repository includes degree-preserving rewiring and a matched random-graph baseline. Its README says, “A null or negative result is a valid answer; this project does not assume that biological topology helps”. That is a research position. It is not a result already obtained.
Listed as future work: Flyhard. Its wording is explicit: “It does not establish that fly topology is better than a shuffled graph or an ordinary policy.”
Protocol present, scores absent: Connectome Fighter. Its status documentation acknowledges that neither a reproducible behavioral improvement from plasticity nor an advantage from biological structure has been demonstrated. Plasticity here means changing connection strengths as the system learns.
Explicitly declared untested: Swat. Its author wrote a sentence I appreciate: “No graph advantage over shuffled wiring has been claimed or tested.”
Actually tested, but only zero-shot: fly-craftax. This was the only project with actual numerical results for a wiring control:
Real graph survived 154.8 steps
Shuffled graph survived 164.9 steps
Disconnected survived 168.0 steps
The author's own verdict was no separation. All three were tied in that reading. The disconnected version even had the highest score. But this was zero-shot: run the system without training it first. The experiment did not train the real graph and the rewired graph separately with equal budgets and then compare their performance.
The other nine projects either compared plasticity on versus off, offered interactive demonstrations of a fixed network, or checked whether inputs causally affected outputs. For example, remove visual input and see whether the actions change. All of that is useful work. None of it answers the topology question.
One distinction nearly tripped me up and deserves its own paragraph: “Does this learning rule help?” and “Does this topology help?” are different questions.
FlyPong reported a negative result. Its dopamine-based plasticity experiment did not improve the paddle's play. Doomfly reported something stronger: learning was harmful. From one test starting point, both the learning branch and the timing-shuffled branch died at 3.657 seconds. The frozen-weight branch survived to the 8-second cap. Erasing memory restored performance to that of the frozen branch. The original wording was the observed difference was harmful.
Both experiments tested plasticity on versus off, rather than topology. They established that these local learning rules brought no benefit in those tests. They did not establish that the wiring brought no benefit. I conflated the two when I first read the project-list descriptions. Reading the repositories corrected that mistake.
2. The circuit I connected
I did not connect the whole brain. I used the mushroom body, the fruit fly's learning center. It is the circuit a fly relies on to remember that a particular smell is followed by an electric shock.
Extracted from MaleCNS, the circuit looked like this:
Projection neurons (ALPN) 686 ← sensory input enters here
Kenyon cells (KC) 4,064 ← sparse-coding middle layer
Mushroom body output neurons 97 ← decision outputs (MBONs)
Dopaminergic neurons (DANs) 340 ← reward and punishment signals
─────
5,187 nodes
The two connection sets in the middle came from the real data: 22,586 PN→KC edges and 61,210 KC→MBON edges.
The biological learning story is fairly direct. An odor activates a subset of Kenyon cells. Only about 200 are active at a time, so the representation is very sparse. If an electric shock arrives then, dopaminergic neurons weaken the connections from those recently active cells to approach-driving output neurons. When the same odor returns, the drive to approach is weaker. The fly avoids it.
The learning rule looks like this:
dw = -eta × KC_activity × (dopamine - baseline)
Three factors, multiplied together. Everything the update uses is local information: presynaptic activity, the postsynaptic channel, and a global dopamine signal. There is no backpropagation. That is the appeal of this family of projects. The learning follows a biological mechanism. The connectome is not merely an initialization matrix handed over to gradient descent for training.
I connected this circuit to CartPole, the classic pole-balancing task, and LunarLander, the lunar landing task.
One caveat belongs up front. It is the shared weak point of all these projects: I built both ends by hand. The conversion from game state to neural input is an engineering choice. So is the conversion from neural activity to a button press. Neither interface comes directly from the fly.
One of the racing projects makes the importance of this layer particularly visible. Its code contains a function named calibrate(target_action_overlap=0.5). In plain language, it adjusts the neural representations of different actions until their overlap reaches 50%. That is a hand-tuned target, not a biological measurement. The author's documentation says that tuning it took the completion rate from single digits to 100 laps per minute.
The interface alone can produce huge performance differences. Those differences have nothing to do with the connectome.
That led to the experimental constraint I took most seriously: the rewired group must use the interface parameters calculated from the real group. No independent recalibration. I enforced this in argument validation. If the script sees an alternative circuit without a frozen interface file, it refuses to run:
if args.circuit != DEFAULT and not args.interface:
p.error('alternative circuit requires --interface; no independent recalibration')
A comment reminds you of a rule. Validation makes you obey it.
3. Two games, ten paired comparisons, a tie
Before looking at game scores, check whether the rewiring actually changed the graph. A direct criterion is the fraction of original edges still present after rewiring:
PN→KC original edges retained 4.72%
KC→MBON original edges retained 25.64%
Every neuron's in-degree and out-degree remained exactly unchanged. So did the distribution of all connection weights. But the destinations were largely replaced. This was not the original graph under a different name.
For CartPole, I trained five initializations for 20,000 steps each. The score is the number of steps the pole stays upright. Higher is better.
Real wiring Random rewiring
seed 1014 51.2 37.2
seed 1015 31.1 32.7
seed 1016 21.6 34.4
seed 1017 11.2 20.0
seed 1018 18.1 17.3
─────────── ───────────────
Mean 26.64 28.32
Real wiring won 2/5 pairs. Its mean was slightly lower.
My first interpretation was that CartPole might be too simple. It has two actions and a four-dimensional input. Perhaps the task offered too little room to distinguish the two circuits. I moved to LunarLander: eight input dimensions, four actions, and dense rewards that can be positive or negative.
Real wiring Random rewiring
seed 1014 -434.7 → -359.6 -459.2 → -339.0
seed 1015 -375.9 → -265.0 -304.9 → -376.1
seed 1016 -450.1 → -263.6 -345.3 → -319.4
seed 1017 -792.4 → -368.9 -438.4 → -363.7
seed 1018 -786.1 → -344.1 -249.1 → -226.3
────────────── ──────────────
Mean -567.8 → -320.2 -359.4 → -324.9
Improvement +247.6 +34.5
There was a trap here, and I nearly stepped into it. The improvement was seven times larger for real wiring. It looked like an overwhelming win.
But the final scores were -320.2 versus -324.9. Almost identical. The real group gained 248 points after starting 208 points lower. That large gain mostly brought it back to a position the random group had occupied with almost no training.
Choose improvement as the metric and real wiring wins by a lot. Choose the final score and the same experiment is a tie. When the starting points differ, these two readings can point in opposite directions. Neither reading requires falsifying a number. The honest response is to report both and explain why they disagree.
An even more important detail: both groups were far worse than random button presses, which scored -201.1. In LunarLander, firing engines at random burns fuel and accumulates penalties. Staying near the ground and doing little can beat flying around badly. Both groups may simply have learned to move less. Neither learned to land. Comparing scores has limited value when neither contestant has solved the task.
4. A third criterion: encoding separability
One possible explanation for tied scores is that the learning rule is the bottleneck. The topology could still help, but the learning process might fail to use it. I therefore bypassed learning and tested the representations produced by the wiring itself.
The procedure was straightforward. Feed in a collection of game states and record their KC encodings. Then ask: can a classifier read the appropriate action out of this representation? Should the agent go left or right here? I trained a linear classifier with cross-validation, so accuracy was measured on held-out examples rather than the examples used to fit it. Higher accuracy means the encoding preserves more of the task information. This is what I mean by encoding separability.
The important constraint is that the labels must be independent of the model being tested. I used the heuristic controller supplied by the gymnasium library. That code belongs to the upstream library and has no connection to my circuit.
On CartPole:
Raw four-dimensional input 98.64%
Real-wiring KC encoding 98.87%
Random-rewiring KC encoding 98.86%
Pure random-projection control 93.59%
The difference between real and rewired was 0.01 percentage points. Notice, though, that the pure random projection lagged by 5 percentage points. This indicates that the sparse encoding itself was useful. The biological wiring provided no additional benefit over degree-preserving rewiring.
On LunarLander, the real group's result finished first. I wrote down an interpretation immediately: real-wiring KC encoding scored 82.98%, compared with 80.26% for the raw eight-dimensional input. A gain of 2.7 percentage points. Finally, I thought, a harder task had created room for the representation to distinguish itself.
Then the controls finished:
Majority-class baseline 33.32%
Raw eight-dimensional input 80.26%
Real-wiring KC encoding 82.98%
Random-rewiring KC encoding 82.96%
Pure random-projection control 82.94%
The majority-class baseline always predicts the most common label. It gives a reference for how much accuracy is available without reading the input at all.
All three projected representations tied. Even pure random projection caught up.
That 2.7-point gain was available from any of these high-dimensional sparse projections. It had nothing to do with the connectome or biological topology. I rushed to explain the treatment group's result before the controls arrived and got slapped by my own data.
Until the control finishes, a difference may be a property of the method itself.
The only stable difference was in encoding sparsity. This measure requires no fitted classifier. It is a direct statistic of the activity:
Real wiring Random rewiring
KCs that never activated 1,601 2,778
Encoding overlap between states 0.175 0.365
Real wiring used more of the available encoding space. Different states produced representations that overlapped less, making those states less easy to confuse at this structural level. That was a real structural advantage. It did not translate into an advantage on any functional metric I tested.
5. My probes fooled me twice
This part is more useful than the headline result. The mistakes transfer to other experiments.
First: the field called “consensus.”
The MaleCNS neurotransmitter table contains several columns: predicted_nt, the prediction for an individual neuron; ground_truth, the experimentally established value; and celltype_predicted_nt, the prediction at the cell-type level. There is also consensus_nt.
I wanted to check how accurate the neurotransmitter predictions were. The neurotransmitter determines whether a neuron is modeled as excitatory or inhibitory. In other words, it determines the sign of its connections. Get the signs wrong and the behavior of the whole network can change.
I selected consensus_nt and checked it against ground_truth. Across 85,000 samples, the accuracy was 1.0000. The confusion matrix was a perfect diagonal. No errors at all.
That was an alarm, not good news. On the same data, predicted_nt achieved only 88.61%. How could the “consensus” column possibly earn a perfect score?
Here is what the audit found: when ground truth was available, consensus_nt matched it 100% of the time. When ground truth was absent, it matched the cell-type prediction 99.6% of the time. This was a composite field. It copied the answer when an answer existed and used a prediction otherwise. Validating it against ground truth meant validating the answer against itself. That is label leakage: the reference answer has already entered the thing being evaluated.
The bait was sweet because the field was called “consensus.” It sounded like exactly the column I should trust most.
After switching to a clean prediction column, I measured a sign-flip rate of 0.1077% at cell-type granularity. That number also forced me to soften an earlier claim. In a perturbation experiment, I had found that artificially flipping 20% of connection signs silenced the network. The cliff began at 10%. I had written that signs were precisely the part of the connectome that had not been measured, so the uncertainty landed on its most vulnerable dimension.
The vulnerability was real. But the actual error was two orders of magnitude below the fatal threshold. My earlier wording had been too strong.
Second: I measured the criterion at the wrong level.
In a separate whole-brain simulation, I added a sanity assertion. If I increased the input rate, the total spike count should rise substantially. Here is what came back:
50Hz input → whole-brain total spikes 447,012
300Hz input → whole-brain total spikes 469,164 up only 5%
The assertion fired: the rate parameter had failed to take effect.
I nearly started fixing the code that passed the parameter through the system. Then I split the measurement by population:
The 60 stimulated neurons: 50.0 Hz → 291.7 Hz parameter took effect
The other 8,386 neurons: 52.9 Hz → 53.9 Hz barely moved
The parameter was working. Recurrent self-excitation accounted for 99.3% of whole-brain activity, independently of the external input. The contribution of those 60 neurons was only 0.6% of the total spike count. It disappeared into the aggregate.
Using a whole-brain total as the criterion let 99.3% of unrelated activity drown out 0.6% of the signal I wanted to measure.
Measure at the level where the tested variable acts directly. Do not default to the largest container that happens to include it. To test the drive applied to a group of neurons, measure that group's firing rate. The whole-network total answers a different question. To test a change to an interface, measure requests passing through that interface. Site-wide queries per second will bury the effect if most requests take other paths.
The false alarm itself turned out to be more valuable than the original check. A 6-fold increase in drive produced only a 1.9% increase in the network-wide response. This was a third independent line of evidence that the model had no operating range, only a self-excited state. Before repairing a probe that raised a false alarm, ask why the alarm was false.
6. How I read these demos
The quality of the projects in this wave varies widely. Several authors, however, describe their own evidence with unusual honesty. Their wording is worth learning from.
The author of the Chrome dinosaur project, Fly Dino, is particularly clear. The design uses a fixed circuit of 80 neurons. Outside it sits a small readout network with 243 parameters, trained using the cross-entropy method. After training, it completed 99/100 runs. Silencing the circuit reduced that to 0/100. This looks like a clean positive result. Yet the documentation explicitly says not superiority of biological topology.
The experiment demonstrates that the external readout can learn and that neural activity is genuinely used. Biological topology being better remains a separate claim. To bridge that gap requires a control the author did not run: replace the 80-neuron circuit with 80 randomly connected neurons, then give that system the same training.
When looking at these demos, ask four questions in order.
First, do the neurons actually participate in the decision? Disable them and see whether behavior changes. Most projects pass this gate. It is also the easiest gate to pass.
Second, where is the learned information stored? If it lives in an external readout—in parameters trained by PPO, CEM, or a bandit method—then the learning happens in an artificial component. PPO is a policy-training method, CEM is the cross-entropy method, and bandit methods learn which choices pay off. The connectome remains a fixed feature extractor.
Third, is there a random-rewiring control? Without it, the evidence reaches “this system can work.” It does not reach “the biological wiring contributes an advantage.”
Fourth, were the control's parameters tuned again? If each group gets its own tuning, the comparison also measures the tuning process. It no longer isolates the wiring. This is easy to overlook because the offending step often hides in a function called calibrate. The name makes it look harmless. Check what it actually changes before treating two runs as a topology-only comparison.
7. Where this experiment is weak
The training budget is severely inadequate. For LunarLander, 20,000 steps is almost nothing. Both groups remained around -320, while random button presses scored -201. Neither learned to land. Any ranking between two systems that both failed to solve the task supports only a limited conclusion.
There are only five seeds. The mean paired difference was +4.7. Variation within the groups was much larger than that. This sample size supports “I did not see a difference.” It cannot support “I proved there is no difference.”
Encoding separability was tested only on CartPole and LunarLander. Both are low-dimensional control tasks. The mushroom body's native job is olfaction: high-dimensional, sparse, categorical input. That structure differs completely from the inputs in these two tasks. I may be testing it with the wrong problems.
The LunarLander probe failed its own acceptance gate. I set three gates: a known-separable control must score highly, shuffled labels must reduce accuracy to guessing, and every fit must converge. All the main four-class criteria passed. Accuracy was 47 percentage points above baseline and dropped to 26-33% after shuffling labels. But an auxiliary binary label I had added failed. It beat the majority-class baseline by only 2.6 percentage points, and the lower confidence bound was negative.
Because of that failure, the probe marked the entire result conclusion_permitted: false. By its own standard, the numbers reported above are observations. They are not approved conclusions.
Every interface parameter came from the real group. This isolates topology as the variable being changed and keeps the comparison controlled. It also narrows the question I actually tested: what happens when random wiring is substituted under an interface calibrated for real wiring? That choice may have favored the real circuit. It may also have hurt it. Both directions are possible, and I did not test which applies.
Mapping neurotransmitters to excitation or inhibition is a modeling assumption. Those signs were not measured separately at every synapse. The sign structure of the entire network rests on that assumption.
8. Next time, run the control in this order
-
Confirm that the variable being tested is a real decision point. Ask whether the system would still produce the right behavior if that step were removed entirely. If it would, testing the step amounts to awarding yourself a prize for something the system does not need.
-
Freeze the control group's parameters. Put the requirement in argument validation, where it can stop an invalid run. A comment is not an enforcement mechanism.
-
Place the criterion where the tested variable acts directly. Before measuring, ask how much of the denominator has nothing to do with the effect you want to detect. Use a measurement that can actually see that effect.
-
Get labels or reference answers from somewhere independent of the tested system. The column with the most authoritative-sounding name may be a composite field that already contains the answers. Check how it was constructed before scoring against it.
-
Give the probe a self-check in both directions. A control known to be separable should score close to perfectly. Shuffled labels should push it back to guessing. Checking only one side leaves the probe's ability to distinguish success from failure untested.
-
Encode permission to conclude as a Boolean field. If a gate fails, report only the observed numbers and explicitly withhold the conclusion. You can change the budget. You cannot loosen the criterion just to make the result pass.
-
Before launching any batch expected to take more than ten minutes, actually run one unit. Measure its duration, multiply by the total number of units, and compare that estimate with your budget. I once skipped this step and chose a stimulated population 130 times larger than the baseline. The job ran for 2 hours and 51 minutes. Completed combinations: 0.
-
Do not explain the treatment group's attractive number before the control finishes. Until that result arrives, the difference may be a property of the method itself.