Putting judgment where the judgment is

In the first experiment, a model called Jev answered a small set of questions about the situation. It estimated things such as the value of food, the threat from a rival and whether the fly should persist or explore.

A separate set of rules turned those estimates into a goal. The existing movement controller then carried out the goal in the simulated body. This let us change the judgment while keeping the movement system in place.

Two simulated flies shared a food patch. One was given a history of hunger, being displaced and failed feeding attempts. The other had recently eaten. We then swapped those histories between the bodies to see whether the assessments followed the past or the body.

In 12 of 14 informative measures, the result followed the history. The direction was revealing: Jev judged the hungry, repeatedly displaced fly to be facing a threatening rival, and that fly eventually retreated. An earlier mock model had instead made the hungry fly persist and win.

A further set of 16 model calls showed why the difference mattered. The mock could return a persistence score above 0.60, but Jev did not exceed 0.45 in that set. Under the original decision thresholds, two goals could not be reached. The thresholds had to fit the range of judgments the model actually produced.

The second experiment used a different simulated fly. Its neural network was derived from a map of fly brain connections and drove a combat simulation. Jev could provide a reward signal, a punishment signal or no signal. It could not directly turn the fly, select an attack or change the escape reflex.

Learning occurred at synapses, the connections through which neurons influence one another. We compared an unchanged network, a network with learning active but no teacher, a hand-written teacher and the Jev teacher. The untaught condition helped distinguish the teacher's effect from changes caused by the learning rule itself.

Jev made 254 teaching calls, and 2,127 synapses changed. We then disconnected the teacher and stopped further learning before testing. The attack-related neural output, a numerical signal produced by the network, rose from about 2.99 in the unchanged network to 5.97 in the Jev-taught network. Yet attack attempts were identical for each paired test seed.

The combat system made its decision by comparing a neural signal with a threshold. In this setup, the resting network was silent, so the threshold calculated from its resting noise was zero. Both the original and taught signals already crossed that threshold. Making the signal larger therefore did not change the decision.

A follow-up searched for a setting where the difference might change engagement. Across 120 paired comparisons, engagement timing still stayed the same. The larger teacher comparison was not run because this necessary behavioral difference did not appear.

The reason lay earlier in the network. Once the relevant output switched on, it stayed on like a latch. That switch happened upstream of the synapses that teaching could change. Teaching altered the signal after it switched on, but not whether or when it did so.

The hand-written teacher and Jev agreed on only 29% of their recorded teaching decisions, yet they changed nearly the same set of connections. Their changed sets had a Jaccard similarity of 0.952, where 1 would mean the sets were identical.

Many connections had reached the lowest strength allowed by the learning rule. Once a connection is at that limit, different teaching histories can leave the same result. The teachers also delivered different total amounts of the reward-related signal in the subset examined, so this comparison could not isolate the quality of their timing.

These experiments found effects at several different points. A fly's history changed its appraisal. Teaching changed synapses and a neural output. But the mapping from that output to action did not preserve a behavioral difference.

The lesson for building such systems is to test the full path from a proposed change to the action it is meant to influence. These are simulations, so their results describe the systems we built rather than learning in living flies.