Giving a coding model a separate controller
A separate controller changed how a local coding model worked, but did not increase the number of tasks it solved.
A coding agent can spend its steps reading the same file again and again without getting closer to a fix. We tested whether a separate controller could make better decisions about when to inspect, edit, test and finish.
Separate the decisions from the code generator
Both versions used the same local Gemma 4 E4B model to generate code and the same twelve-step budget. In one version, the usual control process directed the work. In the other, a separate controller called Jev chose the next kind of action.
The test covered eight coding tasks, with three runs of each task in each mode, for 48 runs in total. It ran locally on an RTX 2060 with 6 GB of video memory. Keeping the generator fixed helped distinguish the controller's choices from the model's ability to write a working patch.
A controller can decide that it is time to run a test, but it does not guarantee that the proposed edit is correct. Conversely, a capable generator may still be held back if the surrounding process gets stuck repeating an unhelpful action. The comparison was designed to make that distinction visible.
The work pattern changed
With the separate controller, average generator calls fell from 13.8 to 4.3 per run. Repeated actions fell from 4.58 to 0.79. The controller tended to make more deliberate choices about inspection and testing, although it sometimes kept reading after an edit instead of testing and finishing.
Both modes solved 2 of 24 runs, although they solved different cases. The controller made the process more orderly and reduced repeated actions. The code generator still had trouble producing working changes, so the number of completed tasks stayed the same.
| Measure | Ordinary control | Separate controller |
|---|---|---|
| Tasks solved | 2 / 24 | 2 / 24 |
| Generator calls | 13.8 | 4.3 |
| Repeated actions | 4.58 | 0.79 |
| Controller calls | 0 | 12 |
A separate controller has its own cost
Fewer calls to the code generator do not necessarily mean less total work. Jev made its own call at each step. A fuller cost comparison would also measure time, token use and controller overhead, alongside the number of tasks completed.
The experiment separates two problems that can look similar from the outside. An agent may waste steps because it chooses the wrong next action, or it may make sensible choices but fail to generate a valid fix. Improving one does not automatically solve the other.
Follow a useful edit through to completion
A next comparison could use more varied tasks and more generator configurations. It could also examine whether an agent tests a promising patch and stops when the task is complete.
In this experiment, the controller reduced repetition and changed the work pattern. The aggregate solve rate stayed the same.