Independent judgment
The same agent that designs a change, writes it, and repairs it is not a neutral witness to whether it worked. Its report that the work is finished is a claim, produced by the party with the strongest reason to believe it. Oversight starts by refusing to treat that claim as proof.
Independence does not mean distrusting agents. It means the standard for done is set outside the agent and checked against the running system, so that trust is the result of the check rather than a precondition for it.
Current methodology
- A requirement is stated and held outside the code the agent controls.
- The running software is exercised and judged against that requirement.
- The decision is computed from evidence, not accepted on assertion.
Open question
- How much of a system can be judged this way as responsibilities get broader and fuzzier.
- How to keep the standard independent when agents help write the standard itself.