Skip to content
Vraelis
Sign inOpen Vraelis

Research

The methods behind trusting agent work.

Vraelis is building the practice of independent oversight for software agents. This page describes the methodology we use today and the questions we are still working through. It does not claim results we have not earned.

Open Vraelis→Read the Method→

Our stance

A builder cannot remain the only judge of its own work.

An agent that plans, writes, and repairs a system will also report that it is finished. Someone independent has to decide whether that claim holds. The directions below are how we approach that decision, and where our thinking is still open.

01

Independent judgment

Why the party doing the work cannot certify it.

↓
02

Responsibility vs implementation

A task and a promise are different objects.

↓
03

Evidence vs confidence

The unit we trust is the observation.

↓
04

Judgment at the boundaries

The small set of moments that need a person.

↓
05

Repair, then verify

Fixed has to mean proven again.

↓
06

Preserved failure history

The failures are the most useful record.

↓
07

Judging the judge

Evaluator failure is not application failure.

↓
08

Autonomy is earned

Independence should be a conclusion, not a setting.

↓

How we read a claim

A completion claim is not evidence.

Before Vraelis accepts that work is done, it separates what an agent asserts from what the running software actually shows.

Expected vs observed

ExpectedAccess remains after the customer signs back in.
ObservedAccess is lost after the customer signs back in.

Illustration. A claim of done that the running software contradicts.

Coverage of a completion claim

Proven by evidenceAsserted, not proven

Illustration. Oversight measures how much of a claim is backed, and treats the rest as unproven rather than safe.

Research directions

Current methodology, and the open questions.

Each direction is stated as what we do today and what we have not solved. The open questions are real. We would rather show them than pretend they are closed.

Direction 01

Independent judgment

The same agent that designs a change, writes it, and repairs it is not a neutral witness to whether it worked. Its report that the work is finished is a claim, produced by the party with the strongest reason to believe it. Oversight starts by refusing to treat that claim as proof.

Independence does not mean distrusting agents. It means the standard for done is set outside the agent and checked against the running system, so that trust is the result of the check rather than a precondition for it.

Current methodology

  • A requirement is stated and held outside the code the agent controls.
  • The running software is exercised and judged against that requirement.
  • The decision is computed from evidence, not accepted on assertion.

Open question

  • How much of a system can be judged this way as responsibilities get broader and fuzzier.
  • How to keep the standard independent when agents help write the standard itself.
Direction 02

Responsibility vs implementation

A task and a responsibility are not the same thing. Add usage based billing is an implementation. Existing customers are never overcharged is a responsibility. Agents are handed implementations and graded on implementations, which is why passing tests can sit right next to a broken promise.

Vraelis tries to keep the responsibility primary and durable, held apart from whatever code happens to satisfy it at a given moment. The implementation is allowed to change. What must remain true should not quietly change with it.

Current methodology

  • A responsibility is recorded as a durable object, distinct from any one change.
  • Evidence is gathered against the responsibility, not only against the diff.

Open question

  • How to elicit the real responsibility when a person states only a task.
  • How responsibilities compose, and conflict, across a large system.
Direction 03

Evidence vs confidence

Confidence is cheap. A model can report high certainty about an outcome it never observed. Evidence is a record of the software actually doing the thing, captured in a way a person can inspect later.

The unit Vraelis trusts is the observation, not the assurance. A green result should reduce to what was exercised, what was seen, and where the boundary of the check sat. The exhibit above is that reduction in miniature: what a claim asserted, set beside what the running software showed.

Current methodology

  • Decisions are backed by execution against the live software, captured as inspectable evidence.
  • The engine runs pinned, on a fixed model and fixed settings, so a result can be re-run under the same conditions.

Open question

  • How to represent the edge of a check, so absence of evidence is never read as evidence of safety.
Direction 04

Judgment at the boundaries

Most of a task can be checked mechanically. A few points cannot, and they tend to be the ones that matter: an irreversible action, a pricing change, a decision that trades one risk for another. These are boundaries where a person should decide, not because the machine failed, but because the choice was never the machine's to make.

The design problem is not raising everything to a person, which no one can sustain, nor raising nothing, which is how bad decisions ship quietly. It is finding the small set of moments that genuinely need judgment and presenting each with enough context to decide well.

Current methodology

  • A plan is approved or refused as a whole, by a person, before a run can start.

Open question

  • How to identify the moments that truly need a person without flooding the queue.
  • How review load should scale as one person oversees many agents.
  • How to raise exactly the sensitive or irreversible moments inside an approved plan, rather than holding the whole plan as one unit.
Direction 05

Repair, then verify

A failure is not the end of a task; it is the middle. The useful question is whether the fix actually holds, and a fix proposed by the same agent that failed is not self certifying. A repair has to be rechecked independently, against the original responsibility, before it counts.

Vraelis treats repair as a structured handoff back to the agent followed by a fresh, independent recheck, so that fixed means proven again rather than asserted again.

Current methodology

  • A failure becomes a structured handoff, not a bug to triage by hand.
  • The repair is re-verified independently against the original requirement.

Open question

  • How many repair cycles are healthy before a responsibility should be escalated instead of retried.
Direction 06

Preserved failure history

When a fix lands, the temptation is to erase what came before. But the earlier failures are the most informative record a system has: they show how this software breaks, which repairs did not survive, and what an agent tends to get wrong.

Vraelis preserves that history rather than overwriting it. A trusted completion is more credible when you can see the failed attempts it replaced, and the record compounds into knowledge about both the system and the agents working on it.

Current methodology

  • Requirements, failures, repairs, and decisions are retained, not overwritten by the latest pass.
  • A completion keeps the trail of what it replaced.

Open question

  • How to turn preserved history into prediction of where the next failure will land.
Direction 07

Judging the judge

An oversight system can fail in two very different ways. The application under test can be broken, or the evaluator itself can be wrong: a check that passes something it should have caught, or flags something correct. Treating those as the same failure is how an overseer quietly loses its authority.

The evaluator has to be held to a higher standard than the software it judges, and its mistakes have to be findable and separable from the application's. An overseer that cannot be audited is just another opinion.

Current methodology

  • The decision is computed and fail-closed, so an uncertain check does not pass by default.
  • Evaluator behavior is inspectable, kept separate from the application result.

Open question

  • How to systematically detect evaluator error, distinct from application error, at scale.
Direction 08

Autonomy is earned

The endpoint people imagine is agents acting with less supervision. That is reasonable, but autonomy should be a conclusion, not a setting. An agent earns room to act by accumulating a record of responsibilities met and failures handled honestly, on a given kind of work.

The direction Vraelis is working toward is oversight that measures reliability over time and lets autonomy expand where it is warranted and contract where it is not, per agent and per domain, rather than granted once and forgotten.

Current methodology

  • Completion is accepted only when the responsibility is met, on every task.

Open question

  • How to measure earned reliability well enough to justify expanding an agent's autonomy.
  • How autonomy should degrade automatically when an agent's record slips.

Notes

Written at length.

Each note takes one part of the argument and works it through, with the limits stated where they apply.

From failure to verified repairA controlled production demonstration: a claim, a discovered failure, evidence returned, an incomplete repair that Vraelis rejected, and only then a verified one.July 22, 2026 / 4 minAI said it was done. Was it?Code generated, tests green, deploy succeeded, and the customer still cannot do the thing. Three different questions get treated as one.July 21, 2026 / 2 minWhat is outcome verification?A claimed result goes in. External evidence determines whether it is true. Everything else is an implementation detail.July 21, 2026 / 2 minWhy AI agents need an independent completion layerAn agent that grades its own work is not reporting a result, it is reporting an intention. The gap between those is where autonomy breaks.July 21, 2026 / 2 minBrowser testing is not the same as verifying an outcomeThe browser is where a claim becomes observable, not where the outcome ends. Confusing the two produces green tests and broken customers.July 21, 2026 / 2 min

Where this goes

Oversight is a practice, not a finished science.

The methodology here is what we apply now. As agents take on more, the standard has to move with them. That work is ongoing, and it is written down elsewhere too.

Read the Vraelis Method→Why Vraelis exists→

Product

PlatformWhat it can reachWhat is builtAgentsIntegrationsPricingEnterprise

Developers

DocumentationAPICLIWebhooks

Company

Who it is forHow this is differentResearchMethodIn publicREADMEChangelogAbout

Trust

SecurityLimitationsPrivacyTermsData rightsSubprocessorsTrademarkContact
© 2026 Vraelis
XLinkedInSecurityPrivacyTerms