Verification ties an answer to recorded work; it does not make a model or a heuristic infallible.
Evidence and checks
Evidence is the record of tool execution: source references, queries, returned rows, Python outputs, charts, and execution errors. Fella’s verifier is local deterministic code. It checks replayable SQL and Python inputs, numeric support, contract execution, result invariants, and whether a chart agrees with its source result. These checks can catch concrete defects, but they cannot prove that every interpretation or conclusion is correct. A finding carries a machine-readable effect, target, and related evidence IDs. A failed check is not automatically a whole-answer veto:
These effects are deliberately different from tool errors. A successful query
that later fails replay remains a successful execution in the trace, but is
marked as excluded from the current answer. A chart can be withheld while its
source result, prose, and other artifacts remain available. An unsupported
claim can be repaired or omitted without discarding the evidence that supports
the rest of the answer.
Structured presentation validation is a separate boundary. If a JSON-render
tree or result binding is invalid, Fella can withhold that view while keeping
the canonical answer, saved analysis run, and evidence. A valid presentation
only establishes that its registered components and data bindings are
well-formed; it does not prove that the prose or interpretation is correct.
Flow
- The model answers from its gathered evidence; the verifier runs deterministic checks against the answer, catalog, analysis contract, and evidence.
- Each actionable finding identifies its effect, target, evidence references, and repair guidance. Informational findings do not trigger repair.
- The agent may make up to three targeted, tool-backed repair passes within the turn’s remaining model-step budget. It preserves unrelated accepted evidence and does not clear the tool-call memo. A new run is allowed when the finding excludes the specific result that needs replacing.
- If the workspace revision changed during the turn, the stale answer is blocked. Other unresolved findings remain attached to the answer for inspection; they do not silently erase unrelated output.
- A cost-gated same-model consistency check may run after a warning. Its
disagreement is informational, not independent proof or an answer-wide
gate. Set
FELLA_SELF_CHECK=0to opt out.
Inspecting a decision
The answer text stays in the conversation. Open Analysis Details to inspect the checks, their target and effect, and which evidence was excluded or withheld. No success/failure badge is repeated on each message. Historical verification records created before typed findings remain readable; their labels are not retroactively treated as authority to block an answer.Verified is reserved for an answer with a grounded interpretation and a
successful replay signal. A replay proves reproducibility at that moment, not
source completeness or truth. An unresolved meaningful check can leave the
answer in review; a whole-answer failure is reserved for an explicit
answer-scoped block.
Limits
- Numeric grounding does not prove every derived percentage or narrative interpretation.
- Heuristic checks can produce false positives and miss semantic mistakes.
- Replaying Python inputs validates its source boundary, not arbitrary Python arithmetic independently.
- A second opinion from the same model is another model output, not an independent judge.
Related
Read the evidence
Follow files, queries, outputs, and checks in the product.
The analytics engine
See where deterministic query results come from.
Built-in tools
Review what is and is not independently rerunnable.