Skip to main content

Test the claim at the right layer

Fella’s tests are organized under tests/: JavaScript and Python unit suites, Rust integration tests, Electron journeys, and provider-backed evaluations. Rust unit tests that need private module details stay beside the source they cover.

Contributor checks

From the repository root:
CI also runs the Electron bridge, updater, chart renderer, release metadata, FQA-Bench adapter, and packaged-app smoke checks. It uses the default SQLite backend with the default PDF and XLSX features.
DuckDB is not a CI path. CI avoids --features duckdb because the bundled build is resource intensive and the DuckDB backend has ingestion-parity gaps. Do not describe it as CI-tested.

Desktop journeys

Playwright configurations and Electron support code are in tests/e2e/. Build the app first, then run the suite relevant to your change:
Credentialed real-model journeys use the configured app provider and incur usage charges:
The last command covers chart families against Luna and Gemma 4 31B. Use Ollama Cloud for Gemma unless the local Ollama server actually hosts that model. Journey completion does not grade numerical correctness or chart suitability.

Model evaluation

FQA-Bench and the model tasks under bench/ capture analytical outcomes and grading criteria. They do not require a particular internal tool order or wording. Run a real provider evaluation with a copied app-data directory:
Provider output is stochastic. Use multiple iterations, report majority accuracy and consistency, and inspect minority failures. Keep model, endpoint, task set, and harness fixed when comparing a change. Never tune grading criteria after seeing candidate output; version approved changes and report the baseline separately.