# Testing — coverage strategy & root guide

The repo-wide test strategy: what is covered, where the gaps are, and the
plan to close them. Per-component run commands live in each product's
`TESTING.md` (linked below); this page is the QA architecture — the
coverage matrix, the risk-ranked gaps, and the roadmap.

**Principle:** all tests run in Docker (host is a Mac, not the source of
truth — [CLAUDE.md](CLAUDE.md)). **Unit tests stub external infra; the demo
scripts are the integration harness; Playwright is E2E.**

> Prereq for `fasten`-linked suites (Go cgo, Analytics API):
> ```bash
> git submodule update --init --recursive
> docker build -t edgebits/fasten-core:latest -f Dockerfile.fasten-core .
> ```

---

## 1. Coverage matrix — product × layer

Counts are test functions present today (`grep 'def test_' / 'func Test' /
'it('`). Grade is a QA judgement of adequacy for the code's risk, not just
count.

| Product / module | Unit | Integration | E2E | Tests | Line cov | Grade |
|---|:--:|:--:|:--:|--:|--:|:--:|
| **Edge Node — gateway** (`edge/gateway`) | ✅ | demo | — | 323 | — | A |
| **Edge Node — event-engine** (`edge/event-engine`) | ✅ | demo | — | 13 | — | B |
| **Edge Node — core/orchestrator** (`edge/core`) | ◑ | demo | — | 3 | — | C |
| **Edge Node — UI** (`edge/ui`) | ◑ formatter only | — | — | 19 | — | D |
| **Edge Sync** (`edge/edge-sync`) | ✅ | demo (DMZ) | — | 25 | — | A |
| **Edge Analytics — API** (`edge-analytics/api`) | ✅ | demo | — | 169 | 75% | A |
| **Edge Analytics — UI** (`edge-analytics/ui`) | — | — | ✅ | 10 | — | B |
| **Edge Manager — API** (`edge-manager/api`) | ✅ | demo (deploy) | ✅ e2e_test | 47 | — | A |
| **Edge Manager — UI** (`edge-manager/ui`) | — | — | — | 0 | — | F |
| **Edge SDK** (`sdk/edge`) | ✅ | — | — | 6 | — | B |
| **Edge-Manager SDK** (`sdk/edge-manager`) | ✅ | — | — | 2 | — | C |
| **Blocks — egress/fabric** (`blocks/egress/fabric`) | ✅ | — | — | 66 | — | A |
| **Blocks — egress/rest** (`blocks/egress/rest`) | ✅ | demo | — | 21 | — | A |
| **Blocks — event-processors/actions** | ✅ | — | — | 7 | — | B |

Legend: ✅ adequate · ◑ partial · — none. **~711 tests total** (602 Python,
80 Go, 29 TS).

Grades: **A** solid · **B** adequate, thin edges · **C** under-covered ·
**D** minimal · **F** none.

---

## 2. Category breakdown

| Category | Where it lives | State |
|---|---|---|
| **Unit** | pytest (real SQLite), Go `httptest`, vitest (Intl) | Strong on backends, weak on frontends |
| **Integration** | `demo/modbus`, `demo/modbus-fleet`, `demo/ccts-*` | The real chain across all planes; asserts data flow + row counts |
| **E2E (UI)** | Playwright (`edge-analytics/ui/e2e`) | Analytics only; Manager + Edge Node UI have none |
| **E2E (API)** | Go `e2e_test.go` (Manager) | Deploy/claim flow |
| **Contract** | edge-sync ↔ Manager mirror test; SDK request-id | Partial — mirror + audit-ship covered; no schema-contract gate |
| **Performance / load** | — | **None** (gap) |
| **Security** | `test_security`, `test_auth`, `test_token_denylist`, lineage/audit-verify | Good on edge gateway |

---

## 3. Gap analysis — risk-ranked

| # | Gap | Risk | Priority |
|---|---|---|:--:|
| 1 | **Edge Manager UI: zero tests** | Pipeline editor, fleet views, deploy flows ship unverified; the reverse-sync fidelity bug went unnoticed | P0 |
| 2 | **Analytics UI: no unit tests** (E2E only) | Component logic (OEE math in `lib/oee`, chart transforms) only exercised through the browser | P1 |
| 3 | **Edge Node UI: formatter only** | 19 vitest cases cover 1 lib file; pipeline-editor schema inference, function-registry untested at unit level | P1 |
| 4 | **Postgres SQL paths** (`aggregate.py` 61%, `telemetry_repo` 24%) | Aggregation correctness only demo-covered; a bad `time_bucket` regression wouldn't fail a unit run | P1 |
| 5 | **No performance/load tests** | Buffer flush, ingest throughput, fleet poll fan-out have no regression guard | P2 |
| 6 | **No schema-contract gate** edge-sync↔Manager↔Analytics | Wire shape drift (e.g. the planned Analytics→Manager shift/downtime config pull) caught only by a hand-written test, not a shared schema | P2 |
| 7 | **core/orchestrator 3 tests** | Connector/egress spawn lifecycle thinly covered | P2 |
| 8 | **datetime call-site sweep** (37 files still on `toLocale*`) | Inconsistent timestamp rendering; formatter exists + tested but not adopted | P2 |

---

## 4. Strategy & roadmap

**Testing philosophy for this platform** (industrial edge, multi-plane):

1. **Backends carry the correctness weight** — services + repos are unit
   tested against real SQLite/httptest, no mocks of the DB. Kept at A grade.
2. **The demo is a first-class test** — it is the integration suite. Every
   cross-plane change must keep `demo/modbus/run.sh` green end-to-end. The
   Postgres SQL paths are deliberately integration-covered here rather than
   unit-mocked.
3. **Frontends need a unit floor + E2E ceiling** — pure logic (OEE math,
   schema inference, formatters) gets vitest; navigation + config flows get
   Playwright. Today only Analytics has the ceiling and only the formatter
   has the floor.

**Roadmap (close gaps in priority order):**

- **P0** — Manager UI: vitest for pipeline `buildCanvas`/`inferSchema` +
  Playwright for the deploy + fleet flows. (Would have caught the
  reverse-sync source/sink drop.)
- **P1** — Analytics + Edge UI vitest floor (`lib/oee`, chart transforms,
  schema inference). Promote the Postgres SQL paths to a Testcontainers
  integration suite so aggregation has a unit-speed regression guard.
- **P2** — a shared JSON-schema contract for the edge-sync↔Manager↔Analytics
  wire, validated on both sides; a k6/vegeta ingest+poll load smoke; the
  datetime call-site sweep.

**Definition of done for a new feature** (per
[.claude/rules/contribution.md](.claude/rules/contribution.md)): greenfield
is test-first; state-changing ops carry an audit test; cross-plane changes
test both sides; the demo stays green.

---

## 5. How to run — per-component guides

| Component | Guide |
|---|---|
| Edge Analytics API | [edge-analytics/api/TESTING.md](edge-analytics/api/TESTING.md) |
| Edge Analytics UI | [edge-analytics/ui/TESTING.md](edge-analytics/ui/TESTING.md) |
| Edge Node UI | [edge/ui/TESTING.md](edge/ui/TESTING.md) |
| Edge Node gateway/core/event-engine | [edge/TESTING.md](edge/TESTING.md) |
| Edge Sync | [edge/edge-sync/TESTING.md](edge/edge-sync/TESTING.md) |
| Edge Manager API | [edge-manager/api/TESTING.md](edge-manager/api/TESTING.md) |
| Edge Manager UI | [edge-manager/ui/TESTING.md](edge-manager/ui/TESTING.md) |
| Edge SDK | [sdk/edge/TESTING.md](sdk/edge/TESTING.md) |
| Edge-Manager SDK | [sdk/edge-manager/TESTING.md](sdk/edge-manager/TESTING.md) |
| Blocks (connectors/egress) | [blocks/TESTING.md](blocks/TESTING.md) |

---

## 6. Integration / end-to-end — the demo chain

The demos are the integration harness — full stack in Docker, data across
every boundary.

```bash
bash demo/modbus/run.sh          # single node (DMZ) — up + seed + verify
bash demo/modbus/run.sh down     # tear down
bash demo/modbus-fleet/run.sh --num-nodes=3 --topology=one-site   # fleet
```

The single demo asserts, in order: Modbus ingest → MQTT → pipeline (enrich →
buffer → REST egress) → Analytics data plane (row counts) → config seeding →
domain compute (reconciliation status) → control plane (edge-sync DMZ poll,
Manager fleet online).

**Direct vs DMZ:** the modbus demo runs **DMZ** (edge-sync polls Manager).
Direct mode (Manager deployer pushes via edge SDK) is in
`demo/ccts-emission-pipeline-steel/{standalone,deploy-dmz}`. Both modes must
produce the identical pipeline on the node — the reverse-sync fidelity bug
([bug-manager-reverse-sync-drops-source-sink](issues/pending/bug-manager-reverse-sync-drops-source-sink.md))
is exactly a violation of that invariant.

Both demos are idempotent (re-run safe; `demo/lib/seed-analytics.sh` upserts).

---

## 7. CI

Auto-triggers (`push`/`pull_request`) are **disabled** repo-wide to avoid
billed runs — `ci.yml` keeps only `workflow_dispatch`. Run suites manually
or locally. New jobs follow the same pattern.