Skip to content

WP-203 — gate, CI and test infrastructure

Engineering log · Phase 19 — Codebase audit remediation

🔗 Depends on WP-201 · Needed by WP-204, WP-205

⬅️ WP-202 · Roadmap · WP-204 ➡️

WP-203 — the gate, the workflows and the test scaffolding, consolidated in one commit

Batch 3 of the codebase-audit roadmap. The audit of 2026-10-21 found a gate target that could skip its own check, the same helpers copied across the repo-content checkers and the test tree, and workflows that spent the critical path on serial pytest. Nothing here changes shipped behaviour except the declared jsonargparse floor. The gate is make gate, as for every package (AGENTS.md sec. 1, D30); it will be run before the commit is made, and this page claims no result for it.

Gate and lint tooling. make golden runs scripts/check_goldens.py unguarded, so a missing harness fails instead of printing a skip line. make test takes COV (default --cov=lucid_yolo --cov-report=term; a CI cell that measures nothing passes COV=) and PYTEST_XARGS (default -n auto). Shared helpers replace per-checker copies: REPO_ROOT, the --repo-root argument, sorted_rglob and the failure printer in scripts/lint/_common.py, and REPO_ROOT and one digest in scripts/_common.py; ten checkers plus check_goldens.py and freeze_goldens.py use them, and the _load_audit copies in the checker tests are one load_script in scripts/_tests/script_loader.py. Checker output and exit codes were diffed against HEAD over 25 invocations and are identical. The no-ipynb-tracked hook is now a stock language: fail hook on \.ipynb$, so audit_no_notebooks_tracked.py and its test are deleted and the hook runs on pre-commit.ci. Three hooks are new: test-name-filler-adverbs (the five padding adverbs of AGENTS.md sec. 2 in test names), tqdm-auto-only (over tests/, scripts/ and notebooks/, which ruff's per-file ignores exempt) and model-card-figures (scripts/lint/audit_model_card_figures.py, the wiring-gate figure on each model card against the golden it cites). tqdm.tqdm joins ruff's banned-api table for src/. The .pre-commit-config.yaml comment that counted twelve local hooks no longer counts.

Test tree. The 45 autouse seed fixtures that only called torch.manual_seed(0) are deleted; tests/conftest.py already seeds every test at the same scope and value, and tests/ptl/test_compile.py::isolate_dynamo, which also resets dynamo, stays. tests/fixtures/ gains markers.py (requires_cuda, requires_accelerator, the BF16 replay expectation), determinism.py (deterministic_algorithms, with two fixtures in tests/ptl/conftest.py), training.py (LogRecorder, has_gradient, tiny_module, collate_unpacked) and tensors.py (assert_bytes, seeded_generator), each replacing the copies the audit listed. Every in-body pytest.skip() became a per-case mark, so none is left under tests/, and the one cuda case of test_module.py that ran nowhere now carries requires_cuda. 91 sites in 18 files use the two markers. 39 test names lose a filler adverb; the one node id cited in a doc, in docs/ROADMAP.md, is updated, and two parametrized dtype tests change their ids (dtypeN to float32 and so on). No assertion changed. On the final tree the offline selection reads 4668 of 5029 collected with 361 deselected, against 4671 of 5031 and 360 at the start of the batch. Helper consolidation alone accounts for 12 fewer selected items (19 doctest items of deleted duplicates, less 8 doctest items of the new helpers, plus the one cuda case that moved to the deselected side) and adds or removes no test function; the rest of the delta is the batch's own test changes: the model-card figure checker's tests and the fixture-cache race and lock tests are added, and the notebook checker's tests are deleted with the checker.

Parallel suite. pytest-xdist>=3.5 joins the dev group and make test runs -n auto; PYTEST_XARGS= runs serially, which is how a failure is read, since xdist interleaves output. The synthetic fixture cache under tests/fixtures/_generated is rebuilt under one fcntl.flock per set and published by atomic rename, so a worker never reads a half-written set and a set any worker has returned is never moved away; a current set is read without taking the lock. The lock makes the test suite POSIX-only, which matches the Linux and macOS CI matrix. Wall time of make test on the 16-core development Mac, same tree and same 4612 passed / 47 skipped: 739 s serial (PYTEST_XARGS=), 207 s with -n auto on a warm cache, 228 s cold, 461 s at -n 4 (the width of a hosted runner); coverage stays 96.52% under xdist. No GitHub-side timing exists yet.

Workflows. The test matrix is ubuntu on 3.11, 3.12 and 3.13 plus one macOS cell on 3.13 (the principal's decision), with coverage measured only on ubuntu 3.11 through COV_ARGS; goldens are their own goldens job and testing-guardian needs the testing, goldens, wheel and workflows jobs. A composite action, .github/actions/wheel-smoke, builds the check shared by ci-tests.yml and release.yml: twine check, a wheel install into a venv outside the checkout, the four --help calls and one packaged config loaded through importlib.resources. Every checkout drops credentials except the docs publish step; docs.yml splits into a read-only build and a publish job that installs only ghp-import; every job has a timeout-minutes set from the recent run history; cache: pip is gone from all five places. ci-tests.yml gains a workflows job running actionlint and zizmor at pinned versions through uvx, so a workflow-audit failure turns the required testing-guardian context red. New: links.yml (weekly lychee run over the README and docs/, not a required check) and .github/dependabot.yml (github-actions, monthly, one group, seven-day cooldown). pyproject.toml raises jsonargparse[signatures] from >=4.28 to >=4.36, the first release that exports auto_cli, which lucid-eval and lucid-predict import; WP-202 had left that floor for this batch. A wheel built from this tree installed against jsonargparse 4.36.0 in a scratch venv printed usage for all four commands.

Left deliberately.

  • Lint helpers, partial. Fence tracking is not unified: audit_doctest_fences follows language-aware fences with offsets and audit_wp_links toggles on three-backtick and tilde fences with a quote prefix, and merging them risks different output for no line saved. REPO_ROOT stays local in audit_frozen_manifest, check_commit_trailers, absolutize_readme, golden_producers, overfit_micro, release_guard and shapes_regression, where the bootstrap needed to import the helper costs more than it removes; audit_frozen_manifest and check_commit_trailers keep their own failure wording. audit_frozen_manifest._load and the audit_licenses fixture keep their own loaders.
  • Cross-task test stories are not merged: tiny_module takes the task and its keyword arguments, and the obb, segment and keypoint files keep a thin binding of their own gains.
  • Skip gaps. The four test_overfit_micro.py tests are gpu and data with no skip, the notebooks full run (test_runs_in_full) is the same, and the mps case in test_module.py is a skip with no gpu mark; all three are outside what this batch scoped and stay as they were.
  • Kept after review. The img_size default pinned in tests/predict/test_predict_obb.py, the signature-bound spy in tests/eval/test_eval_obb.py and the backend default in tests/eval/test_coco_eval.py assert documented public constants or library-boundary defaults and were not rewritten as behaviour tests; the principal may revisit the first.
  • CI proof is local. actionlint and zizmor are clean and the wheel-smoke steps ran by hand, but no GitHub-side run exists: the matrix include, the composite action, the COV_ARGS expression, the docs artifact hop and lychee are untested live. The dropped macOS 3.11 and 3.12 cells must not be required checks in branch protection; testing-guardian is the documented required one.

No frozen golden is touched.