source-academy / py-slang / 29379477368

78%

Build Type

push

github

Committed by web-flow

Commit Message

Testing `sourceacademy-sicp` and py-slang Jest test suite (#255)

* Add opt-in CPython ground-truth test mode (#224)

Reruns py-slang's own Jest test cases through real CPython (via a new
scripts/cpython_batch_runner.py and generateCPythonTestCases in
src/tests/utils.ts) to check the test suite's expected values are
actually true statements about Python, not just internally consistent.
Opt-in via CPYTHON_PATH, not wired into CI.

Along the way this surfaced two real bugs, both fixed here:
- build_llist in the sourceacademy-sicp CPython port validated n >= 0,
  which py-slang's own linked-list.prelude.ts never did.
- build_list has no documented contract anywhere (missing from
  docs/lib/list.js entirely) and is unused, so it's removed along with
  its _gen_list helper (CSE builtin, PVML primitive, and CPython
  mirror).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* Address Gemini review feedback on the CPython test runner

- Catch Exception, not BaseException, in run_case (and widen the
  compile-step's except from SyntaxError alone) so KeyboardInterrupt/
  SystemExit propagate instead of being reported as a test failure,
  and unexpected parse/compile errors (ValueError, RecursionError,
  ...) don't crash the whole batch.
- Reconfigure stdin/stdout to UTF-8 explicitly, since test cases
  include emoji/Unicode literals and the platform default encoding
  isn't guaranteed to be UTF-8 (notably on Windows).
- Compare complex-number results with toBeCloseTo on real/imag parts,
  matching the existing float handling, since transcendental complex
  operations can differ in the last ULP across platforms.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* Make the CPython/sourceacademy-sicp test suite part of CI

sourceacademy-sicp is a shipped, versioned artifact of this repo, not
just test-suite tooling for py-slang -- so testing it thoroughly
shouldn't be optional. The existing test_sicp.py smoke suite (run by
python-package.yml) is ... (continued)

Coverage Stats

2747 of 3804 branches covered (72.21%)

Branch coverage included in aggregate %.

121 of 122 new or added lines in 1 file covered. (99.18%)

10 existing lines in 3 files now uncovered.

6327 of 7824 relevant lines covered (80.87%)

12160.56 hits per line