source-academy / py-slang / 23419444526

59%

push

github

feat: replace hand-written parser with Nearley grammar (#104)

* feat: replace hand-written parser with Nearley grammar

Rewrites the Python parser using the Nearley parser generator with
Moo-based lexer. Adds indentation-aware tokenization, subsumes
validators into resolver, and includes comprehensive parser tests.

* chore: add yarn.lock and remove package-lock.json

Upstream CI uses Yarn with --frozen-lockfile. Generate yarn.lock
for the new nearley/moo dependencies and remove the npm lockfile.

* fix: resolve eslint errors and apply prettier formatting

Adds eslint-disable to generated files (ast-types, python-grammar),
fixes explicit-any types in parser-adapter and token-bridge, removes
unused imports.

* docs: clarify traverse.ts is a standalone utility, not the production path

Updates docstrings to accurately describe the instanceof-based traversal
(not visitor pattern) and warns that runValidators does not pass Environment,
making scope-aware validators like no-reassignment silently no-op.
The main pipeline uses Resolver's inline validation instead.

* fix: add Starred handling to traverseAST, remove broken runValidators export

traverseAST now recurses into ExprNS.Starred children. The standalone
runValidators is removed from public exports — it cannot pass Environment,
silently breaking scope-aware validators like no-reassignment. Use the
Resolver's inline validation pipeline instead.

* fix: allow embedded quotes in triple-quoted string literals

* fix: map hex/octal/binary tokens to BIGINT to match legacy tokenizer

* fix: extend no-reassignment validator to catch AnnAssign

* fix: traverse assignment targets in traverseAST utility

* fix: make compile-grammar idempotent to prevent duplicate @ts-nocheck

Also regenerates python-grammar.ts with current nearleyc output.

* style: format analysis.test.ts to pass prettier

* refactor: rename Moo tokens and rewrite Nearley grammar with spec-aligned names

- Rename lexer tokens: identifier→name... (continued)

276 of 1014 branches covered (27.22%)

Branch coverage included in aggregate %.

560 of 621 new or added lines in 24 files covered. (90.18%)

1214 of 2561 relevant lines covered (47.4%)

52.68 hits per line