29891246868

Committed 22 Jul 2026 04:30AM UTC coverage: 86.112% (-0.07%) from 86.179%

Build # 29891246868

Build Type

Pull #307

github

Committed by

web-flow

Commit Message

Merge bf9c42273 into bda1a2574

Pull Request Pull Request #307: Cross Python ints as DataType.INTEGER, enforce floats-only module boundary

Coverage Stats

4384 of 5477 branches covered (80.04%)

Branch coverage included in aggregate %.

18 of 64 new or added lines in 5 files covered. (28.13%)

30 existing lines in 1 file now uncovered.

9443 of 10580 relevant lines covered (89.25%)

178352.89 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

49.39

/src/conductor/GenericDataHandler.ts

import type { IEvaluator, IInterfacableEvaluator } from "@sourceacademy/conductor/runner";
import {
  ArrayIdentifier,
  ClosureIdentifier,
  DataType,
  ExternCallable,
  IDataHandler,
  IFunctionSignature,
  OpaqueIdentifier,
  PairIdentifier,
  TypedValue,
} from "@sourceacademy/conductor/types";

/**
 * Backward-compatibility shim for `DataType.INTEGER`: conductor added it well after
 * `DataType.NUMBER` already had a whole ecosystem of modules (sound, midi, repeat, ...) built
 * against it, declaring numeric parameters `DataType.NUMBER` and doing raw JS arithmetic on
 * `.value` assuming it's always a `number`. `closure_call`/`closure_call_unchecked` never
 * validated an argument's actual `DataType` against the closure's declared signature (see this
 * class's own doc comment), so once py-slang started tagging a Python `int` as
 * `DataType.INTEGER` (a `bigint`) instead of silently widening it to `DataType.NUMBER`, calling
 * any such module with a plain integer literal (e.g. `sine_sound(220, 2)`) started throwing
 * "Cannot mix BigInt and other types" deep inside the module's own body instead of working like
 * it always did.
 *
 * One-directional by design, per the module-interface contract Martin settled on: values passed
 * *into* a module can be floats or ints, and an int is silently downcast to a float (declared
 * NUMBER/actual INTEGER narrows to a JS `number`). There is deliberately no reverse direction -
 * modules do not get to declare an INTEGER parameter and have py-slang widen a float into a
 * `bigint` for them. Keeping modules bigint-free is the point: it's simpler, keeps arithmetic
 * fast, and doesn't assume every future language backing a module has bigints at all.
 *
 * Deliberately does NOT recurse into a `DataType.PAIR`/`LIST` argument to normalize elements
 * inside it (an earlier version of this function did, to unbreak a `sum_list`-style module that
 * reads raw NUMBER off list elements) - conductor's LIST/PAIR carries no declared element type,
 * so `entry(t)`/`left_branch(t)`/etc. on a binary_tree also declare their tree argument
 * `DataType.LIST`, indistinguishable at this layer from a *flat list of numbers*. Recursing
 * doesn't just fail to help sum_list - it actively walks straight into the tree's own stored
 * value and flattens it back to NUMBER before the module ever sees it, silently reintroducing
 * exactly the "1.0" bug this was all for. A list-consuming module that wants real integers has
 * to become INTEGER-aware itself; there's no safe way to guess it from here.
 */
function coerceArgsToSignature(
  args: TypedValue<DataType>[],
  declaredArgTypes: readonly DataType[],
): TypedValue<DataType>[] {
  return args.map((arg, i) => {
    const declared = declaredArgTypes[i];
    if (declared === DataType.NUMBER && arg.type === DataType.INTEGER) {
      return { type: DataType.NUMBER, value: Number(arg.value) };
    }
    return arg;
  });
}

/**
 * Enforces the other half of the same contract on the way out: "numerical values coming from
 * modules ... are always floats" - unconditionally, not just for modules that happen to declare
 * DataType.NUMBER. A module has no business returning DataType.INTEGER at all under this design,
 * but this is the boundary that actually guarantees it rather than trusting every module author
 * to remember not to.
 */
function coerceReturnValue<T extends DataType>(value: TypedValue<T>): TypedValue<T> {
  if ((value.type as DataType) === DataType.INTEGER) {
    return { type: DataType.NUMBER, value: Number(value.value) } as TypedValue<T>;
  }
  return value;
}

/**
 * `closure_call`/`closure_call_unchecked` hand back an `AsyncGenerator` (the module may yield
 * control back to the host mid-call), so the return-value coercion can't just wrap a value - it
 * has to wrap the generator and coerce whatever it eventually returns, while passing every
 * yielded value through untouched.
 */
async function* coerceGeneratorReturn<T extends DataType>(
  gen: AsyncGenerator<void, TypedValue<T>, undefined>,
): AsyncGenerator<void, TypedValue<T>, undefined> {
  const result = yield* gen;
  return coerceReturnValue(result);
}

/**
 * A conductor `IDataHandler` implementation with no engine-specific logic:
 * pairs, arrays, closures and opaques are all just bookkeeping over plain
 * Maps keyed by an incrementing id, and the list helpers (`list`/`is_list`/
 * `list_to_vec`/`accumulate`/`length`) walk that pair structure generically.
 * The only place an engine's own semantics enter the picture is the
 * `ExternCallable` passed to `closure_make` (authored by that engine's own
 * module-interop layer); `closure_call`/`closure_call_unchecked`/`closure_call_sync` do inspect
 * arguments now, but only to run them through `coerceArgsToSignature` above -
 * everything else about them still passes straight through unexamined.
 *
 * Originally written inline in PyCseEvaluatorBase (see PyCseEvaluator.ts);
 * extracted so every evaluator that talks to conductor modules (CSE, py2js,
 * and eventually WASM/PVML) shares one implementation instead of
 * re-deriving the same identifier-table bookkeeping per engine. An evaluator
 * holds one instance (`private dataHandler = new GenericDataHandler()`) and
 * hands it to both `context.evaluator` (or the engine's equivalent) and
 * `conductor.registerPlugin(ModuleLoaderRunnerPlugin, conductor, dataHandler)`.
 */
export class GenericDataHandler implements IDataHandler {
  hasDataInterface = true as const;
  private pairMap = new Map<
    PairIdentifier,
    { head: TypedValue<DataType>; tail: TypedValue<DataType> }
  >();
  private arrayMap = new Map<
    // eslint-disable-next-line @typescript-eslint/no-explicit-any
    ArrayIdentifier<any>,
    { type: DataType; elements: TypedValue<DataType>[] }
  >();
  private closureMap = new Map<
    // eslint-disable-next-line @typescript-eslint/no-explicit-any
    ClosureIdentifier<any>,
    {
      // eslint-disable-next-line @typescript-eslint/no-explicit-any
      sig: IFunctionSignature<any, any>;
      // eslint-disable-next-line @typescript-eslint/no-explicit-any
      func: ExternCallable<any, any>;
      dependsOn?: (TypedValue<DataType> | null)[];
      isVararg?: boolean;
    }
  >();
  private opaqueMap = new Map<OpaqueIdentifier, { value: unknown; immutable: boolean }>();
  private uniqueId = 0;
  pair_make(
    head: TypedValue<DataType>,
    tail: TypedValue<DataType>,
  ): Promise<TypedValue<DataType.PAIR>> {
    this.pairMap.set(this.uniqueId++ as PairIdentifier, { head, tail });
    return Promise.resolve({ type: DataType.PAIR, value: (this.uniqueId - 1) as PairIdentifier });
  }
  pair_head(p: TypedValue<DataType.PAIR>): Promise<TypedValue<DataType>> {
    const pair = this.pairMap.get(p.value);
    if (!pair) {
      throw new Error(`Invalid pair identifier: ${p.value}`);
    }
    return Promise.resolve(pair.head);
  }
  pair_sethead(p: TypedValue<DataType.PAIR>, tv: TypedValue<DataType>): Promise<void> {
    const pair = this.pairMap.get(p.value);
    if (!pair) {
      throw new Error(`Invalid pair identifier: ${p.value}`);
    }
    pair.head = tv;
    return Promise.resolve();
  }
  pair_tail(p: TypedValue<DataType.PAIR>): Promise<TypedValue<DataType>> {
    const pair = this.pairMap.get(p.value);
    if (!pair) {
      throw new Error(`Invalid pair identifier: ${p.value}`);
    }
    return Promise.resolve(pair.tail);
  }
  pair_settail(p: TypedValue<DataType.PAIR>, tv: TypedValue<DataType>): Promise<void> {
    const pair = this.pairMap.get(p.value);
    if (!pair) {
      throw new Error(`Invalid pair identifier: ${p.value}`);
    }
    pair.tail = tv;
    return Promise.resolve();
  }
  pair_assert(
    p: TypedValue<DataType.PAIR>,
    headType?: DataType,
    tailType?: DataType,
  ): Promise<void> {
    const pair = this.pairMap.get(p.value);
    if (!pair) {
      throw new Error(`Invalid pair identifier: ${p.value}`);
    }
    if (headType && pair.head.type !== headType) {
      throw new Error(`Expected head of type ${headType}, got ${pair.head.type}`);
    }
    if (tailType && pair.tail.type !== tailType) {
      throw new Error(`Expected tail of type ${tailType}, got ${pair.tail.type}`);
    }
    return Promise.resolve();
  }
  array_make<T extends DataType>(
    t: T,
    len: number,
    init?: TypedValue<NoInfer<T>>,
  ): Promise<TypedValue<DataType.ARRAY, NoInfer<T>>> {
    const elements = new Array(len).fill(init ?? { type: t, value: undefined }) as TypedValue<
      NoInfer<T>
    >[];
    const arrayValue: TypedValue<DataType.ARRAY, NoInfer<T>> = {
      type: DataType.ARRAY,
      value: this.uniqueId++ as ArrayIdentifier<typeof t>,
    };
    this.arrayMap.set(arrayValue.value, { type: t, elements });
    return Promise.resolve(arrayValue);
  }
  array_length(a: TypedValue<DataType.ARRAY>): Promise<number> {
    const array = this.arrayMap.get(a.value);
    if (!array) {
      throw new Error(`Invalid array identifier: ${a.value}`);
    }
    return Promise.resolve(array.elements.length);
  }
  array_get<T extends DataType>(
    a: TypedValue<DataType.ARRAY, T>,
    idx: number,
  ): Promise<TypedValue<NoInfer<T>>>;
  array_get(
    a: TypedValue<DataType.ARRAY, DataType.VOID>,
    idx: number,
  ): Promise<TypedValue<DataType>> {
    const array = this.arrayMap.get(a.value);

    if (!array) {
      throw new Error(`Invalid array identifier: ${a.value}`);
    }

    if (idx < 0 || idx >= array.elements.length) {
      throw new Error(`Index out of bounds: ${idx}`);
    }

    const value = array.elements[idx];

    if (!value) {
      throw new Error(`Missing element at index ${idx}`);
    }

    return Promise.resolve(value);
  }

  array_type<T extends DataType>(a: TypedValue<DataType.ARRAY, T>): Promise<NoInfer<T>> {
    const array = this.arrayMap.get(a.value);
    if (array === undefined) {
      throw new Error(`Invalid array identifier: ${a.value}`);
    }
    return Promise.resolve(array.type as NoInfer<T>);
  }
  array_set(
    a: TypedValue<DataType.ARRAY, DataType.VOID>,
    idx: number,
    tv: TypedValue<DataType>,
  ): Promise<void>;
  array_set<T extends DataType>(
    a: TypedValue<DataType.ARRAY, T>,
    idx: number,
    tv: TypedValue<NoInfer<T>>,
  ): Promise<void> {
    const array = this.arrayMap.get(a.value) as
      | { type: T; elements: TypedValue<NoInfer<T>>[] }
      | undefined;

    if (!array) {
      throw new Error(`Invalid array identifier: ${a.value}`);
    }

    if (idx < 0 || idx >= array.elements.length) {
      throw new Error(`Index out of bounds: ${idx}`);
    }

    array.elements[idx] = tv;

    return Promise.resolve();
  }
  array_assert<T extends DataType>(
    a: TypedValue<DataType.ARRAY>,
    type?: T,
    length?: number,
  ): Promise<void> {
    const array = this.arrayMap.get(a.value);
    if (!array) {
      throw new Error(`Invalid array identifier: ${a.value}`);
    }
    if (type !== undefined && array.type !== type) {
      throw new Error(`Expected array of type ${type}, got ${array.type}`);
    }
    if (length !== undefined && array.elements.length !== length) {
      throw new Error(`Expected array of length ${length}, got ${array.elements.length}`);
    }
    return Promise.resolve();
  }
  closure_make<const Arg extends readonly DataType[], const Ret extends DataType>(
    sig: IFunctionSignature<Arg, Ret>,
    func: ExternCallable<Arg, Ret>,
    dependsOn?: (TypedValue<DataType> | null)[],
  ): Promise<TypedValue<DataType.CLOSURE, Ret>> {
    const closureValue: TypedValue<DataType.CLOSURE, Ret> = {
      type: DataType.CLOSURE,
      value: this.uniqueId++ as ClosureIdentifier<Ret>,
    };
    this.closureMap.set(closureValue.value, { sig, func, dependsOn });
    return Promise.resolve(closureValue);
  }
  closure_is_vararg(c: TypedValue<DataType.CLOSURE>): Promise<boolean> {
    return Promise.resolve(this.closureMap.get(c.value)?.isVararg ?? false);
  }
  closure_arity(c: TypedValue<DataType.CLOSURE>): Promise<number> {
    return Promise.resolve(this.closureMap.get(c.value)?.sig.args.length ?? 0);
  }
  closure_call<T extends DataType>(
    c: TypedValue<DataType.CLOSURE, T>,
    args: TypedValue<DataType>[],
    returnType: T,
  ): AsyncGenerator<void, TypedValue<NoInfer<T>>, undefined> {
    const closure = this.closureMap.get(c.value);
    if (closure === undefined) {
      throw new Error(`Invalid closure identifier: ${c.value}`);
    }
    if (closure.sig.returnType !== returnType) {
      throw new Error(`Expected return type ${returnType}, got ${closure.sig.returnType}`);
    }
    const gen = closure.func(...coerceArgsToSignature(args, closure.sig.args)) as AsyncGenerator<
      void,
      TypedValue<NoInfer<T>>,
      undefined
    >;
    return coerceGeneratorReturn(gen);
  }
  closure_call_unchecked<T extends DataType>(
    c: TypedValue<DataType.CLOSURE, T>,
    args: TypedValue<DataType>[],
  ): AsyncGenerator<void, TypedValue<NoInfer<T>>, undefined> {
    const closure = this.closureMap.get(c.value);
    if (closure === undefined) {
      throw new Error(`Invalid closure identifier: ${c.value}`);
    }
    const coercedArgs = coerceArgsToSignature(args, closure.sig.args);
    const gen = closure.func(...coercedArgs) as AsyncGenerator<
      void,
      TypedValue<NoInfer<T>>,
      undefined
    >;
    return coerceGeneratorReturn(gen);
  }
  /**
   * Fast path for a closure that provably never needs to leave the current
   * synchronous call - e.g. a scalar-in/scalar-out wave function sampled
   * 44100x/sec by the sound module. `func` (an `ExternCallable`, normally
   * only callable as an AsyncGenerator per conductor's own contract) may
   * additionally carry a `.sync` escape hatch: a plain function computing
   * the exact same result with no Promise/generator indirection at all. An
   * engine's module-interop layer sets `.sync` only when it can prove the
   * closure never needs a real host round-trip (see py2js's moduleInterop.ts
   * pyClosureFunc); a closure with no such proof (every CSE-machine closure
   * today, or a py2js closure that touches something asyncOnly) simply never
   * gets one.
   *
   * Returns `undefined` when the closure has no sync form - the signal for
   * "fall back to closure_call_unchecked" - which is unambiguous because a
   * TypedValue always wraps a real `{ type, value }` pair, even for
   * DataType.VOID; the bare JS value `undefined` is never a legitimate
   * closure result.
   */
  closure_call_sync<T extends DataType>(
    c: TypedValue<DataType.CLOSURE, T>,
    args: TypedValue<DataType>[],
  ): TypedValue<NoInfer<T>> | undefined {
    const closure = this.closureMap.get(c.value);
    const func = closure?.func as
      | (ExternCallable<DataType[], T> & {
          sync?: (...a: TypedValue<DataType>[]) => TypedValue<DataType> | undefined;
        })
      | undefined;
    const coercedArgs = closure ? coerceArgsToSignature(args, closure.sig.args) : args;
    const result = func?.sync?.(...coercedArgs) as TypedValue<NoInfer<T>> | undefined;
    return result === undefined ? undefined : coerceReturnValue(result);
  }
  closure_arity_assert(c: TypedValue<DataType.CLOSURE>, arity: number): Promise<void> {
    const closure = this.closureMap.get(c.value);
    if (!closure) {
      throw new Error(`Invalid closure identifier: ${c.value}`);
    }
    if (closure.sig.args.length !== arity && !closure.isVararg) {
      throw new Error(`Expected closure of arity ${arity}, got ${closure.sig.args.length}`);
    }
    return Promise.resolve();
  }
  opaque_make(v: unknown, immutable?: boolean): Promise<TypedValue<DataType.OPAQUE>> {
    const opaqueValue: TypedValue<DataType.OPAQUE> = {
      type: DataType.OPAQUE,
      value: this.uniqueId++ as OpaqueIdentifier,
    };
    this.opaqueMap.set(opaqueValue.value, { value: v, immutable: immutable || false });
    return Promise.resolve(opaqueValue);
  }
  opaque_get(o: TypedValue<DataType.OPAQUE>): Promise<unknown> {
    const opaque = this.opaqueMap.get(o.value);
    if (!opaque) {
      throw new Error(`Invalid opaque identifier: ${o.value}`);
    }
    return Promise.resolve(opaque.value);
  }
  opaque_update(o: TypedValue<DataType.OPAQUE>, v: unknown): Promise<void> {
    const opaque = this.opaqueMap.get(o.value);
    if (!opaque) {
      throw new Error(`Invalid opaque identifier: ${o.value}`);
    }
    if (opaque.immutable) {
      throw new Error(`Cannot update immutable opaque value with identifier: ${o.value}`);
    }
    opaque.value = v;
    return Promise.resolve();
  }
  tie(_dependent: TypedValue<DataType>, _dependee: TypedValue<DataType> | null): Promise<void> {
    throw new Error("Method not implemented.");
  }
  untie(_dependent: TypedValue<DataType>, _dependee: TypedValue<DataType> | null): Promise<void> {
    throw new Error("Method not implemented.");
  }
  async list(...elements: TypedValue<DataType>[]): Promise<TypedValue<DataType.LIST>> {
    const list = await elements.reduceRight(
      async (acc, el) => {
        return this.pair_make(el, await acc);
      },
      Promise.resolve({ type: DataType.EMPTY_LIST, value: null }) as Promise<
        TypedValue<DataType.LIST>
      >,
    );
    return list;
  }
  is_list(xs: TypedValue<DataType.LIST>): Promise<boolean> {
    let current: TypedValue<DataType> = xs;
    while (current.type !== DataType.EMPTY_LIST) {
      if (current.type !== DataType.PAIR) {
        return Promise.resolve(false);
      }
      const pair = this.pairMap.get(current.value);
      if (pair === undefined) {
        return Promise.resolve(false);
      }
      current = pair.tail;
    }
    return Promise.resolve(true);
  }
  list_to_vec(xs: TypedValue<DataType.LIST>): Promise<TypedValue<DataType>[]> {
    return new Promise((resolve, reject) => {
      const result: TypedValue<DataType>[] = [];
      let current: TypedValue<DataType> = xs;
      while (current.type !== DataType.EMPTY_LIST) {
        if (current.type !== DataType.PAIR) {
          reject(new Error(`Expected a list, got type ${current.type}`));
          return;
        }
        const pair = this.pairMap.get(current.value);
        if (!pair) {
          reject(new Error(`Invalid pair identifier: ${current.value}`));
          return;
        }
        result.push(pair.head);
        current = pair.tail;
      }
      resolve(result);
    });
  }
  async *accumulate<T extends Exclude<DataType, DataType.VOID>>(
    op: TypedValue<DataType.CLOSURE, T>,
    initial: TypedValue<T>,
    sequence: TypedValue<DataType.LIST>,
    _resultType: T,
  ): AsyncGenerator<void, TypedValue<NoInfer<T>>, undefined> {
    let acc = initial;
    let current: TypedValue<DataType> = sequence;
    while (current.type !== DataType.EMPTY_LIST) {
      if (current.type !== DataType.PAIR) {
        throw new Error(`Expected a list, got type ${current.type}`);
      }
      const pair = this.pairMap.get(current.value);
      if (!pair) {
        throw new Error(`Invalid pair identifier: ${current.value}`);
      }
      acc = yield* this.closure_call_unchecked(op, [acc, pair.head]);
      current = pair.tail;
    }
    return acc;
  }
  length(xs: TypedValue<DataType.LIST>): Promise<number> {
    let length = 0;
    let current: TypedValue<DataType> = xs;
    while (current.type !== DataType.EMPTY_LIST) {
      if (current.type !== DataType.PAIR) {
        throw new Error(`Expected a list, got type ${current.type}`);
      }
      const pair = this.pairMap.get(current.value);
      if (!pair) {
        throw new Error(`Invalid pair identifier: ${current.value}`);
      }
      length++;
      current = pair.tail;
    }
    return Promise.resolve(length);
  }
}

/**
 * `ModuleLoaderRunnerPlugin`'s constructor requires a single object
 * satisfying `IInterfacableEvaluator` (`IEvaluator & IDataHandler`) — but an
 * evaluator built around `GenericDataHandler` has those two halves on two
 * different objects (the evaluator itself, extending `BasicEvaluator`, is
 * the `IEvaluator`; its `dataHandler` field is the `IDataHandler`). A Proxy
 * combines them into the one object the registration call needs, so
 * combining stays a one-line call at each registration site instead of ~20
 * lines of per-evaluator forwarding methods duplicated alongside the
 * bookkeeping this class already centralizes.
 */
export function asInterfacableEvaluator(
  evaluator: IEvaluator,
  dataHandler: GenericDataHandler,
): IInterfacableEvaluator {
  return new Proxy(evaluator, {
    get(target, prop, receiver) {
      if (prop in dataHandler) {
        // Bind so stateful methods (this.uniqueId++ in pair_make etc.) read
        // and write dataHandler, not the proxy — a plain Reflect.get returns
        // the method unbound, so calling it here would set `this` to the
        // proxy and (absent a `set` trap) silently write to `evaluator`.
        const value = Reflect.get(dataHandler, prop, dataHandler);
        return typeof value === "function" ? value.bind(dataHandler) : value;
      }
      return Reflect.get(target, prop, receiver);
    },
  }) as unknown as IInterfacableEvaluator;
}

source-academy / py-slang / 29891246868

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous