sandialabs / LoQS / 34743238027

90%

push

github

Make QuantumProgram checkpointing parallel-safe (#128)

* Rewrite QuantumProgram.run()'s checkpoint mechanism for real concurrent safety

Replaces the driver-side-only checkpoint mechanism -- never exercised
under real worker concurrency, and confirmed unsafe if multiple
processes ever shared one file -- with a design where each worker
checkpoints its own shots directly to its own file, consolidated by the
driver once every dispatched batch is done, and removes the fragile
QuantumProgram._last_results caching mechanism entirely.

- QuantumProgram.run() batches shot dispatch by checkpoint_batch_size
  when checkpointing is enabled, checkpointing each batch inside its own
  worker process (keyed by hostname/pid, batch size 1 recovers fully
  per-shot durability) instead of shipping every shot back to the
  driver before anything is written; gains a lazy_loading_enabled
  parameter controlling whether checkpointed shots stay in memory.
- ProgramResults.checkpoint()/consolidate_checkpoints() drop the old
  strategy/batch-index arithmetic (which could silently misattribute
  shots under out-of-order completion) for a simpler "flush everything
  unwritten" call, and stream one worker's file in at a time on
  consolidation rather than loading everything into memory first.
- Removes QuantumProgram._last_results and its two dead-code dependents
  (fttools.test_program_output's skip_run, pygstitools's deprecated
  _last_results fallback), none of which had real callers left.
- Adds a real multi-process concurrent-write test confirming several OS
  processes checkpointing at once never corrupt or drop data.

* Document parallel checkpointing in the parallel-execution tutorial

Adds a "Checkpointing under parallelism" section to docs/notebooks/parallel.md covering how checkpoint_batch_size and worker identity interact with a shot_executor, the lazy_loading_enabled eviction behavior, and the checkpoint mechanism's scope relative to the other program-level call sit... (continued)

3419 of 4038 branches covered (84.67%)

Branch coverage included in aggregate %.

10717 of 11718 relevant lines covered (91.46%)

0.91 hits per line