pirl-unc / hitlist / 24937135039
80%

DEFAULT BRANCH: main
Ran
Jobs 1
Files 23
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 59.684% (+0.1%) from 59.567%
24937135039

push

github

web-flow 
v1.19.2: Bound ProteomeIndex.from_fasta cache (LRU maxsize=4) (#109) (#158)

Issue #109: PR #101 (closes #86) added an unbounded module-level dict
on ProteomeIndex.from_fasta keyed on (path, size, mtime, lengths,
gene_name, gene_id) to avoid re-indexing shared FASTAs across viral
strain canonicals.  The cache holds every built ProteomeIndex for the
lifetime of the process — and the human proteome alone is ~10 GB
resident.  hitlist data build iterates ~525 canonical species in
the flanking pass; cumulative RSS grows past 16-32 GB and the OS
OOM-kills the build mid-rebuild.

Reproducer trace from the issue:
  fresh Python           rss=0.02 GB
  load_observations      rss=2.81 GB
  + load_binding         rss=3.06 GB
  + human ProteomeIndex  rss=13.21 GB
  + map_peptides         rss=13.87 GB

Three v1.13.3 builds were OOM-killed at the same flanking phase;
disproved as a sandbox-timeout issue when a nohup-detached build
also died.

Fix: bounded LRU using OrderedDict (option (a) from the issue).

- hitlist/proteome.py:
  - _FASTA_INDEX_CACHE is now an OrderedDict with module-level cap
    _FASTA_INDEX_CACHE_MAXSIZE (default 4 — captures the strain-
    variant cache hits motivating #86 since SARS-CoV-2 / EBV / HHV
    canonicals appear close together in build order).
  - Cache hit path calls move_to_end so recency-of-use drives
    eviction order (true LRU, not FIFO).
  - Insertion path evicts least-recently-used entries when over
    capacity; each eviction releases the ~GB-scale numpy arrays so
    peak RSS is bounded by 'maxsize x largest proteome' rather
    than 'sum of all proteomes seen so far'.
  - New public set_fasta_index_cache_maxsize(n) lets callers
    tune the bound (or set 0 to disable caching entirely); shrinking
    evicts down to the new bound immediately.  Negative values
    rejected with ValueError.
  - clear_fasta_index_cache and the size/mtime invalidation
    contract are unchanged.

- tests/test_proteome.py (5 new):
  - 10 distin... (continued)

2490 of 4172 relevant lines covered (59.68%)

0.6 hits per line

Coverage Regressions

Lines Coverage ∆ File
43
79.5
 1.08% proteome.py
Jobs
ID Job ID Ran Files Coverage
1 24937135039.1 23
59.68
 GitHub Action Run
Source Files on build 24937135039