pirl-unc / hitlist
75%

DEFAULT BRANCH: main
Repo Added
Token 2BMOPfDLihNAYkvRMkn4Pz38iRgZjmtXf regen
Build 348 Last
Files 28
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH main
branch: SELECT
CHANGE BRANCH
x
Sync Branches

coverage: 74.898% (+0.4%) from 74.531%
25613983293

push

github

web-flow 
v1.30.46: split gene/protein columns out of observations.parquet (closes #238 partial) (#243)

Drops ``gene_names``, ``gene_ids``, ``protein_ids``, and
``n_source_proteins`` from the observations.parquet schema.  The same
information has always been in ``peptide_mappings.parquet`` (one row
per peptide x protein); we were storing it in TWO places and merging
the long-form mapping back onto every observation row at build time.

## Wins (measured against the v1.30.45 build of the same source data)

| Metric | v1.30.45 | v1.30.46 | Delta |
|---|---|---|---|
| observations.parquet size | 192.2 MB | **117.6 MB** | **-39%, -74.5 MB** |
| Gene/protein columns combined size | 71.1 MB (38.8% of obs) | 0 MB | -71 MB |
| Gene-annotation merge step in build | full-frame merge | skipped | (eliminates the largest single transient memory step in build_observations) |
| ``hitlist pmhc --gene PRAME`` rows loaded | 4.4M (full corpus) | 257 (matched peptides only) | ~17,000x reduction |
| ``hitlist pmhc --gene PRAME`` parquet load time | ~6s | 0.8s | ~7.5x faster |

Build-side memory wins are harder to capture as a single number — the
v1.30.45 build held the full peptide_mappings (~65 MB) AND obs (~190 MB)
in pandas form simultaneously through ``annotate_observations_with_genes``,
which is the largest transient step.  Eliminating that step cuts the
single biggest memory blip in the build pipeline.

## What changed

### Build path (``builder.py``)

Skip ``annotate_observations_with_genes()``.  The peptide_mappings.parquet
sidecar continues to be built independently as it always was.

### Reader path (``observations.py``)

- ``load_observations`` now AUTO-ATTACHES gene/protein columns from
  peptide_mappings.parquet when the caller requests them but the
  parquet doesn't carry them (post-v1.30.46).  Only joins the
  matched-peptides slice (cheap on filtered loads, expensive only on
  full-corpus loads — same complexity as the old build-time merge).
- New entries in ``_DERI... (continued)

4234 of 5653 relevant lines covered (74.9%)

0.75 hits per line

Relevant lines Covered
5653 RELEVANT LINES 4234 COVERED LINES
0.75 HITS PER LINE
Source Files on main

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
25613983293 main v1.30.46: split gene/protein columns out of observations.parquet (closes #238 partial) (#243) Drops ``gene_names``, ``gene_ids``, ``protein_ids``, and ``n_source_proteins`` from the observations.parquet schema. The same information has always be... push web-flow github
74.9
25613896952 split-gene-mapping-from-observations Merge 12f276a1a into ea2cae856 Pull #243 web-flow github
74.9
25601432313 main align version.py with PyPI: 1.30.44 → 1.30.45 (#242) PR #240's amended commit accidentally dropped its version bump (I ran `git commit --amend --message ...` without re-staging hitlist/version.py). The squash merge therefore landed v1.30.45's co... push web-flow github
74.53
25587678819 align-version-with-pypi Merge fc979af1d into f83694dd9 Pull #242 web-flow github
74.53
25587641606 main v1.30.45: predictor-driven multi-allele narrowing in pmhc query (closes #239) (#240) When ``hitlist pmhc --predictor netmhcpan|mhcflurry`` runs against a multi-allele row (every per-donor row from #236, every IEDB row whose ``host_mhc_types`` pop... push web-flow github
74.53
25587100236 predictor-multi-allele-narrowing Merge 60b2d941d into a5200ad85 Pull #240 web-flow github
74.53
25587063807 main v1.30.44: pre-add "" to compressed-categorical categories so fillna("") never TypeErrors (#241) Same bug class as v1.30.41 / v1.30.43, surfaced by ./test.sh --all (integration suite that loads the real observations.parquet). 12 test_export error... push web-flow github
74.4
25586985473 fix-allele-resolution-categorical-fillna Merge 488358408 into 75b189cc0 Pull #241 web-flow github
74.4
25564289983 predictor-multi-allele-narrowing Merge fc3d204ac into 75b189cc0 Pull #240 web-flow github
74.52
25563211782 main v1.30.43: per-donor row split for attributed peptides (closes #236) (#237) * v1.30.43: per-donor row split for attributed peptides (closes #236) When the per-peptide attribution CSV maps a peptide to N matched donors, the scanner now emits N obs... push web-flow github
74.39
See All Builds (348)

Badge your Repo: hitlist

We detected this repo isn’t badged! Grab the embed code to the right, add it to your repo to show off your code coverage, and when the badge is live hit the refresh button to remove this message.
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

Refresh