ory / lumen / 23319341590

68%

Build Type

push

github

Committed by aeneasr

Commit Message

fix: make all e2e and unit tests pass

Key fixes:

1. splitOversizedChunks: account for filepath prefix in embed text
   - Checks now use budget = maxChars - (3 + len(filePath) + 1) so the
     full "// path\n" + content text stays within the model's context
   - Passes the tighter budget to splitChunk and createSubChunks

2. createSubChunks: skip header+overlap when it would exceed budget
   - Adds maxChars parameter; only prepends header+overlap lines if the
     total still fits, preventing non-convergent re-splitting cycles
   - Existing tests updated with maxChars=0 (no limit) to preserve behavior

3. index.go: second splitOversizedChunks pass after mergeUndersizedChunks
   - Prevents oversized chunks from surviving after merge expands them

4. Restore MarkdownChunker and re-register .md/.yaml/.json extensions
   - Languages were removed in a prior commit; restored chunker and tests

5. Fix findEffectiveRoot git-root fallback for SkipDir paths
   - Previously always defaulted to gitRoot; now checks pathCrossesSkipDir
   - testdata/ paths (a Go SkipDir convention) now scope to their own dir,
     preventing testdata/sample-project from indexing the entire repo

6. e2e: add TTL sleeps to TestE2E_IncrementalIndex
   - File changes must wait >1s for the freshness TTL to expire before
     re-searching, matching TestE2E_FreshnessTTLSkipsMerkleWalk pattern

7. e2e lang tests: cap LUMEN_MAX_CHUNK_TOKENS=100 for BERT context window
   - all-minilm uses 4x denser tokenisation than the 4-chars/token estimate

8. Update all cupaloy snapshots to reflect new chunk boundaries

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Run Details

73 of 78 new or added lines in 5 files covered. (93.59%)

285 existing lines in 4 files now uncovered.

2144 of 3101 relevant lines covered (69.14%)

47.82 hits per line