informatics-isi-edu / deriva-mcp-core
90%

Repo Added 25 Mar 2026 07:10AM UTC

Files 39

Badge

Embed ▾

LAST BUILD ON BRANCH main
branch: SELECT

CHANGE BRANCH
x

No branch selected

feat/rag-search-per-rid-data-source-naming

main

Committed 29 Apr 2026 05:48AM UTC coverage: 90.105%. Remained the same

Build # 25093112016

Build Type

push

github

Committed by

mikedarcy

Commit Message

docs(workplan): document RAG data indexing pagination gaps and deferred fixes

  Add notes on two deferred topics identified during code review:

  - Multi-user data: explain why the schema hash deduplication approach does not
    extend to data: sources (row-level vs. schema-level ACL granularity) and
    sketch the shared=True RagDatasetIndexer extension if deduplication becomes
    a practical concern.

  - Pagination gaps: document that rag_index_table silently truncates tables
    larger than 1000 rows (cap inherited from get_entities() pre-Phase 5.6),
    that index_table_data is memory-bound by its list[dict] signature (intentional
    plugin-primitive design), and that _run_dataset_enricher fetches all rows
    before processing begins. Proposed fixes for each: cursor loop in
    rag_index_table, no signature change for index_table_data (callers page),
    and cursor pagination inside _run_dataset_enricher with indexer.limit as a
    total-row cap.

Coverage Stats

3169 of 3517 relevant lines covered (90.11%)

0.9 hits per line

Relevant lines Covered

3517 RELEVANT LINES 3169 COVERED LINES

0.9 HITS PER LINE

Source Files on main

Recent builds

Builds	Branch	Commit	Type	Ran	Committer	Via	Coverage
25093112016	main	docs(workplan): document RAG data indexing pagination gaps and deferred fixes Add notes on two deferred topics identified during code review: - Multi-user data: explain why the schema hash deduplication approach does not extend to data: ...	push	29 Apr 2026 05:51AM UTC	mikedarcy	github	90.11
25091590128	main	fix(rag): correct comment framing and add missing test coverage for per-RID filter The comment added in 55e6eec framed the trailing ":" in own_data_per_rid_prefix as a security defense against a "malicious user_id" causing prefix-overlap. T...	push	29 Apr 2026 04:54AM UTC	mikedarcy	github	90.11
25089657541	main	Merge remote-tracking branch 'origin/main'	push	29 Apr 2026 03:40AM UTC	mikedarcy	github	90.08
25011069162	main	Merge pull request #4 from informatics-isi-edu/feat/rag-search-per-rid-data-source-naming feat(rag): rag_search accepts per-RID data: source naming	push	27 Apr 2026 05:58PM UTC	web-flow	github	90.04
25010740462	feat/rag-search-per-rid-data-source-naming	Merge 55e6eec28 into 0aef9fc0a	Pull #4	27 Apr 2026 05:50PM UTC	web-flow	github	90.04
24969226689	main	fix(rag): enforce per-user and per-catalog isolation in rag_search rag_search applied ACL filtering only to schema: results. Two gaps allowed cross-user or cross-catalog data to appear in scoped searches. data: sources (rag_index_table): s...	push	26 Apr 2026 11:00PM UTC	mikedarcy	github	90.04
24907998680	main	Throttle health check lines in access log to one per 10 minutes	push	24 Apr 2026 07:30PM UTC	mikedarcy	github	90.02
24767922384	main	tests(rag): add TestRagUpdateDocsAsync; wire progress into rag_update_docs_async rag_update_docs_async was missing the task_id_ref pattern and progress_cb wiring that rag_ingest already had, so background tasks showed null progress. Added t...	push	22 Apr 2026 08:19AM UTC	mikedarcy	github	90.01
24766679176	main	rag_status: use full key for enriched dataset sources instead of "enriched"	push	22 Apr 2026 07:48AM UTC	mikedarcy	github	89.5
24766267042	main	rag_status: aggregate indexed_sources by source name to prevent truncation Per-file compound keys ("source:path") produced unbounded output for large indexes. Aggregate to per-source summaries (chunk_count, file_count, last_indexed_at) so t...	push	22 Apr 2026 07:37AM UTC	mikedarcy	github	89.5

See All Builds (44)

informatics-isi-edu / deriva-mcp-core
90%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH main
branch: SELECT

CHANGE BRANCH
x

No branch selected

feat/rag-search-per-rid-data-source-naming

main

CHANGE BRANCH
x

Relevant lines Covered

Source Files on main

Recent builds

informatics-isi-edu / deriva-mcp-core 90%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH main branch: SELECT CHANGE BRANCH x No branch selected feat/rag-search-per-rid-data-source-naming main

CHANGE BRANCH x

Relevant lines Covered

Source Files on main

Recent builds

informatics-isi-edu / deriva-mcp-core
90%

README BADGES
x

LAST BUILD ON BRANCH main
branch: SELECT

CHANGE BRANCH
x

No branch selected

feat/rag-search-per-rid-data-source-naming

main

CHANGE BRANCH
x