benwbrum / fromthepage / 29862605556

74%

push

github

Add per-field CDR and text-only CDR columns to AI disagreement report (#5706)

### Motivation

- Add finer-grained disagreement metrics to the AI disagreement CSV by exposing Character Disagreement Rate (CDR) for each transcription field so reviewers can see per-field differences between Claude and Gemini outputs.
- Provide both verbatim and normalized (text-only) CDR per field to separate punctuation/formatting errors from textual content differences.

### Description

- Discover transcription fields across the work/collection and preload `transcription_fields` with `:spreadsheet_columns`, excluding `description` and `instruction` input types, and build dynamic CSV headers for each field with `"<Field> CDR"` and `"<Field> Text-only CDR"`.
- Add `field_value_for_comparison` helper to extract a field's verbatim value from `transcription_json`, preserving existing spreadsheet-row concatenation logic, and add `normalize_text` for the text-only form.
- For each page, compute per-field verbatim and normalized CDRs only when both Claude and Gemini finished transcriptions exist and the page's collection includes the field; otherwise output blank columns to preserve CSV alignment.
- Use the existing `character_error_rate` implementation via `page.send(:character_error_rate, ...)` to compute CDRs so behavior is consistent with top-level disagreement columns.

### Testing

- Ran a Ruby syntax check: `bundle exec ruby -c lib/tasks/ai_disagreement_report.rake` which returned `Syntax OK`.
- Attempted to run `bundle exec rubocop lib/tasks/ai_disagreement_report.rake` but the `rubocop` executable was not available in the environment so it could not be executed.

2759 of 4321 branches covered (63.85%)

Branch coverage included in aggregate %.

11002 of 14330 relevant lines covered (76.78%)

199.1 hits per line