• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

kobotoolbox / kpi / 21919183404
82%
master: 76%

Build:
Build:
LAST BUILD BRANCH: beccagraber/dev-1748-verification-docs
DEFAULT BRANCH: master
Ran 11 Feb 2026 07:41PM UTC
Jobs 1
Files 879
Run time 2min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

11 Feb 2026 07:08PM UTC coverage: 79.701% (-2.3%) from 82.019%
21919183404

push

github

web-flow
feat(qual): allow un/verification of QA responses DEV-734 (#6690)

### đŸ“Ŗ Summary
Allow users to verify or unverify QA responses, whether manual or
automatic.


### 📖 Description
All responses, manual or automatic, are considered unverified until they
are explicitly verified. This status can be updated via the API, and the
date of verification will be recorded, or removed if it was un-verified.

### 💭 Notes
Originally we were going to reuse the 'accepted' field used by NLP
actions but verification behaves differently than acceptance in many
ways:

1. We cannot unaccept a transcription or translation but we can unverify
a QA response: verification checkbox — it can be checked, unchecked, and
the state is saved along the answer.
2. We show unverified automatic QA responses in the data table and
exports, we do not show unaccepted automatic NLP actions
3. We do not automatically verify manual QA answers
4. 'verified' shows up as its own field in exports, 'accepted' does not

This PR also contains a little refactoring of when/how we deal with
things like acceptance and verification. They are no longer a part of
`run_external_process` since we can verify manual QA responses and, in
any case, verification and acceptance do not actually require external
services.

### 👀 Preview steps
Have AWS_SECRET_ACCESS_KEY and AWS_ACCESS_KEY_ID point to an account
with Bedrock access

1. â„šī¸ have an account and a project with at least one audio question
2. Add a submission
3. Open the audio response for editing
4. Add a transcript
5. Add a QA question and response
6. In a terminal, PATCH the supplement endpoint for that submission with
data
```
{
  "_version": "20250820",
  "{question_xpath}": {
    "manual_qual": {
      "uuid": "{QA question uuid}",
      "verified": true
    }
  }
}
```
7. đŸŸĸ [on PR] The response should contain only one version, with
"verified" = True and "_dateVerified" set to now
8. Enable automatic qual with a text question
9. Request an LLM response t... (continued)

7222 of 11498 branches covered (62.81%)

64 of 66 new or added lines in 11 files covered. (96.97%)

27681 of 34731 relevant lines covered (79.7%)

0.8 hits per line

New Missed Lines in Diff

Lines Coverage ∆ File
1
88.7
0.62% kobo/apps/subsequences/actions/base.py
1
89.51
-0.21% kpi/views/v2/data.py
Jobs
ID Job ID Ran Files Coverage
1 21919183404.1 11 Feb 2026 07:41PM UTC 879
79.7
Source Files on build 21919183404
  • Tree
  • List 879
  • Changed 12
  • Source Changed 0
  • Coverage Changed 12
Coverage ∆ File Lines Relevant Covered Missed Hits/Line Branch Hits Branch Misses
  • Back to Repo
  • bec6d48e on github
  • Prev Build on main (#21906223324)
  • Next Build on main (#21921179405)
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc