kobotoolbox / kpi / 29437572618

push

github

fix(nlp): add configurable Google STT language code overrides and improve batch error handling DEV-2403 (#7248)

### 📣 Summary
Adds configurable language code overrides for Google Speech-to-Text v2
(e.g. mapping Swahili to `auto`) and improves batch transcription error
handling by surfacing per-file Google API errors instead of generic
missing-output failures.

### 📖 Description
#### Problem

Google Speech-to-Text v2 currently fails to transcribe Swahili when
using the standard language codes (`sw` or `sw-KE`), even though the
language is supported. However, using `auto` allows Google to correctly
detect and transcribe the audio.

Additionally, batch transcription failures could sometimes be reported
as "No transcription output written to GCS" even though Google had
already returned a more specific per-file error.

#### Solution

This PR introduces configurable language code overrides for Google
Speech-to-Text and improves error handling for batch transcription
responses.

#### Changes

- Added a configurable `ASR_LANGUAGE_CODE_OVERRIDES` Constance setting
with environment variable support.
- Added parsing for comma-separated language mappings (e.g.
`sw:auto,sw-KE:auto`).
- Applied language overrides only within the Google Speech-to-Text
integration before sending requests to Google, keeping Kobo language
definitions unchanged.
- Added informational logging whenever a configured language override is
applied.
- Improved Speech-to-Text v2 batch error handling by detecting per-file
errors returned by Google before attempting to read output files from
Cloud Storage.
- Added clearer logging and user-facing error messages for per-file
transcription failures.

### 👷 Description for instance maintainers

This PR introduces a default language-code override for Google
Speech-to-Text:
- sw -> auto
- sw-KE -> auto

This workaround is enabled by default because Google Speech-to-Text v2
currently fails to transcribe Swahili when explicitly using `sw` or
`sw-KE`, wh... (continued)

9298 of 12651 branches covered (73.5%)

3 of 55 new or added lines in 2 files covered. (5.45%)

30292 of 37226 relevant lines covered (81.37%)

5.51 hits per line