supabase / supabase / 23208257958

73%

push

github

fix(assistant): sanitize backslash-escaped apostrophes in SQL (#43728)

Fix for the LLM occasionally generating MySQL-style `\'` escapes in SQL,
which are invalid in PostgreSQL.

Example trace where this happened in the wild:
([Braintrust](https://www.braintrust.dev/app/supabase.io/p/Assistant/review?tab=experiment&r=5fcf1b12-8584-455c-9e9a-bdc0fa3ed21c&s=5fcf1b12-8584-455c-9e9a-bdc0fa3ed21c&o=0627ada8-b567-4117-9fe8-49d847cb73a7&review=1))

**Changes**
- Adds `fixSqlBackslashEscapes` to convert `\'` → `''` before SQL is
executed
- Unit tests + adversarial eval dataset case

Compare the results of the adversarial test case:
- `master`: 0% SQL Validity
([Braintrust](https://www.braintrust.dev/app/supabase.io/p/Dev%20(mattrossman%2FAssistant)/trace?object_type=experiment&object_id=b469cbf7-4d6f-429c-9819-6c4099294123&r=dce5a29b-2fde-44c3-80f8-4e14d1f657c0&s=dce5a29b-2fde-44c3-80f8-4e14d1f657c0))
- This branch: 100% SQL Validity
([Braintrust](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=experiment&object_id=160e9ce0-e320-4f6d-8aa7-c5ad7e01fbd2&r=d75ef0e3-90ed-42a7-9ef3-8bf69592f193&s=0eeca492-dbe6-451e-8d81-127caff30320))

Closes AI-400

846 of 959 branches covered (88.22%)

Branch coverage included in aggregate %.

7 of 7 new or added lines in 2 files covered. (100.0%)

3119 of 4438 relevant lines covered (70.28%)

110.04 hits per line