umputun / tg-spam / 35679574726

84%

push

github

Add jev as an optional LLM provider (#453)

adds jev (typesafe.ai) as an optional third LLM provider alongside OpenAI and Gemini, enabled only when a jev token is configured.

jev is a decision model rather than a text generator: a request carries a `state` plus typed questions, and the answers come back typed. This integration asks one yes/no question and receives a spam probability, thresholded in code, instead of asking for a JSON verdict. The existing veto, `--llm.consensus`, history and message-eligibility rules apply unchanged.

covers CLI and database settings, the web settings panel, and a detected-spam filter. Credentials are masked in logs, encrypted by the configured database crypter, and redacted from `GET /settings`. Invalid configuration, nil clients, and missing or null probabilities return errors rather than verdicts.

**development evidence**

a 150-message replay gave comparable observed results, with no reliable quality difference established between the two providers.

on 86 labeled spam messages OpenAI gpt-5-mini detected 79 and jev detected 80 at threshold 0.30. Neither flagged any of the 62 labeled ham; 2 messages were excluded as insufficient-context. The configurations differ in one way worth naming: OpenAI ran with ten messages of history, jev with none. Median jev latency was 307ms.

labels were assigned from message content and revised during review, including a later adjudication under this chat's bot-recommendation policy. This was not an independently labeled holdout.

typesafe.ai currently lists an input price of [$0.042 per million tokens](https://typesafe.ai/). That is jev's published rate, not a measured production cost comparison.

**picking a threshold**

start with `--jev.threshold=0.30`. Raise it if legitimate messages are flagged, or try lowering it toward `0.25` if spam gets through. Raising it can miss more spam; lowering it can flag more legitimate messages.

the default was chosen on the replay data rather t... (continued)

260 of 267 new or added lines in 8 files covered. (97.38%)

9409 of 11111 relevant lines covered (84.68%)

230.47 hits per line