dcdpr / jp / 20600296467

58%

push

github

feat(llm): introduce `EventChain` for stream merging (#345)

The `EventChain` utility provides a way to seamlessly chain multiple LLM
response streams. It detects when a stream ends due to reaching the
maximum token limit and prepares to merge with a subsequent stream.

By buffering the end of the previous stream and the start of the next,
it identifies overlapping content to ensure a continuous and coherent
output. This is particularly useful for long-running generations that
exceed provider-imposed token limits per request.

This module is not yet used in any provider code, but will be in a
future release.

Signed-off-by: Jean Mertz <git@jeanmertz.com>

0 of 176 new or added lines in 1 file covered. (0.0%)

9093 of 17928 relevant lines covered (50.72%)

128.26 hits per line