dcdpr / jp / 20598575745

58%

push

github

enhance(conversation): wrap document content in `CDATA` blocks (#341)

When serializing documents to XML for LLM consumption, we now detect the
presence of `<` or `>` characters and automatically wrap the content in
`CDATA` blocks.

Without `CDATA`, `quick_xml` escapes these and other special characters
into their HTML entity equivalents (e.g., `&lt;` and `&gt;`). This
behavior increases the token count sent to LLM providers, leading to
higher costs. Furthermore, it can negatively impact model performance as
LLMs may struggle to accurately parse and understand heavily escaped
content compared to the original raw text.

---------

Signed-off-by: Jean Mertz <git@jeanmertz.com>

0 of 7 new or added lines in 1 file covered. (0.0%)

9089 of 17747 relevant lines covered (51.21%)

129.57 hits per line