deepset-ai / haystack-experimental / 16936793244
71%

DEFAULT BRANCH: main
Ran
Jobs 1
Files 19
Run time 2min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 89.616% (-4.7%) from 94.318%
16936793244

push

github

web-flow 
feat: Add EmbeddingBasedDocumentSplitter component (#353)

* draft EmbeddingBasedDocumentSplitter

* pylint

* remove test

* fix import in tests

* use document embedder instead of text embedder

* check is warmed up

* add DocumentEmbedder protocol

* handle page breaks

* typing

* Add nltk to test dependencies

* change sentences per group to 3 and update docstrings

* remove unneeded comments from tests, add 2nd integration test

* fix duplicate whitespaces

* preserve whitespaces, don't use strip()

* remove redundant nltk_imports check

* remove redundant input validation

* add asserts for type checks, fix recursionerror

* ignore type check errors

* converting some methods to static, since they don't change/depend on state of the object

* trying to fix macos integration error

* trying to fix macos integration error

* trying to fix macos integration error

* trying to fix macos integration error

* trying to fix macos integration error

* fixing up pyproject.toml

* fixing integrations tests

* fixing pyproject.toml

* forcing integration to run on CPU if MPS is available

* Update pyproject.toml

* Update test/components/preprocessors/test_embedding_based_document_splitter.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* Update test/components/preprocessors/test_embedding_based_document_splitter.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* Update test/components/preprocessors/test_embedding_based_document_splitter.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* fixing alignment issue

* adding init file

* testing for correct page breaks count

* updating README.MD

* refactor: Refactoring in EmbeddingBasedDocumentSplitter (#355)

* Refactoring and slight update to fix in _split_large_splits. Use rstrip since it's not guaranteed the sentence is split on a '.'

* More refactoring to reduce duplicate code

* Fix merge small splits

... (continued)

397 of 443 relevant lines covered (89.62%)

0.9 hits per line

Jobs
ID Job ID Ran Files Coverage
1 16936793244.1 19
89.62
 GitHub Action Run
Source Files on build 16936793244