5
93%
main: 93%

Ran 02 Aug 2023 09:56AM UTC

Files 236

Run time 5s

Badge

Embed ▾

Committed 02 Aug 2023 09:42AM UTC coverage: 22.581% (-0.01%) from 22.593%

Job # 5737255911.5

Build Type

push

github

Committed by

web-flow

Commit Message

feat: enable loading tokenizer for models that are not supported by the transformers library  (#5314)

* add tokenizer load

* change import order

* move imports

* refactor code

* import lib

* remove pretrainedmodel

* fix linting

* update patch

* fix order

* remove tokenizer class

* use tokenizer class

* no copy

* add case for model is an instance

* fix optional

* add ut

* set default to None

* change models

* Update haystack/nodes/prompt/invocation_layer/hugging_face.py

Co-authored-by: bogdankostic <bogdankostic@web.de>

* Update haystack/nodes/prompt/invocation_layer/hugging_face.py

Co-authored-by: bogdankostic <bogdankostic@web.de>

* add unit tests

* add unit tests

* remove lib

* formatting

* formatting

* formatting

* add release note

* Update releasenotes/notes/load-tokenizer-if-not-load-by-transformers-5841cdc9ff69bcc2.yaml

Co-authored-by: bogdankostic <bogdankostic@web.de>

---------

Co-authored-by: bogdankostic <bogdankostic@web.de>

Coverage Stats

5319 of 23555 relevant lines covered (22.58%)

0.23 hits per line

deepset-ai / haystack / 5737255911 / 5
93%
main: 93%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job preview - 5737255911.5

deepset-ai / haystack / 5737255911 / 5 93% main: 93%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job preview - 5737255911.5

deepset-ai / haystack / 5737255911 / 5
93%
main: 93%

README BADGES
x