5737255911
92%

Ran 02 Aug 2023 09:50AM UTC

Jobs 10

Files 236

Run time 7s

Badge

Embed ▾

pending completion

Build # 5737255911

Build Type

push

github

Committed by

web-flow

Commit Message

feat: enable loading tokenizer for models that are not supported by the transformers library  (#5314)

* add tokenizer load

* change import order

* move imports

* refactor code

* import lib

* remove pretrainedmodel

* fix linting

* update patch

* fix order

* remove tokenizer class

* use tokenizer class

* no copy

* add case for model is an instance

* fix optional

* add ut

* set default to None

* change models

* Update haystack/nodes/prompt/invocation_layer/hugging_face.py

Co-authored-by: bogdankostic <bogdankostic@web.de>

* Update haystack/nodes/prompt/invocation_layer/hugging_face.py

Co-authored-by: bogdankostic <bogdankostic@web.de>

* add unit tests

* add unit tests

* remove lib

* formatting

* formatting

* formatting

* add release note

* Update releasenotes/notes/load-tokenizer-if-not-load-by-transformers-5841cdc9ff69bcc2.yaml

Co-authored-by: bogdankostic <bogdankostic@web.de>

---------

Co-authored-by: bogdankostic <bogdankostic@web.de>

Run Details

10960 of 23555 relevant lines covered (46.53%)

2.57 hits per line

Jobs

ID	Job ID	Ran	Coverage
1	pipelines - 5737255911.1	02 Aug 2023 09:51AM UTC	23.79	GitHub Action Run
2	utils - 5737255911.2	02 Aug 2023 09:50AM UTC	20.11	GitHub Action Run
3	modeling - 5737255911.3	02 Aug 2023 09:50AM UTC	19.95	GitHub Action Run
4	document_stores - 5737255911.4	02 Aug 2023 09:50AM UTC	25.66	GitHub Action Run
5	preview - 5737255911.5	02 Aug 2023 09:50AM UTC	22.58	GitHub Action Run
6	nodes - 5737255911.6	02 Aug 2023 09:51AM UTC	30.95	GitHub Action Run
7	cli - 5737255911.7	02 Aug 2023 09:51AM UTC	19.61	GitHub Action Run
8	others - 5737255911.8	02 Aug 2023 09:52AM UTC	22.06	GitHub Action Run
9	prompt - 5737255911.9	02 Aug 2023 09:52AM UTC	24.48	GitHub Action Run
10	agents - 5737255911.10	02 Aug 2023 09:51AM UTC	22.31	GitHub Action Run

Source Files on build 5737255911

Detailed source file information is not available for this build.

deepset-ai / haystack / 5737255911
92%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 5737255911

deepset-ai / haystack / 5737255911 92%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 5737255911

deepset-ai / haystack / 5737255911
92%

README BADGES
x