deepset-ai / haystack-experimental

94%

main: 67%

LAST BUILD ON BRANCH feat/eval-harness-base-class

branch: feat/eval-harness-base-class

CHANGE BRANCH

x

Reset

Sync Branches

• feat/eval-harness-base-class
• LLMMetadataExtractor-use-ChatGenerator
• PR-comments
• add-metadata-extractor
• add-normalize-emb-image-emb
• add-parent-span
• add-pre-commit-hooks
• adding-agent-code-from-haystack-main
• adding-forward-references-llm-metadataextractor
• adding-pipeline-breakpoints
• adding-pipeline-for-easy-PR-review
• adopt-uv
• agent-deserialize-tool-with-utility-function
• agent-refactor-proposal
• agent-usage-example
• agents-update
• align-mypy-haystack
• anthropic_tools
• async-experiments
• auto-merging-retriever
• auto-merging-retriever-update
• bedrock-component-info-update
• bedrock-multimodal
• change-default-model-generators
• changed-ds-deser
• chat_message_store
• chat_msg_retriever
• chat_msg_store
• chat_msg_writer
• chatmessage-retriever-writer-adaptations
• chatmessages-for-tools
• chatpromptbuilder-multimodal
• chore/clean-up-LLMMetadaExtractor
• chore/clean-up-auto-merging-retriever
• chore/clean-up-inmemory-doc-store-async
• chore/prepare_agent_pr
• chore/remove-discontinued-experiments
• chore/remove-pipeline-asyncpipeline
• chore/removing-breakpoints
• ci/release-dev-version
• ci/setup-tests
• cleanup
• david/fix-hybrid-pipeline
• david/fixing-for-more-pipelines
• david/pipeline-testing
• david/remove-sender-in-serialisation
• dependabot/github_actions/actions/checkout-5
• dependabot/github_actions/actions/setup-python-6
• dependencies-tight-up
• deserialize-tools-inplace
• disable_firecrawl
• disable_meteo
• doc-length-router
• doc-to-image-content
• doc-to-image-content-store-file-path-as-string
• doc-to-image-fmt
• document-captioner
• document-type-router
• embedding-based-document-splitter
• embedding-based-document-splitter-new
• enhance-tool-logs
• explicit_param
• feat/add-summarizer
• feat/agent-breakpoints
• feat/agent_component
• feat/agent_examples
• feat/agent_streaming_llm
• feat/async-chat-prompt-builder
• feat/async-inmemory-retrievers
• feat/async_pipeline
• feat/breakpoints-structure
• feat/eval-harness-pipeline-pair
• feat/eval-harness-rag
• feat/function-caller
• feat/multiquery-retriever
• feat/pipeline-checkpoints
• feat/rag-eval-harness-ctx-relevance-metric
• feat/rag-eval-harness-gt-inputs-in-results
• feat/rag-eval-harness-notebook
• feat/ready_made_supercomponents
• feat/summariser-based-on-LLM
• feat/supercomponent_base
• feat/supercomponents
• file-to-image-content
• fix-bedrock-test
• fix-breakpoints
• fix-bug-in-resuming-inputs
• fix-component-tool
• fix-deserialization-errors
• fix-init-params-in-state
• fix-json-dumps
• fix-linting
• fix-linting-error
• fix-llmmetadata-extractor
• fix-logger-message
• fix-openai-empty-content
• fix-validate-input
• fix/adding-mocks-to-tests
• fix/adding-pandas-as-dependency
• fix/auto-merging-retriever-tests
• fix/component_tool_deserialization_in_agent
• fix/deepcopy_component_tool
• fix/document-stores
• fix/haystack-serialization-fix
• fix/init-imports
• fix/naming
• fix/pipeline
• fix/pipeline_run
• fix/pyproject-toml
• fix/rag-eval-harness-bugs
• fix/typo-example
• fix_anthropic_format
• fix_naming
• fix_tests
• fixing-breakpoints-tests
• fixing-breakpoints-tests-file-name-change
• fixing-hierarchical-splitter
• fixing-init-and-test-imports
• fixing/windows-tests-failing
• from_component
• hallucination-risk-calc
• hatch-scripts-unify
• hfapi-tools
• image-content
• image-embedder-experiment
• image-show
• image-to-document-converter
• image-validation
• imagecontent-convenience-class-methods
• improve-multimodal-docstrings
• jinja-chatmessage-extension
• llmmetadata-extractor
• main
• make-LLM-based-evaluators-raise_on_failure_False
• metadata-extractor-update
• more-cleanup
• multi-file-converter-unclassified
• multimodal-add-usage-examples
• mv-imagecontent
• mv-ipython-import
• ollama-pydantic-fix
• ollama-tools
• openai-check-tool-names
• openai-multimodal
• openaichatgenerator-with-tools
• openapi
• openapi_corner_cases_handling
• openapi_filter
• openapitool_error_handling
• pin-transformers
• pipe_tool
• pipeline-wrapper
• py39
• pylint-consistency
• query-expander
• raise-error-for-async-pipeline
• refactor-doc-to-img
• refactor-emb-base-splitter
• refactor-image-converters
• refactor-pipeline
• refactor-serialization
• refactor/restructure-eval-code
• remove-agent
• remove-harness
• remove-multimodality
• remove-pre-made-super-components
• remove-sample-components
• rename-max-loops-allowed
• replaced_refs
• restore-branch
• revert-resume-state
• revert/openai_tests
• rm-multifileconverter-deps
• rm-tool-chatmessage
• sanitize_functions
• sentence-transformers-image-embedder
• separate-integration-tests
• sjrl-patch-1
• skip_flaky_tests_ci
• skip_on_empty_secrets
• state-and-tools-feedback
• structured-output
• super-components-update
• super_component_decorator
• telemetry
• test_ci
• test_flakyness
• tool
• tool-from-fun-flexible
• tool-from-function
• tool-invoke-error
• tool-invoker
• tool-output-handling
• tool_openapi_integration
• tools_normalization
• topk_chat_retriever
• try-pypdfium2
• unpin-haystack
• update-bedrock
• update-example
• update-llmmetadataextract-prompt
• update-ollama
• update-strict-tools-usage
• update_openapi_tests

Build # 9189156861

Build Type

Pull #5

github

Committed by web-flow

Commit Message

Merge b3f329ad2 into d9454fc00

Pull Request Pull Request #5: feat: Implement abstract `EvaluationHarness` class

Run Details

20 of 38 new or added lines in 2 files covered. (52.63%)

279 of 298 relevant lines covered (93.62%)

0.94 hits per line