deepset-ai / haystack-experimental

98%

main: 60%

LAST BUILD ON BRANCH fix/rag-eval-harness-bugs

branch: fix/rag-eval-harness-bugs

CHANGE BRANCH

x

Reset

Sync Branches

• fix/rag-eval-harness-bugs
• LLMMetadataExtractor-use-ChatGenerator
• PR-comments
• add-metadata-extractor
• add-normalize-emb-image-emb
• add-parent-span
• add-pre-commit-hooks
• adding-agent-code-from-haystack-main
• adding-forward-references-llm-metadataextractor
• adding-pipeline-breakpoints
• adding-pipeline-for-easy-PR-review
• adopt-uv
• agent-deserialize-tool-with-utility-function
• agent-mandatory-messages
• agent-memory
• agent-refactor-proposal
• agent-usage-example
• agents-update
• align-mypy-haystack
• anthropic_tools
• async-experiments
• auto-merging-retriever
• auto-merging-retriever-update
• bedrock-component-info-update
• bedrock-multimodal
• build/supply-chain-guardrails
• change-default-model-generators
• changed-ds-deser
• chat_message_store
• chat_msg_retriever
• chat_msg_store
• chat_msg_writer
• chatmessage-retriever-writer-adaptations
• chatmessages-for-tools
• chatpromptbuilder-multimodal
• check-api-ref
• chore/clean-up-LLMMetadaExtractor
• chore/clean-up-auto-merging-retriever
• chore/clean-up-inmemory-doc-store-async
• chore/pin-actions-to-commit-sha
• chore/prepare_agent_pr
• chore/remove-discontinued-experiments
• chore/remove-pipeline-asyncpipeline
• chore/removing-EmbedderBasedDocumentsplitter
• chore/removing-QueryExpander-MultiQueryRetriever
• chore/removing-breakpoints
• ci/release-dev-version
• ci/setup-tests
• ci_pip_upgrade
• claude/e2b-sandbox-integration-3cWWo
• clean/remove-agent-related-code
• cleanup
• david/fix-hybrid-pipeline
• david/fixing-for-more-pipelines
• david/pipeline-testing
• david/remove-sender-in-serialisation
• dependabot/github_actions/actions/checkout-5
• dependabot/github_actions/actions/checkout-6
• dependabot/github_actions/actions/checkout-6.0.2
• dependabot/github_actions/actions/checkout-6.0.3
• dependabot/github_actions/actions/checkout-7.0.0
• dependabot/github_actions/actions/checkout-7.0.1
• dependabot/github_actions/actions/setup-python-6
• dependabot/github_actions/actions/setup-python-6.3.0
• dependabot/github_actions/actions/setup-python-7.0.0
• dependabot/github_actions/aws-actions/configure-aws-credentials-6.1.0
• dependabot/github_actions/aws-actions/configure-aws-credentials-6.1.2
• dependabot/github_actions/aws-actions/configure-aws-credentials-6.1.3
• dependabot/github_actions/aws-actions/configure-aws-credentials-6.2.0
• dependabot/github_actions/aws-actions/configure-aws-credentials-6.2.1
• dependabot/github_actions/aws-actions/configure-aws-credentials-6.2.2
• dependabot/github_actions/aws-actions/configure-aws-credentials-6.2.3
• dependabot/github_actions/coverallsapp/github-action-2.3.8
• dependabot/github_actions/deepset-ai/notify-slack-action-a65def0c8bf91d6520286ab34280151c76a5a008
• dependabot/github_actions/tj-actions/changed-files-47
• dependabot/github_actions/tj-actions/changed-files-47.0.5
• dependabot/github_actions/tj-actions/changed-files-47.0.6
• dependencies-tight-up
• deserialize-tools-inplace
• disable_firecrawl
• disable_meteo
• doc-length-router
• doc-to-image-content
• doc-to-image-content-store-file-path-as-string
• doc-to-image-fmt
• docstrings-fix
• document-captioner
• document-type-router
• docusaurus-api-ref-sync
• drop-39
• drop-pylint
• embedding-based-document-splitter
• embedding-based-document-splitter-new
• enhance-tool-logs
• explicit_param
• feat/add-regex-extractor
• feat/add-summarizer
• feat/agent-breakpoints
• feat/agent_component
• feat/agent_examples
• feat/agent_streaming_llm
• feat/async-chat-prompt-builder
• feat/async-inmemory-retrievers
• feat/async_pipeline
• feat/breakpoints-structure
• feat/eval-harness-base-class
• feat/eval-harness-pipeline-pair
• feat/eval-harness-rag
• feat/function-caller
• feat/multiquery-retriever
• feat/pipeline-checkpoints
• feat/rag-eval-harness-ctx-relevance-metric
• feat/rag-eval-harness-gt-inputs-in-results
• feat/rag-eval-harness-notebook
• feat/ready_made_supercomponents
• feat/remove-RegexTextExtractor
• feat/summariser-based-on-LLM
• feat/supercomponent_base
• feat/supercomponents
• feature/bui-198-provide-chatpromptbuilder-capability-to-agent-component
• feature/md-header-infer
• file-to-image-content
• fix-agent-check
• fix-bedrock-test
• fix-breakpoints
• fix-bug-in-resuming-inputs
• fix-component-tool
• fix-confirmation-strategies
• fix-deserialization-errors
• fix-exeperimental-tests
• fix-hitl-breakpoint
• fix-init-params-in-state
• fix-json-dumps
• fix-linting
• fix-linting-error
• fix-llmmetadata-extractor
• fix-logger-message
• fix-md-header-level-inferrer
• fix-openai-empty-content
• fix-typing-issue-agent
• fix-validate-input
• fix/adding-mocks-to-tests
• fix/adding-pandas-as-dependency
• fix/auto-merging-retriever-tests
• fix/component_tool_deserialization_in_agent
• fix/deepcopy_component_tool
• fix/document-stores
• fix/haystack-serialization-fix
• fix/init-imports
• fix/md-inferrer-skipped-header-level
• fix/naming
• fix/pipeline
• fix/pipeline_run
• fix/pyproject-toml
• fix/typo-example
• fix_anthropic_format
• fix_naming
• fix_tests
• fixing-breakpoints-tests
• fixing-breakpoints-tests-file-name-change
• fixing-hierarchical-splitter
• fixing-init-and-test-imports
• fixing/windows-tests-failing
• fmt-more-forgiving
• from_component
• graph_genereation
• hallucination-risk-calc
• hatch-scripts-unify
• hfapi-tools
• hitl-alternative
• hitl-with-breakpoints
• human-in-the-loop
• image-content
• image-embedder-experiment
• image-show
• image-to-document-converter
• image-validation
• imagecontent-convenience-class-methods
• improve-multimodal-docstrings
• jinja-chatmessage-extension
• llmmetadata-extractor
• main
• make-LLM-based-evaluators-raise_on_failure_False
• make-agent-compatible-with-haystack
• mem0-integration
• mem0-v3
• metadata-extractor-update
• migrate-hitl
• more-cleanup
• multi-file-converter-unclassified
• multimodal-add-usage-examples
• mv-docs
• mv-imagecontent
• mv-ipython-import
• notify-slack
• ollama-pydantic-fix
• ollama-tools
• openai-check-tool-names
• openai-multimodal
• openaichatgenerator-with-tools
• openapi
• openapi_corner_cases_handling
• openapi_filter
• openapitool_error_handling
• patch-api-keys-agent
• pep-585
• pin-transformers
• pipe_tool
• pipeline-wrapper
• py39
• pylint-consistency
• query-expander
• quiet-repo-for-archive
• raise-error-for-async-pipeline
• readme-cleanup
• refactor-doc-to-img
• refactor-emb-base-splitter
• refactor-image-converters
• refactor-pipeline
• refactor-serialization
• refactor/restructure-eval-code
• remove-agent
• remove-doc-indexer-super-component
• remove-harness
• remove-mem0
• remove-multimodality
• remove-pre-made-super-components
• remove-sample-components
• rename-max-loops-allowed
• replace-claude-3.5-sonnet
• replaced_refs
• restore-branch
• revert-resume-state
• revert/openai_tests
• rm-multifileconverter-deps
• rm-tool-chatmessage
• run_context_and_async_conf_strategies
• sanitize_functions
• sentence-transformers-image-embedder
• separate-integration-tests
• simplify-pydoc
• sjrl-patch-1
• skip_flaky_tests_ci
• skip_on_empty_secrets
• small-improvments-based-on-feedback
• state-and-tools-feedback
• structured-output
• summarizer-test-monkeypatch
• super-components-update
• super_component_decorator
• sync-agent-with-haystack-main
• telemetry
• test-310
• test_ci
• test_flakyness
• tool
• tool-from-fun-flexible
• tool-from-function
• tool-invoke-error
• tool-invoker
• tool-output-handling
• tool_openapi_integration
• tools_normalization
• topk_chat_retriever
• try-pypdfium2
• unpin-haystack
• update-agent-serialization
• update-bedrock
• update-chat-message-store
• update-example
• update-hatch
• update-hitl
• update-llmmetadataextract-prompt
• update-ollama
• update-protocol
• update-query-expander-comps
• update-strict-tools-usage
• update-transfomers-dep
• update_openapi_tests
• use-deepcopy-with-exceptions

Build # 9513555861

Build Type

Pull #13

github

Committed by web-flow

Commit Message

Merge 59d5ac8e6 into c5f3ab00c

Pull Request Pull Request #13: fix: Correct parameter name for faithfulness evaluator and fix component names passed to `PipelinePair`

Coverage Stats

20 of 22 new or added lines in 1 file covered. (90.91%)

10 existing lines in 1 file now uncovered.

612 of 626 relevant lines covered (97.76%)

0.98 hits per line