ml6team / fondant / 7183908091

91%

push

github

Fix data io for generic PandasTransformComponent (#721)

I ran into an issue when using the generic `evaluate_ragas` component. 

```Python
component = RetrieverEval(
    module="langchain.llms",
    llm_name="OpenAI",
    llm_kwargs={"openai_api_key": os.environ["OPENAI_KEY"]},
    produces={
        "context_precision": pa.float32(),
        "context_relevancy": pa.float32(),
    },
)
```
Lead to the following error:
```
ValueError: Fields ['context_precision', 'context_relevancy'] defined in output dataset but not found in dataframe
```

Because we were not using the `operation_spec` yet for the
`PandasTransformComponent`. I think we can reduce the chance of similar
errors in the future by only passing the operation_spec to the
component, but then we'll need to include the other information from the
component spec in there as well.

4 of 4 new or added lines in 1 file covered. (100.0%)

1693 of 1837 relevant lines covered (92.16%)

2.76 hits per line