8030025822
91%

Ran 24 Feb 2024 10:49AM UTC

Jobs 3

Files 60

Run time 3s

Badge

Embed ▾

Committed 24 Feb 2024 10:45AM UTC coverage: 91.43% (-0.004%) from 91.434%

Build # 8030025822

Build Type

push

github

Committed by

web-flow

Commit Message

Don't write metadata file (#875)

We reintroduced writing the metadata file in #864 to preserve the
divisions of the data when writing and reading again. We turned this
behavior off in the past, but without proper documentation of the
reason.

I'm now running into issues with Dask workers dying when writing large
datasets though, presumably because of the metadata file, as documented
in these Dask issues:
- https://github.com/dask/dask/issues/6600
- https://github.com/dask/dask/issues/3873
- https://github.com/dask/dask/issues/8901

Also, while I ran into issues with the preservation of divisions before,
I can't reproduce this locally with a small example. Let's turn writing
metadata off again and validate if we are still having issues with this.

Run Details

6049 of 6616 relevant lines covered (91.43%)

0.91 hits per line

Jobs

ID	Job ID	Ran	Coverage
1	test-3.11 - 8030025822.1	24 Feb 2024 10:52AM UTC	91.43
2	test-3.10 - 8030025822.2	24 Feb 2024 10:52AM UTC	91.43
3	test-3.9 - 8030025822.3	24 Feb 2024 10:52AM UTC	91.42

Source Files on build 8030025822

Detailed source file information is not available for this build.

ml6team / fondant / 8030025822
91%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 8030025822

ml6team / fondant / 8030025822 91%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 8030025822

ml6team / fondant / 8030025822
91%

README BADGES
x