grobidOrg / grobid / 32760873987

44%

master: 44%

push

github

Add pdf.pdfalto.discardClippedText option (default off)

Figures embedded as PDF Form XObjects frequently carry the invisible,
clipped text of the manuscript page they were exported from. pdfalto emits
that text as ordinary words, so GROBID extracts whole paragraphs and
section headings two or three times: in a 2,595-article PMC materials-science
corpus, 60% of documents were affected (~9% of paragraphs duplicated); in a
1,943-article PMC life-science corpus, 7%.

pdfalto gains a -discardClippedText flag re-enabling xpdf's clip test
(kermitt2/pdfalto PR to follow); this change lets GROBID pass it, guarded by
a new config key pdf.pdfalto.discardClippedText. Off by default, since stock
pdfalto binaries do not know the flag; enabling it on a supporting build is
output-identical for documents without hidden clipped text.

10131 of 25700 branches covered (39.42%)

Branch coverage included in aggregate %.

21226 of 46371 relevant lines covered (45.77%)

1.86 hits per line