grobidOrg / grobid
44%
master: 44%

LAST BUILD BRANCH: feature/delft-1.0.0-pytorch
DEFAULT BRANCH: master
Repo Added
Token teyP78tECFj7GsoLVxxuREI5tGJlxw8Ge regen
Build 569 Last
Files 338
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH fix/paragraph-split-after-callout
branch: fix/paragraph-split-after-callout
CHANGE BRANCH
x
Sync Branches

coverage: 43.545% (+0.03%) from 43.511%
29900893897

Pull #1517

github

lfoppiano 
fix: split paragraphs after a callout on layout, not the sequence label

Re-addresses the real #1482 case (a paragraph that genuinely ends at a
trailing callout) on top of the clean 0.9.0 base restored by the revert,
without the isBeginning() predicate that made the previous attempt split
at nearly every callout.

The decision comes from the layout of the callout and the resumed text:

- Same line as the callout -> continue. The common case (a callout
  inside a paragraph), and it needs no further evidence.
- A line break alone proves nothing, since a paragraph interrupted near
  the right margin also wraps, so a break must also cross a block
  boundary - blocks being the unit pdfalto groups paragraphs into.
- A block boundary also falls inside a paragraph, so the resumed text
  must additionally read like the start of one (upper case or digit).
  This stops continuations, and the punctuation closing the interrupted
  sentence, from being split off.
- Text resuming higher up the page has wrapped to the top of the next
  column, and a page change makes the coordinates incomparable. Neither
  is evidence of a paragraph break.

Splitting requires positive evidence; anything unknown continues the
paragraph, so the worst case degrades to 0.9.0 rather than a new failure
mode. Only two overloads: the layout-aware one, and the 0.9.0-compatible
2-arg used by Table and external callers.

Two details, both found only by instrumenting. LayoutToken
.isNewLineAfter() cannot detect the line break: it is only populated by
enrichWithNewLineInfo(), which the fulltext path never calls, so it is
always false here - the break is read from the Y coordinates. And the
token before a cluster boundary is very often a synthesised space with
no coordinates (x = y = -1), so the comparison skips back to the last
positioned token or every callout looks like a line break.

Full corpus, 11,945 documents vs 0.9.0: <p> 1,170,384 -> 1,177,912
(+0.64%, master +85.5%); paragraphs opening ... (continued)
Pull Request #1517: Fix regression in split paragraphs after a callout on layout

10152 of 25716 branches covered (39.48%)

Branch coverage included in aggregate %.

21245 of 46387 relevant lines covered (45.8%)

1.86 hits per line

Relevant lines Covered
46387 RELEVANT LINES 21245 COVERED LINES
1.86 HITS PER LINE
Source Files on fix/paragraph-split-after-callout

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
29900893897 fix/paragraph-split-after-callout fix: split paragraphs after a callout on layout, not the sequence label Re-addresses the real #1482 case (a paragraph that genuinely ends at a trailing callout) on top of the clean 0.9.0 base restored by the revert, without the isBeginning() pred... Pull #1517 lfoppiano github
43.54
29872286305 fix/paragraph-split-after-callout chore: spotless Signed-off-by: Luca Foppiano <luca@foppiano.org> Pull #1517 lfoppiano github
43.12
See All Builds (569)

Badge your Repo: grobid

We detected this repo isn’t badged! Grab the embed code to the right, add it to your repo to show off your code coverage, and when the badge is live hit the refresh button to remove this message.
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

Refresh