Skip to content

fix(serializer): resolve page-break markers when serializing a single node - #715

Open
Anai-Guo wants to merge 1 commit into
docling-project:mainfrom
Anai-Guo:fix-page-break-marker-leak
Open

fix(serializer): resolve page-break markers when serializing a single node#715
Anai-Guo wants to merge 1 commit into
docling-project:mainfrom
Anai-Guo:fix-page-break-marker-leak

Conversation

@Anai-Guo

@Anai-Guo Anai-Guo commented Aug 7, 2026

Copy link
Copy Markdown

Fixes #714.

The bug

DocSerializer.serialize() emits an internal sentinel for each page break
(#_#_DOCLING_DOC_PAGE_BREAK_<prev>_<next>_#_#, common.py:481), and every serializer replaces it in
serialize_doc(). But serialize_doc() is only reached from the whole-document path, so a page break
nested inside a group leaked the raw sentinel out of the single-node entry point — and into
HierarchicalChunker, which builds chunk.text from serialize(item=...):

whole document: '- item one\n<!-- page break -->\n- item two'
single group  : '- item one\n#_#_DOCLING_DOC_PAGE_BREAK_1_2_#_#\n- item two'   # before
chunk         : '- item one\n#_#_DOCLING_DOC_PAGE_BREAK_1_2_#_#\n- item two'   # before

The fix

A new DocSerializer._replace_page_breaks() hook, called from serialize() as well. The base
implementation returns the text unchanged; Markdown, DocTags and LaTeX override it with the
context-free replacement they already performed in serialize_doc(), which now just calls the hook.

Two details keep the existing behaviour intact:

  • HTML is deliberately left on the base no-op. HTMLOutputStyle.SPLIT_PAGE needs prev_page /
    next_page and each marker's offset to slice the body into pages, so it can only resolve them once
    the whole document is available. Its sentinel survives to serialize_doc() exactly as before.
  • serialize() skips the substitution when the node being serialized is itself the page break.
    Markers between two top-level items therefore still reach serialize_doc() as their own part. That
    matters for page_break_placeholder="": resolving those markers early would make them empty parts,
    which the if p.text join filters drop, silently removing the blank separators that
    test_md_cross_page_list_page_break_empty pins down.

LaTeX's serialize_doc() resolves markers from self.params.merge_with_patch(patch=kwargs) while
requires_page_break() reads self.params; the override follows requires_page_break() and uses
self.params. A per-call page_break_command override is already only half-honoured today (whether
breaks are emitted at all is decided from self.params), so I left that as-is rather than change it here.

Verification

  • Three regression tests in test/test_serialization.py (Markdown, DocTags, LaTeX) serialize
    #/groups/2 of activities.json — a list spanning pages 1–2 — on its own. All three fail on main
    with the raw sentinel and pass here.
  • No whole-document output changes: I exported all 38 documents under test/data/doc/ as Markdown
    (with page_break_placeholder set and empty), DocTags, LaTeX and HTML SPLIT_PAGE, before and
    after. All 190 outputs are byte-identical.
  • pytest test/ — same 9 pre-existing failures as main in my environment (test_doctags_load,
    test_docling_doc; unrelated to serialization), 372 → 375 passed, the 3 being the new tests.
  • ruff check and ruff format --check clean with the repo config.

🤖 Generated with Claude Code

… node

The internal `#_#_DOCLING_DOC_PAGE_BREAK_<prev>_<next>_#_#` sentinel is
created in `DocSerializer.serialize()` but was only ever replaced in
`serialize_doc()`, which the single-node path never reaches. A page break
nested inside a group therefore leaked the raw sentinel out of
`serializer.serialize(item=...)` and into `HierarchicalChunker` chunk text.

Resolve the markers in `DocSerializer.serialize()` too, via a new
`_replace_page_breaks()` hook that the Markdown, DocTags and LaTeX
serializers override with their existing context-free replacement. The
hook is a no-op in the base class, so HTML `SPLIT_PAGE` -- which needs the
page numbers and the marker offsets, and can only resolve them once the
whole document is available -- keeps the sentinel as before.

Markers that sit between two top-level items are still resolved by
`serialize_doc()`: `serialize()` skips the substitution when the node
being serialized is itself the page break, so an empty
`page_break_placeholder` keeps producing the same separators it does today.

Fixes docling-project#714

Signed-off-by: Tai An <antai12232931@outlook.com>
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

DCO Check Passed

Thanks @Anai-Guo, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🔴 1 of 2 protections blocking · waiting on 👀 reviews

Protection Waiting on
🔴 Require two reviewer for test updates 👀 reviews
🟢 Enforce conventional commit

🔴 Require two reviewer for test updates

Waiting for

  • #approved-reviews-by >= 2
This rule is failing.

When test data is updated, we require two reviewers

  • #approved-reviews-by >= 2

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Page-break marker leaks the internal sentinel when a single node is serialized (and into chunk text)

1 participant