Skip to content

fix(doctags): accept the paragraph tag when loading DocTags - #716

Open
manticore999 wants to merge 1 commit into
docling-project:mainfrom
manticore999:fix/doctags-paragraph-roundtrip
Open

fix(doctags): accept the paragraph tag when loading DocTags#716
manticore999 wants to merge 1 commit into
docling-project:mainfrom
manticore999:fix/doctags-paragraph-roundtrip

Conversation

@manticore999

Copy link
Copy Markdown
Contributor

Problem

export_to_doctags() emits <paragraph> for items labelled PARAGRAPH, but the
tag is missing from the pattern load_from_doctags() matches against, so those
elements are skipped and their text is silently dropped. Export and import
disagree about the format.

PARAGRAPH is produced by the PPTX, JATS, USPTO, AsciiDoc and LaTeX backends, so
this affects ordinary documents. For a PowerPoint deck, every slide body is lost
and only the titles survive:

from docling.document_converter import DocumentConverter
from docling_core.types.doc import DoclingDocument
from docling_core.types.doc.document import DocTagsDocument

doc = DocumentConverter().convert("deck.pptx").document
print(doc.export_to_text())
# Quarterly Report
# Revenue grew by twelve percent this quarter.
# Costs remained flat across all regions.

dt = doc.export_to_doctags()          # contains <paragraph>Revenue grew ...</paragraph>
back = DoclingDocument.load_from_doctags(
    DocTagsDocument.from_doctags_and_image_pairs([dt], [None]), document_name="d")
print(back.export_to_text())
# Quarterly Report          <- slide text is gone

Minimal version, no files needed:

doc = DoclingDocument(name="x")
doc.add_text(label=DocItemLabel.PARAGRAPH, text="PARA")
doc.add_text(label=DocItemLabel.TEXT, text="TEXT")
# round-trip through DocTags -> "TEXT"   ("PARA" dropped)

Fix

Add the tag to the parser's pattern and map it to DocItemLabel.PARAGRAPH, so
both the content and the label survive.

Impact

Measured over the 192 groundtruth DoclingDocuments in the docling repo,
comparing text length before and after a DocTags round-trip:

documents losing >10% of text
before 29
after 17

The six worst cases (patents ipa20200022300, pftaps057006474, pg06442728,
pa20010031492, ipa20180000016, and powerpoint_malformed_pictures) returned
0–3% of their text and now round-trip correctly. The remaining 17 lose text for
unrelated reasons (drawingml, some tables, checkboxes) and are untouched here.

Tests

Two tests in test/test_doctags_load.py, covering the plain and <loc_...>
forms. Both fail without the change and pass with it; the existing doctags tests
are unaffected (12 passed).

Checklist:

  • Documentation has been updated, if necessary.
  • Examples have been added, if necessary.
  • Tests have been added, if necessary.

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

DCO Check Passed

Thanks @manticore999, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🔴 1 of 2 protections blocking · waiting on 👀 reviews

Protection Waiting on
🔴 Require two reviewer for test updates 👀 reviews
🟢 Enforce conventional commit

🔴 Require two reviewer for test updates

Waiting for

  • #approved-reviews-by >= 2
This rule is failing.

When test data is updated, we require two reviewers

  • #approved-reviews-by >= 2

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@manticore999
manticore999 force-pushed the fix/doctags-paragraph-roundtrip branch from 809dae7 to a24a3fe Compare August 10, 2026 16:52
@manticore999

Copy link
Copy Markdown
Contributor Author

Updated: widened from <paragraph> to also cover <handwritten_text> and <reference>.
All three are emitted by export_to_doctags but were missing from the parser's tag
pattern, so they hit the same silent-drop path. REFERENCE is produced by the LaTeX
backend; PARAGRAPH by PPTX, JATS, USPTO, AsciiDoc and LaTeX.

Tests are now parametrized over the three labels and fail without the change.

`export_to_doctags` emits `<paragraph>`, `<handwritten_text>` and `<reference>`
for items carrying those labels, but the pattern `load_from_doctags` matches
against did not list them. The elements were skipped and their text silently
dropped, so export and import disagreed about the format.

PARAGRAPH is produced by the PPTX, JATS, USPTO, AsciiDoc and LaTeX backends and
REFERENCE by the LaTeX backend, so ordinary documents are affected. For a
PowerPoint deck every slide body was lost on a round-trip and only the titles
survived.

Add the three tags to the pattern and map them onto their labels so the label is
preserved as well.

Signed-off-by: manticore999 <hamzabadreddine852@gmail.com>
@manticore999
manticore999 force-pushed the fix/doctags-paragraph-roundtrip branch from a24a3fe to d027b01 Compare August 10, 2026 17:25
@codecov

codecov Bot commented Aug 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@ceberam
ceberam requested a review from vagenas August 17, 2026 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant