knowledge parent-child segmentation

[yanwencheng]

Self Checks

• I have read the Contributing Guide and Language Policy.
• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report, otherwise it will be closed.
• 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
• Please do not modify this template :) and fill in all the required fields.

Dify version

1.4.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

In the parent-child segmentation of the knowledge base, ### is used for child segment segmentation, but the parent block also contains field content. How should this be handled?

After import, the content of the parent block is similar to:

Parent block content ### Child segment 1 ### Child segment 2

✔️ Expected Behavior

In the parent-child segmentation of the knowledge base, ### is used for child segment segmentation, and the parent block should not contain any child segment content.

After import, the content of the parent block is similar to:

Parent block content ### Child segment 1 ### Child segment 2

❌ Actual Behavior

After import, the content of the parent block is similar to:

Parent block content ### Child segment 1 ### Child segment 2

[dosubot[bot]]

The current backend logic for parent-child segmentation in Dify uses your chosen delimiters (like ###) to split documents into parent and child segments. However, the parent chunk will still include all content within its segmented range, even if some of that content is also split out as child segments—this is intentional and not a bug source.

If you want the parent block to exclude child segment content after import, you would need to modify the backend segmentation processor to strip out child ranges from the parent chunk. This would involve changes to the splitting logic in the ParentChildIndexProcessor and related text splitter classes.

As a workaround, you can preprocess your input files to ensure parent and child content are clearly separated before import, or post-process the imported data to remove child content from parent blocks. The frontend configuration for delimiters and segmentation rules only affects how the backend splits the text, not the actual content included in each chunk.

If you want to customize this behavior, focus on the backend splitting algorithm after the initial parent and child chunking steps.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[yanwencheng]

@dosu Will this modification affect recall performance?

[dosubot[bot]]

Modifying the segmentation so that parent blocks do not contain child segment content will affect recall performance. If you strictly separate parent and child segments, queries will only match the relevant segment, which can improve precision and relevance. However, recall for queries that previously matched both parent and child content may decrease, especially if your queries rely on hierarchical context or expect overlapping information. This change helps avoid redundant or unintuitive results and makes retrieval more fine-grained, but you may need to adjust chunk size and overlap parameters to ensure important context isn't lost during segmentation. Dify's retrieval matches the content stored in the vector database, so how you split and index segments directly impacts recall performance source.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[dosubot[bot]]

Yes, modifying the segmentation logic so that parent blocks exclude child segment content will affect recall performance. Currently, parent chunks include all child content, which increases the chance that a query will retrieve relevant context from either the parent or child chunk. If you strip child content from parent blocks, recall may decrease for queries that rely on the broader context provided by the parent, since only the child chunks will contain that information. This makes retrieval more precise but can fragment context, so unless your retrieval strategy always fetches both parent and child chunks as needed, you may lose some recall and coverage for hierarchical documents. For maximum recall, it's generally beneficial for parent chunks to retain child content unless you have a specific use case for strict separation source.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}