webui : put DeepSeek R1 CoT in a collapsible <details> element #11364
Conversation
|
I'm not sure why the CI is failing, Could be due to a difference in the gzip versions used between my computer and the CI? |
There was a problem hiding this comment.
The idea is ok but I don't really like the implementation. Instead of modifying the shape of pendingMsg, you should make a new function that transform content into pair of cot, content and call it from html vue code.
In other words, in a MVC model, what you're changing is M while you can just do it in V with functional programming approach.
I agree that the implementation is a bit hacky for now, I'll try to improve it. |
|
What I added:
Demo: Screen.Recording.2025-01-24.at.00.17.28.movCC @ggerganov for your info |
|
Merging this once the CI pass |
|
Cool! |
…nt (ggml-org#11364) * webui : put DeepSeek R1 CoT in a collapsible <details> element * webui: refactor split * webui: don't use regex to split cot and response * webui: format+qol * webui: no loading icon if the model isn't generating * ui fix, add configs * add jsdoc types * only filter </think> for assistant msg * build * update build --------- Co-authored-by: Xuan Son Nguyen <[email protected]>
| let thinkSplit = content.split('<think>', 2); | ||
| actualContent += thinkSplit[0]; | ||
| while (thinkSplit[1] !== undefined) { | ||
| // <think> tag found | ||
| thinkSplit = thinkSplit[1].split('</think>', 2); | ||
| cot += thinkSplit[0]; | ||
| isThinking = true; | ||
| if (thinkSplit[1] !== undefined) { | ||
| // </think> closing tag found | ||
| isThinking = false; | ||
| thinkSplit = thinkSplit[1].split('<think>', 2); | ||
| actualContent += thinkSplit[0]; | ||
| } | ||
| } |
There was a problem hiding this comment.
I just found out I was mistaken about the behavior of javascript's String.prototype.split() with a limit. I assumed it would keep all the content including the separators in the last element but I was wrong.
This shouldn't cause problems for R1 models since it typically uses the chain of thought only once per reply, but if, for example, the user specifies in the system prompt "You can hide your thoughts to the user by surrounding them with <think> and </think>. You can use it multiple times per answer.", then everything after the second <think> is never going to be displayed.
…nt (ggml-org#11364) * webui : put DeepSeek R1 CoT in a collapsible <details> element * webui: refactor split * webui: don't use regex to split cot and response * webui: format+qol * webui: no loading icon if the model isn't generating * ui fix, add configs * add jsdoc types * only filter </think> for assistant msg * build * update build --------- Co-authored-by: Xuan Son Nguyen <[email protected]>
…nt (ggml-org#11364) * webui : put DeepSeek R1 CoT in a collapsible <details> element * webui: refactor split * webui: don't use regex to split cot and response * webui: format+qol * webui: no loading icon if the model isn't generating * ui fix, add configs * add jsdoc types * only filter </think> for assistant msg * build * update build --------- Co-authored-by: Xuan Son Nguyen <[email protected]>
…nt (ggml-org#11364) * webui : put DeepSeek R1 CoT in a collapsible <details> element * webui: refactor split * webui: don't use regex to split cot and response * webui: format+qol * webui: no loading icon if the model isn't generating * ui fix, add configs * add jsdoc types * only filter </think> for assistant msg * build * update build --------- Co-authored-by: Xuan Son Nguyen <[email protected]>
This hides the
<think>and</think>tags from the user, and moves the content beetween these tags to a collipsable element that closes itself for compactness once the model is done generating the answer.This only works for R1 tags, other reasoning models lik QwQ will not generate those tags by themselves.
It could be interesting to use this work as a basis to remove the previous chains of thoughts from the prompts during multi-turn conversations, as suggested here: #11325 (comment)