[Evals] Fix issue with reading from system prompt in file#104
Merged
[Evals] Fix issue with reading from system prompt in file#104
Conversation
Signed-off-by: SumanthRH <sumanthrh@anyscale.com>
erictang000
approved these changes
Mar 31, 2025
SumanthRH
added a commit
that referenced
this pull request
Apr 17, 2025
yllkryeziu
pushed a commit
to yllkryeziu/adaptive-compute-rewrite
that referenced
this pull request
Dec 12, 2025
…#104) `model_config.system_prompt` can be a string or a `StringInFile` object...Before passing it to the task handler, we need to read the string from the file/ get the underlying `_string` atttribute from StringInFile
yllkryeziu
pushed a commit
to yllkryeziu/adaptive-compute-rewrite
that referenced
this pull request
Dec 12, 2025
# What does this PR do? Fixes an issue with null system prompt introduced in NovaSky-AI#104
lru0612
pushed a commit
to lru0612/SkyThought
that referenced
this pull request
Feb 24, 2026
…ult behavior on thinking tokens accumulation (#178)
In `ppo_base_config`, add `generator.chat_template` to allow users to
define custom chat template. This is particularly helpful for Qwen3
models, where you might want to keep the thinking tokens or remove them.
```.yaml
chat_template:
source: "name" # "name" or "file"
name_or_path: null # e.g., "qwen3_with_thinking" or "/path/to/template.j2"
```
You can either specify a chat template defined in
`skyrl_train.generators.utils.CUSTOM_CHAT_TEMPLATES` (we currently have
`qwen3_with_thinking` and `qwen3_without_thinking`) or load from a file.
Before this PR, the behavior for Qwen3 models are that, they will not
keep the non-last turn thinking tokens and we always retokenize (every
turn and at the end of the trajectory).
After this PR, Qwen3 models by default will accumulate in a TI/TO
manner, just like any other models. Users can modify this behavior by
setting `name_or_path` to `qwen3_without_thinking` to keep the old
behavior.
Related to #179 and address NovaSky-AI#104
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Tyler Griggs <tgriggs@berkeley.edu>
Co-authored-by: Charlie Ruan <charlieruan@berkeley.edu>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes an issue when reading from system prompt in file:
The error itself is hard to read but the root cause is that
model_config.system_promptcan be a string or a StringInFile object...Before passing it to the task handler, we need to read the string from the file/ get the underlying_stringatttribute fromStringInFile.