Skip to content

Fixing DB bloat caused by long stdout.log on many evaluations#140

Merged
RobertTLange merged 6 commits into
SakanaAI:mainfrom
marcopirazzini:webUI_stdout
Jun 25, 2026
Merged

Fixing DB bloat caused by long stdout.log on many evaluations#140
RobertTLange merged 6 commits into
SakanaAI:mainfrom
marcopirazzini:webUI_stdout

Conversation

@marcopirazzini

Copy link
Copy Markdown
Contributor

Summary

  • (1) Added a verbosity option to JobConfig which can suppress printing in evaluation number run_shinka_eval
  • (2) Extended the option to set the numerical computation to single threaded to SLURM

Why

  • (1) When ShinkaEvolve evaluates a program it saves two strings to stdout, which is then also stored both as plain text and in the program database as metadata. If each program is evaluated many times over many iterations, this can generate GBs of useless data.
  • (1) When the database becomes too bloated, the WebUI is very slow or completely fails to load it
  • (2) This was currently only available to local jobs

@RobertTLange

Copy link
Copy Markdown
Collaborator

Pushed a follow-up commit on top of this PR: 0f12639 fix: honor quiet eval result saves.

What changed:

  • Preserves the default behavior: save_json_results() still prints save messages by default.
  • When run_shinka_eval(..., verbose=False) or SHINKA_EVAL_VERBOSE=0 resolves quiet mode, the final result-save stdout lines are now suppressed too.
  • Added regression tests for both default-output and quiet-output behavior.

Local verification:

  • UV_PROJECT_ENVIRONMENT=/tmp/shinka-pr140-venv uv run pytest tests/test_wrap_eval_parallel.py -q -> 6 passed
  • UV_PROJECT_ENVIRONMENT=/tmp/shinka-pr140-venv uv run pytest -q -> 477 passed, 5 warnings
  • UV_PROJECT_ENVIRONMENT=/tmp/shinka-pr140-venv uv run ruff check shinka/core/wrap_eval.py tests/test_wrap_eval_parallel.py -> passed

Notes: full repo ruff check . and mypy shinka are still red on unrelated pre-existing issues outside this patch.

@RobertTLange

Copy link
Copy Markdown
Collaborator

Pushed another follow-up commit: 9c7a0c7 ci: remove claude code review workflow.

Updates:

  • Removed the automatic Claude Code Review pull-request workflow (kept the separate comment-triggered Claude Code workflow intact).
  • Added a TBD changelog entry covering the stdout bloat controls, SLURM numeric thread cap support, quiet eval result-save fix, and removal of the automatic Claude Code Review workflow.

Validation:

  • Confirmed only .github/workflows/claude-code-review.yml was deleted; .github/workflows/claude.yml remains.
  • git diff --check -- CHANGELOG.md .github/workflows/claude-code-review.yml passed.
  • Changelog/deletion sanity check passed.

@RobertTLange RobertTLange merged commit 85f174b into SakanaAI:main Jun 25, 2026
1 check passed
@marcopirazzini marcopirazzini deleted the webUI_stdout branch June 26, 2026 01:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants