v1.7.2 — Windows install fix, MTP load + RAM regression, auto-tune MTP#46
Merged
Conversation
…to-tune MTP - install: require Node >=22.13.0 so node:sqlite loads without the --experimental-sqlite flag (below that, `npx turbollm` crashes with ERR_UNKNOWN_BUILTIN_MODULE despite `node -v` reporting 22.x). Guard in bin/turbollm.mjs + cli.ts, engines field, README. (#40) - engines: probe no longer treats llama.cpp's "argument has been removed" --help notices as supported flags, so --draft-max/--draft-min are no longer passed to a build that rejects them (crash on every MTP load). Falls back to --spec-draft-n-max/--spec-draft-n-min when only those are present. ensureProbed auto-reprobes engines carrying the stale flags so an app upgrade fixes already-registered engines without a manual reprobe. (#43) - mtp: native NextN/MTP (--spec-type draft-mtp) no longer passes --model-draft pointing at the main model file. That made llama.cpp load a full second copy of the weights into RAM (measured +35GB on a 35B MoE, and 54% slower generation) with no error. Native MTP now uses ~460MB extra and runs faster, as intended. The fork's own `nextn` spec-type still gets --model-draft. - bench: auto-tune now tunes with the user's speculative setting active instead of forcing it off, so the offload/KV fit reserves room for MTP's real footprint and the winning profile preserves the user's MTP choice instead of silently resetting it to off on save.
The Developer → Connect a CLI section still showed only the old manual env-var / config-file setup and was missing openclaw and hermes entirely, even though `turbollm launch <cli>` already supports them. - add openclaw and hermes cards (were hitting the default branch → empty steps) - lead opencode/kilo/openclaw/hermes with the one-command `turbollm launch <cli>` step, manual config kept as a fallback - thread the model key through so openclaw/hermes snippets key the provider by the stable model key, matching what cli-launch.ts actually writes
A real model key contains spaces and | (e.g. 'name|Q5_K_M|123'); unquoted in the copy-paste hermes snippet it would split into multiple args / a pipe in the shell. The actual 'turbollm launch hermes' path was already safe (spawn without a shell). Flagged by the v1.7.2 pre-release review (M1).
mohitsoni48
force-pushed
the
release/v1.7.2
branch
from
ddc46c0 to
4e9c1c4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bug-fix release addressing two reported issues (#40, #43) plus MTP correctness/performance and a Developer-screen catch-up.
Fixes
ERR_UNKNOWN_BUILTIN_MODULE: node:sqlite): require Node >=22.13.0, the version wherenode:sqlitebecame available without--experimental-sqlite. The old guard only checkedmajor < 22, so 22.11.0 passed and then crashed on import.--helpnotice as proof that--draft-max/--draft-minwere supported, then passed them to a build that rejects them → engine exits on every MTP load. Probe now skips removed-flag notices and falls back to--spec-draft-n-max/--spec-draft-n-min.ensureProbedauto-reprobes already-registered engines carrying the stale flags.--spec-type draft-mtp) no longer passes--model-draftpointing at the main model file — that made llama.cpp load a second full copy of the weights into RAM (measured +35 GB on a 35B MoE, 54% slower gen), silently. Native MTP now uses ~460 MB extra and runs faster.Feature
openclawandhermescards and lead every launch-capable CLI with the one-commandturbollm launch <cli>flow (manual config kept as fallback).Tests
New:
probe.test.ts,registry.staleness.test.ts,connect-snippets.test.ts, plus nextn/draft-mtp cases inprofile.gen.test.ts. Full suite green (651 pass), typecheck clean. MTP load, RAM footprint, and auto-tune-preserves-MTP verified live on an RTX 5070 Ti with a Qwen3.6-35B-A3B MTP model.