Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 10 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,8 +177,8 @@ the detail:
- **Saved per-model profiles, per engine** — tune once per (model, engine) pair, so
switching engines (or between two installs of the same engine, e.g. a fork) never
overwrites another engine's tuning for the same model.
- **Configurable VRAM headroom** (Settings → Engine, 300 MB–2 GB, default 1 GB) — tell auto-tune
how much VRAM to keep free for other GPU workloads instead of a fixed margin.
- **Configurable VRAM headroom** (Settings → Models & loading → Advanced, 300 MB–2 GB, default
1 GB) — tell auto-tune how much VRAM to keep free for other GPU workloads instead of a fixed margin.

</details>

Expand Down Expand Up @@ -216,7 +216,7 @@ the detail:
in every chat with no restart. A **Research** persona forces multi-step web search and cites sources inline.
- **Tool-call approval gate** — every tool call asks for your approval by default before it runs,
with **Deny**, **Allow**, **Allow for this chat**, or **Always Allow** on an inline bar above the
composer. Set per-tool defaults globally from DeveloperTool permissions.
composer. Set per-tool defaults globally from SettingsTools & safety.

</details>

Expand Down Expand Up @@ -255,7 +255,7 @@ curl http://127.0.0.1:6996/v1/chat/completions \
- **Anthropic-compatible** `/v1/messages` — including **tool use and streaming** — which powers
Claude Code below. No other local host offers this.
- **Structured output** — constrain any response to a **GBNF grammar** (or JSON shape).
- **API-key auth** you can require when sharing over a LAN (Settings → Network).
- **API-key auth** you can require when sharing over a LAN (Settings → Network & sharing).

**The gateway loads models for you.** Most local hosts make you load a model first, then call it.
TurboLLM's gateway reads the `model` field of any incoming request, **fuzzy-matches it to your
Expand All @@ -267,7 +267,7 @@ coding model, a vision model, and an embedder just names each one and it works
`opencode`, `kilo`, `openclaw`, and `hermes` — each gets pointed at TurboLLM the way that tool
expects (config file merge, or its own CLI command) instead of a manual copy-paste setup. Inside
Claude Code, `/model` lists your local models directly (with **Auto Model Swap** on in Settings →
Gateway) so you can switch mid-session instead of only at launch.
Models & loading) so you can switch mid-session instead of only at launch.

</details>

Expand All @@ -285,7 +285,7 @@ fight for memory (and one usually OOMs). TurboLLM can hand the GPU over automati
It's **push-based, not polling** — ComfyUI signals TurboLLM the moment a job starts/ends, so the
handoff is immediate and deterministic (the model is gone *before* ComfyUI executes).

**One-time setup** (Settings → ComfyUI): turn on **Pause for ComfyUI**, enter your ComfyUI folder
**One-time setup** (Settings → Network & sharing → ComfyUI): turn on **Pause for ComfyUI**, enter your ComfyUI folder
(the one containing `custom_nodes`), click **Install gate** (it writes a small custom node wired to
this daemon), then **restart ComfyUI** once. The panel shows a live indicator (rendering / idle /
connected); **Remove** undoes it.
Expand Down Expand Up @@ -333,7 +333,7 @@ daemon any time with **Ctrl+C**.
No other local-LLM app lets you run **whatever inference engine you want**. TurboLLM treats
the engine as a swappable component.

**Add a custom engine** (Engines screen → **Add engine**):
**Add a custom engine** (Engines screen → **Add your own engine**):

1. Compile or download any `llama-server`-compatible binary — stock
[llama.cpp](https://github.com/ggml-org/llama.cpp), a community fork, or your own build.
Expand Down Expand Up @@ -398,13 +398,14 @@ any OpenAI- or Anthropic-compatible tool (Open WebUI, Kilo Code, opencode, …).
## Use it from any device on your network

The UI runs in the browser, so any phone, tablet, or laptop on your LAN can use the model on
your GPU box:
your GPU box — every screen is responsive down to phone widths, with the desktop layout
untouched:

```bash
turbollm --addr 0.0.0.0:6996 # bind all interfaces, then open http://<your-ip>:6996
```

Turn on **Require API key** in Settings → Network when you expose it.
Turn on **Require API key** in Settings → Network & sharing when you expose it.

---

Expand Down
45 changes: 45 additions & 0 deletions turbollm/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,51 @@ published version on npm has a matching `vX.Y.Z` tag in git.

_Nothing yet._

## [1.7.4] - 2026-07-07

**Engines, Library, Developer, and Settings redesigned, and every screen now works on mobile.**

### Added
- Mobile-responsive layouts across every screen (drawer navigation, stacked cards, horizontal
tab strips, split panes that stack vertically) — the desktop/landscape layout is unchanged.
- **Library**: a dedicated Folders dialog with a real folder picker for managing model
directories, plus name search and facet-aware filter chips (a chip only appears if at least
one model has it).
- **Engines**: per-card hardware-fit indicators (green/amber/red: Compatible / runs after a
build / incompatible + reason) and grounded pros/cons for every engine.
- The folder/file picker (used by model folders and "Add your own engine") gained an editable
address bar — paste or type an absolute path and jump there directly, alongside click-to-browse.

### Changed
- **Library**: rows are now a column-aligned two-tier layout (name + one state signal + quiet
capability chips, with aligned Quant/Size/Ctx/Speed columns); multi-quant models fold into one
row with a quant dropdown instead of a collapsible group.
- **Engines**: the active-engine bar and hardware-fit selector merged into one panel with a
run-state traffic light (green running / amber starting / red error / grey stopped); llama.cpp
variants unified under one "Manage GPU builds" card; "Add your own engine" moved into a compact
strip.
- **Developer**: narrowed to connecting external tools — a Connection panel, one-command CLI
setup cards, and a collapsed API reference. Cloud Deploy (RunPod) moved behind an internal
feature flag, off by default.
- **Settings**: reorganized into a two-pane, five-category layout (General · Models & loading ·
Tools & safety · Network & sharing · System) with one sticky Save bar; expert knobs (GPU
layers, VRAM headroom, image/response token caps) moved into a collapsible Advanced section.
Tool permissions moved here from Developer (Settings → Tools & safety).
- "Add your own engine" filesystem browsing is local-only (same restriction as engine scanning)
but can now browse the whole local filesystem, including other drives on Windows — previously
confined to your home directory.

### Fixed
- Every dialog now keeps a gutter on narrow screens instead of rendering flush to the viewport edge.
- Assorted mobile overflow issues: chat header controls clipping on narrow screens, Library
action buttons pushed off-screen, Settings controls clipped at the edge, and long Hugging Face
repo names/inline code overflowing their containers.

### Discord
- New, clearer look for the Engines, Library, Developer, and Settings screens.
- The whole app now works properly in your phone's browser, not just on desktop.
- Picking a model folder or a custom engine now lets you paste in a path directly, or browse to any drive.

## [1.7.3] - 2026-07-06

**The standalone Agents screen is retired — replaced by a shared Skills library used directly in chat, plus a new Agents tab in Customize for editing personas and building your own.**
Expand Down
19 changes: 10 additions & 9 deletions turbollm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,8 +177,8 @@ the detail:
- **Saved per-model profiles, per engine** — tune once per (model, engine) pair, so
switching engines (or between two installs of the same engine, e.g. a fork) never
overwrites another engine's tuning for the same model.
- **Configurable VRAM headroom** (Settings → Engine, 300 MB–2 GB, default 1 GB) — tell auto-tune
how much VRAM to keep free for other GPU workloads instead of a fixed margin.
- **Configurable VRAM headroom** (Settings → Models & loading → Advanced, 300 MB–2 GB, default
1 GB) — tell auto-tune how much VRAM to keep free for other GPU workloads instead of a fixed margin.

</details>

Expand Down Expand Up @@ -216,7 +216,7 @@ the detail:
in every chat with no restart. A **Research** persona forces multi-step web search and cites sources inline.
- **Tool-call approval gate** — every tool call asks for your approval by default before it runs,
with **Deny**, **Allow**, **Allow for this chat**, or **Always Allow** on an inline bar above the
composer. Set per-tool defaults globally from DeveloperTool permissions.
composer. Set per-tool defaults globally from SettingsTools & safety.

</details>

Expand Down Expand Up @@ -255,7 +255,7 @@ curl http://127.0.0.1:6996/v1/chat/completions \
- **Anthropic-compatible** `/v1/messages` — including **tool use and streaming** — which powers
Claude Code below. No other local host offers this.
- **Structured output** — constrain any response to a **GBNF grammar** (or JSON shape).
- **API-key auth** you can require when sharing over a LAN (Settings → Network).
- **API-key auth** you can require when sharing over a LAN (Settings → Network & sharing).

**The gateway loads models for you.** Most local hosts make you load a model first, then call it.
TurboLLM's gateway reads the `model` field of any incoming request, **fuzzy-matches it to your
Expand All @@ -267,7 +267,7 @@ coding model, a vision model, and an embedder just names each one and it works
`opencode`, `kilo`, `openclaw`, and `hermes` — each gets pointed at TurboLLM the way that tool
expects (config file merge, or its own CLI command) instead of a manual copy-paste setup. Inside
Claude Code, `/model` lists your local models directly (with **Auto Model Swap** on in Settings →
Gateway) so you can switch mid-session instead of only at launch.
Models & loading) so you can switch mid-session instead of only at launch.

</details>

Expand All @@ -285,7 +285,7 @@ fight for memory (and one usually OOMs). TurboLLM can hand the GPU over automati
It's **push-based, not polling** — ComfyUI signals TurboLLM the moment a job starts/ends, so the
handoff is immediate and deterministic (the model is gone *before* ComfyUI executes).

**One-time setup** (Settings → ComfyUI): turn on **Pause for ComfyUI**, enter your ComfyUI folder
**One-time setup** (Settings → Network & sharing → ComfyUI): turn on **Pause for ComfyUI**, enter your ComfyUI folder
(the one containing `custom_nodes`), click **Install gate** (it writes a small custom node wired to
this daemon), then **restart ComfyUI** once. The panel shows a live indicator (rendering / idle /
connected); **Remove** undoes it.
Expand Down Expand Up @@ -333,7 +333,7 @@ daemon any time with **Ctrl+C**.
No other local-LLM app lets you run **whatever inference engine you want**. TurboLLM treats
the engine as a swappable component.

**Add a custom engine** (Engines screen → **Add engine**):
**Add a custom engine** (Engines screen → **Add your own engine**):

1. Compile or download any `llama-server`-compatible binary — stock
[llama.cpp](https://github.com/ggml-org/llama.cpp), a community fork, or your own build.
Expand Down Expand Up @@ -398,13 +398,14 @@ any OpenAI- or Anthropic-compatible tool (Open WebUI, Kilo Code, opencode, …).
## Use it from any device on your network

The UI runs in the browser, so any phone, tablet, or laptop on your LAN can use the model on
your GPU box:
your GPU box — every screen is responsive down to phone widths, with the desktop layout
untouched:

```bash
turbollm --addr 0.0.0.0:6996 # bind all interfaces, then open http://<your-ip>:6996
```

Turn on **Require API key** in Settings → Network when you expose it.
Turn on **Require API key** in Settings → Network & sharing when you expose it.

---

Expand Down
2 changes: 1 addition & 1 deletion turbollm/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "turbollm",
"version": "1.7.3",
"version": "1.7.4",
"description": "TurboLLM — local LLM platform: run any inference engine auto-tuned to your GPU, with a web UI and OpenAI/Anthropic-compatible API. Point Claude Code at your own machine in one command.",
"license": "FSL-1.1-ALv2",
"author": "Mohit Soni",
Expand Down
72 changes: 39 additions & 33 deletions turbollm/src/api/routes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import type { Context, Hono } from 'hono'
import { streamSSE } from 'hono/streaming'
import { existsSync, mkdirSync, readFileSync, readdirSync, realpathSync, rmSync, statSync, writeFileSync } from 'node:fs'
import { basename, dirname, join, resolve, sep } from 'node:path'
import { basename, dirname, join, resolve } from 'node:path'
import { GATE_VERSION, gateNodeSource } from '../comfyui/gate-template'
import { randomUUID } from 'node:crypto'
import { homedir, networkInterfaces } from 'node:os'
Expand Down Expand Up @@ -944,26 +944,31 @@ export function registerApi(app: Hono, d: Deps): void {
}
})

// ---- filesystem browser (spec 03 §9): pick an engine binary by navigating the
// disk from the browser. Loopback-only and confined to the user's home dir so a
// page on the LAN cannot read arbitrary files through the daemon.
// ---- filesystem browser (spec 03 §9): pick an engine binary/folder by navigating
// the disk from the browser. LOCAL-ONLY (isLocalRequest, matching /engines/scan): a
// page on the LAN — even with a valid key — cannot read the filesystem through the
// daemon; only the browser on the machine running TurboLLM can. Deliberately NOT
// home-confined, so an engine that lives on another drive (D:\) or outside home is
// reachable — the local user already picks (and executes) arbitrary binaries via scan.
app.get('/api/v1/fs/browse', (c) => {
const home = realHome()
if (!isLocalRequest(c, d))
return err(c, 403, 'forbidden', 'Browsing the filesystem is only available on the machine running TurboLLM.')
const raw = (c.req.query('path') ?? '').trim()
// Resolve the requested path; default to the home dir when none is given.
const target = raw ? resolve(raw) : home
// Canonicalize symlinks before the containment check so a symlink inside home
// that points outside cannot be used to escape. Fall back to the lexical path
// if the target doesn't exist yet (it then fails the readdir below cleanly).
// Windows "This PC": a virtual root above the drive letters, so the user can hop
// between drives (C:\, D:\, …). Reached by going "up" from a drive root.
if (raw === FS_DRIVES_ROOT && process.platform === 'win32') {
return c.json({ path: FS_DRIVES_ROOT, parent: null, entries: listWindowsDrives() })
}
// Default to the home dir (a sensible starting point); any path is allowed.
const target = raw ? resolve(raw) : homedir()
// Canonicalize symlinks; fall back to the lexical path if it doesn't exist yet
// (it then fails the readdir below cleanly).
let real: string
try {
real = realpathSync(target)
} catch {
real = target
}
if (!isWithinHome(real, home)) {
return err(c, 403, 'path_outside_home', 'That folder is outside your home directory.')
}
let entries: { name: string; path: string; isDir: boolean }[]
try {
entries = readdirSync(real, { withFileTypes: true })
Expand All @@ -985,9 +990,11 @@ export function registerApi(app: Hono, d: Deps): void {
} catch {
return err(c, 400, 'fs_read_failed', 'Could not read that folder (permission denied or not a directory).')
}
// Parent is null at the home root or once it would escape home.
const parentDir = real === home ? null : dirname(real)
const parent = parentDir && isWithinHome(parentDir, home) ? parentDir : null
// Parent: the containing dir; at a filesystem/drive root, go up to the Windows drive
// list (so the user can switch drives) or stop (null) on a single-root OS.
const parentDir = dirname(real)
const parent =
parentDir === real ? (process.platform === 'win32' ? FS_DRIVES_ROOT : null) : parentDir
return c.json({ path: real, parent, entries })
})

Expand Down Expand Up @@ -2236,24 +2243,23 @@ function readTail(path: string, n: number): string[] {

// ── filesystem browser helpers (spec 03 §9) ─────────────────────────────────

/** The user's home dir, canonicalized (symlinks resolved) so the containment
* check below compares like-with-like. */
function realHome(): string {
const h = homedir()
try {
return realpathSync(h)
} catch {
return h
}
}
/** Sentinel path for the Windows "This PC" drive list — a virtual root above the
* drive letters (C:\, D:\, …) so the browser can hop between drives. */
const FS_DRIVES_ROOT = '::drives'

/** True when `p` is the home dir itself or a descendant of it. Compares the
* normalized paths and requires a trailing separator on the prefix so
* `/home/bobby` is not treated as inside `/home/bob`. Cross-platform: `sep`
* is `\` on Windows, `/` elsewhere. */
function isWithinHome(p: string, home: string): boolean {
if (p === home) return true
return p.startsWith(home.endsWith(sep) ? home : home + sep)
/** Available Windows drive roots (C:\, D:\, …) as browsable entries. Probes A–Z;
* only the drives that exist on the box respond, missing ones are skipped. */
function listWindowsDrives(): { name: string; path: string; isDir: boolean }[] {
const drives: { name: string; path: string; isDir: boolean }[] = []
for (let i = 65; i <= 90; i++) {
const root = `${String.fromCharCode(i)}:\\`
try {
if (existsSync(root)) drives.push({ name: root, path: root, isDir: true })
} catch {
/* unreadable drive (e.g. an empty removable slot) — skip */
}
}
return drives
}

function getLanIp(): string {
Expand Down
2 changes: 1 addition & 1 deletion turbollm/web/src/components/ModelLoadMenu.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ export function ModelLoadMenu({
return (
<DropdownMenu>
<DropdownMenuTrigger
className="flex h-8 max-w-[260px] items-center gap-1.5 rounded-md border border-border bg-panel px-2.5 text-[13px] text-ink transition-colors hover:border-[color:var(--accent)] disabled:opacity-60"
className="flex h-8 max-w-[160px] items-center gap-1.5 rounded-md border border-border bg-panel px-2.5 text-[13px] text-ink transition-colors hover:border-[color:var(--accent)] disabled:opacity-60 md:max-w-[260px]"
disabled={pending}
>
{pending || ejecting ? (
Expand Down
4 changes: 2 additions & 2 deletions turbollm/web/src/components/common.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ export function ScreenHeader({
actions?: ReactNode
}) {
return (
<div className="mb-6 flex items-start justify-between gap-4">
<div>
<div className="mb-6 flex flex-col gap-3 sm:flex-row sm:items-start sm:justify-between sm:gap-4">
<div className="min-w-0">
<h1 className="text-[18px] font-semibold tracking-[-0.01em] text-ink">{title}</h1>
{description && <p className="mt-1 text-[13px] text-muted">{description}</p>}
</div>
Expand Down
4 changes: 2 additions & 2 deletions turbollm/web/src/components/ui/alert-dialog.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ export const AlertDialogContent = React.forwardRef<
<AlertDialogPrimitive.Content
ref={ref}
className={cn(
'fixed left-1/2 top-1/2 z-50 w-full max-w-md -translate-x-1/2 -translate-y-1/2',
'fixed left-1/2 top-1/2 z-50 w-[calc(100%_-_2rem)] max-w-md -translate-x-1/2 -translate-y-1/2',
'rounded-[var(--radius-lg)] border border-border bg-panel p-5 shadow-[var(--shadow-2)]',
'focus:outline-none',
className,
Expand All @@ -49,7 +49,7 @@ export function AlertDialogFooter({
className,
...props
}: React.HTMLAttributes<HTMLDivElement>) {
return <div className={cn('mt-5 flex justify-end gap-2', className)} {...props} />
return <div className={cn('mt-5 flex flex-wrap justify-end gap-2', className)} {...props} />
}

export const AlertDialogTitle = React.forwardRef<
Expand Down
Loading
Loading