Skip to content

AGENTS.md

Latest commit

 

History

History
85 lines (62 loc) · 4.3 KB

AGENTS.md

File metadata and controls

85 lines (62 loc) · 4.3 KB
Raw

TurboFieldfare

Swift and Metal inference for Gemma 4 26B-A4B on Apple Silicon.

Scope

This checkout is for running and reporting existing behavior. Do not edit source, change runtime defaults, or start optimization work unless the user asks.

Layout and commands

Sources/TurboFieldfareFormat/ owns the Foundation-only .gturbo v1 wire contract. Sources/TurboFieldfare/ is the runtime; Sources/TurboFieldfareRepack/, Sources/TurboFieldfareCLI/, Sources/TurboFieldfareServer/, and Sources/TurboFieldfareApp/ contain the installer, CLI, loopback server, and Mac app. Tests/ contains focused public tests; docs/ contains design, benchmark, and experiment notes.

swift run -c release TurboFieldfareRepack --output scratch/gemma4.gturbo
swift run -c release TurboFieldfareRepack --output scratch/gemma4.gturbo --resume
swift build -c release
.build/release/TurboFieldfareMac
swift run -c release TurboFieldfareCLI \
  --model scratch/gemma4.gturbo \
  --prompt "The capital of France is" \
  --max-new 64

The installer streams the pinned model without staging the full source checkpoint. Set HF_TOKEN only if requested. The download is about 15 GB. Cancellation preserves verified completed ranges; continue them with --resume or remove them with --discard-partial --output scratch/gemma4.gturbo.

Local server

Follow the server guide for launch commands, health checks, client setup, prompt reuse, tool loops, and supported API behavior. Apply the model-process checks below first; never start a second model process or terminate an existing one.

Keep the server on 127.0.0.1; it has no remote authentication or TLS, so do not proxy, tunnel, or expose it. A tool call from the local model never bypasses the client's normal permission policy. Keep the execution session alive while the server is needed, and stop only a server you launched.

Test rules

Before a model run, require macOS 26+, Swift 6.2+, enough disk, acceptable memory_pressure -Q, a completed scratch/gemma4.gturbo, and no process from pgrep -fl 'TurboFieldfareServer|TurboFieldfareMac|TurboFieldfareDecodeService|TurboFieldfareCLI|TurboFieldfarePackageTests|swiftpm-testing-helper|mlx_lm|mlx-lm'. If a check fails, inform the user and stop; do not terminate apps or delete or reinstall the model.

Run package tests through Scripts/test.sh. Run only one app, CLI, or model-using test at a time.

For performance results, build release once and follow the community benchmark guide exactly. Do not enable experimental controls or profiling.

Do not download a full checkpoint, duplicate the .gturbo model, create a worktree, or purge caches just to run tests.

Report the commit, hardware and RAM, macOS, Swift version, exact command, exit code, complete timing footer or error, and every protocol deviation. Treat results as measurements, not performance ceilings.

App controls

The Mac app sends prompts through the pinned Gemma 4 IT chat format. It exposes context length, temperature, Top-K, Top-P, expert-cache slots, prefill, and RDADVISE. The defaults are temperature 0.2, Top-K 64, and Top-P 0.95. Responses can use the context space left after formatting the prompt, and FP16 is the runtime KV format. The HUD shows generation rate, token count, and decode-service memory; Last run also shows time to first token and I/O. Build the app with its sibling TurboFieldfareDecodeService; it never loads a second in-process model. See README and Runtime controls.

Images

Image support is an optional <name>.vision.gturbo companion pack that sits beside the text model. Without it the text runtime behaves exactly as before.

swift run -c release TurboFieldfareRepack \
  --vision-output scratch/gemma4.vision.gturbo \
  --text-model scratch/gemma4.gturbo

The model-free suites cover preprocessing, the tower kernels, the companion format, the installer transaction, prompt rendering, and the server ingress. Cases that need a real installed pack skip themselves when scratch/gemma4.gturbo and its companion are missing, so a checkout without a model still runs green.

Keep the image path fail-closed. If the pack is missing or invalid, say image support is unavailable. Never accept an image and then answer as though it had not been sent.