Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 40 additions & 5 deletions playbooks/core/lmstudio-rocm-llms/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,28 +73,63 @@ Learn how to start chatting with a ChatGPT-grade LLM completely locally.
7. If not in the chat window, press `Ctrl + 1` or click on the 👾 button on the top left of the screen.
8. Send a message and start interacting with the model!

<!-- @os:windows -->
<!-- @test:id=lmstudio-select-gpu-runtime-windows timeout=120 hidden=True -->
```powershell
# CI: pin a GPU (Vulkan) runtime so tests don't fall back to the CPU engine.
lms runtime ls
$rt = ((lms runtime ls) -match 'vulkan' | Select-Object -First 1)
if ($rt) {
lms runtime select (($rt.Trim() -split '\s+')[0])
lms runtime ls | Select-String 'ENGINE|✓'
} else {
Write-Output "WARNING: no Vulkan runtime installed; GPU acceleration unavailable. Install with: lms get <vulkan-runtime>"
}
```
<!-- @test:end -->
<!-- @os:end -->

<!-- @os:windows -->
<!-- @test:id=lmstudio-load-model-windows timeout=1200 hidden=True -->
```powershell
lms unload --all
lms ps
$ID = "${lms_model}-$env:GITHUB_RUN_ID"
Set-Content -Path "$env:TEMP\lmstudio_model_id.txt" -Value $ID -Encoding utf8
# retry once: large-model loads can transiently fail under memory pressure
lms load ${lms_model} --context-length 32768 --gpu max --identifier "$ID" -y
if ($LASTEXITCODE -ne 0) { lms unload --all; Start-Sleep 5; lms load ${lms_model} --context-length 32768 --gpu max --identifier "$ID" -y }
lms ps
lms chat "$ID" -p "Reply with exactly: OK"
```
<!-- @test:end -->
<!-- @os:end -->

<!-- @os:linux -->
Comment thread
lucbruni-amd marked this conversation as resolved.
<!-- @test:id=lmstudio-select-gpu-runtime-linux timeout=120 hidden=True -->
```bash
# CI: pin a GPU (Vulkan) runtime so tests don't fall back to the CPU engine.
lms runtime ls
GPU_RT="$(lms runtime ls 2>/dev/null | awk '/vulkan/{print $1; exit}')"
if [ -n "$GPU_RT" ]; then
lms runtime select "$GPU_RT"
lms runtime ls | grep -E 'ENGINE|✓'
else
echo "WARNING: no Vulkan runtime installed; GPU acceleration unavailable. Install with: lms get <vulkan-runtime>"
fi
```
<!-- @test:end -->
<!-- @os:end -->

<!-- @os:linux -->
<!-- @test:id=lmstudio-load-model-linux timeout=1200 hidden=True -->
```bash
lms unload --all || true
lms ps
ID="${lms_model}-${GITHUB_RUN_ID}"
echo "$ID" > /tmp/lmstudio_model_id.txt
lms load ${lms_model} --context-length 32768 --gpu max --identifier "$ID" -y
# retry once: large-model loads can transiently fail under memory pressure
lms load ${lms_model} --context-length 32768 --gpu max --identifier "$ID" -y || { lms unload --all; sleep 5; lms load ${lms_model} --context-length 32768 --gpu max --identifier "$ID" -y; }
lms ps # Verify model is really loaded
lms chat "$ID" -p "Reply with exactly: OK"
```
Expand Down Expand Up @@ -265,12 +300,12 @@ req = urllib.request.Request(
"model": model_id,
"messages": [{"role":"user","content":"What is 2 + 2? Reply with only the number."}],
"temperature": 0,
"max_tokens": 500
"max_tokens": 64
}).encode("utf-8"),
headers={"Content-Type":"application/json"},
method="POST",
)
with urllib.request.urlopen(req, timeout=60) as r:
with urllib.request.urlopen(req, timeout=120) as r:
print(r.read().decode("utf-8", "replace"))
```
<!-- @test:end -->
Expand All @@ -290,12 +325,12 @@ req = urllib.request.Request(
"model": model_id,
"messages": [{"role":"user","content":"What is 47 + 42? Reply with only the number in words."}],
"temperature": 0,
"max_tokens": 500
"max_tokens": 64
}).encode("utf-8"),
headers={"Content-Type":"application/json"},
method="POST",
)
with urllib.request.urlopen(req, timeout=60) as r:
with urllib.request.urlopen(req, timeout=120) as r:
print(r.read().decode("utf-8", "replace"))
```
<!-- @test:end -->
Expand Down
Loading