Local Sweep next-edit prediction proposes newlines at random locations

[emilebergeron]

Reproduction steps

1. Setup sweep next-edit prediction using the settings recommended here: edit_prediction: Support self-hosted Sweep Next Edit models #51139
2. Point zed to a running Sweep 1.5B server (I used llama.cpp)
3. Write some text in an existing file

Current vs. Expected behavior

Expected behavior: Zed proposed a completion, and then stops
Current behavior: Zed proposes a completion, and then keeps proposing newlines at random locations in the file

For what it's worth, I noticed that in crates/edit_prediction/src/sweep_prompt.rs, trailing newlines are added to the prompt (line 224 and 342), but don't seem to be stripped from the response in clean_response_text.

Zed version and system specs

Zed: v1.15.0+stable.339.e17dc4f9d50db73a458b64dcce50ecd4878b98a3 (Zed)
OS: Linux Wayland fedora 44
Memory: 15.4 GiB
Architecture: x86_64
GPU: NVIDIA RTX A2000 Laptop GPU || NVIDIA || 595.80

Attach Zed log file

Unable to provide log due to company policy

Relevant Zed settings

"edit_predictions": {
"provider": "open_ai_compatible_api",
"open_ai_compatible_api": {
"api_url": "http://127.0.0.1:11434/v1/completions",
"model": "sweep-next-edit",
"prompt_format": "infer",
"max_output_tokens": 512,
},
},

Relevant Keymap

keymap.json

(for AI issues) Model provider details

No response

If you are using WSL on Windows, what flavor of Linux are you using?

None