[Extensions] Per-file language server

[deecewan]

Check for existing issues

• Completed

Describe the feature

For sorbet for Ruby, the language server needs exactly 1 input directory. This is the directory that contains the sorbet/ directory (and by extension, the config).

if I have a project that looks like this, for instance:

root/
  api/
    sorbet/
    some-file.rb
  ui/
    index.js

when I open root/ in Zed, it will start the language server with the cwd set to root/. The problem is, is that Sorbet doesn't find a sorbet/ directory there, so errors. Per-project, I could set a config value to say "load api/" and then update the extension to launch the LSP correctly for that directory. But there's no way to check that, for the file I'm looking at, where that directory should be.

Also, if I have a project that looks like this:

root/
  service-one/
    sorbet/
    some-file.rb
  service-two/
    sorbet/
    some-file.rb

I can't start language servers for both, because a language server is based solely on a root, and there's no way to set config in that case, because it'd need two different values at the same time.

Ideally, I'd like to be able to get the file I'm looking at, and return a language server for that file. In the first instance, I'd get the ruby file, and return a root of root/api. In the second, I'd start 2 servers with unique IDs, and have the IDs cached (somehow...). Then for every file that is opened, my extension can start (or return) a language server that should handle that file.

This is pretty much how neovim supports language servers right now, where the plugin is asked about each file, instead of the language as a whole.

Somewhat relatedly, even if I did set the root/api dir for the first case, Sorbet doesn't return fully-qualified files. Rather, it returns file names relative to the root/api/. When Zed tries to open them (say, for "Go to definition"), the file doesn't exist. It returns some-file.rb instead of api/some-file.rb, for instance.

If applicable, add mockups / screenshots to help present your vision of the feature

No response

[timothyshrugged]

Just out of curiosity: how are you getting Sorbet to work with Zed in the first place?

[deecewan]

@timothyshrugged i created a custom plugin that implemented the LSP plugin interface for zed. Then just called the sorbet language server underneath that.

[notpeter]

I think addressing would improve Zed user experience with monorepos.
Can you help sketch out what the interface for this might look like?
Specifically would this require .zed/settings.json keys?
Would love to hash-out some pseudo settings.

So for example, my current settings might include things like:

  "file_types": {
    "Ruby": ["*.Brewfile"]
  }
  "languages": {
    "Ruby": {
      "language_servers": ["ruby-lsp", "rubocop", "!solargraph", "..."]
    }
  },
  "lsp": {
    "ruby-lsp": {
      "initialization_options": {}
    }
  },

In order to support this we need to specify (at the project / folder level) that the default method of associating LSPs to Language should be ignored/overridden by an alternate path based approach. Ideas?

What about something like:

  "language_paths": [
    {
      "paths": ["webapp", "backend"],
      "Ruby": {
        "language_servers": ["ruby-lsp", "rubocop", "!solargraph", "..."],
      },
      "Markdown": {
        "remove_trailing_whitespace_on_save": false
      }
    },
    { 
      "paths": ["frontend"],
      "Ruby": {
        "language_servers": ["sorbet", "!rubocop", "!solargraph", "..."],
      }
    }
  ]

Another option would be to support .zed/settings.json in subdirectories, but that feels pretty ugly. It's already quite difficult when doing ssh to keep track of: ~/.config/zed/settings.json (local), ~/.config/zed/settings.json (remote) and .zed/settings.json (project) + default.json (baked into the app). I'd hate to increase complexity further.

Definitely open to other suggestions.

[deecewan]

I'm not sure it needs that level of config. I think that a "pool" of language servers, and when a file is opened, Zed calls some method in the language server API which says "hey this particular file has opened". The language server can look for the root, in whatever manner it needs to (in Sorbet's case, sorbet/ being a directory. for ESLint, .eslintrc).

The LSP implementation in Zed can then return an existing running connection (if one exists for that root), or can start a new server in the pool and return that.

[notpeter]

I struggle to imagine how this could be implemented in practice.

Zed calls some method in the language server API which says "hey this particular file has opened". The language server can look for the root, in whatever manner it needs to (in Sorbet's case, sorbet/ being a directory. for ESLint, .eslintrc).

You are proposing an Language Server Protocol extension, which every LSP must implement. Additionally you are requiring a language server self-determine whether it should handle a particular directory which it can't possibly do (e.g. ruby-lsp can't know that sorbet should be used not itself)

It also requires spawning the Sorbet LS in order to determine whether the Sorbet LS should be used. I currently have 15-20 potential LSPs configured for use with Zed, but in normal projects only 1-4 are started, all on demand. This is obviously a non-starter.

---

Can you expand a little further on your use case? It sounds like you're trying to work around a sorbet LSP limitation -- it only supports operating on a workspace which has a /sorbet directory in the root. I don't think this is a problem that Zed can easily solve if your project does not follow this pattern.

[deecewan]

i'm not proposing an extension to LSP - just proposing passing the newly opened file path to the Zed extension, and then having it return a "handle" to an LSP that it should use from that point on.

it's possible to have ruby-lsp and sorbet running together, and I don't think it's up to either of them to decide if they should run based on the other existing. if the user adds both LSPs, both should run.

this is also not requiring sorbet to spawn - the Zed extension itself can determine what to start, in what directory, etc. This is how Neovim currently handles LSP. The neovim plugin looks at the file that has been opened, and says "okay, i have some heuristics to figure out if I should start, and if I do, what command to run, what root to run under, etc".

for a usecase: imagine a monorepo. each project in the repo could be a project of any language. ruby, javascript, rust, go, etc. the root of the monorepo contains nothing but folders and a README. if I open that monorepo in Zed, how can I get an LSP to start for the rust project when I open a rust file, using the rust folder as the root for the LSP to i.e. find config, resolve relative paths, etc. and if I have 2 rust project in the monorepo, I might need 2 different LSPs, if the LSP has startup context based on it's root. You wouldn't want autocomplete to provide you with package information for rust-project-1 in a file belonging to rust-project-2.

So, Zed passes rust-project-2/main.rs to the Zed rust LSP extension. the Zed extension looks at the file and says "okay, let me find the root based on my built-in heuristics. there's a Cargo.toml in rust-project-2, that must be the root for this file". It spawns a new server, somehow maintains a binding to it, and returns a reference to that running server. Zed uses that returned server (probably a rust struct) to ask for autocomplete, etc, from the language server. if a file is opened in rust-project-2 again, the Zed extension can return the same reference. if a file is opened in rust-project-1, the Zed extension spawns a new server, and returns that.

apologies for the long-winded response - i don't think I was clear at all before this.

[deecewan]

i also realise i said "language server API" a lot. I meant the "Zed language server extension API", and not the API of the language server itself.

[qystishere]

I'm having a slightly different problem that may not be specifically Zed-related, but I'll describe it here anyway.

I am using both Deno and standalone TypeScript code in the same project. The Deno language server works fine, and I can exclude files that are not intended for it with deno.json. However, the TypeScript language server ignores the settings from tsconfig.json, and Zed always shows errors in the Deno code (specifically, it can't find Deno imports, even though compilation works fine).

This feature would solve my problem. These issues don't occur in VSCode, though I haven't investigated why in detail.

[notpeter]

@qystishere Thanks for the example. That is exactly the sort of problem I think Zed could reasonably fix.

---

@deecewan

i also realise i said "language server API" a lot. I meant the "Zed language server extension API", and not the API of the language server itself.

Cool. It's actually a similar problem in some way, less bad, but still not great.

The change to Extension API would likely be the addition of a function that allows an Extension to register a callback of the form which_lsps_you_think(path: PathBuf): Vec<String>. If you had 10 language extensions which supported this API Zed would have to async poll 10 extensions every time you opened a buffer to determine which language server(s) should be notified.

The dumb implementation of this could mean that the extension had to hit the filesystem (hunting for a sorbet directory). A complex implementation would have the extension maintaining a cache of some sort to make those decisions quick.

I'm open to other suggestions, but I think it's better to make the decision about which language servers should be spawned to be declarative and kept within the Zed core to avoid the async fan-out required to poll extensions.

I really appreciate you bringing this up because it got me thinking about the settings changes that will be necessary to support operating Zed at the root of a mono-repo. We've got a bunch of work to do but it's definitely a worth goal.

---

I'm going to go ahead and move this to a discussion because there's definitely more design/discussion needed before this is ready for an actual implementation.

Thanks all.

[deecewan]

yeah there might be a way to declaratively specify how to find the root - but i do think the important part is based on the path. something like

sorbet:
  root_patterns: ["sorbet/"]
eslint:
  root_pattens: [".eslintrc", ".eslintrc.json"]

for a very non-exhaustive example. this would look from path up the tree til it finds something that matches one of the items in the list. this could be pretty efficiently cached, too. a directory would need to be checked once, and "it supports sorbet" or "it doesn't" could be stored. only downside is that then you'd have to clear out the cache on any FS change.

[timothyshrugged]

I’ve found Sublime LSP’s model for this to be pretty easy to reason about. You can add your own (named) LSPs via command line args and, in most cases, define the paths that they run against. Sorbet itself looks for a specific folder to run against and you specify that in the args/config as well.

This doesn’t solve specifically for per-file LSPs, but it does move some of the LSP abstraction away from Zed and into “hey, what would you call from a terminal to invoke this LSP how you’d like?”

[osiewicz]

@deecewan just to chime in, what you're describing is exactly the model I'm implementing right now (with a small caveat that you should also be able to have just one global instance of a language server for which workspaceFolders should be modified).
Oddly enough, it was designed independently from this discussion, which I've stumbled upon just now, but it's fun to see us converge on the same thing.