Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New proc-macro server RPC API #19205

Open
Veykril opened this issue Feb 22, 2025 · 0 comments
Open

New proc-macro server RPC API #19205

Veykril opened this issue Feb 22, 2025 · 0 comments
Labels
A-proc-macro proc macro C-Architecture Big architectural things which we need to figure up-front (or suggestions for rewrites :0) )

Comments

@Veykril
Copy link
Member

Veykril commented Feb 22, 2025

The current proc-macro server RPC API is reaching its limits preventing us from implementing more of the proc-macro API.

The current API is effectively a serial JSON based client to server request-response via stdout. That is the client (rust-analyzer, RustRover) makes a request to the proc-macro server process via JSON message and then receives a response from the server. Notably these requests cannot interleave (the proc-macro server is sequential).

This has a couple of downsides:

  • JSON: We have a weird serialization scheme for JSON to bring down the overall data we sent over the wire to combat JSON being fairly bulky. We want to replace this part (presumably with postcard)
  • client to server request-response scheme: A fairly simple messaging style that unfortunately does not suffice for implementing all the proc-macro APIs. Some of the proc-macro functions actually need to access semantic state which lives in the client and hence the server needs to be able to ask the client for this information while it is computing the response for a client's request. That means, we need a request-response scheme from client to server that allows to have server to client request-response pairs while the client is waiting for the response. One can think of this as a client request opening a communication channel from the server to the client, and the server response closing it.
  • sequential server: The current implementation of the server is entirely sequential, it can only serve a single request at any given time. Ideally we would allow the server to serve multiple requests concurrently (that is multiplex the connection), but that alone does not parallelize that well due to proc-macros being able to observe its environment and therefore we can only parallelize proc-macro expansions that share the same environment (which depending on the workload is still a win). An alternative would be to allow for spawning multiple proc-macro servers to interface with (this would more require work in the client than the server).

Notably an implementation of this needs to live side by side with the old protocol for a transition period as the proc-macro server is a rustup component that IDEs rely on.

Relevant crates:

@Veykril Veykril added C-Architecture Big architectural things which we need to figure up-front (or suggestions for rewrites :0) ) A-proc-macro proc macro labels Feb 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-proc-macro proc macro C-Architecture Big architectural things which we need to figure up-front (or suggestions for rewrites :0) )
Projects
None yet
Development

No branches or pull requests

1 participant