Feature: Generate JSON Schema for NEAR RPC API

[frol]

Feature Request

To improve the consistency of NEAR RPC clients, we need to define NEAR RPC Schema.

Ideally, I would generate the schema from the source code to avoid drifts between the schema and the implementation.

There are a number of issues with the current state of things:

• two Rust clients are currently maintained: internal in nearcore for testing, and public one
• too many dependencies for the public Rust crate due to transitive dependencies of nearcore crates, that are not necessarily used in the JSON RPC client contexts
• lack of strict types for non-Rust clients (there are deeply nested types, e.g. Block, Chunks, RPC errors, and Transaction errors)
• impossible to verify API compatibility and catch the potential drifts at the PR to nearcore stage - with autogenerated schema we will be able to write a simple insta-snapshot test on it and make sure that any change there is expected

Implementation Ideas

There is a nice Rust crate that should help us to achieve it: https://graham.cool/schemars/. It was previously discussed back in 2020 in #1972, but since that time Rust crates have evolved quite a bit.

Here is how I see it implemented:

1. All RPC-involved structs should get JsonSchema derive or manual trait implementation (if automatic derive does not play well)
2. near-jsonrpc implementation should aggregate all the JsonSchema results for each exposed method into OpenAPI spec like paperclip or similar crates do. Unfortunately, OpenRPC is has not evolved enough to compete with OpenAPI ecosystem, so I recommend we try to fit OpenAPI our needs by defining the method names as independent POST paths (and I would suggest the RPC implementation ignore the path and let any POST /* requests to be handled by JSON-RPC handler); see how it is already done here: Generate Open RPC specification nervosnetwork/ckb#4149 (comment)
3. near-jsonrpc server should have a method to expose GET /openapi-spec.json endpoint with the derived schema

With that, we can use the generated schema at nearcore compilation to generate internal client on the fly as well as external clients in a many programming languages. There are quite a few codegen solutions for OpenAPI: 1, 2, ... there are more dedicated and better or worse ones to choose from.

Beneficiary

• near-api-js (json-provider subpackage)
• near-jsonrpc-client-rs - we will finally be able to drop direct dependencies on nearcore and cut a stable 1.0 release
• near-jsonrpc-client-internal Rename workspace member near-jsonrpc-client to avoid name collision #12908
• py-near
• and bring the low-level RPC clients with strict types to many other languages

Priority

Let's discuss it here first. This can be sponsored and contributed through NEAR DevHub.

Update: there is also a discussion in Telegram group: https://t.me/NEAR_Tools_Community_Group/4428

Contributing

• Yes, I am interested

[frol]

For visibility: @PolyProgrammist will work on a proof-of-concept to see if we have any major blockers for applying schemars::JsonSchema to the relevant structs to enable automatic introspection.