Reduce size of `InferenceResult` #35784

Job	Run time
proc-macro-srv	1m 34s
changes	6s
Rust (macos-latest)	4m 47s
Typo Check	6s
Rust (ubuntu-latest)	4m 45s
Rust Cross (powerpc-unknown-linux-gnu)	1m 11s
miri	38s
Rust (windows-latest)	7m 14s
rustfmt	19s
Rust Cross (x86_64-unknown-linux-musl)	1m 11s
Rust Cross (wasm32-unknown-unknown)	45s
analysis-stats	9m 55s
TypeScript	0s
conclusion	4s
	32m 35s

Provide feedback