Support debugging use cases for Dataproc and Serverless Spark

[dborowitz]

Prerequisites

• Search the current open issues

What are you trying to do that currently feels hard or impossible?

I'd like to add more tools specifically to support debugging use cases for the existing Serverless Spark source, as well as the closely related Cloud Dataproc Clusters. Unlike the depth-first approach I used for Serverless, now I want to go a bit breadth-first, prioritizing:

• List & get APIs across multiple supported resources (clusters, jobs, sessions)
• Fetching logs from Cloud Logging associated with these resources--not just the logs URL that's already in the tools output, but actually running the LQL query in that link and returning structured JSON as the results.
• (If necessary) exposing paths to other outputs stored on GCS as part of the "get" tools. (Actually fetching the files from GCS is likely not in scope for MCP Toolbox)

Suggested Solution(s)

No response

Alternatives Considered

No response

Additional Details

No response

[averikitsch]

Hi @dborowitz Thanks for the issue. Let us know how we can support you building these tools. I believe you have been in contact with @Yuan325. Typically we would like to review the tools API design before the PR. Thanks!

[dborowitz]

Of course. I didn't write a design for the tools in #2407 since those are basically identical CRUD endpoints to the existing Serverless Spark ones.

The other thing I have prototyped is the logs tools. Agreed that this is not necessarily the right design; I've chatted with Yuan about other options, you may have seen the internal doc about it, but I'd expect some alignment before raising a PR. I just wanted to get something to share with my team for eval purposes sooner rather than later.