I see where you're going now, with this signature, it's really an invitation to reuse custom exception handlers.
A first instinct would be to literally pass the app's existing exception handler, but then only the StarletteContextException are caught, so it's actually about writing a new exception handler, surely inspired by existing custom ones, and passing it there.
typing won't even help in making this caveat obvious here, as a general handler built for the already existing HTTPException, which is a fairly common thing to write, passes typecheck as the StarletteContextException now is a subclass.

def general_error_handler(req: Request, exc: HTTPException) -> Response:
    # built for all HTTPException
    return Response()

m = StarletteContextMiddlewareMixin(error_handler=general_error_handler)

One might think that all HTTPException are handled there.
That can get more confusing with custom plugins who might raise HTTPException as they might be used to everywhere else, and to only get burned as they are not actually caught by the middleware.

if passing such an handler is to be considered canon, then the middlewares should except on HTTPException instead of the StarletteContextException, and type the callback as taking the HTTPException. the StarletteContextException we raise can be considered as internal fine-grained exceptions.
if not, then the StarletteContextException shouldn't inherit from HTTPException to disallow that usage from typing, and enforce (and document) the usage of StarletteContextException exclusively for plugins.

also, it takes an async function, so it should be an Callable[[Request, StarletteContextException], Awaitable[Response]]

One might think that all HTTPException are handled there.

If a custom exception handler is needed and the docs state how to use it, why this is needed, and what kind of errors it supports, then everything should be pretty clear, or at least as clear as it gets.

That can get more confusing with custom plugins who might raise HTTPException as they might be used to everywhere else, and to only get burned as they are not actually caught by the middleware.

Well, indeed that's a disadvantage I didn't consider earlier. I was thinking if this could solve it

    async def dispatch(
        self, request: Request, call_next: RequestResponseEndpoint
    ) -> Response:
        context = await self.set_context(request)
        token: Token = _request_scope_context_storage.set(context)

        try:
            response = await call_next(request)
        except StarletteContextException as e:
            response = await self.create_response_from_exception(request, e)
        else:
            for plugin in self.plugins:
                try:
                    await plugin.enrich_response(response)
                except Exception as e:
                    response = await self.create_response_from_exception(request, e)
                    break
        finally:
            _request_scope_context_storage.reset(token)

        return response

WDYT?

I think it's possible that the application state will be "success" and then it fails on response enrichment. If that happens, there's no clear rollback/cleanup. 🤔 Actually that's the problem in the current implementation as well, I think.

an Exception raised during enrich_response would rise up to the plain exception handler. As data isn't coming from the user by that point (or should have been validated before), it's really a faulty implementation, so getting a 500 seems correct to me . I'm not sure that's something to be concerned about.
And in any case, the situation is fairly different, I don't think it's okay to simply reuse the same handler expecting validation errors from input.

moving the handling to wrap everything? no, that can't be right.
StarletteContextException should only happen during plugin execution anyway.

Well I'm glad I didn't merge it yet. Thanks!

return await self.error_handler(request, exc)
that looks very much like only awaitables are supported. why the union?
(and i'd rather support only one than digging into the can of worms of supporting either)