Skip to content

Use unordered executeBlocking for GraphQL blocking resolvers#54927

Open
phillip-kruger wants to merge 1 commit into
quarkusio:mainfrom
phillip-kruger:fix-graphql-mailer-deadlock
Open

Use unordered executeBlocking for GraphQL blocking resolvers#54927
phillip-kruger wants to merge 1 commit into
quarkusio:mainfrom
phillip-kruger:fix-graphql-mailer-deadlock

Conversation

@phillip-kruger

@phillip-kruger phillip-kruger commented Jun 18, 2026

Copy link
Copy Markdown
Member

When a blocking GraphQL resolver calls a reactive Vert.x client — such as the imperative Mailer — the resolver can deadlock. The root cause is that BlockingHelper.runBlocking() dispatches blocking resolvers via executeBlocking with a RequestScopedTaskQueue that serializes tasks per-request. The reactive client's event-loop callbacks can't dispatch while the ordered task slot is held, creating a circular wait.

This removes the RequestScopedTaskQueue mechanism and always uses executeBlocking(callable, false) (unordered). This is safe because:

  • Mutation ordering is already guaranteed by graphql-java's AsyncSerialExecutionStrategyconfirmed by @jmartisk
  • Query field parallelism is explicitly permitted by the GraphQL spec
  • The WebSocket handler already uses unordered execution
  • The gRPC extension had the identical deadlock and fixed it the same way

Not a revert of #54372

This is not a revert of the RequestScopedTaskQueue change (PR #54372). Before that PR, the code used vc.executeBlocking(callable) with no explicit ordering parameter, which defaulted to ordered=true in Vert.x 4 — globally serializing all requests sharing an event loop (the cause of #54361). This PR uses executeBlocking(callable, false) — explicitly unordered — which is a state that never existed before:

State Code Cross-request Within-request Mailer
Before #54372 executeBlocking(callable) — implicit true Serialized (slow) Serialized Deadlock
After #54372 executeBlocking(callable, queue) Parallel Serialized Deadlock
This PR executeBlocking(callable, false) Parallel Parallel Works

Includes a regression test that sends mail via the imperative Mailer from a @Blocking GraphQL resolver using a real (non-mock) SMTP connection.

Fixes #29141
Fixes #36215

@github-actions

github-actions Bot commented Jun 18, 2026

Copy link
Copy Markdown

🎊 PR Preview 77c9a79 has been successfully built and deployed to https://quarkus-pr-main-54927-preview.surge.sh/version/main/guides/

  • Images of blog posts older than 3 months are not available.
  • Newsletters older than 3 months are not available.

@Doogiemuc

Copy link
Copy Markdown

Hello @phillip-kruger ! Thank you so much for looking into this. Only some hours after my comment. I am a senior java developer with more than three decades of coding experience. But what you are writing up there sounds like magic even to me. So once again: KUDOs and I am looking forward to the next quarkus and/GraphQL fix version! 👍

@phillip-kruger

Copy link
Copy Markdown
Member Author

@Doogiemuc what would help is if you can build against this branch and confirm if this solves the issue for you

@Doogiemuc

Copy link
Copy Markdown

"build against this branch" Does that mean I would need to build quarkus from source? I'd assume this is a big task that would take me a day just to setup all dependencies. Or do I miss something and there is an easy way?

@phillip-kruger

Copy link
Copy Markdown
Member Author

@Doogiemuc it's fairly easy:

  • Get a local quarkus: git clone git@github.com:quarkusio/quarkus.git
  • Then checkout this PR: gh pr checkout 54927 (assuming you have gh cli installed)

Then do https://github.com/quarkusio/quarkus/blob/main/CONTRIBUTING.md#building-main

Once a local version is build you can use that in your app:
https://github.com/quarkusio/quarkus/blob/main/CONTRIBUTING.md#with-maven

@quarkus-bot

This comment has been minimized.

@quarkus-bot

This comment has been minimized.

@quarkus-bot

This comment has been minimized.

cescoffier
cescoffier previously approved these changes Jun 19, 2026

@cescoffier cescoffier left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with this change. I made some comment on the test.

@jponge We had to modify this code heavily for Vert.x 5 - expect the next merge to be chaotic.

@jmartisk

Copy link
Copy Markdown
Contributor

So I'll need a bit more of an explanation here, how is this different from #29306 which I submitted in the past, but it turned out to break Hibernate?

@phillip-kruger

Copy link
Copy Markdown
Member Author

Hi @jmartisk, your PR #29306 was the right fix, the only reason it broke Hibernate was that RequestScopedSessionHolder and RequestScopedStatelessSessionHolder use a HashMap internally, which isn't thread-safe when multiple blocking resolvers run concurrently within the same request. I've now added that fix in this PR as well (switched both to ConcurrentHashMap).

So in short: this PR = your original ordered=false change + the ConcurrentHashMap fix for the Hibernate session holders that was the actual root cause of the breakage.

…urrentHashMap for request-scoped Hibernate session holders
@jmartisk

jmartisk commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

I also proposed the switch to the ConcurrentHashMap in the previous PR (see #29306 (comment)) but it was considered wrong. CC @Sanne @yrodiere

@quarkus-bot

quarkus-bot Bot commented Jun 19, 2026

Copy link
Copy Markdown

Status for workflow Quarkus Documentation CI

This is the status report for running Quarkus Documentation CI on commit 42ed240.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

Warning

There are other workflow runs running, you probably need to wait for their status before merging.

@yrodiere

yrodiere commented Jun 19, 2026

Copy link
Copy Markdown
Member

I also proposed the switch to the ConcurrentHashMap in the previous PR (see #29306 (comment)) but it was considered wrong. CC @Sanne @yrodiere

Indeed. If multiple threads are hitting the same RequestScopedSessionHolder, then you are already in trouble, because they will likely try to use the same session at some point, and that's simply not allowed.

Whatever threads are involved here need to either:

  • each have their own request context
  • OR each use its own transaction

EDIT: though if it's supposed to be atomic/consistent DB accesses, as Sanne suggested you would really want to use a single transaction for all this, which would imply a single thread.

@Sanne

Sanne commented Jun 19, 2026

Copy link
Copy Markdown
Member

It's not quite fair to state "it would break Hibernate". Hibernate could in theory do this, but it's enforcing safety by design - and efficiency I should add: if multiple operations need to happen on the DB, it's a waste of resources to initiate multiple transactional contexts and let them "figure out" any data race condition. It's also extremely error prone as exact impact on the final data representation will become timing dependent, becoming error prone and introducing difficult to reproduce interleavings.

Make an appropriate plan at the client side, decide what it is exactly that needs to happen and send a single coherent plan to the database. Hibernate would help to define such a consistent plan.

@quarkus-bot

quarkus-bot Bot commented Jun 19, 2026

Copy link
Copy Markdown

Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 42ed240.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

You can consult the Develocity build scans.


Flaky tests - Develocity

⚙️ JVM Integration Tests - JDK 17

📦 integration-tests/hibernate-orm-graphql-panache

io.quarkus.it.hibertnate.orm.graphql.panache.HibernateOrmGraphQLPanacheTest.testEndpoint - History

  • 1 expectation failed. JSON path data.authors.id doesn't match. Expected: iterable containing [<1>] Actual: null - java.lang.AssertionError
Details
java.lang.AssertionError: 
1 expectation failed.
JSON path data.authors.id doesn't match.
Expected: iterable containing [<1>]
  Actual: null

	at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:344)
	at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure.validate(ResponseSpecificationImpl.groovy:516)

⚙️ JVM Integration Tests - JDK 17 Windows

📦 integration-tests/grpc-hibernate

com.example.grpc.hibernate.BlockingRawTest.shouldAdd - History

  • Condition with Lambda expression in com.example.grpc.hibernate.BlockingRawTestBase was not fulfilled within 30 seconds. - org.awaitility.core.ConditionTimeoutException
Details
org.awaitility.core.ConditionTimeoutException: Condition with Lambda expression in com.example.grpc.hibernate.BlockingRawTestBase was not fulfilled within 30 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1160)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1129)
	at com.example.grpc.hibernate.BlockingRawTestBase.shouldAdd(BlockingRawTestBase.java:59)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)

📦 integration-tests/hibernate-orm-graphql-panache

io.quarkus.it.hibertnate.orm.graphql.panache.HibernateOrmGraphQLPanacheTest.testEndpoint - History

  • 1 expectation failed. JSON path data.authors.id doesn't match. Expected: iterable containing [<1>] Actual: null - java.lang.AssertionError
Details
java.lang.AssertionError: 
1 expectation failed.
JSON path data.authors.id doesn't match.
Expected: iterable containing [<1>]
  Actual: null

	at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:344)
	at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure.validate(ResponseSpecificationImpl.groovy:516)

⚙️ JVM Integration Tests - JDK 21

📦 integration-tests/hibernate-orm-graphql-panache

io.quarkus.it.hibertnate.orm.graphql.panache.HibernateOrmGraphQLPanacheTest.testEndpoint - History

  • 1 expectation failed. JSON path data.authors.id doesn't match. Expected: iterable containing [<1>] Actual: null - java.lang.AssertionError
Details
java.lang.AssertionError: 
1 expectation failed.
JSON path data.authors.id doesn't match.
Expected: iterable containing [<1>]
  Actual: null

	at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:344)
	at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure.validate(ResponseSpecificationImpl.groovy:516)
  • 1 expectation failed. JSON path data.authors.id doesn't match. Expected: iterable containing [<1>] Actual: null - java.lang.AssertionError
Details
java.lang.AssertionError: 
1 expectation failed.
JSON path data.authors.id doesn't match.
Expected: iterable containing [<1>]
  Actual: null

	at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:344)
	at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure.validate(ResponseSpecificationImpl.groovy:516)
  • 1 expectation failed. JSON path data.books.id doesn't match. Expected: iterable containing [<2>, <3>, <4>, <5>] Actual: null - java.lang.AssertionError
Details
java.lang.AssertionError: 
1 expectation failed.
JSON path data.books.id doesn't match.
Expected: iterable containing [<2>, <3>, <4>, <5>]
  Actual: null

	at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:344)
	at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure.validate(ResponseSpecificationImpl.groovy:516)

⚙️ JVM Integration Tests - JDK 25

📦 integration-tests/hibernate-orm-graphql-panache

io.quarkus.it.hibertnate.orm.graphql.panache.HibernateOrmGraphQLPanacheTest.testEndpoint - History

  • 1 expectation failed. JSON path data.books.id doesn't match. Expected: iterable containing [<2>, <3>, <4>, <5>] Actual: null - java.lang.AssertionError
Details
java.lang.AssertionError: 
1 expectation failed.
JSON path data.books.id doesn't match.
Expected: iterable containing [<2>, <3>, <4>, <5>]
  Actual: null

	at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:344)
	at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure.validate(ResponseSpecificationImpl.groovy:516)

⚙️ JVM Integration Tests - JDK 25 Semeru

📦 integration-tests/hibernate-orm-graphql-panache

io.quarkus.it.hibertnate.orm.graphql.panache.HibernateOrmGraphQLPanacheTest.testEndpoint - History

  • 1 expectation failed. JSON path data.books.id doesn't match. Expected: iterable containing [<2>, <3>, <4>, <5>] Actual: null - java.lang.AssertionError
Details
java.lang.AssertionError: 
1 expectation failed.
JSON path data.books.id doesn't match.
Expected: iterable containing [<2>, <3>, <4>, <5>]
  Actual: null

	at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:344)
	at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure.validate(ResponseSpecificationImpl.groovy:516)
  • 1 expectation failed. JSON path data.books.id doesn't match. Expected: iterable containing [<2>, <3>, <4>, <5>] Actual: null - java.lang.AssertionError
Details
java.lang.AssertionError: 
1 expectation failed.
JSON path data.books.id doesn't match.
Expected: iterable containing [<2>, <3>, <4>, <5>]
  Actual: null

	at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:344)
	at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure.validate(ResponseSpecificationImpl.groovy:516)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Imperative Mailer not working with smallrye-graphql Mailer stuck when sending a message from GraphQL endpoint

6 participants