Best practices for library development with custom features

[kkriske]

I am currently working on enhancing the native-image support for a library that isn't mine. I have no say over the release cadence and the used JDK, so to make sure compilation always works I am adding the org.graalvm.sdk:graal-sdk as an optional dependency:

<dependency>
    <groupId>org.graalvm.sdk</groupId>
    <artifactId>graal-sdk</artifactId>
    <version>22.3.0</version>
    <optional>true</optional>
</dependency>

Is this the proper way to add support for current and upcomming GraalVM releases? In my understanding, for the end-user this would make sure that the graal-sdk is not fetched from maven and the org.graalvm.sdk module of the JDK is used instead. So as long as there are no breaking changes to the API this should continue to function.

In the future, would the proper way be to add a dependency to the most recent LTS release? So that would be GraalVM for JDK 21 starting from september 2023 assuming integration of JDK by then.

In a project using the module system, does adding the requirement to org.graalvm.sdk prevent compilation on non-graalvm JDKs?

/cc @fniephaus

[fniephaus]

Hi @kkriske,

Thanks for raising this. Lots of interesting but complicated questions. Let me try to answer them one by one as well as I can...

Is this the proper way to add support for current and upcoming GraalVM releases?

I'd say it is the proper way to add support for a specific GraalVM release. Maven typically requires a specific version, so you can't say use the latest version of some dependency. If it helps, you could make the GraalVM version adjustable with something like:

  <properties>
    <!-- ... -->
    <graalvm-version>22.3.0</graalvm-version>
  </properties>

  <dependency>
      <groupId>org.graalvm.sdk</groupId>
      <artifactId>graal-sdk</artifactId>
      <version>${graalvm-version}</version>
      <optional>true</optional>
   </dependency>

and then overwrite the version via, for example, mvn -Dgraalvm-version=19.2.0 .... I think you could use this to build multiple versions of the same library against different GraalVM versions on one JDK.

In my understanding, for the end-user this would make sure that the graal-sdk is not fetched from maven and the org.graalvm.sdk module of the JDK is used instead.

I just tried this out myself and indeed, it seems to be the case that the org.graalvm.sdk module from the JDK takes precedence over the Maven dependency, but Maven still downloads the dependency. So technically, you should be able to "link" a library against a specific GraalVM version by building with the corresponding version of GraalVM. If you want to achieve the same without a GraalVM installation, you need to specify the GraalVM version again, either in the POM or dynamically via a property for example.

In the future, would the proper way be to add a dependency to the most recent LTS release?

I guess it depends on the things you need in the library. What if you want to use an API that was introduced or deprecated after the most recent LTS release?

In a project using the module system, does adding the requirement to org.graalvm.sdk prevent compilation on non-graalvm JDKs?

Yes, I think that is the case.

---

I'm afraid I couldn't answer all your questions entirely but I hope this helps. We are in the bizarre situation where GraalVM has both a public API and JDK distributions. As you probably know, we are aligning with the OpenJDK and not all details have been figured out yet (e.g., maybe we need to split org.graalvm.sdk). If you could elaborate on your use case and the problems you'd like to solve, that could help us make the right calls for our users in the future.

[kkriske]

it seems to be the case that the org.graalvm.sdk module from the JDK takes precedence over the Maven dependency, but Maven still downloads the dependency.

For the library developer yes, but for the library consumer I think the graal-sdk should not be downloaded because the dependency is marked optional in the library itself. So it would be up to the consumer to add it if required.

---

For libraries it is often the case that at least all LTS versions need to be supported. So having different releases for different JDK versions is not that great. I'm also strugling a bit with multi-release jars because I cannot write a Feature in a java 8 codebase, but the feature cannot be found by the native-tests of the native-build-tools from the multi-release java 11 folder in the jar.

Having a multirelease JAR with the same feature class for every release would help to provide a feature that specifically works for a certain GraalVM release. At least that will be the case from now on if GraalVM releases follow OpenJDK versioning.

I guess it depends on the things you need in the library. What if you want to use an API that was introduced or deprecated after the most recent LTS release?

That's a general restriction in library development. If you want your library to be usable with java 8 and java 17, you can't use whatever was added in java 9 and later, but you also can't use whatever was removed. This would also possibly be fixed by multi-release jars.

---

We are in the bizarre situation where GraalVM has both a public API and JDK distributions.

I think an important thing would be to integrate the SDK asap, so there is no need for a separate maven artifact, but that's probably not possible without integrating native image itself because having the API but not the implementation in OpenJDK doesn't really make sense.

---

If you could elaborate on your use case and the problems you'd like to solve, that could help us make the right calls for our users in the future.

I am working on enhancing the native-image support of org.xerial:sqlite-jdbc, however I want to do it properly as to not cause maintenance overhead for the maintainers and without disrupting the current established processes such as currend development flows or release cadences. For example, I don't want to introduce changes that will force them to always use a GraalVM JDK, which is apparently problematic for a project using the module system as it requires an explicit dependency on the org.gralvm.sdk module.
xerial/sqlite-jdbc#823

[kkriske]

@fniephaus tag for visibility.

[fniephaus]

Thanks for the ping, @kkriske. I didn't see an immediate question but let me leave some more comments...

---

Having a multirelease JAR with the same feature class for every release would help to provide a feature that specifically works for a certain GraalVM release. At least that will be the case from now on if GraalVM releases follow OpenJDK versioning.

I'm not sure multi-release JARs will be feasible in the future though because the set of features may no longer be identical. In this case, there will probably still be different artifacts to depend on at development time, unless some part of the API becomes part of the JDK. In that latter case, libraries would need to check for feature availability, which I guess would also work today if a library runs on different versions of GraalVM (which includes our SDK), or am I missing something?

---

I think an important thing would be to integrate the SDK asap, so there is no need for a separate maven artifact, but that's probably not possible without integrating native image itself because having the API but not the implementation in OpenJDK doesn't really make sense.

I see why you'd like the integration to be done asap but I'm afraid that's not how OpenJDK projects work.

---

I am working on enhancing the native-image support of org.xerial:sqlite-jdbc, however I want to do it properly as to not cause maintenance overhead for the maintainers and without disrupting the current established processes such as currend development flows or release cadences.

I totally understand and thanks for doing this, we want to support you as well as we can. Nonetheless, project Galahad was announced only a few weeks and hasn't even been created yet. At this point, I'm afraid libraries and frameworks must expect some additional maintenance overhead to support GraalVM until the project is completed.

[pyckle]

it seems to be the case that the org.graalvm.sdk module from the JDK takes precedence over the Maven dependency, but Maven still downloads the dependency.

For the library developer yes, but for the library consumer I think the graal-sdk should not be downloaded because the dependency is marked optional in the library itself. So it would be up to the consumer to add it if required.

Hey @fniephaus, question regarding this.

Is there a situation where this library should be included by maven at runtime, or is it always provided by graalvm and the lib is only necessary for compilation on a non-graalvm jdk? If it is the latter, wouldn't it make more sense to use the "provided" dependency scope in maven rather than optional?