[GR-62463] A language with id '???' is not installed

[bingo-soft]

I'm trying to build the simplest possible language on top of Truffle framework, but fail. This is how my build.gradle.kts looks like:

plugins {
    kotlin("jvm") version "2.0.21"
    id("antlr")
    id("com.strumenta.antlr-kotlin") version "1.0.0"
}

group = "pro.stery.core"
version = "1.0.2"

repositories {
    mavenCentral()
}

dependencies {
    antlr("org.antlr:antlr4:4.13.2")
    implementation("com.strumenta:antlr-kotlin-runtime:1.0.0")
    implementation("org.graalvm.sdk:graal-sdk:24.0.0")
    implementation("org.graalvm.truffle:truffle-api:24.0.0")
    implementation("org.graalvm.compiler:compiler:24.0.0")
    implementation("javax.annotation:javax.annotation-api:1.3.2")
    implementation("com.google.code.findbugs:jsr305:3.0.2")
    annotationProcessor("org.graalvm.truffle:truffle-dsl-processor:24.0.0")
    testImplementation(platform("org.junit:junit-bom:5.10.0"))
    testImplementation(kotlin("test"))
}

sourceSets["main"].java.srcDir("build/generated-src/antlr/main")

tasks.register<com.strumenta.antlrkotlin.gradle.AntlrKotlinTask>("generateKotlinGrammarSource") {
    maxHeapSize = "64m"
    packageName = "pro.stery.core.parser.antlr"
    arguments = listOf("-visitor")
    source = project.objects
        .sourceDirectorySet("antlr", "antlr")
        .srcDir("src/main/antlr").apply {
            include("*.g4")
        }
    outputDirectory = File("build/generated-src/antlr/main")
    //outputDirectory = File("src/build/generated-src")
}

tasks.named("generateGrammarSource").configure {
    enabled = false
}

tasks.named("compileKotlin") {
    dependsOn(tasks.named("generateKotlinGrammarSource"))
}

tasks.test {
    useJUnitPlatform()
    //jvmArgs("-ea", "-Dgraalvm.locatorDisabled=true", "--add-exports", "org.graalvm.truffle/com.oracle.truffle.api=ALL-UNNAMED",
    //"--add-exports", "org.graalvm.truffle/com.oracle.truffle.api.nodes=ALL-UNNAMED", "--add-exports",
    //"org.graalvm.truffle/com.oracle.truffle.api.staticobject=ALL-UNNAMED')")
}

tasks.withType<Test> {
    testLogging {
        events("passed", "skipped", "failed") // Make sure test results are shown
        showStandardStreams = true // This ensures that your println statements are shown
    }
}

kotlin {
    jvmToolchain(21)
}

And these are my classes. First one defines some metadata about language

package pro.stery.core


object LanguageInfo {
    const val ID: String = "stery"
    const val NAME: String = "stery"
    const val IMPLEMENTATION: String = "SteryRuntime"
    const val VERSION: String = "0.0.1"
    const val MIME_TYPE: String = "application/x-stery"
    const val FILE_EXTENSION: String = ".stery"
}

Custome context class:

package pro.stery.core.runtime

import com.oracle.truffle.api.TruffleLanguage.ContextReference
import com.oracle.truffle.api.nodes.Node


class VmContext {
    private lateinit var holder: Holder

    companion object {
        val REFERENCE: ContextReference<VmContext> = ContextReference.create(VmLanguage::class.java)

        fun get(node: Node?): VmContext {
            return REFERENCE[node]
        }
    }

    class Holder(
    ) {
    }

    fun initialize(holder: Holder) {
        this.holder = holder
    }
}

And the language class itself:

package pro.stery.core.runtime

import com.oracle.truffle.api.CallTarget
import com.oracle.truffle.api.TruffleLanguage
import com.oracle.truffle.api.nodes.Node;
import pro.stery.core.LanguageInfo


@TruffleLanguage.Registration(
    id = LanguageInfo.ID,
    name = LanguageInfo.NAME,
    version = LanguageInfo.VERSION,
    characterMimeTypes = [LanguageInfo.MIME_TYPE]
)
class VmLanguage : TruffleLanguage<VmContext>() {

    companion object {
        val REFERENCE: LanguageReference<VmLanguage> = LanguageReference.create(VmLanguage::class.java)

        fun get(node: Node?): VmLanguage {
            return REFERENCE[node]
        }
    }

    override fun createContext(env: Env?): VmContext {
        return VmContext()
    }

    public override fun parse(request: ParsingRequest?): CallTarget {
        throw UnsupportedOperationException("parse")
    }
}

I just minimized it as hard as possible. And this is how I try to test it:

package pro.stery.core.runtime

import org.graalvm.polyglot.Context
import org.graalvm.polyglot.Engine
import pro.stery.core.LanguageInfo

....
val engine = Engine.newBuilder(LanguageInfo.ID).option("engine.WarnInterpreterOnly", "false").build()
val context = Context.newBuilder(LanguageInfo.ID).engine(engine).allowAllAccess(true).build()
context.initialize(LanguageInfo.ID) //this raises error message

And this is the error I get:

java.lang.IllegalArgumentException: A language with id 'stery' is not installed. Installed languages are: [].
	at com.oracle.truffle.polyglot.PolyglotEngineException.illegalArgument(PolyglotEngineException.java:129)
	at com.oracle.truffle.polyglot.PolyglotEngineImpl.throwNotInstalled(PolyglotEngineImpl.java:1173)
	at com.oracle.truffle.polyglot.PolyglotEngineImpl.requirePublicLanguage(PolyglotEngineImpl.java:1180)
	at com.oracle.truffle.polyglot.PolyglotContextImpl.requirePublicLanguage(PolyglotContextImpl.java:1704)
	at com.oracle.truffle.polyglot.PolyglotContextImpl.lookupLanguageContext(PolyglotContextImpl.java:1666)
	at com.oracle.truffle.polyglot.PolyglotContextImpl.initializeLanguage(PolyglotContextImpl.java:1635)
	at com.oracle.truffle.polyglot.PolyglotContextDispatch.initializeLanguage(PolyglotContextDispatch.java:55)
	at org.graalvm.polyglot.Context.initialize(Context.java:579)

Unfortunatelly, I can not find any working example and not sure how to effectively debug it. I wish, someone explained how we can "install" our custom language. In documentation I see, that explicit initialize is not necessary, but I also tried to invoke eval on context, without calling initialize, but still get the very same error. Thanks in advance for any help! PS. I've seen similar issue threads, but to no avail. Documentation also seems to be rather outdated. Probably, I just have to know where to look. Thanks!

[selhagani]

Hi @bingo-soft,

Thank you for reaching out to us about this.
Did you get a chance to look through the following documentation links:
https://www.graalvm.org/latest/graalvm-as-a-platform/implement-language/
https://www.graalvm.org/latest/graalvm-as-a-platform/language-implementation-framework/Languages/

[bingo-soft]

@selhagani Thank you very much! Yes, sure, I've seen this documentation. It is outdated, unfortunatelly. It mentions for example GraalVM Updater tool, which however is no longer present. Probably, the whole problem is that I'm trying to implement my language in Kotlin (not in pure Java) and probably there are some nuances, about it. So, my current setup consist from several lines of code, which is easily reproducible.

First, LanguageInfo.kt:

package pro.stery.core


object LanguageInfo {
    const val ID: String = "stery"
    const val NAME: String = "stery"
    const val IMPLEMENTATION: String = "SteryRuntime"
    const val VERSION: String = "0.0.1"
    const val MIME_TYPE: String = "application/x-stery"
    const val FILE_EXTENSION: String = ".stery"
}

Second, context class:

package pro.stery.core.runtime

import com.oracle.truffle.api.TruffleLanguage.ContextReference
import com.oracle.truffle.api.nodes.Node


class VmContext {
    private lateinit var holder: Holder

    companion object {
        val REFERENCE: ContextReference<VmContext> = ContextReference.create(VmLanguage::class.java)

        fun get(node: Node?): VmContext {
            return REFERENCE[node]
        }
    }

    class Holder(
    ) {
    }

    fun initialize(holder: Holder) {
        this.holder = holder
    }
}

And third, language:

package pro.stery.core.runtime

import com.oracle.truffle.api.CallTarget
import com.oracle.truffle.api.TruffleLanguage
import com.oracle.truffle.api.nodes.Node;
import pro.stery.core.LanguageInfo


@TruffleLanguage.Registration(
    id = LanguageInfo.ID,
    name = LanguageInfo.NAME,
    version = LanguageInfo.VERSION,
    characterMimeTypes = [LanguageInfo.MIME_TYPE]
)
class VmLanguage : TruffleLanguage<VmContext>() {

    companion object {
        val REFERENCE: LanguageReference<VmLanguage> = LanguageReference.create(VmLanguage::class.java)

        fun get(node: Node?): VmLanguage {
            return REFERENCE[node]
        }
    }

    override fun createContext(env: Env?): VmContext {
        return VmContext()
    }

    public override fun parse(request: ParsingRequest?): CallTarget {
        throw UnsupportedOperationException("parse")
    }
}

And I also created first unit test, just to test this initial setup:

package pro.stery.core.runtime

import org.graalvm.polyglot.Context
import org.graalvm.polyglot.Engine
import org.junit.jupiter.api.Assertions.assertEquals
import org.junit.jupiter.api.Test
import pro.stery.core.LanguageInfo

class LanguageTest {
    @Test
    fun `language initialization`() {
        val engine = Engine.newBuilder(LanguageInfo.ID).option("engine.WarnInterpreterOnly", "false").build()
        val context = Context.newBuilder(LanguageInfo.ID).engine(engine).allowAllAccess(true).build()
        context.initialize(LanguageInfo.ID)
        assertEquals(1, 1)
    }
}

Again, I get this notorious error message:

A language with id 'stery' is not installed. Installed languages are: [].
java.lang.IllegalArgumentException: A language with id 'stery' is not installed. Installed languages are: [].
	at com.oracle.truffle.polyglot.PolyglotEngineException.illegalArgument(PolyglotEngineException.java:129)
	at com.oracle.truffle.polyglot.PolyglotEngineImpl.throwNotInstalled(PolyglotEngineImpl.java:1173)
	at com.oracle.truffle.polyglot.PolyglotEngineImpl.requirePublicLanguage(PolyglotEngineImpl.java:1180)
	at com.oracle.truffle.polyglot.PolyglotContextImpl.requirePublicLanguage(PolyglotContextImpl.java:1704)
	at com.oracle.truffle.polyglot.PolyglotContextImpl.lookupLanguageContext(PolyglotContextImpl.java:1666)
	at com.oracle.truffle.polyglot.PolyglotContextImpl.initializeLanguage(PolyglotContextImpl.java:1635)
	at com.oracle.truffle.polyglot.PolyglotContextDispatch.initializeLanguage(PolyglotContextDispatch.java:55)
	at org.graalvm.polyglot.Context.initialize(Context.java:579)
	at pro.stery.core.runtime.LanguageTest.language initialization(LanguageTest.kt:14)

@selhagani If you have any ideas beyond the scope of this outdated documentation, I would be really glad, if you could shed some light on this. Thanks in advance!

[bingo-soft]

I also tried other dependencies - graal versions 21 - 23. But they result into a bigger list of errors, even more shaky. So, 24 version looks nice, the only problem with it is how to "install" a custom language. Probably, there are some public methods to do this or some API to test these internals. I do not know this code flow, when some language is supposed to finally be installed. @selhagani You will do a big favour, if you could share your know-how. Thanks for your time!

[bingo-soft]

If you have any troubles with reproducing the error, let me know! Thank you!

[bingo-soft]

I minimized it even more. So, I checked two cases, first one - calling custom language implemented in Java. Here is its source code:

import com.oracle.truffle.api.CallTarget;
import com.oracle.truffle.api.TruffleLanguage;

@TruffleLanguage.Registration(id = "test", name = "Test")
public final class TestLanguage extends TruffleLanguage<Void> {

    @Override
    protected CallTarget parse(ParsingRequest request) throws Exception {
        throw new UnsupportedOperationException("unsupported");
    }

    @Override
    protected Void createContext(Env env) {
        return null;
    }
}

And the second case - calling custom language implemented in Kotlin:

import com.oracle.truffle.api.*

@TruffleLanguage.Registration(id = "ktl", name = "Ktl")
class KtlLanguage : TruffleLanguage<Void?>() {
    override fun parse(request: ParsingRequest): CallTarget {
        throw UnsupportedOperationException("unsupported")
    }

    override fun createContext(env: Env): Void? {
        return null
    }
}

And the way I test them - is load them to a separate project like so:

dependencies {
    ...
    implementation(project(":testLang"))
    implementation(project(":ktlLang"))
    testImplementation(kotlin("test"))
}

And the actual test:

import org.graalvm.polyglot.Context

...

Context.newBuilder("ktl") // replace with test and it works
            .allowHostAccess(HostAccess.ALL)
            .out(System.out)
            .err(System.err)
            .build()
            .use { context ->
                val bindings = context.getBindings("ktl") // replace with test and it works
           }

If I try to apply "ktl" language I get an error, that it is not installed, if however I try "test" language, it works. So, at first glance, it looks like @TruffleLanguage.Registration does not work as expected for Kotlin language class. If you have any ideas, how to fix it, that would be so great!

[selhagani]

Thank you for sharing the details with us!
I shared this with out team and I'll make sure to keep you updated.

[bingo-soft]

Hi there! @selhagani Are there any updates on this?

[chumer]

@bingo-soft sorry for the delay!

There can be two reasons for this problem:

1. The truffle-dsl-processor did not run as an annotation processor. The annotation processor generates the necessary service classes for you. You can easily check that by a class KtlLanguageProvider is contained in the build output.

See https://github.com/graalvm/simplelanguage/blob/master/language/pom.xml#L84

You probably know better than me how to configure the Kotlin compiler to pick up an annotation processor.

2. You have an issue you your classpath or you are using an outdated JDK (accidentally). I would need the output of your java -version and the full Java command line you are running to find out what could be wrong there.

You can also compare your language implementation to our example language SimpleLangauge:
https://github.com/graalvm/simplelanguage

Note that we do not recommend using Kotlin to implement Truffle languages. It may work, but you might see problems with certain more advanced Kotlin features when compiling or when using the annotation processors.

[bingo-soft]

Thank you, @chumer Christian! Yeah, the whole problem was with annotation processor, which did not generate necessary classes. I tried kapt processor, but it is almost deprecated. So my only working solution now is to create service classes by hand. I actually tried it and it works as expected.