How does Spark in Java/Scala communicate with the Native C++ Implementation of Velox Backend

[VaibhavFRI]

Wanted to know how gluten helps in offloading spark workload to velox backend and how it communicates

[zhztheplayer]

Hi @VaibhavFRI, this question seems to be a little bit too broad. Though if you are about to look at the code, you may started from the JNI wrapper files (E.g., JniWraper.scala / JniWraper.cc) which are responsible for handling communications between Java code and C++ code.

[VaibhavFRI]

Hi @zhztheplayer thanks will look into those files.
Is there a way to debug the native C++ code so that the function call stack can be obtained when spark runs with gluten?

[zhztheplayer]

I think a easy way is to gdb attach to the JVM process, but make sure Velox / Gluten libraries are built with debug information.