Issue found on page 'Android'

[gis-wes]

Please describe the problem you encountered in the DuckDB documentation and include the "Page URL" link shown below.

The documented cmake command works for the listed extensions. It fails when "spatial" is added to the list. It can't find the dependency modules. Fails on "proj" first with:

22.21 CMake Error at $BUILD_PATH/_deps/spatial_extension_fc-src/CMakeLists.txt:52 (find_package):
22.21   Could not find a package configuration file provided by "PROJ" with any of
22.21   the following names:
22.21
22.21     PROJConfig.cmake
22.21     proj-config.cmake
22.21
22.21   Add the installation prefix of "PROJ" to CMAKE_PREFIX_PATH or set
22.21   "PROJ_DIR" to a directory containing one of the above files.  If "PROJ"
22.21   provides a separate development package or SDK, be sure it has been
22.21   installed.
22.21
22.21
22.22 -- Configuring incomplete, errors occurred!

Note: only create an issue if you wish to report a problem with the DuckDB documentation. For questions about DuckDB or the use of certain DuckDB features, use GitHub Discussions, Stack Overflow, or Discord.

Page URL: https://duckdb.org/docs/stable/dev/building/android.html

Seems like config difference is required with the dependencies. Still figuring it out.

[gis-wes]

Appears the difference is vcpkg not being utilized for the in-tree extensions. ZLIB is currently being found in the NDK tree and Proj is first dependency that needs to be fetched.

I've switched around the ndk and vcpkg toolchain files but only changes that ZLIB fails first when not looking in the NDK tree. Have considered building the duckdb-spatial separately. Ran into issues with the cmake version so far. There are PRs for "fixing android" on duckdb-spatial so this may just be in a bad state. Seems it's been that way for a while.

[szarnyasg]

Thanks for raising this. I'm cross-linking duckdb/duckdb-spatial#456 and duckdb/duckdb-spatial#345. Will improve the documentation.

[gis-wes]

The shortest path I found to a successful build is below: (still struggling with loading lib)

1. Clone spatial repo too:

+RUN git clone https://github.com/duckdb/duckdb-spatial.git -b $DUCKDB_VERSION

2. Install with vcpkg:

+RUN export ANDROID_NDK_HOME=${ANDROID_NDK_ROOT} && \
+     cd duckdb-spatial && \
+     vcpkg install --triplet arm64-android

3. Separately install what is missing from vcpkg.json config?

+RUN export ANDROID_NDK_HOME=${ANDROID_NDK_ROOT} && \
+    vcpkg install curl:arm64-android

4. Use vcpkg toolchain with cmake: (and other dependency related config)

-    -DCMAKE_TOOLCHAIN_FILE=${ANDROID_NDK_ROOT}/build/cmake/android.toolchain.cmake \
+    -DCMAKE_TOOLCHAIN_FILE=${VCPKG_TOOLCHAIN_PATH} \
+    -DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=${ANDROID_NDK_ROOT}/build/cmake/android.toolchain.cmake \
+    -DVCPKG_TARGET_TRIPLET=arm64-android \
+    -DANDROID_NDK=${ANDROID_NDK_ROOT} \
+    -DVCPKG_INSTALLED_DIR=${DUCKDB_BUILD_ROOT}/duckdb-spatial/vcpkg_installed \
+    -DCURL_INCLUDE_DIR=${VCPKG_ROOT}/installed/arm64-android/include/curl \
+    -DCURL_LIBRARY=${VCPKG_ROOT}/installed/arm64-android/lib/libcurl.a \

This builds fine, but it crashes when used. It's looking for "libdl.so.2" when libduckdb.so is loaded. A symptom of have something x86 related involved in the build I believe. I'll update again when I figure it out.

[gis-wes]

It seems the libdl.so.2 expectation is actually from the jdbc driver from duckdb-java repo.

It needed to be rebuilt for android.

git clone https://github.com/duckdb/duckdb-java.git -b $DUCKDB_VERSION
cd duckdb-java

sed -i '/project(/i \
if(NOT "$ENV{ANDROID_NDK}" STREQUAL "")\n\
    message(STATUS "Patching JNI build for Android NDK: $ENV{ANDROID_NDK}")\n\
    set(CMAKE_TOOLCHAIN_FILE "$ENV{ANDROID_NDK}/build/cmake/android.toolchain.cmake")\n\
    set(ANDROID_PLATFORM "$ENV{ANDROID_VERSION}")\n\
    set(ANDROID_ABI "$ENV{ANDROID_ABI}")\n\
    set(ANDROID_NDK "$ENV{ANDROID_NDK}")\n\
    set(JNI_EXTRA_LINK_FLAGS \n\
        "-llog" \n\
    )\n\
    add_compile_definitions(ANDROID)\n\
else()\n\
    set(JNI_EXTRA_LINK_FLAGS "")\n\
endif()\n\
' CMakeLists.txt

make release

The driver works afterward. The nested ".so" file causes a warning. Could load it separately.

[gis-wes]

Still can't query the spatial extension. Last hurdle is getting the spatial extension linked as a static library like other extensions in the build:

Linking CXX static library extension/parquet/libparquet_extension.a
Linking CXX static library extension/json/libjson_extension.a
Linking CXX static library extension/icu/libicu_extension.a

instead of

Linking CXX shared library extension/spatial/spatial.duckdb_extension

The spatial extension isn't detected by

SELECT extension_name, installed, install_mode, description FROM duckdb_extensions()

but it appears to be included when inspecting the shared library.

No success at overriding the linking behavior in the build. Might be able to activate it thru JNI.