Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix build for macos #2397

Merged
merged 5 commits into from
Jan 21, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Add check to directly use ANN Search when filters match all docs. (#2320)[https://github.com/opensearch-project/k-NN/pull/2320]
- Use one formula to calculate cosine similarity (#2357)[https://github.com/opensearch-project/k-NN/pull/2357]
- Add WithFieldName implementation to KNNQueryBuilder (#2398)[https://github.com/opensearch-project/k-NN/pull/2398]
- Make the build work for M series MacOS without manual code changes and local JAVA_HOME config (#2397)[https://github.com/opensearch-project/k-NN/pull/2397]
### Bug Fixes
* Fixing the bug when a segment has no vector field present for disk based vector search (#2282)[https://github.com/opensearch-project/k-NN/pull/2282]
* Fixing the bug where search fails with "fields" parameter for an index with a knn_vector field (#2314)[https://github.com/opensearch-project/k-NN/pull/2314]
Expand Down
15 changes: 15 additions & 0 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,21 @@ export OPENSEARCH_HOME=the directory of opensearch...
export JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:$OPENSEARCH_HOME/plugins/opensearch-knn/lib
```

CMAKE will use as JAVA_HOME environment whatever your gradle is currently using. For example:
```bash
Java home directory found by gradle: /opt/homebrew/Cellar/openjdk@21/21.0.5/libexec/openjdk.jdk/Contents/Home
=======================================
OpenSearch Build Hamster says Hello!
Gradle Version : 8.4
OS Info : Mac OS X 14.4 (aarch64)
JDK Version : 21 (Homebrew JDK)
JAVA_HOME : /opt/homebrew/Cellar/openjdk@21/21.0.5/libexec/openjdk.jdk/Contents/Home
Random Testing Seed : 8AB32A4719AA345E
In FIPS 140 mode : false
=======================================
```
The JAVA_HOME used by gradle will be the default that the project will be using.

#### Environment

Currently, the plugin only supports Linux on x64 and arm platforms.
Expand Down
8 changes: 8 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
* SPDX-License-Identifier: Apache-2.0
*/


import org.gradle.internal.jvm.Jvm
import org.opensearch.gradle.test.RestIntegTestTask
import org.opensearch.gradle.testclusters.OpenSearchCluster
import org.apache.tools.ant.taskdefs.condition.Os
Expand Down Expand Up @@ -336,13 +338,19 @@ task cmakeJniLib(type:Exec) {
args.add("-DAVX512_ENABLED=${avx512_enabled}")
args.add("-DCOMMIT_LIB_PATCHES=${commit_lib_patches}")
args.add("-DAPPLY_LIB_PATCHES=${apply_lib_patches}")
def javaHome = Jvm.current().getJavaHome()
logger.lifecycle("Java home directory used by gradle: $javaHome")
if (Os.isFamily(Os.FAMILY_MAC)) {
environment('JAVA_HOME',javaHome)
}
if (Os.isFamily(Os.FAMILY_WINDOWS)) {
dependsOn windowsPatches
args.add("-G")
args.add("Unix Makefiles")
args.add("-DBLAS_LIBRARIES=$rootDir\\src\\main\\resources\\windowsDependencies\\libopenblas.dll")
args.add("-DLAPACK_LIBRARIES=$rootDir\\src\\main\\resources\\windowsDependencies\\libopenblas.dll")
}

commandLine args
}

Expand Down
39 changes: 29 additions & 10 deletions jni/cmake/init-faiss.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -54,18 +54,37 @@ if(NOT DEFINED APPLY_LIB_PATCHES OR "${APPLY_LIB_PATCHES}" STREQUAL true)
endforeach()
endif()

if (${CMAKE_SYSTEM_NAME} STREQUAL Darwin)
if(CMAKE_C_COMPILER_ID MATCHES "Clang\$")
set(OpenMP_C_FLAGS "-Xpreprocessor -fopenmp")
set(OpenMP_C_LIB_NAMES "omp")
set(OpenMP_omp_LIBRARY /usr/local/opt/libomp/lib/libomp.dylib)
endif()
if (APPLE)
message(STATUS "darwin macos detected")
if(CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64")
message(STATUS "detected Mac with ARM architecture")
if(CMAKE_C_COMPILER_ID MATCHES "Clang\$")
set(OpenMP_C_FLAGS "-Xpreprocessor -fopenmp")
set(OpenMP_C_LIB_NAMES "omp")
set(OpenMP_omp_LIBRARY /opt/homebrew/opt/libomp/lib/libomp.dylib)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this also going to be system dependent?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes this is system independent. Newer version of brew will use that path by default while the old ones use the old path that was hard coded.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern is more that users may not use brew to install omp

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The guideline in the instructions right now for Mac is to use brew, if this guideline is followed the build will fail unless the above is set.

endif()

if(CMAKE_CXX_COMPILER_ID MATCHES "Clang\$")
set(OpenMP_CXX_FLAGS "-Xpreprocessor -fopenmp -I/opt/homebrew/opt/libomp/include")
set(OpenMP_CXX_LIB_NAMES "omp")
set(OpenMP_omp_LIBRARY /opt/homebrew/opt/libomp/lib/libomp.dylib)
endif()
else()
message(STATUS "detected Mac with x86 architecture")
if(CMAKE_C_COMPILER_ID MATCHES "Clang\$")
set(OpenMP_C_FLAGS "-Xpreprocessor -fopenmp")
set(OpenMP_C_LIB_NAMES "omp")
set(OpenMP_omp_LIBRARY /usr/local/opt/libomp/lib/libomp.dylib)
endif()

if(CMAKE_CXX_COMPILER_ID MATCHES "Clang\$")
set(OpenMP_CXX_FLAGS "-Xpreprocessor -fopenmp -I/usr/local/opt/libomp/include")
set(OpenMP_CXX_LIB_NAMES "omp")
set(OpenMP_omp_LIBRARY /usr/local/opt/libomp/lib/libomp.dylib)
if(CMAKE_CXX_COMPILER_ID MATCHES "Clang\$")
set(OpenMP_CXX_FLAGS "-Xpreprocessor -fopenmp -I/usr/local/opt/libomp/include")
set(OpenMP_CXX_LIB_NAMES "omp")
set(OpenMP_omp_LIBRARY /usr/local/opt/libomp/lib/libomp.dylib)
endif()
endif()


endif()

find_package(ZLIB REQUIRED)
Expand Down
Loading