[Question] Could not install via pip or build from source code as a model of python

[Mikachu2333]

What I met

1. Could not install this package via pip or build from source code
2. I want to use the model HanLP in python and it need this

SYS infos

1. Win11 24h2 Pro (26100.2605, Official iso, amd x86_64, Lenovo ThinkBook 14s yoga)
2. Msys2 installed
3. Python 3.12 installed
4. MSVC toolchain (Visual Studio) installed
   • .NET desktop development
   • Desktop development with C++
   • Windows application development
5. Cmake and make installed
6. GNU target installed (mingw-w64-ucrt-x86_64)

$PATH$

%SYSTEMROOT%\System32\OpenSSH;
%SYSTEMROOT%\System32\WindowsPowerShell\v1.0;
%SystemRoot%;
%SystemRoot%\System32\Wbem;
%SystemRoot%\system32;
C:\Program Files\Eclipse Adoptium\jdk-21.0.4.7-hotspot\bin;
C:\Program Files\Graphviz\bin;
C:\Program Files\OpenSSL-Win64\bin;
C:\Program Files\PowerShell\7;
C:\Program Files\dotnet;
C:\msys64;
C:\msys64\usr\bin;
C:\msys64\ucrt64\bin;
C:\msys64\ucrt\include;
D:\Common_Process;
D:\Git\cmd;
D:\Microsoft VS Code\bin;
D:\Windows Kits\10\Windows Performance Toolkit\

MSYS2 softwares

base 2022.06-1
base-devel 2024.11-1
bash 5.2.037-1
bash-completion 2.16.0-1
binutils 2.43.1-1
bison 3.8.2-5
brotli 1.1.0-1
bsdtar 3.7.7-1
bzip2 1.0.8-4
ca-certificates 20240203-2
cmake 3.31.3-1
coreutils 8.32-5
cppdap 1.58.0a-3
curl 8.11.1-1
dash 0.5.12-1
db 6.2.32-5
diffstat 1.67-1
diffutils 3.10-1
dos2unix 7.5.2-1
file 5.46-1
filesystem 2024.12.18-1
findutils 4.10.0-2
flex 2.6.4-3
gawk 5.3.1-1
gcc-libs 13.3.0-1
gdbm 1.24-1
getent 2.18.90-5
gettext 0.22.4-1
git 2.47.1-1
gmp 6.3.0-1
gnupg 2.4.7-1
grep 13.0-6
gzip 1.13-1
heimdal 7.8.0-5
heimdal-libs 7.8.0-5
inetutils 2.5-2
info 7.1.1-1
isl 0.27-1
jsoncpp 1.9.6-2
less 668-1
libarchive 3.7.7-1
libargp 20241207-1
libasprintf 0.22.4-1
libassuan 2.5.7-1
libbz2 1.0.8-4
libcbor 0.11.0-1
libcurl 8.11.1-1
libdb 6.2.32-5
libedit 20240808_3.1-1
libexpat 2.6.4-1
libffi 3.4.6-1
libfido2 1.15.0-1
libgcrypt 1.11.0-1
libgdbm 1.24-1
libgettextpo 0.22.4-1
libgnutls 3.8.8-1
libgpg-error 1.51-1
libhogweed 3.10.1-1
libiconv 1.18-1
libidn2 2.3.7-1
libintl 0.22.4-1
libksba 1.6.7-1
liblz4 1.10.0-1
liblzma 5.6.3-1
libnettle 3.10.1-1
libnghttp2 1.64.0-1
libnpth 1.8-1
libopenssl 3.4.0-1
libp11-kit 0.25.5-2
libpcre 8.45-4
libpcre2_8 10.44-1
libpsl 0.21.5-2
libreadline 8.2.013-1
librhash 1.4.4-1
libsqlite 3.46.1-1
libssh2 1.11.1-1
libtasn1 4.19.0-1
libunistring 1.2-1
libutil-linux 2.35.2-4
libuv 1.49.2-1
libxcrypt 4.4.37-1
libzstd 1.5.6-1
m4 1.4.19-2
make 4.4.1-2
mingw-w64-cross-common-binutils 2.43.1-2
mingw-w64-cross-mingw64-binutils 2.43.1-2
mingw-w64-cross-mingw64-crt 12.0.0.r250.gc6bf4bdf6-2
mingw-w64-cross-mingw64-gcc 14.2.0-3
mingw-w64-cross-mingw64-headers 12.0.0.r250.gc6bf4bdf6-1
mingw-w64-cross-mingw64-windows-default-manifest 6.4-5
mingw-w64-cross-mingw64-winpthreads 12.0.0.r250.gc6bf4bdf6-1
mingw-w64-ucrt-x86_64-binutils 2.43.1-1
mingw-w64-ucrt-x86_64-bzip2 1.0.8-3
mingw-w64-ucrt-x86_64-crt-git 12.0.0.r459.g63f3f2846-1
mingw-w64-ucrt-x86_64-expat 2.6.4-1
mingw-w64-ucrt-x86_64-gcc 14.2.0-2
mingw-w64-ucrt-x86_64-gcc-libgfortran 14.2.0-2
mingw-w64-ucrt-x86_64-gcc-libs 14.2.0-2
mingw-w64-ucrt-x86_64-gdb 15.2-2
mingw-w64-ucrt-x86_64-gdb-multiarch 15.2-2
mingw-w64-ucrt-x86_64-gettext-runtime 0.23.1-1
mingw-w64-ucrt-x86_64-gmp 6.3.0-2
mingw-w64-ucrt-x86_64-gperf 3.1-2
mingw-w64-ucrt-x86_64-headers-git 12.0.0.r459.g63f3f2846-3
mingw-w64-ucrt-x86_64-isl 0.27-1
mingw-w64-ucrt-x86_64-libffi 3.4.6-1
mingw-w64-ucrt-x86_64-libiconv 1.18-1
mingw-w64-ucrt-x86_64-libmangle-git 12.0.0.r459.g63f3f2846-1
mingw-w64-ucrt-x86_64-libsystre 1.0.1-6
mingw-w64-ucrt-x86_64-libtre 0.9.0-1
mingw-w64-ucrt-x86_64-libwinpthread-git 12.0.0.r459.g63f3f2846-1
mingw-w64-ucrt-x86_64-make 4.4.1-3
mingw-w64-ucrt-x86_64-mpc 1.3.1-2
mingw-w64-ucrt-x86_64-mpdecimal 4.0.0-1
mingw-w64-ucrt-x86_64-mpfr 4.2.1-2
mingw-w64-ucrt-x86_64-ncurses 6.5.20241228-2
mingw-w64-ucrt-x86_64-openblas 0.3.28-2
mingw-w64-ucrt-x86_64-opencc 1.1.9-1
mingw-w64-ucrt-x86_64-openssl 3.4.0-1
mingw-w64-ucrt-x86_64-pkgconf 12.3.0-1
mingw-w64-ucrt-x86_64-python 3.12.8-1
mingw-w64-ucrt-x86_64-python-argcomplete 3.5.3-1
mingw-w64-ucrt-x86_64-python-arrow 1.3.0-3
mingw-w64-ucrt-x86_64-python-babel 2.16.0-2
mingw-w64-ucrt-x86_64-python-certifi 2024.8.30-2
mingw-w64-ucrt-x86_64-python-charset-normalizer 3.4.1-1
mingw-w64-ucrt-x86_64-python-click 8.1.7-2
mingw-w64-ucrt-x86_64-python-colorama 0.4.6-3
mingw-w64-ucrt-x86_64-python-dateutil 2.9.0-2
mingw-w64-ucrt-x86_64-python-distlib 0.3.8-3
mingw-w64-ucrt-x86_64-python-docutils 0.20.1-3
mingw-w64-ucrt-x86_64-python-idna 3.8-2
mingw-w64-ucrt-x86_64-python-imagesize 1.4.1-3
mingw-w64-ucrt-x86_64-python-jinja 3.1.5-1
mingw-w64-ucrt-x86_64-python-markupsafe 2.1.5-3
mingw-w64-ucrt-x86_64-python-numpy 2.2.0-1
mingw-w64-ucrt-x86_64-python-packaging 24.2-1
mingw-w64-ucrt-x86_64-python-pandas 2.2.2-4
mingw-w64-ucrt-x86_64-python-pip 24.3.1-1
mingw-w64-ucrt-x86_64-python-pipx 1.7.1-2
mingw-w64-ucrt-x86_64-python-platformdirs 4.3.6-2
mingw-w64-ucrt-x86_64-python-pygments 2.18.0-2
mingw-w64-ucrt-x86_64-python-pytz 2024.2-2
mingw-w64-ucrt-x86_64-python-requests 2.32.3-2
mingw-w64-ucrt-x86_64-python-setuptools 75.6.0-1
mingw-w64-ucrt-x86_64-python-six 1.16.0-6
mingw-w64-ucrt-x86_64-python-snowballstemmer 2.2.0-4
mingw-w64-ucrt-x86_64-python-sphinx 8.1.3-2
mingw-w64-ucrt-x86_64-python-sphinx-alabaster-theme 1.0.0-2
mingw-w64-ucrt-x86_64-python-sphinxcontrib-applehelp 2.0.0-2
mingw-w64-ucrt-x86_64-python-sphinxcontrib-devhelp 2.0.0-2
mingw-w64-ucrt-x86_64-python-sphinxcontrib-htmlhelp 2.1.0-2
mingw-w64-ucrt-x86_64-python-sphinxcontrib-jsmath 1.0.1-5
mingw-w64-ucrt-x86_64-python-sphinxcontrib-moderncmakedomain 3.29.0-2
mingw-w64-ucrt-x86_64-python-sphinxcontrib-qthelp 2.0.0-2
mingw-w64-ucrt-x86_64-python-sphinxcontrib-serializinghtml 2.0.0-2
mingw-w64-ucrt-x86_64-python-types-python-dateutil 2.9.0.20241003-1
mingw-w64-ucrt-x86_64-python-tzdata 2024.2-2
mingw-w64-ucrt-x86_64-python-urllib3 1.26.20-2
mingw-w64-ucrt-x86_64-python-userpath 1.9.2-2
mingw-w64-ucrt-x86_64-readline 8.2.013-1
mingw-w64-ucrt-x86_64-sqlite3 3.47.2-1
mingw-w64-ucrt-x86_64-tcl 8.6.13-1
mingw-w64-ucrt-x86_64-termcap 1.3.1-7
mingw-w64-ucrt-x86_64-tk 8.6.13-1
mingw-w64-ucrt-x86_64-tools-git 12.0.0.r459.g63f3f2846-1
mingw-w64-ucrt-x86_64-tzdata 2024b-1
mingw-w64-ucrt-x86_64-windows-default-manifest 6.4-4
mingw-w64-ucrt-x86_64-winpthreads-git 12.0.0.r459.g63f3f2846-1
mingw-w64-ucrt-x86_64-winstorecompat-git 12.0.0.r459.g63f3f2846-1
mingw-w64-ucrt-x86_64-xxhash 0.8.2-2
mingw-w64-ucrt-x86_64-xz 5.6.3-3
mingw-w64-ucrt-x86_64-zlib 1.3.1-1
mingw-w64-ucrt-x86_64-zstd 1.5.6-2
mingw-w64-x86_64-binutils 2.43.1-1
mingw-w64-x86_64-bzip2 1.0.8-3
mingw-w64-x86_64-crt-git 12.0.0.r459.g63f3f2846-1
mingw-w64-x86_64-expat 2.6.4-1
mingw-w64-x86_64-gcc 14.2.0-2
mingw-w64-x86_64-gcc-libs 14.2.0-2
mingw-w64-x86_64-gcc-rust 14.2.0-2
mingw-w64-x86_64-gettext-runtime 0.23.1-1
mingw-w64-x86_64-gmp 6.3.0-2
mingw-w64-x86_64-headers-git 12.0.0.r459.g63f3f2846-3
mingw-w64-x86_64-isl 0.27-1
mingw-w64-x86_64-libffi 3.4.6-1
mingw-w64-x86_64-libiconv 1.18-1
mingw-w64-x86_64-libsystre 1.0.1-6
mingw-w64-x86_64-libtre 0.9.0-1
mingw-w64-x86_64-libwinpthread-git 12.0.0.r459.g63f3f2846-1
mingw-w64-x86_64-mpc 1.3.1-2
mingw-w64-x86_64-mpdecimal 4.0.0-1
mingw-w64-x86_64-mpfr 4.2.1-2
mingw-w64-x86_64-ncurses 6.5.20241228-2
mingw-w64-x86_64-openssl 3.4.0-1
mingw-w64-x86_64-python 3.12.8-1
mingw-w64-x86_64-python-regex 2024.11.6-1
mingw-w64-x86_64-readline 8.2.013-1
mingw-w64-x86_64-sqlite3 3.47.2-1
mingw-w64-x86_64-tcl 8.6.13-1
mingw-w64-x86_64-termcap 1.3.1-7
mingw-w64-x86_64-tk 8.6.13-1
mingw-w64-x86_64-tzdata 2024b-1
mingw-w64-x86_64-windows-default-manifest 6.4-4
mingw-w64-x86_64-winpthreads-git 12.0.0.r459.g63f3f2846-1
mingw-w64-x86_64-xz 5.6.3-3
mingw-w64-x86_64-zlib 1.3.1-1
mingw-w64-x86_64-zstd 1.5.6-2
mintty 13.7.7-1
mpc 1.3.1-1
mpdecimal 4.0.0-1
mpfr 4.2.1-1
msys2-keyring 120241007-1
msys2-launcher 1.5-3
msys2-runtime 3.5.4-8
nano 8.2-1
nasm 2.16.03-1
ncurses 6.5.20240831-2
nettle 3.10.1-1
openssh 9.9p1-1
openssl 3.4.0-1
p11-kit 0.25.5-2
pacman 6.1.0-10
pacman-contrib 1.10.6-1
pacman-mirrors 20241217-1
patch 2.7.6-2
perl 5.38.2-2
perl-Authen-SASL 2.1700-1
perl-Clone 0.47-1
perl-Convert-BinHex 1.125-2
perl-Encode-Locale 1.05-2
perl-Error 0.17029-2
perl-File-Listing 6.16-1
perl-HTML-Parser 3.83-1
perl-HTML-Tagset 3.24-1
perl-HTTP-Cookies 6.11-1
perl-HTTP-Daemon 6.16-1
perl-HTTP-Date 6.06-1
perl-HTTP-Message 7.00-1
perl-HTTP-Negotiate 6.01-3
perl-IO-HTML 1.004-2
perl-IO-Socket-SSL 2.089-1
perl-IO-Stringy 2.113-2
perl-LWP-MediaTypes 6.04-2
perl-MIME-tools 5.515-1
perl-MailTools 2.22-1
perl-Net-HTTP 6.23-1
perl-Net-SMTP-SSL 1.04-2
perl-Net-SSLeay 1.94-1
perl-TermReadKey 2.38-6
perl-TimeDate 2.33-2
perl-Try-Tiny 0.32-1
perl-URI 5.31-1
perl-WWW-RobotRules 6.02-3
perl-http-cookiejar 0.014-1
perl-libwww 6.77-1
pinentry 1.3.1-1
pkgconf 2.3.0-1
python 3.12.8-1
python-setuptools 75.2.0-1
rebase 4.5.0-4
sed 4.9-1
tar 1.35-2
texinfo 7.1.1-1
texinfo-tex 7.1.1-1
time 1.9-3
tzcode 2024a-1
util-linux 2.35.2-4
wget 1.25.0-1
which 2.21-4
xz 5.6.3-1
zlib 1.3.1-1
zstd 1.5.6-1

Python list

Package Version
alabaster 1.0.0
argcomplete 3.5.3
arrow 1.3.0
babel 2.16.0
certifi 2024.8.30
charset-normalizer 3.4.1
click 8.1.7
cnlunar 0.2.0
colorama 0.4.6
distlib 0.3.8
docutils 0.20.1
hanlp_common 0.0.22
hanlp_restful 0.0.24
idna 3.8
imagesize 1.4.1
Jinja2 3.1.5
MarkupSafe 2.1.5
numpy 2.2.0
packaging 24.2
pandas 2.2.2
phrasetree 0.0.9
pip 24.3.1
pipx 1.7.1
platformdirs 4.3.6
Pygments 2.18.0
python-dateutil 2.9.0
pytz 2024.2
requests 2.32.3
setuptools 75.6.0
six 1.16.0
snowballstemmer 2.2.0
Sphinx 8.1.3
sphinxcontrib-applehelp 2.0.0
sphinxcontrib-devhelp 2.0.0
sphinxcontrib-htmlhelp 2.1.0
sphinxcontrib-jsmath 1.0.1
sphinxcontrib-moderncmakedomain 3.29.0
sphinxcontrib-qthelp 2.0.0
sphinxcontrib-serializinghtml 2.0.0
types-python-dateutil 2.9.0.20241003
tzdata 2024.2
urllib3 1.26.20
userpath 1.9.2
wheel 0.45.1

Error info

pip

Collecting sentencepiece==0.2.0
  Using cached https://mirrors.ustc.edu.cn/pypi/packages/c9/d2/b9c7ca067c26d8ff085d252c89b5f69609ca93fb85a00ede95f4857865d4/sentencepiece-0.2.0.tar.gz (2.6 MB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [33 lines of output]
      CMake Deprecation Warning at CMakeLists.txt:15 (cmake_minimum_required):
        Compatibility with CMake < 3.10 will be removed from a future version of
        CMake.

        Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
        to tell CMake that the project requires at least <min> but has been updated
        to work with policies introduced by <max> or earlier.


      -- VERSION: 0.2.0
      CMake Error at CMakeLists.txt:23 (project):
        Generator

          Unix Makefiles

        does not support platform specification, but platform

          x64

        was specified.


      CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
      CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
      -- Configuring incomplete, errors occurred!
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "C:/msys64/tmp/pip-install-4gp49ze2/sentencepiece_bc98403eeed743409f481dd760c49335/setup.py", line 128, in <module>
          subprocess.check_call([
        File "C:/msys64/ucrt64/lib/python3.12/subprocess.py", line 413, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['cmake', 'sentencepiece', '-A', 'x64', '-B', 'build', '-DSPM_ENABLE_SHARED=OFF', '-DCMAKE_INSTALL_PREFIX=build\\root']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

build from source （MSVC）

PS D:\Temp\vcpkg> ./vcpkg install sentencepiece
Computing installation plan...
The following packages will be built and installed:
  * abseil:[email protected]
  * protobuf:[email protected]
    sentencepiece:[email protected]#1
  * utf8-range:[email protected]
  * vcpkg-cmake:x64-windows@2024-04-23
  * vcpkg-cmake-config:x64-windows@2024-05-23
Additional packages (*) will be modified to complete this operation.
Detecting compiler hash for triplet x64-windows...
error: while detecting compiler information:
The log file content at "D:\Temp\vcpkg\buildtrees\detect_compiler\stdout-x64-windows.log" is:
CMake Error at D:/Temp/vcpkg/scripts/ports.cmake:143 (message):
  Cannot find port:

    Directory does not exist: D;/Temp/vcpkg/scripts/detect_compiler


error: vcpkg was unable to detect the active compiler's information. See above for the CMake failure output.

build from source (GNU)

CMake Deprecation Warning at CMakeLists.txt:15 (cmake_minimum_required):
  Compatibility with CMake < 3.10 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.


-- VERSION: 0.2.1
-- The CXX compiler identification is unknown
CMake Error at CMakeLists.txt:23 (project):
  The CMAKE_CXX_COMPILER:

    C

  is not a full path and was not found in the PATH.

  Tell CMake where to find the compiler by setting either the environment
  variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path
  to the compiler, or to the compiler name if it is in the PATH.


-- Configuring incomplete, errors occurred!
make: *** No targets specified and no makefile found.  Stop.

What I've tried

1. I guess this is due to the inability to find the specified compiler, so I specified the gcc.exe and g++.exe of the GNU target (UCRT64) as the via through environment variables, but it still reported this error.
2. I've also tried cl.exe but nothing happened
3. I explore these issues: A library that conflicts with the use of protobuf in vcpkg #927 Tips for Termux installation #950 coredump when build with CXXFLAGS -Wp,-D_GLIBCXX_ASSERTIONS #987 Compatibility Issue when using v0.2.0 with transformers and tensorflow #1060 The pip command to install the SentencePiece Python module fails. #1070 Windows pip Dependancy Installation Error #990 install command line tools without sudo #1025 #
4. I tried to use cmake .. -DCMAKE_CXX_COMPILER=C:/msys2/usr/bin/g++.exe -DCMAKE_C_COMPILER=C:/msys2/usr/bin/gcc.exe to fix error but nothing happened

What I want

Install this project as python model successfully

[Mikachu2333]

I installed successfully on WSL(debian) but still failed on Windows

[dsnehitha]

i'm trying to install sentencepiece on mac and getting the issue with wheel building.