Merge pull request #1037 from Nagico2/feat/win-arm-build

Add support for windows arm64

Author: taku910

Diff for: .github/workflows/wheel.yml

@@ -45,6 +45,8 @@ jobs:
           cmake --build ${{github.workspace}}/build_win32 --config Release --target install --parallel 8
           cmake -A x64   -B ${{github.workspace}}/build_amd64 -DSPM_ENABLE_SHARED=OFF -DCMAKE_INSTALL_PREFIX=${{github.workspace}}/build/root_amd64
           cmake --build ${{github.workspace}}/build_amd64 --config Release --target install --parallel 8
+          cmake -A arm64 -B ${{github.workspace}}/build_arm64 -DSPM_ENABLE_SHARED=OFF -DCMAKE_INSTALL_PREFIX=${{github.workspace}}/build/root_arm64
+          cmake --build ${{github.workspace}}/build_arm64 --config Release --target install --parallel 8
 
       - name: Build for Mac
         if: runner.os == 'macOS'
@@ -66,6 +68,7 @@ jobs:
         env:
           CIBW_ARCHS_LINUX: auto aarch64
           CIBW_ARCHS_MACOS: x86_64 universal2 arm64
+          CIBW_ARCHS_WINDOWS: auto ARM64
           CIBW_SKIP: "pp* *-musllinux_*"
           CIBW_BUILD_VERBOSITY: 1
 

Diff for: python/README.md

@@ -3,7 +3,7 @@
 Python wrapper for SentencePiece. This API will offer the encoding, decoding and training of Sentencepiece.
 
 ## Build and Install SentencePiece
-For Linux (x64/i686), macOS, and Windows(win32/x64) environment, you can simply use pip command to install SentencePiece python module.
+For Linux (x64/i686), macOS, and Windows(win32/x64/arm64) environment, you can simply use pip command to install SentencePiece python module.
 
 ```
 % pip install sentencepiece
@@ -27,6 +27,20 @@ If you don’t have write permission to the global site-packages directory or do
 % python setup.py install --user
 ```
 
+For Windows users who want to build from source, you can build and install the Python wrapper using Visual Studio. First, you need to install the `pwsh.exe` (Powershell 7). Use `winget install --id Microsoft.Powershell --source winget` to install directly. Then open the `Developer PowerShell for VS 2022`, and execute the following commands. 
+```
+git clone https://github.com/google/sentencepiece.git
+cd sentencepiece
+mkdir build
+cd build
+cmake .. -DSPM_ENABLE_SHARED=OFF -DCMAKE_INSTALL_PREFIX=".\root" 
+cmake --build . --config Release --target install
+cd ../python
+pip install wheel
+python setup.py bdist_wheel
+Get-ChildItem .\dist\sentencepiece*.whl | ForEach-Object { pip install $_.FullName }
+```
+
 ## Usage
 
 See [this google colab page](https://github.com/google/sentencepiece/blob/master/python/sentencepiece_python_module_example.ipynb) to run sentencepiece interactively.

Diff for: python/setup.py

@@ -19,6 +19,7 @@
 import string
 import subprocess
 import sys
+import platform
 from setuptools import Extension, setup
 from setuptools.command.build_ext import build_ext as _build_ext
 from setuptools.command.build_py import build_py as _build_py
@@ -103,11 +104,21 @@ def build_extension(self, ext):
     _build_ext.build_extension(self, ext)
 
 
-if os.name == 'nt':
-  # Must pre-install sentencepice into build directory.
+def get_win_arch():
   arch = 'win32'
   if sys.maxsize > 2**32:
     arch = 'amd64'
+  if 'arm' in platform.machine().lower():
+    arch = 'arm64'
+  if os.getenv('PYTHON_ARCH', '') == 'ARM64':
+    # Special check for arm64 under ciwheelbuild, see https://github.com/pypa/cibuildwheel/issues/1942
+    arch = 'arm64'
+  return arch
+
+
+if os.name == 'nt':
+  # Must pre-install sentencepice into build directory.
+  arch = get_win_arch()
   if os.path.exists('..\\build\\root_{}\\lib'.format(arch)):
     cflags = ['/std:c++17', '/I..\\build\\root_{}\\include'.format(arch)]
     libs = [
@@ -125,6 +136,8 @@ def build_extension(self, ext):
     cmake_arch = 'Win32'
     if arch == 'amd64':
       cmake_arch = 'x64'
+    elif arch == "arm64":
+      cmake_arch = "ARM64"
     subprocess.check_call([
         'cmake',
         'sentencepiece',