Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

various willow upgrades #38

Merged
merged 9 commits into from
Jun 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
176 changes: 49 additions & 127 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,40 +7,12 @@ on: [
]

env:
# Important to pin the clang version, cause we also use it for linting
CLANG_VERSION: 17
CLANG_TIDY_JOBS: 4
# LLVM MinGW download
LLVM_MINGW_VERSION: llvm-mingw-20231128-msvcrt-ubuntu-20.04-x86_64
LLVM_MINGW_DOWNLOAD: https://github.com/mstorsjo/llvm-mingw/releases/download/20231128/llvm-mingw-20231128-msvcrt-ubuntu-20.04-x86_64.tar.xz
# xwin settings
XWIN_VERSION: xwin-0.5.0-x86_64-unknown-linux-musl
XWIN_DOWNLOAD: https://github.com/Jake-Shadle/xwin/releases/download/0.5.0/xwin-0.5.0-x86_64-unknown-linux-musl.tar.gz
LLVM_MINGW_VERSION: llvm-mingw-20240619-msvcrt-ubuntu-20.04-x86_64
LLVM_MINGW_DOWNLOAD: https://github.com/mstorsjo/llvm-mingw/releases/download/20240619/llvm-mingw-20240619-msvcrt-ubuntu-20.04-x86_64.tar.xz

jobs:
cache-clang:
runs-on: windows-latest

steps:
- name: Cache Clang
uses: actions/cache@v4
id: cache-clang
with:
path: C:\Program Files\LLVM
key: ${{ runner.os }}-clang-${{ env.CLANG_VERSION }}
lookup-only: true

- name: Setup Clang
if: steps.cache-clang.outputs.cache-hit != 'true'
uses: egor-tensin/setup-clang@v1
with:
version: ${{ env.CLANG_VERSION }}

# ==============================================================================

build-windows:
runs-on: windows-latest
needs: cache-clang

strategy:
fail-fast: false
Expand All @@ -53,13 +25,9 @@ jobs:
]

steps:
- name: Restore Clang Cache
- name: Setup Clang
if: startswith(matrix.preset, 'clang')
uses: actions/cache/restore@v4
with:
path: C:\Program Files\LLVM
key: ${{ runner.os }}-clang-${{ env.CLANG_VERSION }}
fail-on-cache-miss: true
uses: egor-tensin/setup-clang@v1

- name: Add MSVC to PATH
if: startswith(matrix.preset, 'msvc')
Expand Down Expand Up @@ -94,7 +62,8 @@ jobs:
run: cmake --build out/build/${{ matrix.preset }}

build-ubuntu:
runs-on: ubuntu-latest
# Require at least 24 for the mingw build
runs-on: ubuntu-24.04

strategy:
fail-fast: false
Expand All @@ -104,98 +73,66 @@ jobs:
"clang-cross-ue4-x64-release",
"llvm-mingw-ue3-x86-release",
"llvm-mingw-ue4-x64-release",
# Currently, ubuntu-latest is 22.04, whose mingw version is too old, so disabling these
# builds for now
# Not sure of the exact threshold, 13.1.0 works
# "mingw-ue3-x86-release",
# "mingw-ue4-x64-release",
"mingw-ue3-x86-release",
"mingw-ue4-x64-release",
]

steps:
- name: Setup CMake and Ninja
uses: lukka/get-cmake@latest

- name: Setup msitools
- name: Setup apt packages
uses: awalsh128/cache-apt-pkgs-action@latest
with:
packages: msitools
packages: msitools python3-requests
version: ${{ runner.os }}-apt

# Both Clang and MinGW install quick enough that it's not worth caching
# Caching would also lose the +x - so we'd have to tar before caching/untar after, making it
# even slower
- name: Setup Clang
if: startswith(matrix.preset, 'clang')
run: |
wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
sudo ./llvm.sh ${{ env.CLANG_VERSION }}

sudo update-alternatives --install \
/usr/bin/clang \
clang \
/usr/bin/clang-${{ env.CLANG_VERSION }} \
200
sudo update-alternatives --install \
/usr/bin/clang++ \
clang++ \
/usr/bin/clang++-${{ env.CLANG_VERSION }} \
200
sudo update-alternatives --install \
/usr/bin/llvm-rc \
llvm-rc \
/usr/bin/llvm-rc-${{ env.CLANG_VERSION }} \
200

- name: Setup LLVM MinGW
if: startswith(matrix.preset, 'llvm-mingw')
run: |
wget -nv ${{ env.LLVM_MINGW_DOWNLOAD }}
tar -xf ${{ env.LLVM_MINGW_VERSION }}.tar.xz -C ~/
echo $(readlink -f ~/${{ env.LLVM_MINGW_VERSION }}/bin) >> $GITHUB_PATH

- name: Set up MinGW
- name: Setup MinGW
if: startswith(matrix.preset, 'mingw')
uses: egor-tensin/setup-mingw@v2
with:
platform: ${{ fromJSON('["x86", "x64"]')[contains(matrix.preset, 'x64')] }}

# xwin does take long enough that caching's worth it
- name: Restore xwin cache
if: contains(matrix.preset, 'cross')
- name: Setup Clang
if: startswith(matrix.preset, 'clang-cross')
uses: egor-tensin/setup-clang@v1

- name: Restore win sdk cache
if: startswith(matrix.preset, 'clang-cross')
uses: actions/cache@v4
id: cache-xwin
id: cache-win-sdk
with:
path: ~/xwin
key: ${{ runner.os }}-xwin
path: ~/win-sdk
key: ${{ runner.os }}-win-sdk

- name: Setup xwin
if: contains(matrix.preset, 'cross') && steps.cache-xwin.outputs.cache-hit != 'true'
- name: Setup win sdk
if: startswith(matrix.preset, 'clang-cross') && steps.cache-win-sdk.outputs.cache-hit != 'true'
run: |
wget -nv ${{ env.XWIN_DOWNLOAD }}
tar -xf ${{ env.XWIN_VERSION }}.tar.gz
${{ env.XWIN_VERSION }}/xwin \
--accept-license \
--arch x86,x86_64 \
splat \
--include-debug-libs \
--output ~/xwin
git clone https://github.com/mstorsjo/msvc-wine.git
msvc-wine/vsdownload.py --accept-license --dest ~/win-sdk Microsoft.VisualStudio.Workload.VCTools
msvc-wine/install.sh ~/win-sdk
rm -r msvc-wine

- name: Checkout repository and submodules
uses: actions/checkout@v4
with:
submodules: recursive

- name: Configure build
- name: Configure CMake
working-directory: ${{ env.GITHUB_WORKSPACE }}
run: |
pip install requests

cmake . \
--preset ${{ matrix.preset }} \
-G Ninja \
-DXWIN_DIR=$(readlink -f ~)/xwin
# The extra xwin dir arg will be ignored if we don't need it
# The extra msvc wine arg won't do anything if we're not cross compiling
run: >
cmake .
--preset ${{ matrix.preset }}
-G Ninja
-DMSVC_WINE_ENV_SCRIPT=$(readlink -f ~)/win-sdk/bin/${{ fromJSON('["x86", "x64"]')[contains(matrix.preset, 'x64')] }}/msvcenv.sh

- name: Build
working-directory: ${{ env.GITHUB_WORKSPACE }}
Expand All @@ -205,7 +142,6 @@ jobs:

clang-tidy:
runs-on: windows-latest
needs: cache-clang

strategy:
fail-fast: false
Expand All @@ -216,37 +152,34 @@ jobs:
]

steps:
- name: Restore Clang Cache
uses: actions/cache/restore@v4
with:
path: C:\Program Files\LLVM
key: ${{ runner.os }}-clang-${{ env.CLANG_VERSION }}
fail-on-cache-miss: true
- name: Setup Clang
if: startswith(matrix.preset, 'clang')
uses: egor-tensin/setup-clang@v1

- name: Setup CMake and Ninja
uses: lukka/get-cmake@latest

# Need newer python to run the python lib downloader script
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: ">=3.10"

# Needed pyyaml for clang tidy to enable `-export-fixes` and requests for the python lib downloader
- name: Install pip packages
run: pip install pyyaml requests

- name: Checkout repository and submodules
uses: actions/checkout@v4
with:
submodules: recursive

- name: Configure build
- name: Configure CMake
working-directory: ${{ env.GITHUB_WORKSPACE }}
# Also need pyyaml for clang tidy to enable `-export-fixes`
run: |
pip install pyyaml requests

cmake . `
--preset ${{ matrix.preset }} `
-G Ninja `
-DCMAKE_DISABLE_PRECOMPILE_HEADERS=True
run: cmake . --preset ${{ matrix.preset }} -DCMAKE_DISABLE_PRECOMPILE_HEADERS=On

- name: Remove `.modmap`s from compile commands
run: |
(Get-Content "out\build\${{ matrix.preset }}\compile_commands.json") `
-replace "@CMakeFiles.+?\.modmap", "" `
| Set-Content `
Expand All @@ -256,33 +189,22 @@ jobs:
working-directory: ${{ env.GITHUB_WORKSPACE }}
run: |
python (Get-Command run-clang-tidy).Source `
-j ${{ env.CLANG_TIDY_JOBS }} `
-p "out\build\${{ matrix.preset }}" `
-export-fixes clang-tidy-fixes.yml `
$([Regex]::Escape("$pwd\src") + ".+\.(c|cpp|h|hpp)$") `
-extra-arg="-Wno-unknown-pragmas"
# For some reason, the above started giving unknown pragma errors in library headers (both
# unrealsdk and python) in clang-tidy 17
# It compiles fine, doesn't show up in clangd, and doesn't happen in the unrealsdk build, so
# just suppressing it for now
$([Regex]::Escape("$pwd\src") + ".+\.(c|cpp|h|hpp)$")

- name: Process clang-tidy warnings
uses: asarium/clang-tidy-action@v1
with:
fixesFile: clang-tidy-fixes.yml


clang-format:
runs-on: windows-latest
needs: cache-clang

steps:
- name: Restore Clang Cache
uses: actions/cache/restore@v4
with:
path: C:\Program Files\LLVM
key: ${{ runner.os }}-clang-${{ env.CLANG_VERSION }}
fail-on-cache-miss: true
- name: Setup Clang
if: startswith(matrix.preset, 'clang')
uses: egor-tensin/setup-clang@v1

- name: Checkout repository
uses: actions/checkout@v4
Expand Down
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
cmake_minimum_required(VERSION 3.24)

project(pyunrealsdk VERSION 1.1.1)
project(pyunrealsdk VERSION 1.2.0)

function(_pyunrealsdk_add_base_target_args target_name)
target_compile_features(${target_name} PUBLIC cxx_std_20)
Expand Down
52 changes: 52 additions & 0 deletions changelog.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,57 @@
# Changelog

## v1.2.0 (Upcoming)

Also see the unrealsdk v1.2.0 changelog [here](https://github.com/bl-sdk/unrealsdk/blob/master/changelog.md#v120).

- Added bindings for the new classes introduced in unrealsdk v1.2.0 - `UByteAttributeProperty`,
`UComponentProperty`, `UFloatAttributeProperty`, `UIntAttributeProperty`, and
`UByteProperty::Enum`.

[ab211486](https://github.com/bl-sdk/pyunrealsdk/commit/ab211486)

- Getting a byte property which has an associated enum will now return the appropriate Python enum,
in the same way as an enum property does. Byte properties without an enum still return an int.

[ab211486](https://github.com/bl-sdk/pyunrealsdk/commit/ab211486)

- Fixed that it was impossible to set Python properties on unreal objects.

[8b75fbbf](https://github.com/bl-sdk/pyunrealsdk/commit/8b75fbbf)

- Changed the log level specific printers, `unrealsdk.logging.error` et al., to each use their own,
logger objects rather than modifying `sys.stdout` in place.

[285e276a](https://github.com/bl-sdk/pyunrealsdk/commit/285e276a)

- Updated various docstrings and type stubs to be more accurately.

[d66295ef](https://github.com/bl-sdk/pyunrealsdk/commit/d66295ef),
[0df05cea](https://github.com/bl-sdk/pyunrealsdk/commit/0df05cea),
[285e276a](https://github.com/bl-sdk/pyunrealsdk/commit/285e276a)

- Restructured CMake to allow you to define the Python version to link against directly within it,
similarly to unrealsdk.

```cmake
set(UNREALSDK_ARCH x64)
set(UNREALSDK_UE_VERSION UE4)
set(EXPLICIT_PYTHON_ARCH win64)
set(EXPLICIT_PYTHON_VERSION 3.12.3)

add_subdirectory(libs/pyunrealsdk)
```
[abca72b3](https://github.com/bl-sdk/pyunrealsdk/commit/abca72b3)

- Release the GIL during unreal function calls, to try avoid a deadlock when running with
`UNREALSDK_LOCKING_PROCESS_EVENT`.

[31fdb4ee](https://github.com/bl-sdk/pyunrealsdk/commit/31fdb4ee)

- Upgraded pybind.

[b1335304](https://github.com/bl-sdk/pyunrealsdk/commit/b1335304)

## v1.1.1
- Updated CI and stubs to Python 3.12

Expand Down
2 changes: 1 addition & 1 deletion libs/pybind11
Submodule pybind11 updated 163 files
2 changes: 1 addition & 1 deletion libs/unrealsdk
Loading
Loading