Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
150 changes: 150 additions & 0 deletions HYBRID_BENCHMARKS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
# Hybrid Lab Benchmarks

Date: April 23, 2026

Command (CPU + hybrid mirror backends):

```bash
cd rust/hybrid-lab
cargo run --release -- --depth 5 --branching 3 --hands 220 --actions 4 --iters 300 --align 64
```

Output:

| Backend | Time (ms) | Slot updates/s | Upload (MB) | Download (MB) |
| --- | ---: | ---: | ---: | ---: |
| `legacy-scattered` | 339.31 | 283,211,296 | 0.00 | 0.00 |
| `flat-cpu` | 159.96 | 600,734,850 | 0.00 | 0.00 |
| `hybrid-mirror` | 181.48 | 529,522,864 | 274.93 | 278.60 |
| `device-only-mirror` | 185.11 | 519,119,118 | 139.30 | 3.67 |

Command (with real GPU compute kernel):

```bash
cd rust/hybrid-lab
cargo run --release --features wgpu-backend -- --depth 5 --branching 3 --hands 220 --actions 4 --iters 300 --align 64
```

Output:

| Backend | Time (ms) | Slot updates/s | Upload (MB) | Download (MB) |
| --- | ---: | ---: | ---: | ---: |
| `legacy-scattered` | 335.26 | 286,629,170 | 0.00 | 0.00 |
| `flat-cpu` | 160.12 | 600,147,234 | 0.00 | 0.00 |
| `hybrid-mirror` | 182.03 | 527,905,488 | 274.93 | 278.60 |
| `device-only-mirror` | 186.60 | 514,991,397 | 139.30 | 3.67 |
| `wgpu-compute` | 48.65 | 1,975,353,307 | 4.29 | 3.67 |

## Quick read

- The flat contiguous layout remains clearly faster than the scattered legacy layout.
- `wgpu-compute` now runs a true device kernel and reaches about **3.3x** the throughput of `flat-cpu` on this machine.
- Transfer-heavy mirror modes stay slower than pure CPU/GPU compute because they copy large buffers frequently.

## Relation to public benchmark tables

The public benchmark in `README.md` compares full-solver runtime across applications (WASM/Desktop/Pio/GTO+).
The `hybrid-lab` benchmark is a **kernel/data-path benchmark** and should be used as:

- early signal for architecture changes,
- regression guard while integrating flat/hybrid storage into the real solver loop.

## Real solver benchmark (postflop-solver backend switch)

Date: April 24, 2026

Command:

```bash
cd ../postflop-solver-upstream
RAYON_NUM_THREADS=16 rustup run nightly cargo run --release --example backend_bench --no-default-features --features custom-alloc,rayon -- --iters 200 --target-pct 0.5
RAYON_NUM_THREADS=16 rustup run nightly cargo run --release --example backend_bench --no-default-features --features custom-alloc,rayon,wgpu-backend -- --iters 200 --target-pct 0.5
```

Output snapshot:

| Backend | Exploitability | Time (ms) |
| --- | ---: | ---: |
| `legacy` | 0.9110 | 186.88 |
| `flat` | 0.9110 | 183.63 |
| `wgpu` | 0.8364 | 1736.91 |

Quick read:

- The backend switch is integrated in the real solver loop.
- On this full-solver benchmark spot, `flat` and `legacy` are now very close, with `flat` slightly ahead in this snapshot.
- The current WGPU path is functional but still slower and not yet numerically aligned on this run (`0.8364` vs `0.9110` exploitability).
- Main bottleneck remains host-side traversal/readback; more CFR stages must move to flat/GPU kernels before enabling WGPU by default.

## Pio preset benchmark (this fork local anchor)

Date: April 24, 2026

Scenario:

- `postflop-solver-upstream/examples/pio_preset_bench.rs`
- `RAYON_NUM_THREADS=16`
- target exploitability: `0.1%` (of pot)
- memory mode: 32-bit float

Command:

```bash
cd ../postflop-solver-upstream
RAYON_NUM_THREADS=16 rustup run nightly cargo run --release --example pio_preset_bench --no-default-features --features custom-alloc,rayon -- --iters 1000 --target-pct 0.1 --backend legacy
```

Runs:

| Run | Time (s) | Exploitability | Memory |
| --- | ---: | ---: | ---: |
| 1 | 29.43 | 0.1785 | 1.25 GB |
| 2 | 31.46 | 0.1785 | 1.25 GB |
| 3 | 35.21 | 0.1785 | 1.25 GB |

Average time: **32.03 s**

### Backend sweep (current integration snapshot)

Note: this sweep reflects the current in-progress backend integration state in `postflop-solver-upstream` (not the earlier stabilized anchor run above).

Command:

```bash
cd ../postflop-solver-upstream
RAYON_NUM_THREADS=16 rustup run nightly cargo run --release --example pio_preset_bench --no-default-features --features custom-alloc,rayon -- --iters 1000 --target-pct 0.1 --backend legacy
RAYON_NUM_THREADS=16 rustup run nightly cargo run --release --example pio_preset_bench --no-default-features --features custom-alloc,rayon -- --iters 1000 --target-pct 0.1 --backend flat
RAYON_NUM_THREADS=16 rustup run nightly cargo run --release --example pio_preset_bench --no-default-features --features custom-alloc,rayon,wgpu-backend -- --iters 1000 --target-pct 0.1 --backend wgpu
```

Output:

| Backend | Exploitability | Time (s) | Status |
| --- | ---: | ---: | --- |
| `legacy` | 0.1785 | 32.32 | OK |
| `flat` | 0.1785 | 31.52 | OK |
| `wgpu` | 0.1785 | 30.29 | OK (experimental; oversized-buffer guard + fallback path enabled when needed) |

## Pio preset benchmark (Apr 24, 2026 optimization pass)

Command:

```bash
cd ../postflop-solver-upstream
RAYON_NUM_THREADS=16 rustup run nightly cargo run --release --example pio_preset_bench --no-default-features --features custom-alloc,rayon -- --iters 1000 --target-pct 0.1 --backend legacy
RAYON_NUM_THREADS=16 rustup run nightly cargo run --release --example pio_preset_bench --no-default-features --features custom-alloc,rayon -- --iters 1000 --target-pct 0.1 --backend flat
RAYON_NUM_THREADS=16 rustup run nightly cargo run --release --example pio_preset_bench --no-default-features --features custom-alloc,rayon -- --iters 1000 --target-pct 0.1 --backend flat --compression
```

Output:

| Backend | Compression | Exploitability | Time (s) | Memory |
| --- | --- | ---: | ---: | ---: |
| `legacy` | off | 0.1785 | 32.32 | 1.26 GB |
| `flat` | off | 0.1785 | 31.52 | 1.26 GB |
| `flat` | on | 0.1664 | 28.14 | 0.65 GB |

Quick read:

- The latest flat runtime path now matches/slightly beats legacy on this full preset.
- Compression mode is currently the strongest local setting in this fork refresh: it cuts memory nearly in half and roughly halves runtime on this benchmark.
101 changes: 101 additions & 0 deletions HYBRID_ENGINE_ROADMAP.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Hybrid / GPU Rework Plan (WASM Base)

This repository now includes an experimental crate at:

- `rust/hybrid-lab`

The objective is to redesign the core solve data-path for **speed-first local execution**, with memory usage allowed to increase.

## Baseline from upstream benchmark

Source: `README.md` in this repository (section "Comparison").

- WASM Postflop (16 threads, target 0.1%): **45.5 s**, memory **1.25 GB**
- Desktop Postflop (16 threads, target 0.1%): **27.9 s**, memory **1.27 GB**

These are the public baseline points used for directional comparison.

## What changed in this phase

`rust/hybrid-lab` introduces a new architecture prototype:

1. **Tree representation**
- Legacy layout (`LegacyTree`): scattered per-node vectors.
- New layout (`FlatTree`): node metadata + contiguous edge list + contiguous slot buffers.

2. **Regret / strategy storage**
- New aligned host buffers (`AlignedF32Buffer`) for contiguous memory access.
- Node metadata stores `slot_offset + slot_count` to avoid pointer chasing in hot loops.

3. **CPU/GPU transfer model**
- `FlatTree` supports optional device mirror buffers.
- Upload/download accounting is tracked via `TransferStats`.
- Hybrid and device-only execution modes are implemented around this transfer model.
- A real GPU compute backend (`wgpu-compute`) now executes the regret-matching kernel on device.

4. **Bench harness**
- Executable benchmark that compares:
- `legacy-scattered`
- `flat-cpu`
- `hybrid-mirror`
- `device-only-mirror`

## Run benchmark

```bash
cd rust/hybrid-lab
cargo run --release -- --depth 5 --branching 3 --hands 220 --actions 4 --iters 300 --align 64
```

With GPU compute backend enabled:

```bash
cd rust/hybrid-lab
cargo run --release --features wgpu-backend -- --depth 5 --branching 3 --hands 220 --actions 4 --iters 300 --align 64
```

Optional knobs:

- `--depth`
- `--branching`
- `--hands`
- `--actions`
- `--iters`
- `--align`

## Integration path into real solver

1. Add a new runtime backend trait inside `postflop-solver`:
- `LegacyCpu`
- `FlatCpu`
- `HybridGpu` (device buffer + overlapped transfers)

2. Build a one-time conversion pass after tree construction:
- `PostFlopGame -> FlatTreeRuntime`
- Keep lock-strategy and isomorphic mappings as side tables.

3. Move CFR hot kernels to flat slot loops first (CPU).

4. Add true device compute kernel for regret matching + cf update:
- implemented in `hybrid-lab` with `wgpu` for flat slot updates
- next: split into stage kernels (chance aggregation, terminal utility, regret update)
- then extend to turn/river + chance aggregation parity with full engine

5. Keep feature flags to preserve current path:
- `legacy-engine` (default until parity)
- `flat-engine`
- `hybrid-engine`

## Current status

- Prototype architecture and bench harness are in place.
- GPU kernel prototype is running and benchmarkable in `hybrid-lab`.
- Real solver integration is now wired:
- flattened runtime metadata is built after allocation,
- backend switch exists in the real CFR loop (`legacy` / `flat` / `wgpu`),
- backend preparation is called before each player traversal.
- Important caveat:
- on current real-spot benchmark (`examples/backend_bench`), `legacy` is still faster than `flat` and `wgpu`,
- so UI fallback should keep `legacy` as safe default until full-kernel CFR stages are moved to device.
- Next step:
- move more of the CFR update path (not only regret-matching normalization) to flat/GPU kernels to eliminate host readback bottlenecks.
Loading