You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Pure Go bindings for [ONNX Runtime](https://github.com/microsoft/onnxruntime) using [ebitengine/purego](https://github.com/ebitengine/purego).
4
5
5
6
This library provides a pure Go interface to ONNX Runtime without requiring cgo, enabling cross-platform machine learning inference in Go applications.
6
7
7
8
NOTE: This project is currently unstable. APIs may change without notice.
8
9
9
-
## ONNX Runtime GenAI Support (Experimental)
10
+
## Supported Versions
10
11
11
-
This library also includes experimental support for [ONNX Runtime GenAI](https://github.com/microsoft/onnxruntime-genai), enabling text generation with large language models. See `examples/genai/` for usage examples.
12
+
| Library | Supported Version |
13
+
|---------|-------------------|
14
+
| ONNX Runtime | 1.23.x |
15
+
| ONNX Runtime GenAI | 0.11.x |
16
+
17
+
## ONNX Runtime GenAI Support
18
+
19
+
This library also includes experimental support for [ONNX Runtime GenAI](https://github.com/microsoft/onnxruntime-genai), enabling text generation with large language models. See [`examples/`](./examples/) for usage examples.
12
20
13
21
## Prerequisites
14
22
@@ -36,13 +44,4 @@ go get github.com/shota3506/onnxruntime-purego
36
44
37
45
## Examples
38
46
39
-
The `examples/` directory contains complete examples:
40
-
41
-
-**resnet**: Image classification using ResNet
42
-
-**roberta-sentiment**: Sentiment analysis using RoBERTa
43
-
-**yolov10**: Object detection using YOLOv10
44
-
45
-
-**genai/phi3**: Text generation using Phi-3
46
-
-**genai/phi3.5-vision**: Multimodal vision-language processing using Phi-3.5-Vision
47
-
48
-
See each example's README for detailed instructions.
47
+
See the [`examples/`](./examples/) directory for complete usage examples.
Copy file name to clipboardExpand all lines: examples/genai/phi3.5-vision/README.md
+5-7Lines changed: 5 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,11 +2,11 @@
2
2
3
3
This example demonstrates multimodal vision-language processing using ONNX Runtime GenAI with the Phi-3.5-Vision model. The model can analyze images and answer questions about their content.
4
4
5
-
## Prerequisites
5
+
See [examples/README.md](../README.md) for common GenAI prerequisites and troubleshooting.
6
6
7
-
1.**ONNX Runtime GenAI Library**: Download from [releases](https://github.com/microsoft/onnxruntime-genai/releases)
-`-model`: Path to model directory containing `genai_config.json` (required)
49
44
-`-max-length`: Maximum tokens to generate (default: 200)
50
45
51
-
## Environment Variables
52
-
53
-
-`ONNXRUNTIME_GENAI_LIB_PATH`: Path to ONNX Runtime GenAI shared library (required)
54
-
55
46
## Expected Output
56
47
57
48
```
@@ -65,26 +56,6 @@ What is the capital of France? The capital of France is Paris.
65
56
66
57
## Troubleshooting
67
58
68
-
1.**Library not found**: Ensure `ONNXRUNTIME_GENAI_LIB_PATH` is set correctly and the library is accessible.
69
-
70
-
2.**Model loading fails**: Verify the model directory contains `genai_config.json` and the ONNX model files.
71
-
72
-
3.**Slow generation**: The INT4 CPU model runs at ~20 tokens/sec. Consider using GPU acceleration for faster inference.
59
+
1.**Model loading fails**: Verify the model directory contains `genai_config.json` and the ONNX model files.
73
60
74
-
4.**macOS ARM64 crash on library load**: The pre-built macOS ARM64 binaries have a known issue where they crash during static initialization with `std::runtime_error` in `Ort::InitApi()`. This is a problem with the released binaries, not with the Go bindings.
75
-
76
-
**Workarounds:**
77
-
- Build ONNX Runtime GenAI from source for macOS ARM64
78
-
- Use Linux x64 environment for testing
79
-
- Check [GitHub Issues](https://github.com/microsoft/onnxruntime-genai/issues) for updates
80
-
81
-
**Debug output:**
82
-
```
83
-
libc++abi: terminating due to uncaught exception of type std::runtime_error
84
-
SIGABRT: abort
85
-
```
86
-
87
-
This crash occurs in:
88
-
```
89
-
libonnxruntime-genai.dylib`Ort::InitApi()
90
-
```
61
+
2.**Slow generation**: The INT4 CPU model runs at ~20 tokens/sec. Consider using GPU acceleration for faster inference.
0 commit comments