Skip to content

Commit a8a783e

Browse files
authored
Merge pull request #10 from shota3506/update-readme
update readme
2 parents 4961ab2 + 6d2bb60 commit a8a783e

File tree

4 files changed

+57
-53
lines changed

4 files changed

+57
-53
lines changed

README.md

Lines changed: 11 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,22 @@
11
# onnxruntime-purego
2+
[![Go Reference](https://pkg.go.dev/badge/github.com/shota3506/onnxruntime-purego.svg)](https://pkg.go.dev/github.com/shota3506/onnxruntime-purego)
23

34
Pure Go bindings for [ONNX Runtime](https://github.com/microsoft/onnxruntime) using [ebitengine/purego](https://github.com/ebitengine/purego).
45

56
This library provides a pure Go interface to ONNX Runtime without requiring cgo, enabling cross-platform machine learning inference in Go applications.
67

78
NOTE: This project is currently unstable. APIs may change without notice.
89

9-
## ONNX Runtime GenAI Support (Experimental)
10+
## Supported Versions
1011

11-
This library also includes experimental support for [ONNX Runtime GenAI](https://github.com/microsoft/onnxruntime-genai), enabling text generation with large language models. See `examples/genai/` for usage examples.
12+
| Library | Supported Version |
13+
|---------|-------------------|
14+
| ONNX Runtime | 1.23.x |
15+
| ONNX Runtime GenAI | 0.11.x |
16+
17+
## ONNX Runtime GenAI Support
18+
19+
This library also includes experimental support for [ONNX Runtime GenAI](https://github.com/microsoft/onnxruntime-genai), enabling text generation with large language models. See [`examples/`](./examples/) for usage examples.
1220

1321
## Prerequisites
1422

@@ -36,13 +44,4 @@ go get github.com/shota3506/onnxruntime-purego
3644

3745
## Examples
3846

39-
The `examples/` directory contains complete examples:
40-
41-
- **resnet**: Image classification using ResNet
42-
- **roberta-sentiment**: Sentiment analysis using RoBERTa
43-
- **yolov10**: Object detection using YOLOv10
44-
45-
- **genai/phi3**: Text generation using Phi-3
46-
- **genai/phi3.5-vision**: Multimodal vision-language processing using Phi-3.5-Vision
47-
48-
See each example's README for detailed instructions.
47+
See the [`examples/`](./examples/) directory for complete usage examples.

examples/README.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Examples
2+
3+
This directory contains example applications demonstrating the usage of onnxruntime-purego.
4+
5+
## ONNX Runtime Examples
6+
7+
- [**resnet**](./resnet/): Image classification using ResNet
8+
- [**roberta-sentiment**](./roberta-sentiment/): Sentiment analysis using RoBERTa
9+
- [**yolov10**](./yolov10/): Object detection using YOLOv10
10+
11+
## ONNX Runtime GenAI Examples
12+
13+
- [**genai/phi3**](./genai/phi3/): Text generation using Phi-3
14+
- [**genai/phi3.5-vision**](./genai/phi3.5-vision/): Multimodal vision-language processing using Phi-3.5-Vision
15+
16+
### GenAI Prerequisites
17+
18+
1. **ONNX Runtime GenAI Library**
19+
20+
Download from [ONNX Runtime GenAI Releases](https://github.com/microsoft/onnxruntime-genai/releases):
21+
- macOS: `libonnxruntime-genai.dylib`
22+
- Linux: `libonnxruntime-genai.so`
23+
- Windows: `onnxruntime-genai.dll`
24+
25+
2. **Environment Variable**
26+
27+
Set the library path before running GenAI examples:
28+
```bash
29+
export ONNXRUNTIME_GENAI_LIB_PATH=/path/to/libonnxruntime-genai.dylib
30+
```
31+
32+
### GenAI Troubleshooting
33+
34+
1. **Library not found**: Ensure `ONNXRUNTIME_GENAI_LIB_PATH` is set correctly and the library is accessible.
35+
36+
2. **macOS ARM64 crash on library load**: See [onnxruntime-genai issues](https://github.com/microsoft/onnxruntime-genai/issues).

examples/genai/phi3.5-vision/README.md

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,11 @@
22

33
This example demonstrates multimodal vision-language processing using ONNX Runtime GenAI with the Phi-3.5-Vision model. The model can analyze images and answer questions about their content.
44

5-
## Prerequisites
5+
See [examples/README.md](../README.md) for common GenAI prerequisites and troubleshooting.
66

7-
1. **ONNX Runtime GenAI Library**: Download from [releases](https://github.com/microsoft/onnxruntime-genai/releases)
7+
## Prerequisites
88

9-
2. **Phi-3.5-Vision Model** (~3.2GB):
9+
1. **Phi-3.5-Vision Model** (~3.2GB):
1010
```bash
1111
huggingface-cli download microsoft/Phi-3.5-vision-instruct-onnx \
1212
--include "cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4/*" \
@@ -49,8 +49,6 @@ export ONNXRUNTIME_GENAI_LIB_PATH=/path/to/libonnxruntime-genai.dylib
4949

5050
Reference: [GitHub Issue #1146](https://github.com/microsoft/onnxruntime-genai/issues/1146)
5151

52-
2. **Library not found**: Ensure `ONNXRUNTIME_GENAI_LIB_PATH` is set correctly.
53-
54-
3. **Model loading fails**: Verify the model directory contains all required files (genai_config.json, processor_config.json, and three ONNX models).
52+
2. **Model loading fails**: Verify the model directory contains all required files (genai_config.json, processor_config.json, and three ONNX models).
5553

56-
4. **Out of memory during generation**: Reduce `-max-length` or close other applications.
54+
3. **Out of memory during generation**: Reduce `-max-length` or close other applications.

examples/genai/phi3/README.md

Lines changed: 5 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -2,16 +2,11 @@
22

33
This example demonstrates basic text generation using ONNX Runtime GenAI with the Phi-3-mini model.
44

5-
## Prerequisites
6-
7-
1. **ONNX Runtime GenAI Library**
5+
See [examples/README.md](../README.md) for common GenAI prerequisites and troubleshooting.
86

9-
Download from [ONNX Runtime GenAI Releases](https://github.com/microsoft/onnxruntime-genai/releases):
10-
- macOS: `libonnxruntime-genai.dylib`
11-
- Linux: `libonnxruntime-genai.so`
12-
- Windows: `onnxruntime-genai.dll`
7+
## Prerequisites
138

14-
2. **Phi-3-mini Model**
9+
1. **Phi-3-mini Model**
1510

1611
Download using Hugging Face CLI:
1712
```bash
@@ -48,10 +43,6 @@ export ONNXRUNTIME_GENAI_LIB_PATH=/path/to/libonnxruntime-genai.dylib
4843
- `-model`: Path to model directory containing `genai_config.json` (required)
4944
- `-max-length`: Maximum tokens to generate (default: 200)
5045

51-
## Environment Variables
52-
53-
- `ONNXRUNTIME_GENAI_LIB_PATH`: Path to ONNX Runtime GenAI shared library (required)
54-
5546
## Expected Output
5647

5748
```
@@ -65,26 +56,6 @@ What is the capital of France? The capital of France is Paris.
6556

6657
## Troubleshooting
6758

68-
1. **Library not found**: Ensure `ONNXRUNTIME_GENAI_LIB_PATH` is set correctly and the library is accessible.
69-
70-
2. **Model loading fails**: Verify the model directory contains `genai_config.json` and the ONNX model files.
71-
72-
3. **Slow generation**: The INT4 CPU model runs at ~20 tokens/sec. Consider using GPU acceleration for faster inference.
59+
1. **Model loading fails**: Verify the model directory contains `genai_config.json` and the ONNX model files.
7360

74-
4. **macOS ARM64 crash on library load**: The pre-built macOS ARM64 binaries have a known issue where they crash during static initialization with `std::runtime_error` in `Ort::InitApi()`. This is a problem with the released binaries, not with the Go bindings.
75-
76-
**Workarounds:**
77-
- Build ONNX Runtime GenAI from source for macOS ARM64
78-
- Use Linux x64 environment for testing
79-
- Check [GitHub Issues](https://github.com/microsoft/onnxruntime-genai/issues) for updates
80-
81-
**Debug output:**
82-
```
83-
libc++abi: terminating due to uncaught exception of type std::runtime_error
84-
SIGABRT: abort
85-
```
86-
87-
This crash occurs in:
88-
```
89-
libonnxruntime-genai.dylib`Ort::InitApi()
90-
```
61+
2. **Slow generation**: The INT4 CPU model runs at ~20 tokens/sec. Consider using GPU acceleration for faster inference.

0 commit comments

Comments
 (0)