Skip to content

Commit 8e24b43

Browse files
committed
Start adding content for GenAI docs
1 parent 8b8a6fe commit 8e24b43

File tree

2 files changed

+78
-0
lines changed

2 files changed

+78
-0
lines changed

docs/genai/howto/install.md

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
---
2+
title: Install ONNX Runtime GenAI
3+
description: Instructions to install ONNX Runtime GenAI on your target platform in your environment
4+
has_children: false
5+
nav_order: 1
6+
---
7+
8+
# Install ONNX Runtime GenAI
9+
10+
## Python package
11+
12+
(Coming soon) `pip install onnxruntime-genai`
13+
14+
(Temporary)
15+
1. Build from source
16+
17+
Follow the instructions in [build-from-source.md]
18+
19+
2. Install wheel
20+
21+
```bash
22+
cd build/wheel
23+
pip install onnxruntime-genai*.whl
24+
```
25+
26+
## C# package
27+
28+
(Coming soon) `dotnet add package Microsoft.ML.OnnxRuntime.GenAI`
29+
30+
(Temporary)
31+
1. Build from source
32+
33+
Follow the instructions in [build-from-source.md]
34+
35+
2. Build nuget package
36+
37+
```cmd
38+
nuget.exe pack Microsoft.ML.OnnxRuntimeGenAI.nuspec -Prop version=0.1.0 -Prop id="Microsoft.ML.OnnxRuntimeGenAI.Gpu"
39+
```
40+
41+
3. Install the nuget package
42+
43+
```cmd
44+
dotnet add package .. local instructions
45+
```
46+
47+
48+
## C artifacts
49+
50+
(Coming soon) Download release archive
51+
52+
Unzip archive
53+
54+
(Temporary)
55+
1. Build from source
56+
57+
Follow the instructions in [build-from-source.md]
58+
59+
60+
2. Use the following include locations to build your C application
61+
62+
*
63+
64+
3. Use the following library locations to build your C application
65+
66+
*
67+
68+
69+

docs/genai/index.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Generative AI with ONNX Runtime
2+
3+
Run generative AI models with ONNX Runtime.
4+
5+
This library provides the generative AI loop for ONNX models, including inference with ONNX Runtime, logits processing, search and sampling, and KV cache management.
6+
7+
Users can call a high level `generate()` method, or run each iteration of the model in a loop, generating one token at a time, and optionally updating generation parameters inside the loop.
8+
9+
It has support for greedy/beam search and TopP, TopK sampling to generate token sequences and built-in logits processing like repetition penalties. You can also easily add custom scoring.

0 commit comments

Comments
 (0)