Skip to content

Commit d3cab27

Browse files
YUNQIUGUOrachguorachguo
authored
[Mobile] Add basic react native expo sample (#308)
* add initial contents for basic react native usage app * updates to readme * minor updates * update * push model * update * update * address pr comments * minor update * add loading model from bytes code * minor update * format --------- Co-authored-by: rachguo <[email protected]> Co-authored-by: rachguo <[email protected]>
1 parent b9fb5b0 commit d3cab27

File tree

15 files changed

+5886
-1
lines changed

15 files changed

+5886
-1
lines changed

‎mobile/README.md

+6
Original file line numberDiff line numberDiff line change
@@ -75,3 +75,9 @@ This example shows how to use ORT to do speech recognition using the [Whisper](h
7575

7676
- [Azure Whisper](examples/whisper/azure/android)
7777
- [Local Whisper](examples/whisper/local/android)
78+
79+
### React Native Expo Sample
80+
81+
This is an example React Native Expo project which demonstrates basic usage of ORT such as loading onnx models and creating inference sessions, etc.
82+
83+
- [React Native Basic Sample](examples/React_Native/ort-rn-basic-usage/)

‎mobile/examples/React_Native/ort-rn-basic-usage/App.tsx

+81
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
import { StatusBar } from 'expo-status-bar';
2+
import { Alert, Button, StyleSheet, Text, View } from 'react-native';
3+
4+
import * as ort from 'onnxruntime-react-native';
5+
import { Asset } from 'expo-asset';
6+
7+
// Note: These modules are used for reading model into bytes
8+
// import RNFS from 'react-native-fs';
9+
// import base64 from 'base64-js';
10+
11+
let myModel: ort.InferenceSession;
12+
13+
async function loadModel() {
14+
try {
15+
// Note: `.onnx` model files can be viewed in Netron (https://github.com/lutzroeder/netron) to see
16+
// model inputs/outputs detail and data types, shapes of those, etc.
17+
const assets = await Asset.loadAsync(require('./assets/mnist.onnx'));
18+
const modelUri = assets[0].localUri;
19+
if (!modelUri) {
20+
Alert.alert('failed to get model URI', `${assets[0]}`);
21+
} else {
22+
// load model from model url path
23+
myModel = await ort.InferenceSession.create(modelUri);
24+
Alert.alert(
25+
'model loaded successfully',
26+
`input names: ${myModel.inputNames}, output names: ${myModel.outputNames}`);
27+
28+
// loading model from bytes
29+
// const base64Str = await RNFS.readFile(modelUri, 'base64');
30+
// const uint8Array = base64.toByteArray(base64Str);
31+
// myModel = await ort.InferenceSession.create(uint8Array);
32+
}
33+
} catch (e) {
34+
Alert.alert('failed to load model', `${e}`);
35+
throw e;
36+
}
37+
}
38+
39+
async function runModel() {
40+
try {
41+
// Prepare model input data
42+
// Note: In real use case, you must set the inputData to the actual input values
43+
const inputData = new Float32Array(28 * 28);
44+
const feeds:Record<string, ort.Tensor> = {};
45+
feeds[myModel.inputNames[0]] = new ort.Tensor(inputData, [1, 28, 28]);
46+
// Run inference session
47+
const fetches = await myModel.run(feeds);
48+
// Process output
49+
const output = fetches[myModel.outputNames[0]];
50+
if (!output) {
51+
Alert.alert('failed to get output', `${myModel.outputNames[0]}`);
52+
} else {
53+
Alert.alert(
54+
'model inference successfully',
55+
`output shape: ${output.dims}, output data: ${output.data}`);
56+
}
57+
} catch (e) {
58+
Alert.alert('failed to inference model', `${e}`);
59+
throw e;
60+
}
61+
}
62+
63+
export default function App() {
64+
return (
65+
<View style={styles.container}>
66+
<Text>ONNX Runtime React Native Basic Usage</Text>
67+
<Button title='Load model' onPress={loadModel}></Button>
68+
<Button title='Run' onPress={runModel}></Button>
69+
<StatusBar style="auto" />
70+
</View>
71+
);
72+
}
73+
74+
const styles = StyleSheet.create({
75+
container: {
76+
flex: 1,
77+
backgroundColor: '#fff',
78+
alignItems: 'center',
79+
justifyContent: 'center',
80+
},
81+
});

‎mobile/examples/React_Native/ort-rn-basic-usage/README.md

+128
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# ONNX Runtime React Native Expo Example Application
2+
3+
This is a [React Native](https://reactnative.dev/docs/getting-started) application for [ONNX Runtime](https://github.com/microsoft/onnxruntime) via [Expo](https://docs.expo.dev/) platform. The demo app demonstrates how to accomplish simple tasks such as loading onnx models and creating inference sessions, etc. in a react native expo application.
4+
5+
## Prerequisites
6+
7+
1. Install [Node.js](https://nodejs.org/en)
8+
2. Install [Expo-CLI](https://docs.expo.dev/more/expo-cli/)
9+
```sh
10+
npm install -g expo-cli
11+
```
12+
3. Install [Yarn](https://classic.yarnpkg.com/en/docs/install#mac-stable) (Recommended)
13+
```sh
14+
npm install -g yarn
15+
```
16+
17+
18+
## Validate your React Native Environment
19+
20+
Run the example expo app as-is for validating local React Native environment setup.
21+
22+
**Steps:**
23+
24+
1. Run `yarn install` to set up JavaScript dependencies.
25+
```sh
26+
yarn install
27+
```
28+
29+
2. Install NPM `onnxruntime-react-native` package.
30+
```sh
31+
expo install onnxruntime-react-native@latest
32+
```
33+
34+
3. Run the following command in `<PROJECT_ROOT>` to generate Android and iOS project.
35+
```sh
36+
expo prebuild
37+
```
38+
The generated Android and iOS project code will be updated automatically.
39+
40+
4. Build and run the app.
41+
42+
Run the following command to build and launch the app:
43+
44+
- In `<PROJECT_ROOT>`, run the following command to launch for Android:
45+
46+
```sh
47+
expo run:android
48+
```
49+
50+
- In `<PROJECT_ROOT>`, run the following command to launch for iOS:
51+
```sh
52+
expo run:ios
53+
```
54+
55+
## Steps that were done to add onnxruntime-react-native to the example app
56+
57+
The following steps were done in this sample for using onxnruntime-react-native. These can be replicated as a reference when setting up your own react native expo application.
58+
59+
1. NPM `onnxruntime-react-native` package.
60+
61+
Note: By default, we install the current latest release version of ORT react native npm package(Recommended). Can also update to dev version npm package if necessary.
62+
63+
[Link to npm `onnxruntime-react-native` packages](https://www.npmjs.com/package/onnxruntime-react-native?activeTab=versions)
64+
65+
2. Prepare the model.
66+
67+
- Model files are usually placed under `<PROJECT_ROOT>/assets`.
68+
69+
In this sample application, a simple ONNX MNIST model (`mnist.onnx`) is provided by default.
70+
71+
- Add file `metro.config.js` under `<PROJECT_ROOT>`. This file adds the extension `.onnx` to the bundler's asset extension list, which allows the bundler to include the model into assets.
72+
73+
`metro.config.js` file in this sample application looks like:
74+
75+
```js
76+
const { getDefaultConfig } = require('@expo/metro-config');
77+
const defaultConfig = getDefaultConfig(__dirname);
78+
defaultConfig.resolver.assetExts.push('onnx');
79+
module.exports = defaultConfig;
80+
```
81+
82+
3. Generate Android and iOS directories native code to run your React Native app.
83+
84+
In this sample project, it generates the native code automatically by using package `onnxruntime-react-native` as an Expo plugin. (Recommended approach.)
85+
86+
- In `<PROJECT_ROOT>/app.json`, add the following line to section `expo`:
87+
```
88+
"plugins": ["onnxruntime-react-native"],
89+
```
90+
Note: The plugin is added by default in `app.json` in the sample.
91+
92+
- Run the following command in `<PROJECT_ROOT>` to generate Android and iOS project: More info about [Expo CLI Prebuild](https://docs.expo.dev/workflow/prebuild/).
93+
```sh
94+
expo prebuild
95+
```
96+
The generated Android and iOS project code will be updated automatically.
97+
98+
99+
[Optional] Set up manually.
100+
101+
1. First run `expo prebuild` to generate Android and iOS Native code.
102+
103+
Note: In this tutorial we use `ai.onnxruntime.example.reactnative.basicusage` as Android package name/iOS bundle identifier.
104+
Expo requires a manual configuration on package name and bundle identifier.
105+
106+
2. [Android]: Add `onnxruntime-react-native` as Gradle dependencies.
107+
108+
In `<PROJECT_ROOT>/android/app/build.gradle`, add the following line to section `dependencies`:
109+
110+
```
111+
implementation project(':onnxruntime-react-native')
112+
```
113+
114+
3. [iOS]: Add `onnxruntime-react-native` as CocoaPods dependencies.
115+
116+
In `<PROJECT_ROOT>/ios/Podfile`, add the following line to section `target 'OrtReactNativeBasicUsage'`:
117+
118+
```sh
119+
pod 'onnxruntime-react-native', :path => '../node_modules/onnxruntime-react-native'
120+
```
121+
122+
Run the following command in `<PROJECT_ROOT>/ios` to install pod:
123+
124+
```sh
125+
pod install
126+
```
127+
**NOTE:**
128+
If you are interested in creating a new react native expo project from scratch, refer to instructions: https://docs.expo.dev/get-started/create-a-project/

‎mobile/examples/React_Native/ort-rn-basic-usage/app.json

+35
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
{
2+
"expo": {
3+
"name": "OrtReactNativeBasicUsage",
4+
"slug": "OrtReactNativeBasicUsage",
5+
"plugins": ["onnxruntime-react-native"],
6+
"version": "1.0.0",
7+
"orientation": "portrait",
8+
"icon": "./assets/icon.png",
9+
"splash": {
10+
"image": "./assets/splash.png",
11+
"resizeMode": "contain",
12+
"backgroundColor": "#ffffff"
13+
},
14+
"updates": {
15+
"fallbackToCacheTimeout": 0
16+
},
17+
"assetBundlePatterns": [
18+
"**/*"
19+
],
20+
"ios": {
21+
"supportsTablet": true,
22+
"bundleIdentifier": "ai.onnxruntime.example.reactnative.basicusage"
23+
},
24+
"android": {
25+
"adaptiveIcon": {
26+
"foregroundImage": "./assets/adaptive-icon.png",
27+
"backgroundColor": "#FFFFFF"
28+
},
29+
"package": "ai.onnxruntime.example.reactnative.basicusage"
30+
},
31+
"web": {
32+
"favicon": "./assets/favicon.png"
33+
}
34+
}
35+
}

‎mobile/examples/React_Native/ort-rn-basic-usage/assets/adaptive-icon.png

17.1 KB
Loading

‎mobile/examples/React_Native/ort-rn-basic-usage/assets/favicon.png

1.43 KB
Loading

‎mobile/examples/React_Native/ort-rn-basic-usage/assets/icon.png

21.9 KB
Loading

‎mobile/examples/React_Native/ort-rn-basic-usage/assets/mnist.onnx

25.8 KB
Binary file not shown.

‎mobile/examples/React_Native/ort-rn-basic-usage/assets/splash.png

46.2 KB
Loading

‎mobile/examples/React_Native/ort-rn-basic-usage/babel.config.js

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
module.exports = function(api) {
2+
api.cache(true);
3+
return {
4+
presets: ['babel-preset-expo'],
5+
};
6+
};

‎mobile/examples/React_Native/ort-rn-basic-usage/metro.config.js

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
const { getDefaultConfig } = require('@expo/metro-config');
2+
const defaultConfig = getDefaultConfig(__dirname);
3+
defaultConfig.resolver.assetExts.push('onnx');
4+
module.exports = defaultConfig;

‎mobile/examples/React_Native/ort-rn-basic-usage/package.json

+28
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
{
2+
"name": "ort-rn-basic-usage",
3+
"version": "1.0.0",
4+
"scripts": {
5+
"start": "expo start --dev-client",
6+
"android": "expo run:android",
7+
"ios": "expo run:ios",
8+
"web": "expo start --web",
9+
"eject": "expo eject"
10+
},
11+
"dependencies": {
12+
"expo": "~49.0.0",
13+
"expo-splash-screen": "~0.20.5",
14+
"expo-status-bar": "~1.2.0",
15+
"onnxruntime-react-native": "^1.16.0",
16+
"react": "18.2.0",
17+
"react-dom": "18.2.0",
18+
"react-native": "0.72.5",
19+
"react-native-web": "0.17.1"
20+
},
21+
"devDependencies": {
22+
"@babel/core": "^7.20.20",
23+
"@types/react": "~18.2.21",
24+
"@types/react-native": "~0.72.2",
25+
"typescript": "~4.8.4"
26+
},
27+
"private": true
28+
}

‎mobile/examples/React_Native/ort-rn-basic-usage/tsconfig.json

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"extends": "expo/tsconfig.base",
3+
"compilerOptions": {
4+
"strict": true
5+
}
6+
}

0 commit comments

Comments
 (0)