Skip to content

feat: streaming in speech to text transcription #168

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: v0.4.0-rc1
Choose a base branch
from
Open

feat: streaming in speech to text transcription #168

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a3394df
feat: llama with whisper demo (#179)
mdydek Apr 22, 2025
a4c262a
fixed S2T streaming
May 8, 2025
846ac06
Merge branch 'v0.4.0-rc1' into @md/s2t_streaming
mkopcins May 15, 2025
135eb72
missing import
May 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions examples/llm/assets/icons/mic_icon.svg
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions examples/llm/assets/icons/stop_icon.svg
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
228 changes: 159 additions & 69 deletions examples/llm/ios/Podfile.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
PODS:
- boost (1.84.0)
- DoubleConversion (1.1.6)
- EXConstants (17.1.5):
- EXConstants (17.1.6):
- ExpoModulesCore
- Expo (53.0.8):
- DoubleConversion
Expand Down Expand Up @@ -67,6 +67,8 @@ PODS:
- ReactCommon/turbomodule/bridging
- ReactCommon/turbomodule/core
- Yoga
- ExpoSpeech (13.1.4):
- ExpoModulesCore
- fast_float (6.1.4)
- FBLazyVector (0.79.2)
- fmt (11.0.2)
Expand Down Expand Up @@ -1399,7 +1401,7 @@ PODS:
- React-jsiexecutor
- React-RCTFBReactNativeSpec
- ReactCommon/turbomodule/core
- react-native-executorch (0.3.2):
- react-native-executorch (0.3.3):
- DoubleConversion
- glog
- hermes-engine
Expand Down Expand Up @@ -1821,6 +1823,82 @@ PODS:
- React-logger (= 0.79.2)
- React-perflogger (= 0.79.2)
- React-utils (= 0.79.2)
- RNAudioAPI (0.5.7):
- DoubleConversion
- glog
- hermes-engine
- RCT-Folly (= 2024.11.18.00)
- RCTRequired
- RCTTypeSafety
- React-Core
- React-debug
- React-Fabric
- React-featureflags
- React-graphics
- React-hermes
- React-ImageManager
- React-jsi
- React-NativeModulesApple
- React-RCTFabric
- React-renderercss
- React-rendererdebug
- React-utils
- ReactCodegen
- ReactCommon/turbomodule/bridging
- ReactCommon/turbomodule/core
- RNAudioAPI/audioapi (= 0.5.7)
- Yoga
- RNAudioAPI/audioapi (0.5.7):
- DoubleConversion
- glog
- hermes-engine
- RCT-Folly (= 2024.11.18.00)
- RCTRequired
- RCTTypeSafety
- React-Core
- React-debug
- React-Fabric
- React-featureflags
- React-graphics
- React-hermes
- React-ImageManager
- React-jsi
- React-NativeModulesApple
- React-RCTFabric
- React-renderercss
- React-rendererdebug
- React-utils
- ReactCodegen
- ReactCommon/turbomodule/bridging
- ReactCommon/turbomodule/core
- RNAudioAPI/audioapi/ios (= 0.5.7)
- Yoga
- RNAudioAPI/audioapi/ios (0.5.7):
- DoubleConversion
- glog
- hermes-engine
- RCT-Folly (= 2024.11.18.00)
- RCTRequired
- RCTTypeSafety
- React-Core
- React-debug
- React-Fabric
- React-featureflags
- React-graphics
- React-hermes
- React-ImageManager
- React-jsi
- React-NativeModulesApple
- React-RCTFabric
- React-renderercss
- React-rendererdebug
- React-utils
- ReactCodegen
- ReactCommon/turbomodule/bridging
- ReactCommon/turbomodule/core
- Yoga
- RNLiveAudioStream (1.1.1):
- React
- RNReanimated (3.17.5):
- DoubleConversion
- glog
Expand Down Expand Up @@ -2009,6 +2087,7 @@ DEPENDENCIES:
- ExpoFont (from `../node_modules/expo-font/ios`)
- ExpoKeepAwake (from `../node_modules/expo-keep-awake/ios`)
- ExpoModulesCore (from `../node_modules/expo-modules-core`)
- ExpoSpeech (from `../node_modules/expo-speech/ios`)
- fast_float (from `../node_modules/react-native/third-party-podspecs/fast_float.podspec`)
- FBLazyVector (from `../node_modules/react-native/Libraries/FBLazyVector`)
- fmt (from `../node_modules/react-native/third-party-podspecs/fmt.podspec`)
Expand Down Expand Up @@ -2080,6 +2159,8 @@ DEPENDENCIES:
- ReactAppDependencyProvider (from `build/generated/ios`)
- ReactCodegen (from `build/generated/ios`)
- ReactCommon/turbomodule/core (from `../node_modules/react-native/ReactCommon`)
- RNAudioAPI (from `../node_modules/react-native-audio-api`)
- RNLiveAudioStream (from `../node_modules/react-native-live-audio-stream`)
- RNReanimated (from `../node_modules/react-native-reanimated`)
- RNSVG (from `../node_modules/react-native-svg`)
- Yoga (from `../node_modules/react-native/ReactCommon/yoga`)
Expand Down Expand Up @@ -2112,6 +2193,8 @@ EXTERNAL SOURCES:
:path: "../node_modules/expo-keep-awake/ios"
ExpoModulesCore:
:path: "../node_modules/expo-modules-core"
ExpoSpeech:
:path: "../node_modules/expo-speech/ios"
fast_float:
:podspec: "../node_modules/react-native/third-party-podspecs/fast_float.podspec"
FBLazyVector:
Expand Down Expand Up @@ -2251,6 +2334,10 @@ EXTERNAL SOURCES:
:path: build/generated/ios
ReactCommon:
:path: "../node_modules/react-native/ReactCommon"
RNAudioAPI:
:path: "../node_modules/react-native-audio-api"
RNLiveAudioStream:
:path: "../node_modules/react-native-live-audio-stream"
RNReanimated:
:path: "../node_modules/react-native-reanimated"
RNSVG:
Expand All @@ -2259,91 +2346,94 @@ EXTERNAL SOURCES:
:path: "../node_modules/react-native/ReactCommon/yoga"

SPEC CHECKSUMS:
boost: 1dca942403ed9342f98334bf4c3621f011aa7946
DoubleConversion: f16ae600a246532c4020132d54af21d0ddb2a385
EXConstants: 26f334022fc0b454b7ecdc7c12d36ffa68c7c397
Expo: 54adb30ffa0a086c1a4bb87364d69b7917caeb49
ExpoAsset: 3ea3275cca6a7793b3d36fbf1075c590f803fbcb
ExpoBrightness: 05e750736f8886dcf235212b0caf85b0f605fc88
ExpoCalendar: 660542dc1c5ef98f46bedcc8745aa707df5d501a
ExpoFileSystem: ff47f2208da552fbee872dedc4e74fb7c6ac5907
ExpoFont: 312c73403bbd4f98e1d6a5330641a56292583cd2
ExpoKeepAwake: e8dedc115d9f6f24b153ccd2d1d8efcdfd68a527
ExpoModulesCore: 0bf5158ae9fb71e575582a8a48dbd26ced42f306
boost: 7e761d76ca2ce687f7cc98e698152abd03a18f90
DoubleConversion: cb417026b2400c8f53ae97020b2be961b59470cb
EXConstants: 9f310f44bfedba09087042756802040e464323c0
Expo: 769ab5c190382eedebc733af6708bbc9ca5f643b
ExpoAsset: 3bc9adb7dbbf27ae82c18ca97eb988a3ae7e73b1
ExpoBrightness: c335c6ccc082d5249a4b38dba5cd9a08aa0bf62b
ExpoCalendar: f5f94ea8dcd957b1434beb4e1c0da1af063322e6
ExpoFileSystem: 0f3f466ecd3560f55768cd3f94ac3a17f093b8e6
ExpoFont: abbb91a911eb961652c2b0a22eef801860425ed6
ExpoKeepAwake: bf0811570c8da182bfb879169437d4de298376e7
ExpoModulesCore: 3ac17421302df62928fc99c133cf25bdbcf0b004
ExpoSpeech: 4db7ef7888b9edc39ca9afee54e9c4b3df269ccb
fast_float: 06eeec4fe712a76acc9376682e4808b05ce978b6
FBLazyVector: 84b955f7b4da8b895faf5946f73748267347c975
fmt: 01b82d4ca6470831d1cc0852a1af644be019e8f6
glog: 08b301085f15bcbb6ff8632a8ebaf239aae04e6a
fmt: a40bb5bd0294ea969aaaba240a927bd33d878cdd
glog: 5683914934d5b6e4240e497e0f4a3b42d1854183
hermes-engine: 314be5250afa5692b57b4dd1705959e1973a8ebe
opencv-rne: 63e933ae2373fc91351f9a348dc46c3f523c2d3f
RCT-Folly: 36fe2295e44b10d831836cc0d1daec5f8abcf809
RCT-Folly: e78785aa9ba2ed998ea4151e314036f6c49e6d82
RCTDeprecation: 83ffb90c23ee5cea353bd32008a7bca100908f8c
RCTRequired: eb7c0aba998009f47a540bec9e9d69a54f68136e
RCTTypeSafety: 659ae318c09de0477fd27bbc9e140071c7ea5c93
React: c2d3aa44c49bb34e4dfd49d3ee92da5ebacc1c1c
React-callinvoker: 1bdfb7549b5af266d85757193b5069f60659ef9d
React-Core: 7150cf9b6a5af063b37003062689f1691e79c020
React-CoreModules: 15a85e6665d61678942da6ae485b351f4c699049
React-cxxreact: 74f9de59259ac951923f5726aa14f0398f167af9
React-Core: 10597593fdbae06f0089881e025a172e51d4a769
React-CoreModules: 6907b255529dd46895cf687daa67b24484a612c2
React-cxxreact: a9f5b8180d6955bc3f6a3fcd657c4d9b4d95c1f6
React-debug: e74e76912b91e08d580c481c34881899ccf63da9
React-defaultsnativemodule: 628285212bbd65417d40ad6a9f8781830fda6c98
React-domnativemodule: 185d9808198405c176784aaf33403d713bd24fb7
React-Fabric: c814804affbe1952e16149ddd20256e1bccae67e
React-FabricComponents: 81ef47d596966121784afec9924f9562a29b1691
React-FabricImage: f14f371d678aa557101def954ac3ba27e48948ff
React-defaultsnativemodule: 11f6ee2cf69bf3af9d0f28a6253def33d21b5266
React-domnativemodule: f940bbc4fa9e134190acbf3a4a9f95621b5a8f51
React-Fabric: 6f5c357bf3a42ff11f8844ad3fc7a1eb04f4b9de
React-FabricComponents: 10e0c0209822ac9e69412913a8af1ca33573379b
React-FabricImage: f582e764072dfa4715ae8c42979a5bace9cbcc12
React-featureflags: d5facceff8f8f6de430e0acecf4979a9a0839ba9
React-featureflagsnativemodule: 96f0ab285382d95c90f663e02526a5ceefa95a11
React-graphics: 1a66ee0a3f093b125b853f6370296fadcaf6f233
React-hermes: 8b86e5f54a65ecb69cdf22b3a00a11562eda82d2
React-idlecallbacksnativemodule: 5c25ab145c602264d00cb26a397ab52e0efa031c
React-ImageManager: 15e34bd5ef1ac4a18e96660817ef70a7f99ee8c2
React-jserrorhandler: 02cdf2cd45350108be1ffd2b164578936dbbdff7
React-jsi: 6af1987cfbb1b6621664fdbf6c7b62bd4d38c923
React-jsiexecutor: 51f372998e0303585cb0317232b938d694663cbd
React-jsinspector: 3539ad976d073bfaa8a7d2fa9bef35e70e55033e
React-jsinspectortracing: e8dbacaf67c201f23052ca1c2bae2f7b84dec443
React-jsitooling: 95a34f41e3c249d42181de13b4f8d854f178ca9f
React-jsitracing: 25b029cf5cad488252d46da19dd8c4c134fd5fe4
React-logger: 368570a253f00879a1e4fea24ed4047e72e7bbf3
React-Mapbuffer: c04fcda1c6281fc0a6824c7dcc1633dd217ac1ec
React-microtasksnativemodule: ca2804a25fdcefffa0aa942aa23ab53b99614a34
react-native-executorch: aa97983ed344f0cbfb47f531f0c24638c579f1c1
react-native-safe-area-context: 00d03dc688ba86664be66f9e3f203fc7d747d899
React-NativeModulesApple: 452b86b29fae99ed0a4015dca3ad9cd222f88abf
React-featureflagsnativemodule: a7dd141f1ef4b7c1331af0035689fbc742a49ff4
React-graphics: 36ae3407172c1c77cea29265d2b12b90aaef6aa0
React-hermes: 9116d4e6d07abeb519a2852672de087f44da8f12
React-idlecallbacksnativemodule: ae7f5ffc6cf2d2058b007b78248e5b08172ad5c3
React-ImageManager: 9daee0dc99ad6a001d4b9e691fbf37107e2b7b54
React-jserrorhandler: 1e6211581071edaf4ecd5303147328120c73f4dc
React-jsi: 753ba30c902f3a41fa7f956aca8eea3317a44ee6
React-jsiexecutor: 47520714aa7d9589c51c0f3713dfbfca4895d4f9
React-jsinspector: cfd27107f6d6f1076a57d88c932401251560fe5f
React-jsinspectortracing: 76a7d791f3c0c09a0d2bf6f46dfb0e79a4fcc0ac
React-jsitooling: 995e826570dd58f802251490486ebd3244a037ab
React-jsitracing: 094ae3d8c123cea67b50211c945b7c0443d3e97b
React-logger: 8edfcedc100544791cd82692ca5a574240a16219
React-Mapbuffer: c3f4b608e4a59dd2f6a416ef4d47a14400194468
React-microtasksnativemodule: 054f34e9b82f02bd40f09cebd4083828b5b2beb6
react-native-executorch: d0c3dffa0a4a4111ea9c7b97f3fbf088a48d3b2a
react-native-safe-area-context: 562163222d999b79a51577eda2ea8ad2c32b4d06
React-NativeModulesApple: 2c4377e139522c3d73f5df582e4f051a838ff25e
React-oscompat: ef5df1c734f19b8003e149317d041b8ce1f7d29c
React-perflogger: 6fd2f6811533e9c19a61e855c3033eecbf4ad2a0
React-performancetimeline: abf31259d794c9274b3ea19c5016186925eec6c4
React-perflogger: 9a151e0b4c933c9205fd648c246506a83f31395d
React-performancetimeline: 5b0dfc0acba29ea0269ddb34cd6dd59d3b8a1c66
React-RCTActionSheet: a499b0d6d9793886b67ba3e16046a3fef2cdbbc3
React-RCTAnimation: 2595dcb10a82216a511b54742f8c28d793852ac6
React-RCTAppDelegate: f03604b70f57c9469a84a159d8abecf793a5bcff
React-RCTBlob: e00f9b4e2f151938f4d9864cf33ebf24ac03328a
React-RCTFabric: 3945d116fd271598db262d4e6ed5691d431ed9e8
React-RCTFBReactNativeSpec: 0f4d4f0da938101f2ca9d5333a8f46e527ad2819
React-RCTImage: dac5e9f8ec476aefe6e60ee640ebc1dfaf1a4dbe
React-RCTLinking: 494b785a40d952a1dfbe712f43214376e5f0e408
React-RCTNetwork: b3d7c30cd21793e268db107dd0980cb61b3c1c44
React-RCTRuntime: a8ff419d437228e7b8a793b14f9d711e1cbb82af
React-RCTSettings: a060c7e381a3896104761b8eed7e284d95e37df3
React-RCTText: 4f272b72dbb61f390d8c8274528f9fdbff983806
React-RCTVibration: 0e5326220719aca12473d703aa46693e3b4ce67a
React-RCTAnimation: cc64adc259aabc3354b73065e2231d796dfce576
React-RCTAppDelegate: 9d523da768f1c9e84c5f3b7e3624d097dfb0e16b
React-RCTBlob: e727f53eeefded7e6432eb76bd22b57bc880e5d1
React-RCTFabric: 58590aa4fdb4ad546c06a7449b486cf6844e991f
React-RCTFBReactNativeSpec: 9064c63d99e467a3893e328ba3612745c3c3a338
React-RCTImage: 7159cbdbb18a09d97ba1a611416eced75b3ccb29
React-RCTLinking: 46293afdb859bccc63e1d3dedc6901a3c04ef360
React-RCTNetwork: 4a6cd18f5bcd0363657789c64043123a896b1170
React-RCTRuntime: 5ab904fd749aa52f267ef771d265612582a17880
React-RCTSettings: 61e361dc85136d1cb0e148b7541993d2ee950ea7
React-RCTText: abd1e196c3167175e6baef18199c6d9d8ac54b4e
React-RCTVibration: 490e0dcb01a3fe4a0dfb7bc51ad5856d8b84f343
React-rendererconsistency: 351fdbc5c1fe4da24243d939094a80f0e149c7a1
React-renderercss: d333f2ada83969591100d91ec6b23ca2e17e1507
React-rendererdebug: 039e5949b72ba63c703de020701e3fd152434c61
React-renderercss: 3438814bee838ae7840a633ab085ac81699fd5cf
React-rendererdebug: 0ac2b9419ad6f88444f066d4b476180af311fb1e
React-rncore: 57ed480649bb678d8bdc386d20fee8bf2b0c307c
React-RuntimeApple: 344a5e1105256000afabaa8df12c3e4cab880340
React-RuntimeCore: 0e48fb5e5160acc0334c7a723a42d42cef4b58b6
React-RuntimeApple: 8b7a9788f31548298ba1990620fe06b40de65ad7
React-RuntimeCore: e03d96fbd57ce69fd9bca8c925942194a5126dbc
React-runtimeexecutor: d60846710facedd1edb70c08b738119b3ee2c6c2
React-RuntimeHermes: 064286a03871d932c99738e0f8ef854962ab4b99
React-runtimescheduler: e917ab17ae08c204af1ebf8f669b7e411b0220c8
React-RuntimeHermes: aab794755d9f6efd249b61f3af4417296904e3ba
React-runtimescheduler: c3cd124fa5db7c37f601ee49ca0d97019acd8788
React-timing: a90f4654cbda9c628614f9bee68967f1768bd6a5
React-utils: 51c4e71608b8133fecc9a15801d244ae7bdf3758
ReactAppDependencyProvider: d5dcc564f129632276bd3184e60f053fcd574d6b
ReactCodegen: fda99a79c866370190e162083a35602fdc314e5d
ReactCommon: 4d0da92a5eb8da86c08e3ec34bd23ab439fb2461
RNReanimated: 8b24b49fc13fce9a6e1729ccff645a63d2b7a6d1
RNSVG: ee32efbed652c5151fd3f98bed13c68af285bc38
React-utils: a612d50555b6f0f90c74b7d79954019ad47f5de6
ReactAppDependencyProvider: 04d5eb15eb46be6720e17a4a7fa92940a776e584
ReactCodegen: c63eda03ba1d94353fb97b031fc84f75a0d125ba
ReactCommon: 76d2dc87136d0a667678668b86f0fca0c16fdeb0
RNAudioAPI: 2e3fd4bf75aa5717791babb30126707504996f09
RNLiveAudioStream: 93ac2bb6065be9018d0b00157b220f11cebc1513
RNReanimated: 2313402fe27fecb7237619e9c6fcee3177f08a65
RNSVG: 794f269526df9ddc1f79b3d1a202b619df0368e3
SocketRocket: d4aabe649be1e368d1318fdf28a022d714d65748
Yoga: 9f110fc4b7aa538663cba3c14cbb1c335f43c13f
Yoga: c758bfb934100bb4bf9cbaccb52557cee35e8bdf

PODFILE CHECKSUM: bba19a069e673f2259009e9d2caab44374fdebcf

Expand Down
Loading
Loading