Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(vertexai): Add Live streaming feature #16991

Merged
merged 44 commits into from
Mar 25, 2025
Merged
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
f53b00c
init
cynthiajoan Jan 13, 2025
0e5a21d
more testing for the audio player
cynthiajoan Jan 20, 2025
cd59d74
organize example page and functions
cynthiajoan Jan 23, 2025
8b3285d
Merge branch 'vertexai/example_update' into vertexai/bidi
cynthiajoan Jan 23, 2025
c5bb860
get bidi demo working under new example layout
cynthiajoan Jan 23, 2025
9575c5e
Merge branch 'main' into vertexai/bidi
cynthiajoan Jan 27, 2025
3ff550e
kinda playable
cynthiajoan Jan 28, 2025
d2cbaff
working audio output
cynthiajoan Jan 29, 2025
76aa22b
use firebase project for the connection
cynthiajoan Jan 30, 2025
c1695e5
record to file and load file to stream, no response yet
cynthiajoan Feb 3, 2025
b361009
try different ways to test sending audio
cynthiajoan Feb 3, 2025
9988f72
try realtime input
cynthiajoan Feb 6, 2025
2d37054
Merge branch 'main' into vertexai/bidi
cynthiajoan Feb 12, 2025
c9476e0
stream recording, try local file dump
cynthiajoan Feb 18, 2025
89c7190
first kinda working version
cynthiajoan Feb 21, 2025
8b3bc15
much better
cynthiajoan Feb 21, 2025
fe2073b
Merge branch 'main' into vertexai/bidi
cynthiajoan Feb 21, 2025
d53655f
Make function calling working
cynthiajoan Feb 26, 2025
fd128b2
add new autopush backend
cynthiajoan Feb 27, 2025
2541969
Some update after api doc
cynthiajoan Mar 9, 2025
1326935
more update on api
cynthiajoan Mar 11, 2025
7ee188c
more clean up for api
cynthiajoan Mar 11, 2025
0ca9d5a
Let's still keep image modality
cynthiajoan Mar 11, 2025
b675e6a
minor json fix
cynthiajoan Mar 12, 2025
e5920c2
Use IOWebSocketChannel class to pass in headers
cynthiajoan Mar 12, 2025
eff222c
unit test for live_api
cynthiajoan Mar 14, 2025
8275927
more controllable receive logic
cynthiajoan Mar 17, 2025
2a8164b
test fix after model library refactor
cynthiajoan Mar 17, 2025
07e18f9
documentation and try catch in the example
cynthiajoan Mar 17, 2025
202418e
Somehow working continuously conversation
cynthiajoan Mar 17, 2025
6c53ab3
function calling fix
cynthiajoan Mar 18, 2025
fdb006a
changes after bugbash
cynthiajoan Mar 19, 2025
04d1b40
Merge branch 'main' into vertexai/bidi
cynthiajoan Mar 19, 2025
22ae77e
fix after merge main
cynthiajoan Mar 19, 2025
9689825
more fixes after merge
cynthiajoan Mar 19, 2025
7de0ebb
fix analyzing issues
cynthiajoan Mar 24, 2025
3d78da0
more fix for analyzer
cynthiajoan Mar 24, 2025
7113e44
fix analyzer
cynthiajoan Mar 24, 2025
104714e
two more analyzer
cynthiajoan Mar 24, 2025
bda672a
fix the live test for function call
cynthiajoan Mar 24, 2025
d03df57
Address review comment
cynthiajoan Mar 25, 2025
ce683ee
More review comments to address
cynthiajoan Mar 25, 2025
c8bf5fe
review comments on audio player and recorder
cynthiajoan Mar 25, 2025
d2951e7
Update code to handle optional id for FunctionCall parse
cynthiajoan Mar 25, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
test fix after model library refactor
cynthiajoan committed Mar 17, 2025
commit 2a8164b9d1379f2edbd9db78c31c7855b2ee3825
Original file line number Diff line number Diff line change
@@ -13,7 +13,7 @@
// limitations under the License.
import 'package:firebase_core/firebase_core.dart';
import 'package:firebase_vertexai/firebase_vertexai.dart';
import 'package:firebase_vertexai/src/generative_model.dart';
import 'package:firebase_vertexai/src/base_model.dart';
import 'package:flutter_test/flutter_test.dart';

import 'mock.dart';
Original file line number Diff line number Diff line change
@@ -11,18 +11,11 @@
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import 'dart:async';
import 'dart:convert';
import 'dart:typed_data';

import 'package:firebase_app_check/firebase_app_check.dart';
import 'package:firebase_auth/firebase_auth.dart';
import 'package:firebase_core/firebase_core.dart';
import 'package:firebase_vertexai/src/content.dart';
import 'package:firebase_vertexai/src/error.dart';
import 'package:firebase_vertexai/src/live_api.dart';
import 'package:firebase_vertexai/src/live_model.dart';
import 'package:firebase_vertexai/src/live_session.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:mockito/mockito.dart';
import 'package:web_socket_channel/web_socket_channel.dart';
Original file line number Diff line number Diff line change
@@ -13,7 +13,7 @@
// limitations under the License.
import 'package:firebase_core/firebase_core.dart';
import 'package:firebase_vertexai/firebase_vertexai.dart';
import 'package:firebase_vertexai/src/generative_model.dart';
import 'package:firebase_vertexai/src/base_model.dart';
import 'package:flutter_test/flutter_test.dart';

import 'mock.dart';