VinF Hybrid Inference #1: Update the API #8874

erikeldridge · 2025-03-27T17:35:39Z

This change updates the getGenerativeModel getter to accept ModelParams or HybridParams. The HybridParams.mode field indicates whether hybrid inference is intended.

This change just updates the inputs. All existing tests pass, so the diff should be relatively easy to understand. The next change will use the inputs.

changeset-bot · 2025-03-27T17:35:42Z

⚠️ No Changeset found

Latest commit: 3ec2857

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

google-oss-bot · 2025-03-27T17:46:07Z

Size Report ¹

Affected Products

@firebase/vertexai
Type Base (dcc62c0) Merge (a63dbe4) Diff
browser 34.7 kB 35.0 kB +272 B (+0.8%)
main 35.7 kB 36.0 kB +272 B (+0.8%)
module 34.7 kB 35.0 kB +272 B (+0.8%)
firebase
Type Base (dcc62c0) Merge (a63dbe4) Diff
firebase-vertexai.js 28.3 kB 28.4 kB +168 B (+0.6%)

Test Logs

https://storage.googleapis.com/firebase-sdk-metric-reports/YLWP6MxbEo.html

google-oss-bot · 2025-03-27T17:57:30Z

Size Analysis Report ¹

Affected Products

`@firebase/vertexai`

GenerativeModel
Size
Type Base (dcc62c0) Merge (a63dbe4) Diff
size 18.9 kB 19.0 kB +57 B (+0.3%)
size-with-ext-deps 37.9 kB 37.9 kB +57 B (+0.2%)

`ImagenModel`

Size

Type	Base (`dcc62c0`)	Merge (`a63dbe4`)	Diff
size	10.1 kB	20.7 kB	+10.6 kB (+105.8%)
size-with-ext-deps	28.1 kB	39.7 kB	+11.6 kB (+41.5%)

Dependency

Type Base (dcc62c0) Merge (a63dbe4) Diff

functions

constructRequest
createPredictRequestBody
getClientHeaders
getHeaders
handlePredictResponse
makeRequest
registerVertex

26 dependencies

addHelpers
aggregateResponses
assignRoleToPartsAndValidateSendMessageRequest
constructRequest
countTokens
createEnhancedContentResponse
createPredictRequestBody
formatBlockErrorMessage
formatGenerateContentInput
formatNewContent
formatSystemInstruction
generateContent
generateContentStream
generateResponseSequence
getClientHeaders
getFunctionCalls
getHeaders
getResponsePromise
getResponseStream
getText
hadBadFinishReason
handlePredictResponse
makeRequest
processStream
registerVertex
validateChatHistory

19 dependency diffs

+ addHelpers
+ aggregateResponses
+ assignRoleToPartsAndValidateSendMessageRequest
+ countTokens
+ createEnhancedContentResponse
+ formatBlockErrorMessage
+ formatGenerateContentInput
+ formatNewContent
+ formatSystemInstruction
+ generateContent
+ generateContentStream
+ generateResponseSequence
+ getFunctionCalls
+ getResponsePromise
+ getResponseStream
+ getText
+ hadBadFinishReason
+ processStream
+ validateChatHistory

classes

ImagenModel
RequestUrl
VertexAIError
VertexAIModel
VertexAIService

ChatSession
GenerativeModel
ImagenModel
RequestUrl
VertexAIError
VertexAIModel
VertexAIService

+ ChatSession
+ GenerativeModel

variables

24 dependencies

BlockReason
DEFAULT_API_VERSION
DEFAULT_BASE_URL
DEFAULT_FETCH_TIMEOUT_MS
DEFAULT_LOCATION
FinishReason
FunctionCallingMode
HarmBlockMethod
HarmBlockThreshold
HarmCategory
HarmProbability
HarmSeverity
ImagenAspectRatio
ImagenPersonFilterLevel
ImagenSafetyFilterLevel
LANGUAGE_TAG
Modality
PACKAGE_VERSION
SchemaType
Task
VERTEX_TYPE
logger
name
version

31 dependencies

BlockReason
DEFAULT_API_VERSION
DEFAULT_BASE_URL
DEFAULT_FETCH_TIMEOUT_MS
DEFAULT_LOCATION
FinishReason
FunctionCallingMode
HarmBlockMethod
HarmBlockThreshold
HarmCategory
HarmProbability
HarmSeverity
ImagenAspectRatio
ImagenPersonFilterLevel
ImagenSafetyFilterLevel
LANGUAGE_TAG
Modality
PACKAGE_VERSION
POSSIBLE_ROLES
SILENT_ERROR
SchemaType
Task
VALID_PARTS_PER_ROLE
VALID_PART_FIELDS
VALID_PREVIOUS_CONTENT_ROLES
VERTEX_TYPE
badFinishReasons
logger
name
responseLineRE
version

+ POSSIBLE_ROLES
+ SILENT_ERROR
+ VALID_PARTS_PER_ROLE
+ VALID_PART_FIELDS
+ VALID_PREVIOUS_CONTENT_ROLES
+ badFinishReasons
+ responseLineRE

External Dependency

Module	Base (`dcc62c0`)	Merge (`a63dbe4`)	Diff
`tslib`		__asyncGenerator __await	+ __asyncGenerator + __await

`VertexAIModel`

Size

Type	Base (`dcc62c0`)	Merge (`a63dbe4`)	Diff
size	5.23 kB	19.0 kB	+13.7 kB (+262.6%)
size-with-ext-deps	23.2 kB	37.9 kB	+14.8 kB (+63.7%)

Dependency

Type Base (dcc62c0) Merge (a63dbe4) Diff

functions

registerVertex

24 dependencies

addHelpers
aggregateResponses
assignRoleToPartsAndValidateSendMessageRequest
constructRequest
countTokens
createEnhancedContentResponse
formatBlockErrorMessage
formatGenerateContentInput
formatNewContent
formatSystemInstruction
generateContent
generateContentStream
generateResponseSequence
getClientHeaders
getFunctionCalls
getHeaders
getResponsePromise
getResponseStream
getText
hadBadFinishReason
makeRequest
processStream
registerVertex
validateChatHistory

23 dependency diffs

+ addHelpers
+ aggregateResponses
+ assignRoleToPartsAndValidateSendMessageRequest
+ constructRequest
+ countTokens
+ createEnhancedContentResponse
+ formatBlockErrorMessage
+ formatGenerateContentInput
+ formatNewContent
+ formatSystemInstruction
+ generateContent
+ generateContentStream
+ generateResponseSequence
+ getClientHeaders
+ getFunctionCalls
+ getHeaders
+ getResponsePromise
+ getResponseStream
+ getText
+ hadBadFinishReason
+ makeRequest
+ processStream
+ validateChatHistory

classes

VertexAIError
VertexAIModel
VertexAIService

ChatSession
GenerativeModel
RequestUrl
VertexAIError
VertexAIModel
VertexAIService

+ ChatSession
+ GenerativeModel
+ RequestUrl

variables

18 dependencies

BlockReason
DEFAULT_LOCATION
FinishReason
FunctionCallingMode
HarmBlockMethod
HarmBlockThreshold
HarmCategory
HarmProbability
HarmSeverity
ImagenAspectRatio
ImagenPersonFilterLevel
ImagenSafetyFilterLevel
Modality
SchemaType
Task
VERTEX_TYPE
name
version

31 dependencies

BlockReason
DEFAULT_API_VERSION
DEFAULT_BASE_URL
DEFAULT_FETCH_TIMEOUT_MS
DEFAULT_LOCATION
FinishReason
FunctionCallingMode
HarmBlockMethod
HarmBlockThreshold
HarmCategory
HarmProbability
HarmSeverity
ImagenAspectRatio
ImagenPersonFilterLevel
ImagenSafetyFilterLevel
LANGUAGE_TAG
Modality
PACKAGE_VERSION
POSSIBLE_ROLES
SILENT_ERROR
SchemaType
Task
VALID_PARTS_PER_ROLE
VALID_PART_FIELDS
VALID_PREVIOUS_CONTENT_ROLES
VERTEX_TYPE
badFinishReasons
logger
name
responseLineRE
version

13 dependency diffs

+ DEFAULT_API_VERSION
+ DEFAULT_BASE_URL
+ DEFAULT_FETCH_TIMEOUT_MS
+ LANGUAGE_TAG
+ PACKAGE_VERSION
+ POSSIBLE_ROLES
+ SILENT_ERROR
+ VALID_PARTS_PER_ROLE
+ VALID_PART_FIELDS
+ VALID_PREVIOUS_CONTENT_ROLES
+ badFinishReasons
+ logger
+ responseLineRE

External Dependency

Module	Base (`dcc62c0`)	Merge (`a63dbe4`)	Diff
`tslib`		__asyncGenerator __await	+ __asyncGenerator + __await

getGenerativeModel
Size
Type Base (dcc62c0) Merge (a63dbe4) Diff
size 19.1 kB 19.2 kB +153 B (+0.8%)
size-with-ext-deps 38.1 kB 38.2 kB +153 B (+0.4%)

`getImagenModel`

Size

Type	Base (`dcc62c0`)	Merge (`a63dbe4`)	Diff
size	10.2 kB	20.9 kB	+10.7 kB (+104.1%)
size-with-ext-deps	28.2 kB	39.9 kB	+11.6 kB (+41.3%)

Dependency

Type Base (dcc62c0) Merge (a63dbe4) Diff

functions

constructRequest
createPredictRequestBody
getClientHeaders
getHeaders
getImagenModel
handlePredictResponse
makeRequest
registerVertex

27 dependencies

addHelpers
aggregateResponses
assignRoleToPartsAndValidateSendMessageRequest
constructRequest
countTokens
createEnhancedContentResponse
createPredictRequestBody
formatBlockErrorMessage
formatGenerateContentInput
formatNewContent
formatSystemInstruction
generateContent
generateContentStream
generateResponseSequence
getClientHeaders
getFunctionCalls
getHeaders
getImagenModel
getResponsePromise
getResponseStream
getText
hadBadFinishReason
handlePredictResponse
makeRequest
processStream
registerVertex
validateChatHistory

19 dependency diffs

+ addHelpers
+ aggregateResponses
+ assignRoleToPartsAndValidateSendMessageRequest
+ countTokens
+ createEnhancedContentResponse
+ formatBlockErrorMessage
+ formatGenerateContentInput
+ formatNewContent
+ formatSystemInstruction
+ generateContent
+ generateContentStream
+ generateResponseSequence
+ getFunctionCalls
+ getResponsePromise
+ getResponseStream
+ getText
+ hadBadFinishReason
+ processStream
+ validateChatHistory

classes

ImagenModel
RequestUrl
VertexAIError
VertexAIModel
VertexAIService

ChatSession
GenerativeModel
ImagenModel
RequestUrl
VertexAIError
VertexAIModel
VertexAIService

+ ChatSession
+ GenerativeModel

variables

24 dependencies

BlockReason
DEFAULT_API_VERSION
DEFAULT_BASE_URL
DEFAULT_FETCH_TIMEOUT_MS
DEFAULT_LOCATION
FinishReason
FunctionCallingMode
HarmBlockMethod
HarmBlockThreshold
HarmCategory
HarmProbability
HarmSeverity
ImagenAspectRatio
ImagenPersonFilterLevel
ImagenSafetyFilterLevel
LANGUAGE_TAG
Modality
PACKAGE_VERSION
SchemaType
Task
VERTEX_TYPE
logger
name
version

31 dependencies

BlockReason
DEFAULT_API_VERSION
DEFAULT_BASE_URL
DEFAULT_FETCH_TIMEOUT_MS
DEFAULT_LOCATION
FinishReason
FunctionCallingMode
HarmBlockMethod
HarmBlockThreshold
HarmCategory
HarmProbability
HarmSeverity
ImagenAspectRatio
ImagenPersonFilterLevel
ImagenSafetyFilterLevel
LANGUAGE_TAG
Modality
PACKAGE_VERSION
POSSIBLE_ROLES
SILENT_ERROR
SchemaType
Task
VALID_PARTS_PER_ROLE
VALID_PART_FIELDS
VALID_PREVIOUS_CONTENT_ROLES
VERTEX_TYPE
badFinishReasons
logger
name
responseLineRE
version

+ POSSIBLE_ROLES
+ SILENT_ERROR
+ VALID_PARTS_PER_ROLE
+ VALID_PART_FIELDS
+ VALID_PREVIOUS_CONTENT_ROLES
+ badFinishReasons
+ responseLineRE

External Dependency

Module	Base (`dcc62c0`)	Merge (`a63dbe4`)	Diff
`tslib`		__asyncGenerator __await	+ __asyncGenerator + __await

Test Logs

https://storage.googleapis.com/firebase-sdk-metric-reports/IaJu6SujRz.html

github-actions · 2025-04-03T20:59:15Z

Vertex AI Mock Responses Check ⚠️

A newer major version of the mock responses for Vertex AI unit tests is available. update_vertexai_responses.sh should be updated to clone the latest version of the responses: v8.0

hsubox76 · 2025-04-09T18:51:44Z

repo-scripts/changelog-generator/tsconfig.json

@@ -3,7 +3,8 @@
    "strict": true,
    "outDir": "dist",
    "lib": [
-      "ESNext"
+      "ESNext",
+      "dom"


Why is this file changed?

Without this, adding the @types/dom-chromium-ai dependency causes the postsubmit script to fail. I'll try reproing the issue on a stand-alone branch.

Created PR #8916 to repro in CI env.

…ldridge-vertex-api

dlarocque · 2025-04-15T19:31:53Z

packages/vertexai/src/types/requests.ts

@@ -213,3 +213,28 @@ export interface FunctionCallingConfig {
  mode?: FunctionCallingMode;
  allowedFunctionNames?: string[];
 }
+
+/**
+ * Toggles hybrid inference.


Could you add the @public tag here and in other JSDoc comments for public APIs?

dlarocque · 2025-04-15T19:43:41Z

packages/vertexai/src/models/generative-model.ts

+  /**
+   * Defines the name of the default in-cloud model to use for hybrid inference.
+   */
+  static DEFAULT_HYBRID_IN_CLOUD_MODEL = 'gemini-2.0-flash-lite';


Could we avoid defining a default model? If this model string is deprecated in the future, and a user is using an old version of the SDK, would using this default result in an error?

erikeldridge added 8 commits March 27, 2025 10:14

Define HybridParams

f9faf27

Copy over most types from @types package

10f99a7

Document HybridParams and InferenceMode

cb88d70

Trim unused AI types

7752cb0

Assert HybridParams sets the model name

0d628cf

Use dom-chromium-ai package directly

9302d67

Use multi-line JSDoc comments for enum

eb6ae3c

Use the existing name of the model params input

b4aeabc

erikeldridge requested a review from hsubox76 March 27, 2025 17:35

erikeldridge requested a review from a team as a code owner March 27, 2025 17:35

Run docgen:all

2c8915e

erikeldridge requested a review from a team as a code owner March 27, 2025 21:24

erikeldridge mentioned this pull request Mar 27, 2025

VinF Hybrid Inference #2: Introduce ChromeAdapter class #8877

Closed

erikeldridge added 4 commits March 27, 2025 16:41

Use in-cloud phrasing

ae1c254

Parameterize default in-cloud model name

559bd13

Use type for inference mode and update docs

814a1dc

Run yarn format

78fed95

hsubox76 reviewed Apr 9, 2025

View reviewed changes

This was referenced Apr 9, 2025

VinF Hybrid Inference #0: add dom-chromium-ai dep #8916

Closed

VinF Hybrid Inference: @types/dom-chromium-ai #8920

Open

hsubox76 and others added 4 commits April 14, 2025 16:07

rebased and updated deps

8302584

revert deps

d8f8fb3

Merge remote-tracking branch 'public/ch-test-dom-chromium' into erike…

d4ae4e0

…ldridge-vertex-api

Remove erroneous type dep

3ec2857

dlarocque requested changes Apr 15, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

VinF Hybrid Inference #1: Update the API #8874

VinF Hybrid Inference #1: Update the API #8874

erikeldridge commented Mar 27, 2025

changeset-bot bot commented Mar 27, 2025 •

edited

Loading

google-oss-bot commented Mar 27, 2025 •

edited

Loading

`@firebase/vertexai`

`firebase`

google-oss-bot commented Mar 27, 2025 •

edited

Loading

`@firebase/vertexai`

`GenerativeModel`

`ImagenModel`

`VertexAIModel`

`getGenerativeModel`

`getImagenModel`

github-actions bot commented Apr 3, 2025

hsubox76 Apr 9, 2025

erikeldridge Apr 9, 2025

erikeldridge Apr 9, 2025

dlarocque Apr 15, 2025

dlarocque Apr 15, 2025

VinF Hybrid Inference #1: Update the API #8874

Are you sure you want to change the base?

VinF Hybrid Inference #1: Update the API #8874

Conversation

erikeldridge commented Mar 27, 2025

changeset-bot bot commented Mar 27, 2025 • edited Loading

⚠️ No Changeset found

google-oss-bot commented Mar 27, 2025 • edited Loading

Size Report 1

Affected Products

@firebase/vertexai

firebase

Test Logs

google-oss-bot commented Mar 27, 2025 • edited Loading

Size Analysis Report 1

Affected Products

@firebase/vertexai

GenerativeModel

ImagenModel

VertexAIModel

getGenerativeModel

getImagenModel

Test Logs

github-actions bot commented Apr 3, 2025

Vertex AI Mock Responses Check ⚠️

hsubox76 Apr 9, 2025

Choose a reason for hiding this comment

erikeldridge Apr 9, 2025

Choose a reason for hiding this comment

erikeldridge Apr 9, 2025

Choose a reason for hiding this comment

dlarocque Apr 15, 2025

Choose a reason for hiding this comment

dlarocque Apr 15, 2025

Choose a reason for hiding this comment

changeset-bot bot commented Mar 27, 2025 •

edited

Loading

google-oss-bot commented Mar 27, 2025 •

edited

Loading

Size Report ¹

`@firebase/vertexai`

`firebase`

google-oss-bot commented Mar 27, 2025 •

edited

Loading

Size Analysis Report ¹

`@firebase/vertexai`

`GenerativeModel`

`ImagenModel`

`VertexAIModel`

`getGenerativeModel`

`getImagenModel`