-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Add function sample for server-side Remote Config + Vertex AI Gemini API #1135
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 3 commits
36ebaa8
642d2b9
4eb2b14
fdb9d14
a94f4c2
8580b48
85df0ba
0113036
82611fd
28a8ce1
9fa0056
547306d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
# Use server-side Remote Config with Cloud Functions and Vertex AI | ||
|
||
This Cloud Function (`generateWithVertex`) demonstrates how to generate text | ||
using Google's Vertex AI Gemini API. It uses | ||
the Firebase Admin SDK for Node.js and Remote Config to manage model parameters, | ||
safety settings, and feature flags. | ||
|
||
## Setting up the sample | ||
|
||
Follow the [User server-side Remote Config with Cloud Functions and Vertex AI | ||
guide](https://firebase.google.com/docs/remote-config/solution-server) to: | ||
|
||
* Set up your Firebase project. | ||
* Enable required APIs and SDKs. | ||
* Configure IAM permissions. | ||
* Test your function in the Firebase emulator. | ||
* Deploy your function. | ||
|
||
Important: Vertex AI and Cloud Functions require a billing account. Review | ||
[Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and | ||
[Cloud Functions pricing](https://firebase.google.com/pricing) before running | ||
this function. If you're new to Firebase and Google Cloud, check to see if | ||
you're eligible for a | ||
[$300 credit](https://firebase.google.com/support/faq#pricing-free-trial) and | ||
a Free Trial Cloud Billing account. | ||
|
||
## Next steps | ||
|
||
Learn more about Remote Config server implementations at | ||
[Use Remote Config in server | ||
environments](https://firebase.google.com/docs/remote-config/server). |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
{ | ||
"functions": { | ||
"codebase": "remote-config-server-with-vertex" | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,119 @@ | ||
// [START remote_config_server_vertex_init] | ||
const { onRequest } = require("firebase-functions/v2/https"); | ||
const logger = require("firebase-functions/logger"); | ||
|
||
const { initializeApp } = require("firebase-admin/app"); | ||
const { VertexAI } = require('@google-cloud/vertexai'); | ||
const { getRemoteConfig } = require("firebase-admin/remote-config"); | ||
|
||
// Set and check environment variables. | ||
const project = process.env.GCLOUD_PROJECT; | ||
|
||
// Initialize Firebase. | ||
const app = initializeApp(); | ||
// [END remote_config_server_vertex_init] | ||
|
||
// [START remote_config_server_vertex_default_values] | ||
// Define default (fallback) parameter values for Remote Config. | ||
const defaultConfig = { | ||
|
||
// Default values for Vertex AI. | ||
model_name: "gemini-1.5-flash-preview-0514", | ||
generation_config: [{ | ||
"stopSequences": [], "temperature": 0.7, | ||
"maxOutputTokens": 64, "topP": 0.1, "topK": 20 | ||
}], | ||
prompt: "I'm a developer who wants to learn about Firebase and you are a \ | ||
helpful assistant who knows everything there is to know about Firebase!", | ||
safety_settings: [{ | ||
"category": | ||
"HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT", | ||
"threshold": "HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE" | ||
}], | ||
location: 'us-central1', | ||
|
||
// Disable Vertex AI Gemini API access for testing. | ||
vertex_enabled: false | ||
}; | ||
// [END remote_config_server_vertex_default_values] | ||
|
||
// [START remote_config_server_vertex_create_function] | ||
// Export the function. | ||
exports.generateWithVertex = onRequest(async (request, response) => { | ||
|
||
try { | ||
|
||
// Set up Remote Config. | ||
const rc = getRemoteConfig(app); | ||
|
||
// Get the Remote Config template and assign default values. | ||
const template = await rc.getServerTemplate({ | ||
defaultConfig: defaultConfig | ||
}); | ||
|
||
// Add the template evaluation to a constant. | ||
const config = template.evaluate(); | ||
|
||
// Obtain values from Remote Config. | ||
const textModel = config.getString("model_name") || | ||
defaultConfig.model_name; | ||
const textPrompt = config.getString("prompt") || defaultConfig.prompt; | ||
const generationConfig = config.getString("generation_config") || | ||
defaultConfig.generation_config; | ||
const safetySettings = config.getString("safety_settings") || | ||
defaultConfig.safety_settings; | ||
const location = config.getString("location") || | ||
defaultConfig.location; | ||
const vertexEnabled = config.getBoolean("is_vertex_enabled") || | ||
defaultConfig.vertex_enabled; | ||
// [END remote_config_server_vertex_create_function] | ||
|
||
// [START remote_config_server_vertex_function_logic] | ||
// Allow user input. | ||
const userInput = request.query.prompt || ''; | ||
|
||
// Instantiate Vertex AI. | ||
const vertex_ai = new VertexAI({ project: project, location: location }); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are project and location needed, or is that covered by Application Default Credentials? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep, looks like they are needed and it doesn't automatically pull ADC. :/ Thinking it's because you can use Vertex in a different locale than your project. |
||
const generativeModel = vertex_ai.getGenerativeModel({ | ||
model: textModel, | ||
safety_settings: safetySettings, | ||
generation_config: generationConfig, | ||
}); | ||
|
||
// Create the chat; append user input to Remote Config-defined prompt. | ||
const chat = generativeModel.startChat(); | ||
jenh marked this conversation as resolved.
Show resolved
Hide resolved
|
||
const chatInput = textPrompt + " " + userInput; | ||
|
||
if (!chatInput) { | ||
return res.status(400).send('Missing text prompt'); | ||
} | ||
// If vertexEnabled isn't true, do not send queries to Vertex AI. | ||
if (vertexEnabled !== true) { | ||
response.status(200).send({ | ||
message: "Vertex AI call skipped. Vertex is not enabled." | ||
}); | ||
return; | ||
} | ||
|
||
console.log("\nRunning with model ", textModel, ", prompt: ", textPrompt, | ||
jenh marked this conversation as resolved.
Show resolved
Hide resolved
|
||
", generationConfig: ", generationConfig, ", safetySettings: ", | ||
safetySettings, " in ", location, "\n"); | ||
|
||
const result = await chat.sendMessageStream(chatInput); | ||
response.writeHead(200, { 'Content-Type': 'text/plain' }); | ||
|
||
for await (const item of result.stream) { | ||
const chunk = item.candidates[0].content.parts[0].text; | ||
console.log("Received chunk:", chunk); | ||
response.write(chunk); | ||
} | ||
|
||
response.end(); | ||
|
||
} catch (error) { | ||
console.error(error); | ||
response.status(500).send('Internal server error'); | ||
} | ||
}); | ||
// [END remote_config_server_vertex_function_logic] | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
{ | ||
"name": "functions", | ||
jenh marked this conversation as resolved.
Show resolved
Hide resolved
|
||
"description": "Cloud Functions for Firebase", | ||
"scripts": { | ||
"serve": "firebase emulators:start --only functions", | ||
"shell": "firebase functions:shell", | ||
"start": "npm run shell", | ||
"deploy": "firebase deploy --only functions", | ||
"logs": "firebase functions:log" | ||
}, | ||
"engines": { | ||
"node": "18" | ||
}, | ||
"main": "index.js", | ||
"dependencies": { | ||
"@google-cloud/vertexai": "^1.1.0", | ||
"firebase-admin": "^12.1.0", | ||
"firebase-functions": "^4.3.0" | ||
}, | ||
"devDependencies": { | ||
"firebase-functions-test": "^3.1.0" | ||
}, | ||
"private": true | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be great to add some kind of auth check here to encourage developers to protect their AI endpoints. #1134 (comment) does that with
onCall
+ AppCheck + Auth, but this could check Auth with something like: https://firebase.google.com/docs/auth/web/service-worker-sessions#server_side_changesThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, you do have the
vertex_enabled
flag already. Auth would be ideal, but that flag is probably ok for nowThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But the likelihood of folks leaving vertex_enabled to true exists...so I will get some auth in here! I think the service-worker-sessions method would actually work really well with the tutorial, so I'll try this out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So I wasn't able to get this working, but it looks like what I really want for this function (verifying the service account since this example uses a service account) is coming soon in firebase/firebase-admin-node#2466 - - so I cheated here and added a caveat to the README and created another callable function in call-vertex-remote-config-server that demonstrates App Check. And then I can plan to jump back in here and add the google-auth service account verification once it's in.
Do you think that works? Gonna ask for re-review because there are a lot of changes. :D