Skip to content

Add function sample for server-side Remote Config + Vertex AI Gemini API #1135

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Jun 4, 2024
31 changes: 31 additions & 0 deletions Node/remote-config-server-with-vertex/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Use server-side Remote Config with Cloud Functions and Vertex AI

This Cloud Function (`generateWithVertex`) demonstrates how to generate text
using Google's Vertex AI Gemini API. It uses
the Firebase Admin SDK for Node.js and Remote Config to manage model parameters,
safety settings, and feature flags.

## Setting up the sample

Follow the [User server-side Remote Config with Cloud Functions and Vertex AI
guide](https://firebase.google.com/docs/remote-config/solution-server) to:

* Set up your Firebase project.
* Enable required APIs and SDKs.
* Configure IAM permissions.
* Test your function in the Firebase emulator.
* Deploy your function.

Important: Vertex AI and Cloud Functions require a billing account. Review
[Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and
[Cloud Functions pricing](https://firebase.google.com/pricing) before running
this function. If you're new to Firebase and Google Cloud, check to see if
you're eligible for a
[$300 credit](https://firebase.google.com/support/faq#pricing-free-trial) and
a Free Trial Cloud Billing account.

## Next steps

Learn more about Remote Config server implementations at
[Use Remote Config in server
environments](https://firebase.google.com/docs/remote-config/server).
5 changes: 5 additions & 0 deletions Node/remote-config-server-with-vertex/firebase.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"functions": {
"codebase": "remote-config-server-with-vertex"
}
}
119 changes: 119 additions & 0 deletions Node/remote-config-server-with-vertex/functions/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
// [START remote_config_server_vertex_init]
const { onRequest } = require("firebase-functions/v2/https");
const logger = require("firebase-functions/logger");

const { initializeApp } = require("firebase-admin/app");
const { VertexAI } = require('@google-cloud/vertexai');
const { getRemoteConfig } = require("firebase-admin/remote-config");

// Set and check environment variables.
const project = process.env.GCLOUD_PROJECT;

// Initialize Firebase.
const app = initializeApp();
// [END remote_config_server_vertex_init]

// [START remote_config_server_vertex_default_values]
// Define default (fallback) parameter values for Remote Config.
const defaultConfig = {

// Default values for Vertex AI.
model_name: "gemini-1.5-flash-preview-0514",
generation_config: [{
"stopSequences": [], "temperature": 0.7,
"maxOutputTokens": 64, "topP": 0.1, "topK": 20
}],
prompt: "I'm a developer who wants to learn about Firebase and you are a \
helpful assistant who knows everything there is to know about Firebase!",
safety_settings: [{
"category":
"HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT",
"threshold": "HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE"
}],
location: 'us-central1',

// Disable Vertex AI Gemini API access for testing.
vertex_enabled: false
};
// [END remote_config_server_vertex_default_values]

// [START remote_config_server_vertex_create_function]
// Export the function.
exports.generateWithVertex = onRequest(async (request, response) => {

try {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to add some kind of auth check here to encourage developers to protect their AI endpoints. #1134 (comment) does that with onCall + AppCheck + Auth, but this could check Auth with something like: https://firebase.google.com/docs/auth/web/service-worker-sessions#server_side_changes

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, you do have the vertex_enabled flag already. Auth would be ideal, but that flag is probably ok for now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the likelihood of folks leaving vertex_enabled to true exists...so I will get some auth in here! I think the service-worker-sessions method would actually work really well with the tutorial, so I'll try this out.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I wasn't able to get this working, but it looks like what I really want for this function (verifying the service account since this example uses a service account) is coming soon in firebase/firebase-admin-node#2466 - - so I cheated here and added a caveat to the README and created another callable function in call-vertex-remote-config-server that demonstrates App Check. And then I can plan to jump back in here and add the google-auth service account verification once it's in.

Do you think that works? Gonna ask for re-review because there are a lot of changes. :D


// Set up Remote Config.
const rc = getRemoteConfig(app);

// Get the Remote Config template and assign default values.
const template = await rc.getServerTemplate({
defaultConfig: defaultConfig
});

// Add the template evaluation to a constant.
const config = template.evaluate();

// Obtain values from Remote Config.
const textModel = config.getString("model_name") ||
defaultConfig.model_name;
const textPrompt = config.getString("prompt") || defaultConfig.prompt;
const generationConfig = config.getString("generation_config") ||
defaultConfig.generation_config;
const safetySettings = config.getString("safety_settings") ||
defaultConfig.safety_settings;
const location = config.getString("location") ||
defaultConfig.location;
const vertexEnabled = config.getBoolean("is_vertex_enabled") ||
defaultConfig.vertex_enabled;
// [END remote_config_server_vertex_create_function]

// [START remote_config_server_vertex_function_logic]
// Allow user input.
const userInput = request.query.prompt || '';

// Instantiate Vertex AI.
const vertex_ai = new VertexAI({ project: project, location: location });
Copy link
Collaborator

@jhuleatt jhuleatt May 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are project and location needed, or is that covered by Application Default Credentials?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, looks like they are needed and it doesn't automatically pull ADC. :/ Thinking it's because you can use Vertex in a different locale than your project.

const generativeModel = vertex_ai.getGenerativeModel({
model: textModel,
safety_settings: safetySettings,
generation_config: generationConfig,
});

// Create the chat; append user input to Remote Config-defined prompt.
const chat = generativeModel.startChat();
const chatInput = textPrompt + " " + userInput;

if (!chatInput) {
return res.status(400).send('Missing text prompt');
}
// If vertexEnabled isn't true, do not send queries to Vertex AI.
if (vertexEnabled !== true) {
response.status(200).send({
message: "Vertex AI call skipped. Vertex is not enabled."
});
return;
}

console.log("\nRunning with model ", textModel, ", prompt: ", textPrompt,
", generationConfig: ", generationConfig, ", safetySettings: ",
safetySettings, " in ", location, "\n");

const result = await chat.sendMessageStream(chatInput);
response.writeHead(200, { 'Content-Type': 'text/plain' });

for await (const item of result.stream) {
const chunk = item.candidates[0].content.parts[0].text;
console.log("Received chunk:", chunk);
response.write(chunk);
}

response.end();

} catch (error) {
console.error(error);
response.status(500).send('Internal server error');
}
});
// [END remote_config_server_vertex_function_logic]

24 changes: 24 additions & 0 deletions Node/remote-config-server-with-vertex/functions/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"name": "functions",
"description": "Cloud Functions for Firebase",
"scripts": {
"serve": "firebase emulators:start --only functions",
"shell": "firebase functions:shell",
"start": "npm run shell",
"deploy": "firebase deploy --only functions",
"logs": "firebase functions:log"
},
"engines": {
"node": "18"
},
"main": "index.js",
"dependencies": {
"@google-cloud/vertexai": "^1.1.0",
"firebase-admin": "^12.1.0",
"firebase-functions": "^4.3.0"
},
"devDependencies": {
"firebase-functions-test": "^3.1.0"
},
"private": true
}
Loading
Loading