Skip to content

feat: add create_http_options to ContextCacheConfig for cache creation timeout control #4703

@abhinavmaddineni

Description

@abhinavmaddineni

Problem

CachedContent.create() calls to Vertex AI can take 30-40 seconds, blocking the user's request for the entire duration. There is currently no way to set a timeout or other HTTP options on the cache creation call — ADK constructs the CreateCachedContentConfig internally without exposing the http_options field.

Proposed Solution

Add a create_http_options: Optional[types.HttpOptions] field to ContextCacheConfig that gets passed through to the CreateCachedContentConfig when creating a cache. This allows users to control timeout and other HTTP settings on cache creation calls.

When the cache creation call exceeds the configured timeout, it fails gracefully and the request proceeds without caching — the same behavior as if caching were disabled for that turn.

Usage Example

from google.genai import types
from google.adk.agents import ContextCacheConfig

# 10-second timeout on cache creation
cache_config = ContextCacheConfig(
    cache_intervals=3,
    ttl_seconds=3600,
    min_tokens=1024,
    create_http_options=types.HttpOptions(timeout=10000),  # milliseconds
)

Why not use client-level timeout?

The genai.Client accepts http_options at construction time (and ADK 1.26 supports bring-your-own-client), but a client-level timeout applies to all API calls — generate_content, caches.create, caches.delete, etc. Cache creation is uniquely slow (~38s) while other calls are fast, so a per-operation timeout is needed.

Design

  • Zero behavior change when unset: create_http_options=None (default) preserves existing behavior exactly
  • Graceful degradation: Timeout → cache creation fails → request proceeds uncached (existing error handling)
  • No global state: Pure passthrough, no background tasks or in-memory registries
  • Leverages existing SDK support: CreateCachedContentConfig already accepts http_options

Metadata

Metadata

Labels

needs review[Status] The PR/issue is awaiting review from the maintainerservices[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions