Claude Prompt Caching + MIPROv2?

[MattKravetz]

Hi all,

I'm curious if anyone has had success using Claude Prompt Caching (https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching) with MIPROv2?

My use case involves processing (potentially large) press articles with a series of dspy.Modules. I am able to leverage prompt caching in my unoptimized program by instantiating a new dspy.Claude object. Roughly:

for article in articles:
    
    # Creates a dspy.Claude instance, with a system prompt that includes the article text and the required cache control headers
    cached_article_llm = create_claude_model_with_cached_article_text(article_text=article)
    
    # Use this lm with cached context when running my custom dspy modules 
    with dspy.context(lm=cached_article_llm):
        summarizer = ArticleProcessor() 
        result = summarizer()
        ...

I'm now trying to leverage a similar approach with MIPROv2, but I'm hitting issues as it seems the LMs need to be passed into the optimizer constructor (https://github.com/stanfordnlp/dspy/blob/main/dspy/teleprompt/mipro_optimizer_v2.py#L99-L100).
Since I'd like to include multiple different articles in my optimization pass, it's not clear if my unoptimized approach (creating a new lm instance for each distinct article) will work here.

Would anyone happen to know if this is possible?