[Bug]: The OpenAI LLM retry decorator has a hardcoded value for timeout

[ramarnat]

Bug Description

As discussed in this discord thread

Version

All

Steps to Reproduce

Set a higher timeout value than 60 for openai and max_retries to higher than 0, and it will use the hard coded 60 seconds for the retry

PR Fix is here for review:
#17755

[dosubot]

Hey @ramarnat! I'm here to assist you with any bugs, questions, or contributions you have. I'll help you while we wait for a human maintainer.

The issue with the OpenAI LLM retry decorator using a hardcoded timeout value of 60 seconds is due to the stop_after_delay_seconds parameter in the create_retry_decorator function. This parameter is set to 60 seconds, which means the retry mechanism will stop attempting retries after a total delay of 60 seconds, regardless of any higher timeout value set elsewhere ^[1].

To override this hardcoded value, you can use the create_retry_decorator function and specify a custom max_seconds parameter. Here's an example:

from llama_index.llms.openai.utils import create_retry_decorator

# Create a retry decorator with a custom max_seconds value
retry_decorator = create_retry_decorator(
    max_retries=5,  # specify the number of retries
    random_exponential=True,  # choose the wait strategy
    stop_after_delay_seconds=None,  # optional stop after delay
    min_seconds=1,  # minimum wait time
    max_seconds=120  # custom maximum wait time
)

By setting the max_seconds parameter to your desired value (e.g., 120 seconds), you can override the default 60-second timeout ^[2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}