(Docs) LiteLLM container networking config not working as documented

[jordantgh]

What happened?

I spent a while trying to follow the docs here:

https://www.librechat.ai/blog/2023-11-30_litellm

I expected to be able to chat with models listed in litellm-config.yaml, but was unable to. In the LibreChat error logs, the error was as shown below.

I established that the LiteLLM container was working fine with some curls. On the suggestion of Claude, I changed from the documented baseURL: "host.docker.internal:4000" to baseURL: "http://litellm:8000" in librechat.yaml. That worked! Everything is fine now, seemingly. I even have Langfuse monitoring. However, since I spent a long time on this today, I'm wondering if it is worth updating the docs. Do you think the issue I experienced is because the docs are outdated, or a bespoke issue on my side (or even just that http://litellm:8000 is the safer option anyway)?

Steps to Reproduce

docker compose up in LibreChat repo dir

navigate to LiteLLM endpoint, try to interact with default model

Relevant log output

"level":"error","message":"Failed to fetch models from Monitored API The request was made but no response was received: timeout of 5000ms exceeded. Error Request:\n","request":{"_currentRequest":{"_closed":false,"_contentLength":0,"_defaultKeepAlive":true,"_ended":false,"_events":{},"_eventsCount":7,"_hasBody":true,"_header":"GET /models HTTP/1.1\r\nAccept: application/json, text/plain, */*\r\nAuthorization: Bearer [REDACTED] axios/1.7.7\r\nAccept-Encoding: gzip, compress, deflate, br\r\nHost: host.docker.internal:4000\r\nConnection: keep-alive\r\n\r\n",
... 

Code of Conduct

• I agree to follow this project's Code of Conduct

[rio100]

Jordan, check out this Discord chat:

https://discord.com/channels/1086345563026489514/1335943424850788422

[rio100]

Jordan, check out this Discord chat:

https://discord.com/channels/1086345563026489514/1335943424850788422