[Task]: Add rules to match by type of request, i.e. chat or FIM #944
Assignees
Labels
Comments
aponcedeleonch
added a commit
that referenced
this issue
Feb 6, 2025
Closes: #885 and #944 This PR introduces the necessary changes for: 1. Muxing to match on a specific filename (`main.py`) or file type (`.py`) 2. Muxing on a specific request type, i.e. chat or FIM The matching is tightly coupled with the priority of the muxing rules. A match will occur on the first possible rule. Example: Muxing rule 1 -> match all `.py` files -> go to `chat-gpt` Muxing rule 2 -> match all `.md` files -> go to `ollama` If in a request we recieve a `README.md` and a `main.py` file the request will be routed to `chat-gpt` since it's the match with the highest priority in the list. To introduce above changes there were some minor changes: 1. Separate FIM detection into its own class. Before it used to be part of `BaseProvider` 2. Detect if a request is FIM or not before calling `process_request` method. Both of them were necessary in order to re-use the logic to create the matcher for request type
aponcedeleonch
added a commit
that referenced
this issue
Feb 6, 2025
Closes: #885 and #944 This PR introduces the necessary changes for: 1. Muxing to match on a specific filename (`main.py`) or file type (`.py`) 2. Muxing on a specific request type, i.e. chat or FIM The matching is tightly coupled with the priority of the muxing rules. A match will occur on the first possible rule. Example: Muxing rule 1 -> match all `.py` files -> go to `chat-gpt` Muxing rule 2 -> match all `.md` files -> go to `ollama` If in a request we recieve a `README.md` and a `main.py` file the request will be routed to `chat-gpt` since it's the match with the highest priority in the list. To introduce above changes there were some minor changes: 1. Separate FIM detection into its own class. Before it used to be part of `BaseProvider` 2. Detect if a request is FIM or not before calling `process_request` method. Both of them were necessary in order to re-use the logic to create the matcher for request type
aponcedeleonch
added a commit
that referenced
this issue
Feb 7, 2025
Closes: #885 and #944 This PR introduces the necessary changes for: 1. Muxing to match on a specific filename (`main.py`) or file type (`.py`) 2. Muxing on a specific request type, i.e. chat or FIM The matching is tightly coupled with the priority of the muxing rules. A match will occur on the first possible rule. Example: Muxing rule 1 -> match all `.py` files -> go to `chat-gpt` Muxing rule 2 -> match all `.md` files -> go to `ollama` If in a request we recieve a `README.md` and a `main.py` file the request will be routed to `chat-gpt` since it's the match with the highest priority in the list. To introduce above changes there were some minor changes: 1. Separate FIM detection into its own class. Before it used to be part of `BaseProvider` 2. Detect if a request is FIM or not before calling `process_request` method. Both of them were necessary in order to re-use the logic to create the matcher for request type
aponcedeleonch
added a commit
that referenced
this issue
Feb 7, 2025
Closes: #885 and #944 This PR introduces the necessary changes for: 1. Muxing to match on a specific filename (`main.py`) or file type (`.py`) 2. Muxing on a specific request type, i.e. chat or FIM The matching is tightly coupled with the priority of the muxing rules. A match will occur on the first possible rule. Example: Muxing rule 1 -> match all `.py` files -> go to `chat-gpt` Muxing rule 2 -> match all `.md` files -> go to `ollama` If in a request we recieve a `README.md` and a `main.py` file the request will be routed to `chat-gpt` since it's the match with the highest priority in the list. To introduce above changes there were some minor changes: 1. Separate FIM detection into its own class. Before it used to be part of `BaseProvider` 2. Detect if a request is FIM or not before calling `process_request` method. Both of them were necessary in order to re-use the logic to create the matcher for request type
|
This is done |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
This is a low hanging fruit that can be implemented with our initial set of rules
Additional Context
No response
The text was updated successfully, but these errors were encountered: