diff --git a/hub/.doctrees/concepts/unify_api.doctree b/hub/.doctrees/concepts/unify_api.doctree index 3d8630269f..30014fdee7 100644 Binary files a/hub/.doctrees/concepts/unify_api.doctree and b/hub/.doctrees/concepts/unify_api.doctree differ diff --git a/hub/.doctrees/docs/unify/unify/unify.utils.doctree b/hub/.doctrees/docs/unify/unify/unify.utils.doctree index 7e6765862f..717e298aaa 100644 Binary files a/hub/.doctrees/docs/unify/unify/unify.utils.doctree and b/hub/.doctrees/docs/unify/unify/unify.utils.doctree differ diff --git a/hub/.doctrees/environment.pickle b/hub/.doctrees/environment.pickle index 0f07163e60..c52d535fec 100644 Binary files a/hub/.doctrees/environment.pickle and b/hub/.doctrees/environment.pickle differ diff --git a/hub/.doctrees/home/home.doctree b/hub/.doctrees/home/home.doctree index d8cc4329da..a15c6cfd4e 100644 Binary files a/hub/.doctrees/home/home.doctree and b/hub/.doctrees/home/home.doctree differ diff --git a/hub/.doctrees/index.doctree b/hub/.doctrees/index.doctree index 2c59727b74..3267ed8c2c 100644 Binary files a/hub/.doctrees/index.doctree and b/hub/.doctrees/index.doctree differ diff --git a/hub/_sources/concepts/unify_api.rst.txt b/hub/_sources/concepts/unify_api.rst.txt index e88fff6a20..6bf3c4f04c 100644 --- a/hub/_sources/concepts/unify_api.rst.txt +++ b/hub/_sources/concepts/unify_api.rst.txt @@ -10,7 +10,7 @@ Our Universal API provides: - A single, common interface for all models and providers 🟒 - One account, with one balance and one API key πŸ”‘ -To get your universal API key, simply `sign up`_! +To get your universal API key, simply `sign up `_! Querying the API ---------------- @@ -37,9 +37,55 @@ Run the following command in a terminal (replacing :code:`$UNIFY_KEY` with your "messages": [{"role": "user", "content": "Say hello."}] }' -The :code:`model` field is used to specify both the model and the provider, in the format :code:`model@provider`. You can find a list of models and providers either in our `chat `_ interface, or through our `runtime benchmarks `_. +The :code:`model` field is used to specify both the model and the provider, in the format :code:`model@provider`. -Requests can be made from any language, for example using Python: +You can find a list of all models, all providers and all endpoints (model + provider pairs) using the following commands: + +.. code-block:: + + curl -X 'GET' \ + 'https://api.unify.ai/v0/models' \ + -H 'Authorization: Bearer $UNIFY_KEY' \ + -H 'accept: application/json' + +.. code-block:: + + curl -X 'GET' \ + 'https://api.unify.ai/v0/providers' \ + -H 'Authorization: Bearer $UNIFY_KEY' \ + -H 'accept: application/json' + +.. code-block:: + + curl -X 'GET' \ + 'https://api.unify.ai/v0/endpoints' \ + -H 'Authorization: Bearer $UNIFY_KEY' \ + -H 'accept: application/json' + +You can also pass models and providers as arguments to the above functions, to limit the returned list, like so: + +.. code-block:: + + curl -X 'GET' \ + 'https://api.unify.ai/v0/models?provider=' \ + -H 'Authorization: Bearer $UNIFY_KEY' \ + -H 'accept: application/json' + +.. code-block:: + + curl -X 'GET' \ + 'https://api.unify.ai/v0/providers?model=' \ + -H 'Authorization: Bearer $UNIFY_KEY' \ + -H 'accept: application/json' + +.. code-block:: + + curl -X 'GET' \ + 'https://api.unify.ai/v0/endpoints?=' \ + -H 'Authorization: Bearer $UNIFY_KEY' \ + -H 'accept: application/json' + +Requests can easily be made from any language, for example using Python: .. code-block:: python @@ -93,20 +139,24 @@ You can specify all of the parameters that OpenAI supports, but they may not be Unify Python Package ^^^^^^^^^^^^^^^^^^^^ -First, download our `Python package `_ with :code:`pip install unifyai`. - -There is complete documentation in the `readme `_. -Sample inference +First, download our `Python package `_ with :code:`pip install unifyai`. You can then quickly get started like so: .. code-block:: python - from unify import Unify - client = Unify("llama-3-8b-chat@fireworks-ai", api_key="$UNIFY_KEY") - response = client.generate("Say hi.") + import unify + client = unify.Unify("llama-3-8b-chat@fireworks-ai", api_key="$UNIFY_KEY") + response = client.generate("hello world!") + +If you save your API key to the environment variables :code:`UNIFY_KEY`, +then you don't need to specify the :code:`api_key` argument in the example above. + +You can list the models, providers and endpoints using the functions :code:`unify.utils.list_models()`, +:code:`unify.utils.list_providers()` and :code:`unify.utils.list_endpoints()` OpenAI Python Package ^^^^^^^^^^^^^^^^^^^^^ -The Unify API is designed to be compatible with the OpenAI standard, so if you have existing code that uses the OpenAI Python package, it's straightforward to try out our API. +The Unify API is designed to be compatible with the OpenAI standard, so if you have existing code that uses the OpenAI Python package, +it's straightforward to try out our API. .. code-block:: python @@ -129,7 +179,7 @@ Remember that the :code:`model` field needs to contain a string of the form :cod OpenAI NodeJS Package ^^^^^^^^^^^^^^^^^^^^^ -Likewise, if you have existing code that uses the OpenAI NodeJS package, it's straightforward to try out our API. +Likewise, if you have existing code that uses the OpenAI NodeJS package, it's again very straightforward to try out our API. .. code-block:: javascript @@ -152,7 +202,7 @@ You only have to manage the balance and billing details for your Unify account, You can see your balance, top-up your balance, and set automatic refill on the `billing page `_. -You can get your current credit balance with a HTTP request: +You can also get your current credit balance with a HTTP request: .. code-block:: @@ -170,8 +220,16 @@ which will return something like: "credits": 232.32 } +This is also supported in our Python client: + +.. code-block:: python + + import unify + credits = unify.utils.get_credits() + Advanced features ----------------- + Custom endpoints ^^^^^^^^^^^^^^^^^ If you have a custom model which is deployed as an endpoint on (for example a fine-tuned model with OpenAI or Together AI) you can `add your own custom endpoint `_. diff --git a/hub/_sources/home/home.rst.txt b/hub/_sources/home/home.rst.txt index 52786a2ce0..c41b1b2f58 100644 --- a/hub/_sources/home/home.rst.txt +++ b/hub/_sources/home/home.rst.txt @@ -4,9 +4,9 @@ Welcome to Unify! We're on a mission to unify and simplify the LLM landscape. Unify lets you: * **πŸ”‘ Use any LLM from any Provider**: With a single interface, you can use all LLMs from all providers by simply changing one string. No need to manage several API keys or handle different input-output formats. Unify handles all of that for you! -| + * **πŸ“Š Improve LLM Performance**: Add your own custom tests and evals, and benchmark your own prompts on all models and providers. Comparing quality, cost and speed, and iterate on your system prompt until all test cases pass, and you can deploy your app! -| + * **πŸ”€ Route to the Best LLM**: Improve quality, cost and speed by routing to the perfect model and provider for each individual prompt. Quick Start diff --git a/hub/concepts/unify_api.html b/hub/concepts/unify_api.html index 48ff2d0900..c7b3f01298 100644 --- a/hub/concepts/unify_api.html +++ b/hub/concepts/unify_api.html @@ -335,7 +335,7 @@

Universal API`sign up<https://console.unify.ai>`_!

+

To get your universal API key, simply sign up!

Querying the API#

You can interact with the API via:

@@ -358,8 +358,46 @@

HTTP Requests}' -

The model field is used to specify both the model and the provider, in the format model@provider. You can find a list of models and providers either in our chat interface, or through our runtime benchmarks.

-

Requests can be made from any language, for example using Python:

+

The model field is used to specify both the model and the provider, in the format model@provider.

+

You can find a list of all models, all providers and all endpoints (model + provider pairs) using the following commands:

+
curl -X 'GET' \
+  'https://api.unify.ai/v0/models' \
+  -H 'Authorization: Bearer $UNIFY_KEY' \
+  -H 'accept: application/json'
+
+
+
curl -X 'GET' \
+  'https://api.unify.ai/v0/providers' \
+  -H 'Authorization: Bearer $UNIFY_KEY' \
+  -H 'accept: application/json'
+
+
+
curl -X 'GET' \
+  'https://api.unify.ai/v0/endpoints' \
+  -H 'Authorization: Bearer $UNIFY_KEY' \
+  -H 'accept: application/json'
+
+
+

You can also pass models and providers as arguments to the above functions, to limit the returned list, like so:

+
curl -X 'GET' \
+  'https://api.unify.ai/v0/models?provider=<some_provider>' \
+  -H 'Authorization: Bearer $UNIFY_KEY' \
+  -H 'accept: application/json'
+
+
+
curl -X 'GET' \
+  'https://api.unify.ai/v0/providers?model=<some_model>' \
+  -H 'Authorization: Bearer $UNIFY_KEY' \
+  -H 'accept: application/json'
+
+
+
curl -X 'GET' \
+  'https://api.unify.ai/v0/endpoints?<model or provider>=<some_model or some_provider>' \
+  -H 'Authorization: Bearer $UNIFY_KEY' \
+  -H 'accept: application/json'
+
+
+

Requests can easily be made from any language, for example using Python:

import requests
 
 url = "https://api.unify.ai/v0/chat/completions"
@@ -408,18 +446,21 @@ 
HTTP Requests
 
Unify Python Package#

-
First, download our Python package with pip install unifyai.

-
There is complete documentation in the readme.
-Sample inference

-
from unify import Unify
-client = Unify("llama-3-8b-chat@fireworks-ai", api_key="$UNIFY_KEY")
-response = client.generate("Say hi.")
+
First, download our Python package with pip install unifyai. You can then quickly get started like so:

+
import unify
+client = unify.Unify("llama-3-8b-chat@fireworks-ai", api_key="$UNIFY_KEY")
+response = client.generate("hello world!")
 

 

+
If you save your API key to the environment variables UNIFY_KEY,
+then you don’t need to specify the api_key argument in the example above.

+
You can list the models, providers and endpoints using the functions unify.utils.list_models(),
+unify.utils.list_providers() and unify.utils.list_endpoints()

OpenAI Python Package#

-

The Unify API is designed to be compatible with the OpenAI standard, so if you have existing code that uses the OpenAI Python package, it’s straightforward to try out our API.

+

The Unify API is designed to be compatible with the OpenAI standard, so if you have existing code that uses the OpenAI Python package, +it’s straightforward to try out our API.

from openai import OpenAI
 
 client = OpenAI(
@@ -440,7 +481,7 @@ 
OpenAI Python Package
 

 
OpenAI NodeJS Package#

-
Likewise, if you have existing code that uses the OpenAI NodeJS package, it’s straightforward to try out our API.

+
Likewise, if you have existing code that uses the OpenAI NodeJS package, it’s again very straightforward to try out our API.

 
const openai = new OpenAI({
   baseUrl: "https://api.unify.ai/v0/",
   apiKey: "YOUR_UNIFY_KEY"
@@ -460,7 +501,7 @@ 
OpenAI NodeJS PackageBilling#

 
You only have to manage the balance and billing details for your Unify account, and we handle the spending with each provider behind the scenes.

 
You can see your balance, top-up your balance, and set automatic refill on the billing page.

-
You can get your current credit balance with a HTTP request:

+
You can also get your current credit balance with a HTTP request:

 
curl -X 'GET' \
 'https://api.unify.ai/v0/get_credits' \
 -H 'accept: application/json' \
@@ -474,6 +515,11 @@ 
Billing#<
 }
 

 

+
This is also supported in our Python client:

+
import unify
+credits = unify.utils.get_credits()
+

+

 

 

 
Advanced features#

diff --git a/hub/docs/unify/unify/unify.utils.html b/hub/docs/unify/unify/unify.utils.html
index 5f443d3fa8..3db2a38491 100644
--- a/hub/docs/unify/unify/unify.utils.html
+++ b/hub/docs/unify/unify/unify.utils.html
@@ -419,6 +419,26 @@
 
 
 
+

+

+unify.utils.get_credits(api_key=None)#

+
Returns the credits remaining in the user account, in USD.

+

+
Return type:

+
float

+

+

+

+
Args:
api_key (str): If specified, unify API key to be used. Defaults
+to the value in the UNIFY_KEY environment variable.

+

+
Returns:
float: The credits remaining in USD.

+

+
Raises:
ValueError: If there was an HTTP error.

+

+

+

+
 

 

 unify.utils.list_datasets(api_key=None)#

@@ -620,6 +640,7 @@
 
  • delete_evaluation()
    • 
 
  • download_dataset()
    • 
 
  • evaluate()
    • 
+
  • get_credits()
    • 
 
  • list_datasets()
    • 
 
  • list_endpoints()
    • 
 
  • list_evaluations()
    • 
diff --git a/hub/genindex.html b/hub/genindex.html
index 4758bb8705..9f2df3032b 100644
--- a/hub/genindex.html
+++ b/hub/genindex.html
@@ -418,6 +418,8 @@ 
    G
    
         
  • (unify.clients.Unify method)
 
    • 
       
+      
  • get_credits() (in module unify.utils)
+
    • 
   
 
 
diff --git a/hub/home/home.html b/hub/home/home.html
index 161481b0f7..fc32eb499d 100644
--- a/hub/home/home.html
+++ b/hub/home/home.html
@@ -330,17 +330,7 @@ 
    Welcome to Unify!
 
  • πŸ”‘ Use any LLM from any Provider: With a single interface, you can use all LLMs from all providers by simply changing one string. No need to manage several API keys or handle different input-output formats. Unify handles all of that for you!
    • 
-
-
    
-

    
-
    
-
      
 
    • πŸ“Š Improve LLM Performance: Add your own custom tests and evals, and benchmark your own prompts on all models and providers. Comparing quality, cost and speed, and iterate on your system prompt until all test cases pass, and you can deploy your app!
      • 
-
    
-
    
-

    
-
    
-
      
 
    • πŸ”€ Route to the Best LLM: Improve quality, cost and speed by routing to the perfect model and provider for each individual prompt.
      • 
 
    
 
    
diff --git a/hub/index.html b/hub/index.html
index 68dbc68a13..d1c91ab819 100644
--- a/hub/index.html
+++ b/hub/index.html
@@ -305,17 +305,7 @@ 
    Welcome to Unify!
 
  • πŸ”‘ Use any LLM from any Provider: With a single interface, you can use all LLMs from all providers by simply changing one string. No need to manage several API keys or handle different input-output formats. Unify handles all of that for you!
    • 
-
-
    
-

    
-
    
-
      
 
    • πŸ“Š Improve LLM Performance: Add your own custom tests and evals, and benchmark your own prompts on all models and providers. Comparing quality, cost and speed, and iterate on your system prompt until all test cases pass, and you can deploy your app!
      • 
-
    
-
    
-

    
-
    
-
      
 
    • πŸ”€ Route to the Best LLM: Improve quality, cost and speed by routing to the perfect model and provider for each individual prompt.
      • 
 
    
 
    
diff --git a/hub/objects.inv b/hub/objects.inv
index 8d25c4c463..2bd4fd8e47 100644
Binary files a/hub/objects.inv and b/hub/objects.inv differ
diff --git a/hub/searchindex.js b/hub/searchindex.js
index f8ffced5eb..e47140dca9 100644
--- a/hub/searchindex.js
+++ b/hub/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"docnames": ["concepts/benchmarks", "concepts/router", "concepts/unify_api", "console/building_router", "console/connecting_stack", "console/running_benchmarks", "docs/unify/unify", "docs/unify/unify/unify.chat", "docs/unify/unify/unify.clients", "docs/unify/unify/unify.exceptions", "docs/unify/unify/unify.utils", "home/home", "index", "on_prem/on_prem_access", "on_prem/sso"], "filenames": ["concepts/benchmarks.rst", "concepts/router.rst", "concepts/unify_api.rst", "console/building_router.rst", "console/connecting_stack.rst", "console/running_benchmarks.rst", "docs/unify/unify.rst", "docs/unify/unify/unify.chat.rst", "docs/unify/unify/unify.clients.rst", "docs/unify/unify/unify.exceptions.rst", "docs/unify/unify/unify.utils.rst", "home/home.rst", "index.rst", "on_prem/on_prem_access.rst", "on_prem/sso.rst"], "titles": ["Benchmarking", "Routing", "Universal API", "Building a custom router", "Connecting your stack", "Benchmarking endpoints", "<no title>", "Chat", "Clients", "Exceptions", "Utils", "Welcome to Unify!", "Home", "On-Prem access", "SSO (Single Sign On)"], "terms": {"when": [0, 1, 2, 3, 8], "compar": [0, 5, 11, 12], "llm": [0, 4, 5, 7, 11, 12], "i": [0, 2, 3, 4, 5, 7, 8, 10, 14], "constant": 0, "tradeoff": 0, "make": [0, 3, 8], "between": [0, 1, 2, 3, 8], "qualiti": [0, 1, 3, 11, 12], "cost": [0, 1, 2, 3, 5, 11, 12], "latenc": [0, 1, 3, 5], "stronger": 0, "model": [0, 1, 2, 3, 4, 5, 7, 8, 10, 11, 12, 14], "ar": [0, 1, 3, 4, 5], "gener": [0, 1, 2, 8, 11, 12, 14], "slower": 0, "more": [0, 1, 5, 8], "expens": 0, "sometim": [0, 2], "overkil": 0, "task": [0, 1, 3, 5], "hand": [0, 5], "complic": 0, "matter": 0, "further": [0, 3, 8], "new": [0, 1, 2, 3, 4, 5], "releas": [0, 1], "weekli": 0, "each": [0, 1, 2, 3, 5, 10, 11, 12], "claim": 0, "state": 0, "art": 0, "data": [0, 3], "let": [0, 3, 4, 5, 11, 12], "you": [0, 1, 2, 3, 4, 5, 11, 12, 14], "see": [0, 2, 3, 5], "differ": [0, 1, 2, 3, 5, 11, 12], "perform": [0, 3, 5, 11, 12], "can": [0, 1, 2, 3, 4, 5, 11, 12, 14], "relat": 0, "live": [0, 1, 2], "stat": 0, "pull": [0, 14], "from": [0, 1, 2, 3, 5, 7, 8, 10, 11, 12, 14], "our": [0, 2, 3, 4, 5, 11, 12, 13, 14], "runtim": [0, 1, 2], "come": [0, 1, 2], "out": [0, 1, 2], "simpli": [0, 2, 5, 11, 12], "re": [0, 3, 5, 11, 12, 14], "run": [0, 2, 3, 4, 7, 14], "thei": [0, 1, 2], "first": [0, 1, 2, 3, 5], "creat": [0, 2, 3, 4], "which": [0, 1, 2, 3, 5, 11, 12, 14], "repres": [0, 7], "want": [0, 3, 4, 5], "evalu": [0, 5, 10], "need": [0, 1, 2, 3, 4, 5, 11, 12], "list": [0, 2, 3, 5, 8, 10, 11, 12], "prompt": [0, 1, 2, 4, 5, 8, 10, 11, 12], "option": [0, 1, 3, 4, 7, 8, 10], "includ": [0, 3, 4, 5], "refer": [0, 1, 4], "gold": 0, "standard": [0, 2, 4], "answer": [0, 1, 4], "contain": [0, 1, 2, 4, 5, 8, 10], "tend": [0, 5], "get": [0, 1, 2, 7, 8, 10, 11, 12, 13], "accur": 0, "The": [0, 1, 2, 3, 4, 5, 7, 8, 14], "file": [0, 1, 4, 10, 14], "itself": 0, "should": [0, 2, 4, 5, 14], "jsonl": [0, 1, 4], "format": [0, 1, 2, 7, 8, 11, 12], "one": [0, 1, 2, 4, 5, 11, 12], "entri": [0, 1], "per": [0, 3], "line": 0, "exampl": [0, 1, 2, 3, 4], "below": [0, 3, 4, 5], "thi": [0, 1, 2, 3, 4, 5, 10, 14], "ref_answ": [0, 1], "second": [0, 3], "us": [0, 1, 2, 3, 4, 5, 7, 8, 10, 11, 12, 14], "least": [0, 5], "50": 0, "most": [0, 1, 4], "result": [0, 1, 3, 4, 5], "current": [0, 2, 3, 5], "an": [0, 1, 2, 3, 4, 7, 8, 10], "maximum": 0, "limit": [0, 1, 2], "500": [0, 9], "we": [0, 1, 2, 3, 4, 5, 11, 12], "don": [0, 3], "t": [0, 1, 3, 5], "much": [0, 1], "extra": 0, "detail": [0, 2, 3, 14], "past": 0, "250": 0, "In": [0, 1, 2, 3, 4, 5], "dashboard": [0, 1, 3, 5], "click": [0, 1, 3, 4, 5], "select": [0, 1, 3, 5], "open": [0, 3, 5], "interfac": [0, 1, 2, 4, 5, 11, 12], "upload": [0, 1, 3, 4, 5, 10], "finish": [0, 1], "ll": [0, 1, 3, 4, 5], "receiv": [0, 1, 3], "email": [0, 1, 3], "graph": [0, 1, 3, 5], "displai": [0, 1, 3, 5], "x": [0, 2, 3, 5, 14], "axi": [0, 5], "set": [0, 2, 3, 4, 7, 8], "time": [0, 1, 2, 3, 5], "token": [0, 1, 2, 3, 5, 8], "inter": [0, 1, 3], "either": [0, 2, 3, 10], "linear": 0, "log": 0, "scale": 0, "gpt4o": 0, "judg": 0, "cf": 0, "http": [0, 8, 9, 10, 14], "arxiv": 0, "org": 0, "ab": 0, "2306": 0, "05685": 0, "": [0, 1, 2, 3, 4, 5, 8, 11, 12, 14], "respons": [0, 1, 2, 8, 10, 14], "learn": [1, 3, 4, 5], "how": [1, 3, 4, 5], "allow": [1, 3, 5], "flexibl": 1, "about": [1, 5], "handl": [1, 2, 11, 12], "advantag": 1, "sever": [1, 11, 12], "reason": [1, 2], "small": 1, "faster": 1, "cheaper": 1, "wherea": 1, "bigger": 1, "capabl": 1, "often": 1, "rang": 1, "difficulti": 1, "provid": [1, 2, 3, 4, 5, 7, 8, 10, 11, 12], "have": [1, 2, 3, 4, 5, 11, 12, 14], "chang": [1, 3, 5, 11, 12], "over": [1, 5], "everi": [1, 2], "week": 1, "strength": 1, "weak": 1, "smaller": 1, "continu": 1, "improv": [1, 11, 12], "ride": 1, "wave": 1, "abil": 1, "minimis": 1, "base": [1, 3, 7, 8, 9], "statist": 1, "purpos": 1, "wide": 1, "varieti": [1, 4], "To": [1, 2, 5, 13], "queri": [1, 3, 4], "green": 1, "star": [1, 5], "chat": [1, 2, 3, 14], "copi": [1, 3], "string": [1, 2, 8, 11, 12], "For": [1, 3, 4, 5], "q": 1, "1": [1, 8], "c": 1, "4": [1, 3, 8], "65e": 1, "03": 1, "2": [1, 8], "08e": 1, "05": 1, "07e": 1, "paramet": [1, 2], "stand": 1, "rel": 1, "weigh": 1, "complet": [1, 2, 3, 5, 14], "customiz": 1, "meaning": 1, "configur": [1, 3, 5], "directli": 1, "restrict": 1, "specifi": [1, 2, 3, 4, 5, 10], "them": [1, 3, 5], "follow": [1, 2], "model1": 1, "model2": 1, "model3": 1, "provider1": 1, "provider2": 1, "specif": [1, 5, 10], "rate": 1, "hit": 1, "move": [1, 3], "next": [1, 2, 3, 4, 5], "best": [1, 3, 11, 12], "mean": [1, 14], "higher": [1, 8], "than": [1, 5], "avail": [1, 4, 10, 11, 12], "singl": [1, 2, 8, 11, 12], "ve": [1, 4, 5], "found": [1, 9], "similar": 1, "those": [1, 4], "go": [1, 2, 3, 5, 11, 12], "product": [1, 2], "visualis": 1, "your": [1, 2, 8, 11, 12, 14], "same": [1, 3], "benchmark": [1, 2, 3, 4, 11, 12], "e": [1, 3, 5], "form": [1, 2], "xxx": 1, "yyi": 1, "accuraci": 1, "final": [1, 3], "system": [1, 2, 8, 11, 12], "after": [1, 3], "onc": [1, 3, 4, 5, 14], "here": [1, 3], "abl": 1, "choos": [1, 3], "api": [1, 3, 4, 7, 8, 10, 11, 12, 14], "There": [2, 4, 5], "ever": 2, "expand": [2, 5], "ocean": 2, "all": [2, 3, 4, 5, 9, 10, 11, 12, 14], "requir": [2, 3], "uniqu": 2, "kei": [2, 4, 7, 8, 10, 11, 12], "payment": 2, "process": 2, "test": [2, 3, 5, 11, 12], "quickli": 2, "becom": 2, "veri": [2, 3], "cumbersom": 2, "manag": [2, 11, 12], "nuanc": 2, "A": [2, 3, 8, 10], "common": [2, 4], "One": 2, "account": [2, 3, 5, 8], "balanc": [2, 3], "sign": [2, 11, 12], "up": [2, 8, 11, 12], "consol": [2, 3, 4, 5], "ai": [2, 4], "_": 2, "interact": [2, 7, 8], "via": [2, 4], "ani": [2, 3, 5, 11, 12], "program": 2, "languag": 2, "own": [2, 4, 11, 12], "command": 2, "termin": 2, "replac": 2, "unify_kei": [2, 7, 8, 10, 11, 12], "curl": [2, 14], "post": [2, 14], "v0": [2, 5], "h": [2, 14], "author": 2, "bearer": 2, "content": [2, 8, 10, 14], "type": [2, 7, 8, 10, 14], "applic": [2, 3, 9, 14], "json": [2, 8, 10, 14], "d": [2, 3, 14], "llama": [2, 3, 5], "3": [2, 3, 5, 14], "8b": 2, "firework": 2, "messag": [2, 8, 14], "role": [2, 14], "user": [2, 3, 8, 14], "sai": 2, "hello": [2, 11, 12], "field": 2, "both": [2, 3], "find": [2, 5], "through": [2, 3, 4, 5], "made": 2, "import": [2, 11, 12, 14], "url": 2, "header": 2, "unify_api_kei": 2, "payload": 2, "togeth": 2, "print": 2, "text": 2, "look": [2, 3, 14], "someth": [2, 5], "like": [2, 3, 4, 5, 8, 11, 12, 14], "1718888877": 2, "id": [2, 3], "896bfc1ae84271aa": 2, "lhr": 2, "object": [2, 7, 8], "usag": 2, "completion_token": 2, "25": 2, "prompt_token": 2, "13": 2, "total_token": 2, "38": 2, "7": 2, "6e": 2, "06": [2, 3], "choic": [2, 11, 12], "finish_reason": 2, "stop": [2, 8], "index": 2, "0": [2, 3, 8], "It": [2, 11, 12], "nice": 2, "meet": [2, 3], "help": 2, "would": [2, 3, 4, 5], "assist": 2, "seed": 2, "11563975138181362140": 2, "output": [2, 5, 8, 11, 12], "access": [2, 3, 5, 7, 8, 14], "number": [2, 8], "well": [2, 3, 4], "total": [2, 10], "u": [2, 5, 13], "other": [2, 3, 5], "temperatur": [2, 8], "stream": [2, 8], "max_token": [2, 8], "support": [2, 10], "mai": [2, 3], "compat": 2, "download": [2, 10], "pip": [2, 11, 12], "instal": [2, 11, 12], "unifyai": [2, 11, 12, 14], "document": 2, "readm": 2, "sampl": [2, 8], "infer": 2, "client": [2, 7, 11, 12, 14], "api_kei": [2, 7, 8, 10, 11, 12, 14], "hi": [2, 14], "design": [2, 5], "so": [2, 3, 4, 5, 11, 12], "exist": 2, "code": 2, "straightforward": 2, "try": 2, "base_url": [2, 14], "your_unify_kei": 2, "mistral": [2, 5], "7b": [2, 5], "instruct": [2, 5], "true": [2, 8], "chunk": [2, 8], "delta": 2, "end": 2, "rememb": 2, "likewis": 2, "const": 2, "baseurl": 2, "apikei": 2, "await": 2, "again": [2, 4], "onli": [2, 3, 5], "spend": 2, "behind": [2, 5], "scene": 2, "top": [2, 5], "automat": [2, 3], "refil": 2, "page": [2, 3, 5], "credit": [2, 7, 8], "get_credit": 2, "accept": [2, 14], "your_api_kei": 2, "return": [2, 7, 8, 10], "your_user_id": 2, "232": 2, "32": 2, "If": [2, 3, 4, 5, 7, 8, 10], "deploi": [2, 3, 5, 11, 12], "fine": [2, 3], "tune": [2, 3, 5], "add": [2, 3, 4, 5, 11, 12], "relev": 2, "name": [2, 3, 4, 5, 7, 8, 10], "individu": [2, 11, 12], "outag": 2, "disrupt": 2, "workflow": 2, "combat": 2, "down": 2, "fail": 2, "some": [2, 3, 5], "until": [2, 11, 12], "succe": 2, "reach": 2, "tag": 2, "model_a": 2, "provider_a": 2, "model_b": 2, "provider_b": 2, "actual": 2, "went": 2, "section": [3, 4, 5], "window": 3, "enabl": 3, "endpoint": [3, 7, 8, 10, 11, 12], "rout": [3, 4, 11, 12], "dataset": [3, 5, 10], "gpt": [3, 11, 12], "claud": [3, 14], "llama3": 3, "call": [3, 13], "intend": 3, "earlier": [3, 5], "variant": 3, "correspond": [3, 5], "dropdown": [3, 5], "notic": 3, "becaus": 3, "doe": 3, "depend": 3, "button": [3, 4], "submit": [3, 5], "job": [3, 5], "ha": 3, "schedul": 3, "behalf": [3, 4], "alreadi": [3, 4], "been": 3, "ahead": 3, "work": 3, "duplic": 3, "previous": 3, "calls1": [3, 5], "call2": 3, "q1": 3, "q2": 3, "repeat": 3, "howev": 3, "yet": 3, "70b": 3, "g": [3, 5], "trigger": [3, 5], "request": [3, 5, 9, 14], "readi": [3, 4, 11, 12], "manual": 3, "track": [3, 5], "progress": [3, 5], "With": [3, 11, 12], "done": [3, 5], "stage": 3, "accordingli": 3, "now": [3, 4, 5, 11, 12, 14], "visual": [3, 5], "step": [3, 4, 14], "explor": [3, 5], "variou": [3, 5], "possibl": 3, "trade": 3, "off": [3, 4], "speed": [3, 5, 11, 12], "variat": 3, "These": 3, "As": [3, 5], "befor": [3, 5], "point": [3, 5], "plot": [3, 5], "view": [3, 5], "take": 3, "constrain": 3, "search": 3, "space": 3, "wai": [3, 4], "certain": 3, "deploy": 3, "environ": [3, 7, 8, 10, 11, 12], "ensur": [3, 5], "guarante": 3, "Of": 3, "cours": 3, "visibl": 3, "remov": [3, 5, 10], "presum": 3, "anthrop": [3, 14], "properli": 3, "save": [3, 10, 11, 12], "moment": 3, "therefor": 3, "appli": 3, "altern": [3, 4], "had": 3, "overwritten": 3, "clauc": 3, "place": 3, "legend": [3, 5], "underneath": 3, "its": 3, "parent": 3, "slightli": 3, "reduc": 3, "amount": 3, "assum": [3, 4], "decid": 3, "stick": 3, "decis": 3, "avoid": [3, 11, 12], "immedi": 3, "futur": 3, "appear": [3, 5], "full": 3, "frequenc": 3, "gpt4": 3, "42": 3, "29": 3, "calls2": [3, 5], "18": 3, "11": 3, "across": [3, 4, 5], "metric": [3, 5], "ax": 3, "verifi": 3, "session": 3, "perman": 3, "happi": [3, 5], "forget": 3, "where": [3, 5, 8], "inform": [3, 5], "also": [3, 4, 5, 11, 12], "delet": [3, 10], "anytim": 3, "press": [3, 5], "besid": 3, "clipboard": 3, "case": [3, 5, 11, 12], "anthropic_8": 3, "28e": 3, "03_4": 3, "66e": 3, "4_1": 3, "00e": 3, "unifi": [3, 4, 7, 8, 9, 10, 14], "show": [3, 4, 5, 7], "along": 3, "default": [3, 5, 7, 8, 10], "That": [3, 4, 5, 11, 12], "shelf": 4, "azur": 4, "ml": 4, "catalog": 4, "vertex": 4, "garden": 4, "aw": 4, "bedrock": 4, "host": 4, "whole": 4, "do": 4, "major": 4, "cloud": 4, "regardless": 4, "expos": [4, 5], "adher": 4, "openai": [4, 7, 8, 11, 12], "integr": 4, "got": 4, "said": 4, "screen": 4, "local": [4, 10, 14], "build": 4, "label": [4, 5], "preview": 4, "imag": [4, 5], "know": [4, 5], "router": [4, 5], "navig": 5, "custom": [5, 9, 11, 12], "ones": 5, "methodologi": 5, "head": 5, "By": 5, "six": 5, "openherm": 5, "shown": 5, "On": 5, "against": 5, "scatter": 5, "replot": 5, "similarili": 5, "els": 5, "score": 5, "ttft": 5, "updat": 5, "remain": [5, 8], "averag": 5, "explain": [5, 14], "granular": 5, "outsid": 5, "popular": 5, "period": 5, "suit": 5, "dedic": 5, "valu": [5, 7, 8, 10], "tabl": 5, "region": 5, "sequenc": [5, 8], "length": 5, "latest": [5, 14], "sort": 5, "icon": 5, "column": 5, "reveal": 5, "pattern": 5, "sec": 5, "figur": 5, "fit": 5, "haven": 5, "recommend": 5, "resum": 5, "ad": 5, "previou": 5, "privat": 5, "server": [5, 9], "vari": 5, "alwai": 5, "send": 5, "fastest": 5, "under": 5, "ip": 5, "address": 5, "lock": 5, "indic": 5, "order": 5, "train": 5, "just": 5, "necessari": 5, "subsect": 5, "separ": 5, "pair": 5, "check": 5, "entir": [5, 14], "histori": [5, 7, 8], "mixtral": 5, "financ": 5, "alongsid": 5, "foundat": 5, "class": [7, 8, 9], "chatbot": 7, "none": [7, 8, 10], "agent": 7, "__init__": [7, 8], "initi": [7, 8], "arg": [7, 8, 10], "str": [7, 8, 10], "uploaded_bi": [7, 8], "model_nam": [7, 8], "provider_nam": [7, 8], "must": [7, 8, 10], "attempt": [7, 8], "retriev": [7, 8], "variabl": [7, 8, 10, 11, 12], "rais": [7, 8, 9, 10], "unifyerror": [7, 8, 9], "miss": [7, 8], "clear_chat_histori": 7, "clear": 7, "properti": [7, 8], "dar201": [7, 8], "noqa": [7, 8], "show_credit": 7, "fals": [7, 8], "show_provid": 7, "start": 7, "loop": 7, "bool": [7, 8], "whether": 7, "consumpt": 7, "set_client": 7, "dar101": [7, 8], "set_endpoint": [7, 8], "set_model": [7, 8], "set_provid": [7, 8], "asyncunifi": 8, "asynchron": 8, "async": 8, "user_prompt": 8, "system_prompt": 8, "union": 8, "asyncgener": 8, "dict": [8, 10], "dictionari": [8, 10], "convers": 8, "int": 8, "max": 8, "float": 8, "what": [8, 14], "8": 8, "random": [8, 11, 12], "while": 8, "lower": 8, "focus": 8, "determinist": 8, "yield": 8, "error": [8, 9, 10], "occur": 8, "dure": 8, "get_credit_bal": 8, "left": 8, "success": [8, 10], "otherwis": [8, 10], "badrequesterror": [8, 9, 10], "wa": [8, 10, 14], "valueerror": [8, 10], "pars": [8, 10], "1024": 8, "authenticationerror": 9, "401": 9, "unauthor": 9, "400": 9, "bad": 9, "conflicterror": 9, "409": 9, "conflict": 9, "internalservererror": 9, "intern": 9, "notfounderror": 9, "404": 9, "Not": 9, "permissiondeniederror": 9, "403": 9, "forbidden": 9, "ratelimiterror": 9, "429": 9, "too": 9, "mani": 9, "unprocessableentityerror": 9, "422": 9, "unprocess": 9, "entiti": 9, "delete_dataset": 10, "platform": 10, "given": 10, "path": 10, "info": 10, "msg": 10, "delete_evalu": 10, "whose": 10, "download_dataset": 10, "list_dataset": 10, "fetch": 10, "list_endpoint": [10, 11, 12], "empti": 10, "list_evalu": 10, "list_model": 10, "list_provid": 10, "associ": 10, "upload_dataset_from_dictionari": 10, "upload_dataset_from_fil": 10, "mission": [11, 12], "simplifi": [11, 12], "landscap": [11, 12], "No": [11, 12], "input": [11, 12], "eval": [11, 12], "iter": [11, 12], "pass": [11, 12], "app": [11, 12], "perfect": [11, 12], "easiest": [11, 12], "python": [11, 12, 14], "packag": [11, 12], "Then": [11, 12, 14], "4o": [11, 12], "your_kei": [11, 12], "world": [11, 12], "argument": [11, 12], "util": [11, 12], "fingertip": [11, 12], "offer": 13, "book": 13, "prem": 14, "microservic": 14, "grant": 14, "docker": 14, "dt": 14, "p": 14, "8000": 14, "env": 14, "env_file_path": 14, "deepinfra_api_kei": 14, "stability_api_kei": 14, "mistral_ai_api_kei": 14, "perplexity_ai_api_kei": 14, "anyscale_api_kei": 14, "octoai_api_kei": 14, "openai_api_kei": 14, "together_ai_api_kei": 14, "replicate_api_kei": 14, "lepton_ai_api_kei": 14, "fireworks_ai_api_kei": 14, "anthropic_api_kei": 14, "vertexai_service_acc_json": 14, "vertexai_gcloud_path": 14, "vertexai_project": 14, "vertexai_loc": 14, "servic": 14, "weather": 14, "san": 14, "francisco": 14, "tokyo": 14, "pari": 14, "localhost": 14, "haiku": 14, "random_key_wont_be_us": 14, "life": 14, "who": 14, "newton": 14, "theori": 14, "gravit": 14, "give": 14, "long": 14, "pleas": 14, "achiev": 14}, "objects": {"unify": [[7, 0, 0, "-", "chat"], [8, 0, 0, "-", "clients"], [9, 0, 0, "-", "exceptions"], [10, 0, 0, "-", "utils"]], "unify.chat": [[7, 1, 1, "", "ChatBot"]], "unify.chat.ChatBot": [[7, 2, 1, "", "__init__"], [7, 2, 1, "", "clear_chat_history"], [7, 3, 1, "", "client"], [7, 3, 1, "", "endpoint"], [7, 3, 1, "", "model"], [7, 3, 1, "", "provider"], [7, 2, 1, "", "run"], [7, 2, 1, "", "set_client"], [7, 2, 1, "", "set_endpoint"], [7, 2, 1, "", "set_model"], [7, 2, 1, "", "set_provider"]], "unify.clients": [[8, 1, 1, "", "AsyncUnify"], [8, 1, 1, "", "Unify"]], "unify.clients.AsyncUnify": [[8, 2, 1, "", "__init__"], [8, 3, 1, "", "endpoint"], [8, 2, 1, "", "generate"], [8, 2, 1, "", "get_credit_balance"], [8, 3, 1, "", "model"], [8, 3, 1, "", "provider"], [8, 2, 1, "", "set_endpoint"], [8, 2, 1, "", "set_model"], [8, 2, 1, "", "set_provider"]], "unify.clients.Unify": [[8, 2, 1, "", "__init__"], [8, 3, 1, "", "endpoint"], [8, 2, 1, "", "generate"], [8, 2, 1, "", "get_credit_balance"], [8, 3, 1, "", "model"], [8, 3, 1, "", "provider"], [8, 2, 1, "", "set_endpoint"], [8, 2, 1, "", "set_model"], [8, 2, 1, "", "set_provider"]], "unify.exceptions": [[9, 4, 1, "", "AuthenticationError"], [9, 4, 1, "", "BadRequestError"], [9, 4, 1, "", "ConflictError"], [9, 4, 1, "", "InternalServerError"], [9, 4, 1, "", "NotFoundError"], [9, 4, 1, "", "PermissionDeniedError"], [9, 4, 1, "", "RateLimitError"], [9, 4, 1, "", "UnifyError"], [9, 4, 1, "", "UnprocessableEntityError"]], "unify.utils": [[10, 5, 1, "", "delete_dataset"], [10, 5, 1, "", "delete_evaluation"], [10, 5, 1, "", "download_dataset"], [10, 5, 1, "", "evaluate"], [10, 5, 1, "", "list_datasets"], [10, 5, 1, "", "list_endpoints"], [10, 5, 1, "", "list_evaluations"], [10, 5, 1, "", "list_models"], [10, 5, 1, "", "list_providers"], [10, 5, 1, "", "upload_dataset_from_dictionary"], [10, 5, 1, "", "upload_dataset_from_file"]]}, "objtypes": {"0": "py:module", "1": "py:class", "2": "py:method", "3": "py:property", "4": "py:exception", "5": "py:function"}, "objnames": {"0": ["py", "module", "Python module"], "1": ["py", "class", "Python class"], "2": ["py", "method", "Python method"], "3": ["py", "property", "Python property"], "4": ["py", "exception", "Python exception"], "5": ["py", "function", "Python function"]}, "titleterms": {"benchmark": [0, 5], "prepar": [0, 1], "your": [0, 3, 4, 5], "dataset": [0, 1, 4], "how": 0, "doe": 0, "work": 0, "rout": 1, "what": 1, "i": 1, "llm": [1, 2], "foundat": 1, "router": [1, 3], "filter": 1, "endpoint": [1, 2, 4, 5], "maximis": 1, "throughput": 1, "fine": 1, "tune": 1, "custom": [1, 2, 3, 4], "train": [1, 3], "beta": [1, 3, 4, 5], "univers": 2, "api": 2, "queri": 2, "http": 2, "request": 2, "unifi": [2, 11, 12], "python": 2, "packag": 2, "openai": 2, "nodej": 2, "bill": 2, "advanc": 2, "featur": 2, "fallback": 2, "build": 3, "round": [3, 4, 5], "up": [3, 4, 5], "connect": 4, "stack": 4, "prerequisit": 4, "ad": 4, "qualiti": 5, "runtim": 5, "run": 5, "own": 5, "chat": 7, "client": 8, "except": 9, "util": 10, "welcom": [11, 12], "quick": [11, 12], "start": [11, 12], "On": [13, 14], "prem": 13, "access": 13, "sso": 14, "singl": 14, "sign": 14}, "envversion": {"sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "nbsphinx": 4, "sphinx": 60}, "alltitles": {"Routing": [[1, "routing"]], "What is LLM routing?": [[1, "what-is-llm-routing"]], "Foundation router": [[1, "foundation-router"]], "Filtering endpoints": [[1, "filtering-endpoints"]], "Maximising throughput": [[1, "maximising-throughput"]], "Fine-tuned custom router": [[1, "fine-tuned-custom-router"]], "Preparing a dataset": [[1, "preparing-a-dataset"]], "Training a router (beta)": [[1, "training-a-router-beta"]], "Universal API": [[2, "universal-api"]], "Querying the API": [[2, "querying-the-api"]], "HTTP Requests": [[2, "http-requests"]], "Unify Python Package": [[2, "unify-python-package"]], "OpenAI Python Package": [[2, "openai-python-package"]], "OpenAI NodeJS Package": [[2, "openai-nodejs-package"]], "Billing": [[2, "billing"]], "Advanced features": [[2, "advanced-features"]], "Custom endpoints": [[2, "custom-endpoints"]], "LLM Fallbacks": [[2, "llm-fallbacks"]], "Benchmarking": [[0, "benchmarking"]], "Preparing your dataset": [[0, "preparing-your-dataset"]], "Benchmarking your dataset": [[0, "benchmarking-your-dataset"]], "How does it work?": [[0, "how-does-it-work"]], "Building a custom router": [[3, "building-a-custom-router"]], "Training a custom router (Beta)": [[3, "training-a-custom-router-beta"]], "Customizing your router (Beta)": [[3, "customizing-your-router-beta"]], "Round Up": [[3, "round-up"], [5, "round-up"], [4, "round-up"]], "Benchmarking endpoints": [[5, "benchmarking-endpoints"]], "Quality benchmarks": [[5, "quality-benchmarks"], [5, "id4"]], "Runtime benchmarks": [[5, "runtime-benchmarks"], [5, "id2"]], "Running your own benchmarks (Beta)": [[5, "running-your-own-benchmarks-beta"]], "Connecting your stack": [[4, "connecting-your-stack"]], "Custom endpoints (Beta)": [[4, "custom-endpoints-beta"]], "Prerequisite": [[4, "prerequisite"]], "Adding the endpoints": [[4, "adding-the-endpoints"]], "Custom datasets": [[4, "custom-datasets"]], "SSO (Single Sign On)": [[14, "sso-single-sign-on"]], "Welcome to Unify!": [[12, "welcome-to-unify"], [11, "welcome-to-unify"]], "Quick Start": [[12, "quick-start"], [11, "quick-start"]], "On-Prem access": [[13, "on-prem-access"]], "Exceptions": [[9, "module-unify.exceptions"]], "Utils": [[10, "module-unify.utils"]], "Chat": [[7, "module-unify.chat"]], "Clients": [[8, "module-unify.clients"]]}, "indexentries": {"chatbot (class in unify.chat)": [[7, "unify.chat.ChatBot"]], "__init__() (unify.chat.chatbot method)": [[7, "unify.chat.ChatBot.__init__"]], "clear_chat_history() (unify.chat.chatbot method)": [[7, "unify.chat.ChatBot.clear_chat_history"]], "client (unify.chat.chatbot property)": [[7, "unify.chat.ChatBot.client"]], "endpoint (unify.chat.chatbot property)": [[7, "unify.chat.ChatBot.endpoint"]], "model (unify.chat.chatbot property)": [[7, "unify.chat.ChatBot.model"]], "module": [[7, "module-unify.chat"], [8, "module-unify.clients"], [9, "module-unify.exceptions"], [10, "module-unify.utils"]], "provider (unify.chat.chatbot property)": [[7, "unify.chat.ChatBot.provider"]], "run() (unify.chat.chatbot method)": [[7, "unify.chat.ChatBot.run"]], "set_client() (unify.chat.chatbot method)": [[7, "unify.chat.ChatBot.set_client"]], "set_endpoint() (unify.chat.chatbot method)": [[7, "unify.chat.ChatBot.set_endpoint"]], "set_model() (unify.chat.chatbot method)": [[7, "unify.chat.ChatBot.set_model"]], "set_provider() (unify.chat.chatbot method)": [[7, "unify.chat.ChatBot.set_provider"]], "unify.chat": [[7, "module-unify.chat"]], "asyncunify (class in unify.clients)": [[8, "unify.clients.AsyncUnify"]], "unify (class in unify.clients)": [[8, "unify.clients.Unify"]], "__init__() (unify.clients.asyncunify method)": [[8, "unify.clients.AsyncUnify.__init__"]], "__init__() (unify.clients.unify method)": [[8, "unify.clients.Unify.__init__"]], "endpoint (unify.clients.asyncunify property)": [[8, "unify.clients.AsyncUnify.endpoint"]], "endpoint (unify.clients.unify property)": [[8, "unify.clients.Unify.endpoint"]], "generate() (unify.clients.asyncunify method)": [[8, "unify.clients.AsyncUnify.generate"]], "generate() (unify.clients.unify method)": [[8, "unify.clients.Unify.generate"]], "get_credit_balance() (unify.clients.asyncunify method)": [[8, "unify.clients.AsyncUnify.get_credit_balance"]], "get_credit_balance() (unify.clients.unify method)": [[8, "unify.clients.Unify.get_credit_balance"]], "model (unify.clients.asyncunify property)": [[8, "unify.clients.AsyncUnify.model"]], "model (unify.clients.unify property)": [[8, "unify.clients.Unify.model"]], "provider (unify.clients.asyncunify property)": [[8, "unify.clients.AsyncUnify.provider"]], "provider (unify.clients.unify property)": [[8, "unify.clients.Unify.provider"]], "set_endpoint() (unify.clients.asyncunify method)": [[8, "unify.clients.AsyncUnify.set_endpoint"]], "set_endpoint() (unify.clients.unify method)": [[8, "unify.clients.Unify.set_endpoint"]], "set_model() (unify.clients.asyncunify method)": [[8, "unify.clients.AsyncUnify.set_model"]], "set_model() (unify.clients.unify method)": [[8, "unify.clients.Unify.set_model"]], "set_provider() (unify.clients.asyncunify method)": [[8, "unify.clients.AsyncUnify.set_provider"]], "set_provider() (unify.clients.unify method)": [[8, "unify.clients.Unify.set_provider"]], "unify.clients": [[8, "module-unify.clients"]], "authenticationerror": [[9, "unify.exceptions.AuthenticationError"]], "badrequesterror": [[9, "unify.exceptions.BadRequestError"]], "conflicterror": [[9, "unify.exceptions.ConflictError"]], "internalservererror": [[9, "unify.exceptions.InternalServerError"]], "notfounderror": [[9, "unify.exceptions.NotFoundError"]], "permissiondeniederror": [[9, "unify.exceptions.PermissionDeniedError"]], "ratelimiterror": [[9, "unify.exceptions.RateLimitError"]], "unifyerror": [[9, "unify.exceptions.UnifyError"]], "unprocessableentityerror": [[9, "unify.exceptions.UnprocessableEntityError"]], "unify.exceptions": [[9, "module-unify.exceptions"]], "delete_dataset() (in module unify.utils)": [[10, "unify.utils.delete_dataset"]], "delete_evaluation() (in module unify.utils)": [[10, "unify.utils.delete_evaluation"]], "download_dataset() (in module unify.utils)": [[10, "unify.utils.download_dataset"]], "evaluate() (in module unify.utils)": [[10, "unify.utils.evaluate"]], "list_datasets() (in module unify.utils)": [[10, "unify.utils.list_datasets"]], "list_endpoints() (in module unify.utils)": [[10, "unify.utils.list_endpoints"]], "list_evaluations() (in module unify.utils)": [[10, "unify.utils.list_evaluations"]], "list_models() (in module unify.utils)": [[10, "unify.utils.list_models"]], "list_providers() (in module unify.utils)": [[10, "unify.utils.list_providers"]], "unify.utils": [[10, "module-unify.utils"]], "upload_dataset_from_dictionary() (in module unify.utils)": [[10, "unify.utils.upload_dataset_from_dictionary"]], "upload_dataset_from_file() (in module unify.utils)": [[10, "unify.utils.upload_dataset_from_file"]]}})
\ No newline at end of file
+Search.setIndex({"docnames": ["concepts/benchmarks", "concepts/router", "concepts/unify_api", "console/building_router", "console/connecting_stack", "console/running_benchmarks", "docs/unify/unify", "docs/unify/unify/unify.chat", "docs/unify/unify/unify.clients", "docs/unify/unify/unify.exceptions", "docs/unify/unify/unify.utils", "home/home", "index", "on_prem/on_prem_access", "on_prem/sso"], "filenames": ["concepts/benchmarks.rst", "concepts/router.rst", "concepts/unify_api.rst", "console/building_router.rst", "console/connecting_stack.rst", "console/running_benchmarks.rst", "docs/unify/unify.rst", "docs/unify/unify/unify.chat.rst", "docs/unify/unify/unify.clients.rst", "docs/unify/unify/unify.exceptions.rst", "docs/unify/unify/unify.utils.rst", "home/home.rst", "index.rst", "on_prem/on_prem_access.rst", "on_prem/sso.rst"], "titles": ["Benchmarking", "Routing", "Universal API", "Building a custom router", "Connecting your stack", "Benchmarking endpoints", "<no title>", "Chat", "Clients", "Exceptions", "Utils", "Welcome to Unify!", "Home", "On-Prem access", "SSO (Single Sign On)"], "terms": {"when": [0, 1, 2, 3, 8], "compar": [0, 5, 11, 12], "llm": [0, 4, 5, 7, 11, 12], "i": [0, 2, 3, 4, 5, 7, 8, 10, 14], "constant": 0, "tradeoff": 0, "make": [0, 3, 8], "between": [0, 1, 2, 3, 8], "qualiti": [0, 1, 3, 11, 12], "cost": [0, 1, 2, 3, 5, 11, 12], "latenc": [0, 1, 3, 5], "stronger": 0, "model": [0, 1, 2, 3, 4, 5, 7, 8, 10, 11, 12, 14], "ar": [0, 1, 3, 4, 5], "gener": [0, 1, 2, 8, 11, 12, 14], "slower": 0, "more": [0, 1, 5, 8], "expens": 0, "sometim": [0, 2], "overkil": 0, "task": [0, 1, 3, 5], "hand": [0, 5], "complic": 0, "matter": 0, "further": [0, 3, 8], "new": [0, 1, 2, 3, 4, 5], "releas": [0, 1], "weekli": 0, "each": [0, 1, 2, 3, 5, 10, 11, 12], "claim": 0, "state": 0, "art": 0, "data": [0, 3], "let": [0, 3, 4, 5, 11, 12], "you": [0, 1, 2, 3, 4, 5, 11, 12, 14], "see": [0, 2, 3, 5], "differ": [0, 1, 2, 3, 5, 11, 12], "perform": [0, 3, 5, 11, 12], "can": [0, 1, 2, 3, 4, 5, 11, 12, 14], "relat": 0, "live": [0, 1, 2], "stat": 0, "pull": [0, 14], "from": [0, 1, 2, 3, 5, 7, 8, 10, 11, 12, 14], "our": [0, 2, 3, 4, 5, 11, 12, 13, 14], "runtim": [0, 1], "come": [0, 1, 2], "out": [0, 1, 2], "simpli": [0, 2, 5, 11, 12], "re": [0, 3, 5, 11, 12, 14], "run": [0, 2, 3, 4, 7, 14], "thei": [0, 1, 2], "first": [0, 1, 2, 3, 5], "creat": [0, 2, 3, 4], "which": [0, 1, 2, 3, 5, 11, 12, 14], "repres": [0, 7], "want": [0, 3, 4, 5], "evalu": [0, 5, 10], "need": [0, 1, 2, 3, 4, 5, 11, 12], "list": [0, 2, 3, 5, 8, 10, 11, 12], "prompt": [0, 1, 2, 4, 5, 8, 10, 11, 12], "option": [0, 1, 3, 4, 7, 8, 10], "includ": [0, 3, 4, 5], "refer": [0, 1, 4], "gold": 0, "standard": [0, 2, 4], "answer": [0, 1, 4], "contain": [0, 1, 2, 4, 5, 8, 10], "tend": [0, 5], "get": [0, 1, 2, 7, 8, 10, 11, 12, 13], "accur": 0, "The": [0, 1, 2, 3, 4, 5, 7, 8, 10, 14], "file": [0, 1, 4, 10, 14], "itself": 0, "should": [0, 2, 4, 5, 14], "jsonl": [0, 1, 4], "format": [0, 1, 2, 7, 8, 11, 12], "one": [0, 1, 2, 4, 5, 11, 12], "entri": [0, 1], "per": [0, 3], "line": 0, "exampl": [0, 1, 2, 3, 4], "below": [0, 3, 4, 5], "thi": [0, 1, 2, 3, 4, 5, 10, 14], "ref_answ": [0, 1], "second": [0, 3], "us": [0, 1, 2, 3, 4, 5, 7, 8, 10, 11, 12, 14], "least": [0, 5], "50": 0, "most": [0, 1, 4], "result": [0, 1, 3, 4, 5], "current": [0, 2, 3, 5], "an": [0, 1, 2, 3, 4, 7, 8, 10], "maximum": 0, "limit": [0, 1, 2], "500": [0, 9], "we": [0, 1, 2, 3, 4, 5, 11, 12], "don": [0, 2, 3], "t": [0, 1, 2, 3, 5], "much": [0, 1], "extra": 0, "detail": [0, 2, 3, 14], "past": 0, "250": 0, "In": [0, 1, 2, 3, 4, 5], "dashboard": [0, 1, 3, 5], "click": [0, 1, 3, 4, 5], "select": [0, 1, 3, 5], "open": [0, 3, 5], "interfac": [0, 1, 2, 4, 5, 11, 12], "upload": [0, 1, 3, 4, 5, 10], "finish": [0, 1], "ll": [0, 1, 3, 4, 5], "receiv": [0, 1, 3], "email": [0, 1, 3], "graph": [0, 1, 3, 5], "displai": [0, 1, 3, 5], "x": [0, 2, 3, 5, 14], "axi": [0, 5], "set": [0, 2, 3, 4, 7, 8], "time": [0, 1, 2, 3, 5], "token": [0, 1, 2, 3, 5, 8], "inter": [0, 1, 3], "either": [0, 2, 3, 10], "linear": 0, "log": 0, "scale": 0, "gpt4o": 0, "judg": 0, "cf": 0, "http": [0, 8, 9, 10, 14], "arxiv": 0, "org": 0, "ab": 0, "2306": 0, "05685": 0, "": [0, 1, 2, 3, 4, 5, 8, 11, 12, 14], "respons": [0, 1, 2, 8, 10, 14], "learn": [1, 3, 4, 5], "how": [1, 3, 4, 5], "allow": [1, 3, 5], "flexibl": 1, "about": [1, 5], "handl": [1, 2, 11, 12], "advantag": 1, "sever": [1, 11, 12], "reason": [1, 2], "small": 1, "faster": 1, "cheaper": 1, "wherea": 1, "bigger": 1, "capabl": 1, "often": 1, "rang": 1, "difficulti": 1, "provid": [1, 2, 3, 4, 5, 7, 8, 10, 11, 12], "have": [1, 2, 3, 4, 5, 11, 12, 14], "chang": [1, 3, 5, 11, 12], "over": [1, 5], "everi": [1, 2], "week": 1, "strength": 1, "weak": 1, "smaller": 1, "continu": 1, "improv": [1, 11, 12], "ride": 1, "wave": 1, "abil": 1, "minimis": 1, "base": [1, 3, 7, 8, 9], "statist": 1, "purpos": 1, "wide": 1, "varieti": [1, 4], "To": [1, 2, 5, 13], "queri": [1, 3, 4], "green": 1, "star": [1, 5], "chat": [1, 2, 3, 14], "copi": [1, 3], "string": [1, 2, 8, 11, 12], "For": [1, 3, 4, 5], "q": 1, "1": [1, 8], "c": 1, "4": [1, 3, 8], "65e": 1, "03": 1, "2": [1, 8], "08e": 1, "05": 1, "07e": 1, "paramet": [1, 2], "stand": 1, "rel": 1, "weigh": 1, "complet": [1, 2, 3, 5, 14], "customiz": 1, "meaning": 1, "configur": [1, 3, 5], "directli": 1, "restrict": 1, "specifi": [1, 2, 3, 4, 5, 10], "them": [1, 3, 5], "follow": [1, 2], "model1": 1, "model2": 1, "model3": 1, "provider1": 1, "provider2": 1, "specif": [1, 5, 10], "rate": 1, "hit": 1, "move": [1, 3], "next": [1, 2, 3, 4, 5], "best": [1, 3, 11, 12], "mean": [1, 14], "higher": [1, 8], "than": [1, 5], "avail": [1, 4, 10, 11, 12], "singl": [1, 2, 8, 11, 12], "ve": [1, 4, 5], "found": [1, 9], "similar": 1, "those": [1, 4], "go": [1, 2, 3, 5, 11, 12], "product": [1, 2], "visualis": 1, "your": [1, 2, 8, 11, 12, 14], "same": [1, 3], "benchmark": [1, 3, 4, 11, 12], "e": [1, 3, 5], "form": [1, 2], "xxx": 1, "yyi": 1, "accuraci": 1, "final": [1, 3], "system": [1, 2, 8, 11, 12], "after": [1, 3], "onc": [1, 3, 4, 5, 14], "here": [1, 3], "abl": 1, "choos": [1, 3], "api": [1, 3, 4, 7, 8, 10, 11, 12, 14], "There": [2, 4, 5], "ever": 2, "expand": [2, 5], "ocean": 2, "all": [2, 3, 4, 5, 9, 10, 11, 12, 14], "requir": [2, 3], "uniqu": 2, "kei": [2, 4, 7, 8, 10, 11, 12], "payment": 2, "process": 2, "test": [2, 3, 5, 11, 12], "quickli": 2, "becom": 2, "veri": [2, 3], "cumbersom": 2, "manag": [2, 11, 12], "nuanc": 2, "A": [2, 3, 8, 10], "common": [2, 4], "One": 2, "account": [2, 3, 5, 8, 10], "balanc": [2, 3], "sign": [2, 11, 12], "up": [2, 8, 11, 12], "interact": [2, 7, 8], "via": [2, 4], "ani": [2, 3, 5, 11, 12], "program": 2, "languag": 2, "own": [2, 4, 11, 12], "command": 2, "termin": 2, "replac": 2, "unify_kei": [2, 7, 8, 10, 11, 12], "curl": [2, 14], "post": [2, 14], "ai": [2, 4], "v0": [2, 5], "h": [2, 14], "author": 2, "bearer": 2, "content": [2, 8, 10, 14], "type": [2, 7, 8, 10, 14], "applic": [2, 3, 9, 14], "json": [2, 8, 10, 14], "d": [2, 3, 14], "llama": [2, 3, 5], "3": [2, 3, 5, 14], "8b": 2, "firework": 2, "messag": [2, 8, 14], "role": [2, 14], "user": [2, 3, 8, 10, 14], "sai": 2, "hello": [2, 11, 12], "field": 2, "both": [2, 3], "find": [2, 5], "pair": [2, 5], "accept": [2, 14], "also": [2, 3, 4, 5, 11, 12], "pass": [2, 11, 12], "argument": [2, 11, 12], "abov": 2, "function": 2, "return": [2, 7, 8, 10], "like": [2, 3, 4, 5, 8, 11, 12, 14], "so": [2, 3, 4, 5, 11, 12], "some_provid": 2, "some_model": 2, "easili": 2, "made": 2, "import": [2, 11, 12, 14], "url": 2, "header": 2, "unify_api_kei": 2, "payload": 2, "togeth": 2, "print": 2, "text": 2, "look": [2, 3, 14], "someth": [2, 5], "1718888877": 2, "id": [2, 3], "896bfc1ae84271aa": 2, "lhr": 2, "object": [2, 7, 8], "usag": 2, "completion_token": 2, "25": 2, "prompt_token": 2, "13": 2, "total_token": 2, "38": 2, "7": 2, "6e": 2, "06": [2, 3], "choic": [2, 11, 12], "finish_reason": 2, "stop": [2, 8], "index": 2, "0": [2, 3, 8], "It": [2, 11, 12], "nice": 2, "meet": [2, 3], "help": 2, "would": [2, 3, 4, 5], "assist": 2, "seed": 2, "11563975138181362140": 2, "output": [2, 5, 8, 11, 12], "access": [2, 3, 5, 7, 8, 14], "number": [2, 8], "well": [2, 3, 4], "total": [2, 10], "u": [2, 5, 13], "other": [2, 3, 5], "temperatur": [2, 8], "stream": [2, 8], "max_token": [2, 8], "support": [2, 10], "mai": [2, 3], "compat": 2, "download": [2, 10], "pip": [2, 11, 12], "instal": [2, 11, 12], "unifyai": [2, 11, 12, 14], "start": [2, 7], "client": [2, 7, 11, 12, 14], "api_kei": [2, 7, 8, 10, 11, 12, 14], "world": [2, 11, 12], "If": [2, 3, 4, 5, 7, 8, 10], "save": [2, 3, 10, 11, 12], "environ": [2, 3, 7, 8, 10, 11, 12], "variabl": [2, 7, 8, 10, 11, 12], "util": [2, 11, 12], "list_model": [2, 10], "list_provid": [2, 10], "list_endpoint": [2, 10, 11, 12], "design": [2, 5], "exist": 2, "code": 2, "straightforward": 2, "try": 2, "base_url": [2, 14], "your_unify_kei": 2, "mistral": [2, 5], "7b": [2, 5], "instruct": [2, 5], "hi": [2, 14], "true": [2, 8], "chunk": [2, 8], "delta": 2, "end": 2, "rememb": 2, "likewis": 2, "again": [2, 4], "const": 2, "baseurl": 2, "apikei": 2, "await": 2, "onli": [2, 3, 5], "spend": 2, "behind": [2, 5], "scene": 2, "top": [2, 5], "automat": [2, 3], "refil": 2, "page": [2, 3, 5], "credit": [2, 7, 8, 10], "get_credit": [2, 10], "your_api_kei": 2, "your_user_id": 2, "232": 2, "32": 2, "deploi": [2, 3, 5, 11, 12], "fine": [2, 3], "tune": [2, 3, 5], "add": [2, 3, 4, 5, 11, 12], "relev": 2, "name": [2, 3, 4, 5, 7, 8, 10], "individu": [2, 11, 12], "outag": 2, "disrupt": 2, "workflow": 2, "combat": 2, "down": 2, "fail": 2, "some": [2, 3, 5], "until": [2, 11, 12], "succe": 2, "reach": 2, "tag": 2, "model_a": 2, "provider_a": 2, "model_b": 2, "provider_b": 2, "actual": 2, "went": 2, "section": [3, 4, 5], "through": [3, 4, 5], "consol": [3, 4, 5], "window": 3, "enabl": 3, "endpoint": [3, 7, 8, 10, 11, 12], "rout": [3, 4, 11, 12], "dataset": [3, 5, 10], "gpt": [3, 11, 12], "claud": [3, 14], "llama3": 3, "call": [3, 13], "intend": 3, "earlier": [3, 5], "variant": 3, "correspond": [3, 5], "dropdown": [3, 5], "notic": 3, "becaus": 3, "doe": 3, "depend": 3, "button": [3, 4], "submit": [3, 5], "job": [3, 5], "ha": 3, "schedul": 3, "behalf": [3, 4], "alreadi": [3, 4], "been": 3, "ahead": 3, "work": 3, "duplic": 3, "previous": 3, "calls1": [3, 5], "call2": 3, "q1": 3, "q2": 3, "repeat": 3, "howev": 3, "yet": 3, "70b": 3, "g": [3, 5], "trigger": [3, 5], "request": [3, 5, 9, 14], "readi": [3, 4, 11, 12], "manual": 3, "track": [3, 5], "progress": [3, 5], "With": [3, 11, 12], "done": [3, 5], "stage": 3, "accordingli": 3, "now": [3, 4, 5, 11, 12, 14], "visual": [3, 5], "step": [3, 4, 14], "explor": [3, 5], "variou": [3, 5], "possibl": 3, "trade": 3, "off": [3, 4], "speed": [3, 5, 11, 12], "variat": 3, "These": 3, "As": [3, 5], "befor": [3, 5], "point": [3, 5], "plot": [3, 5], "view": [3, 5], "take": 3, "constrain": 3, "search": 3, "space": 3, "wai": [3, 4], "certain": 3, "deploy": 3, "ensur": [3, 5], "guarante": 3, "Of": 3, "cours": 3, "visibl": 3, "remov": [3, 5, 10], "presum": 3, "anthrop": [3, 14], "properli": 3, "moment": 3, "therefor": 3, "appli": 3, "altern": [3, 4], "had": 3, "overwritten": 3, "clauc": 3, "place": 3, "legend": [3, 5], "underneath": 3, "its": 3, "parent": 3, "slightli": 3, "reduc": 3, "amount": 3, "assum": [3, 4], "decid": 3, "stick": 3, "decis": 3, "avoid": [3, 11, 12], "immedi": 3, "futur": 3, "appear": [3, 5], "full": 3, "frequenc": 3, "gpt4": 3, "42": 3, "29": 3, "calls2": [3, 5], "18": 3, "11": 3, "across": [3, 4, 5], "metric": [3, 5], "ax": 3, "verifi": 3, "session": 3, "perman": 3, "happi": [3, 5], "forget": 3, "where": [3, 5, 8], "inform": [3, 5], "delet": [3, 10], "anytim": 3, "press": [3, 5], "besid": 3, "clipboard": 3, "case": [3, 5, 11, 12], "anthropic_8": 3, "28e": 3, "03_4": 3, "66e": 3, "4_1": 3, "00e": 3, "unifi": [3, 4, 7, 8, 9, 10, 14], "show": [3, 4, 5, 7], "along": 3, "default": [3, 5, 7, 8, 10], "That": [3, 4, 5, 11, 12], "shelf": 4, "azur": 4, "ml": 4, "catalog": 4, "vertex": 4, "garden": 4, "aw": 4, "bedrock": 4, "host": 4, "whole": 4, "do": 4, "major": 4, "cloud": 4, "regardless": 4, "expos": [4, 5], "adher": 4, "openai": [4, 7, 8, 11, 12], "integr": 4, "got": 4, "said": 4, "screen": 4, "local": [4, 10, 14], "build": 4, "label": [4, 5], "preview": 4, "imag": [4, 5], "know": [4, 5], "router": [4, 5], "navig": 5, "custom": [5, 9, 11, 12], "ones": 5, "methodologi": 5, "head": 5, "By": 5, "six": 5, "openherm": 5, "shown": 5, "On": 5, "against": 5, "scatter": 5, "replot": 5, "similarili": 5, "els": 5, "score": 5, "ttft": 5, "updat": 5, "remain": [5, 8, 10], "averag": 5, "explain": [5, 14], "granular": 5, "outsid": 5, "popular": 5, "period": 5, "suit": 5, "dedic": 5, "valu": [5, 7, 8, 10], "tabl": 5, "region": 5, "sequenc": [5, 8], "length": 5, "latest": [5, 14], "sort": 5, "icon": 5, "column": 5, "reveal": 5, "pattern": 5, "sec": 5, "figur": 5, "fit": 5, "haven": 5, "recommend": 5, "resum": 5, "ad": 5, "previou": 5, "privat": 5, "server": [5, 9], "vari": 5, "alwai": 5, "send": 5, "fastest": 5, "under": 5, "ip": 5, "address": 5, "lock": 5, "indic": 5, "order": 5, "train": 5, "just": 5, "necessari": 5, "subsect": 5, "separ": 5, "check": 5, "entir": [5, 14], "histori": [5, 7, 8], "mixtral": 5, "financ": 5, "alongsid": 5, "foundat": 5, "class": [7, 8, 9], "chatbot": 7, "none": [7, 8, 10], "agent": 7, "__init__": [7, 8], "initi": [7, 8], "arg": [7, 8, 10], "str": [7, 8, 10], "uploaded_bi": [7, 8], "model_nam": [7, 8], "provider_nam": [7, 8], "must": [7, 8, 10], "attempt": [7, 8], "retriev": [7, 8], "rais": [7, 8, 9, 10], "unifyerror": [7, 8, 9], "miss": [7, 8], "clear_chat_histori": 7, "clear": 7, "properti": [7, 8], "dar201": [7, 8], "noqa": [7, 8], "show_credit": 7, "fals": [7, 8], "show_provid": 7, "loop": 7, "bool": [7, 8], "whether": 7, "consumpt": 7, "set_client": 7, "dar101": [7, 8], "set_endpoint": [7, 8], "set_model": [7, 8], "set_provid": [7, 8], "asyncunifi": 8, "asynchron": 8, "async": 8, "user_prompt": 8, "system_prompt": 8, "union": 8, "asyncgener": 8, "dict": [8, 10], "dictionari": [8, 10], "convers": 8, "int": 8, "max": 8, "float": [8, 10], "what": [8, 14], "sampl": 8, "8": 8, "random": [8, 11, 12], "while": 8, "lower": 8, "focus": 8, "determinist": 8, "yield": 8, "error": [8, 9, 10], "occur": 8, "dure": 8, "get_credit_bal": 8, "left": 8, "success": [8, 10], "otherwis": [8, 10], "badrequesterror": [8, 9, 10], "wa": [8, 10, 14], "valueerror": [8, 10], "pars": [8, 10], "1024": 8, "authenticationerror": 9, "401": 9, "unauthor": 9, "400": 9, "bad": 9, "conflicterror": 9, "409": 9, "conflict": 9, "internalservererror": 9, "intern": 9, "notfounderror": 9, "404": 9, "Not": 9, "permissiondeniederror": 9, "403": 9, "forbidden": 9, "ratelimiterror": 9, "429": 9, "too": 9, "mani": 9, "unprocessableentityerror": 9, "422": 9, "unprocess": 9, "entiti": 9, "delete_dataset": 10, "platform": 10, "given": 10, "path": 10, "info": 10, "msg": 10, "delete_evalu": 10, "whose": 10, "download_dataset": 10, "usd": 10, "list_dataset": 10, "fetch": 10, "empti": 10, "list_evalu": 10, "associ": 10, "upload_dataset_from_dictionari": 10, "upload_dataset_from_fil": 10, "mission": [11, 12], "simplifi": [11, 12], "landscap": [11, 12], "No": [11, 12], "input": [11, 12], "eval": [11, 12], "iter": [11, 12], "app": [11, 12], "perfect": [11, 12], "easiest": [11, 12], "python": [11, 12, 14], "packag": [11, 12], "Then": [11, 12, 14], "4o": [11, 12], "your_kei": [11, 12], "fingertip": [11, 12], "offer": 13, "book": 13, "prem": 14, "microservic": 14, "grant": 14, "docker": 14, "dt": 14, "p": 14, "8000": 14, "env": 14, "env_file_path": 14, "deepinfra_api_kei": 14, "stability_api_kei": 14, "mistral_ai_api_kei": 14, "perplexity_ai_api_kei": 14, "anyscale_api_kei": 14, "octoai_api_kei": 14, "openai_api_kei": 14, "together_ai_api_kei": 14, "replicate_api_kei": 14, "lepton_ai_api_kei": 14, "fireworks_ai_api_kei": 14, "anthropic_api_kei": 14, "vertexai_service_acc_json": 14, "vertexai_gcloud_path": 14, "vertexai_project": 14, "vertexai_loc": 14, "servic": 14, "weather": 14, "san": 14, "francisco": 14, "tokyo": 14, "pari": 14, "localhost": 14, "haiku": 14, "random_key_wont_be_us": 14, "life": 14, "who": 14, "newton": 14, "theori": 14, "gravit": 14, "give": 14, "long": 14, "pleas": 14, "achiev": 14}, "objects": {"unify": [[7, 0, 0, "-", "chat"], [8, 0, 0, "-", "clients"], [9, 0, 0, "-", "exceptions"], [10, 0, 0, "-", "utils"]], "unify.chat": [[7, 1, 1, "", "ChatBot"]], "unify.chat.ChatBot": [[7, 2, 1, "", "__init__"], [7, 2, 1, "", "clear_chat_history"], [7, 3, 1, "", "client"], [7, 3, 1, "", "endpoint"], [7, 3, 1, "", "model"], [7, 3, 1, "", "provider"], [7, 2, 1, "", "run"], [7, 2, 1, "", "set_client"], [7, 2, 1, "", "set_endpoint"], [7, 2, 1, "", "set_model"], [7, 2, 1, "", "set_provider"]], "unify.clients": [[8, 1, 1, "", "AsyncUnify"], [8, 1, 1, "", "Unify"]], "unify.clients.AsyncUnify": [[8, 2, 1, "", "__init__"], [8, 3, 1, "", "endpoint"], [8, 2, 1, "", "generate"], [8, 2, 1, "", "get_credit_balance"], [8, 3, 1, "", "model"], [8, 3, 1, "", "provider"], [8, 2, 1, "", "set_endpoint"], [8, 2, 1, "", "set_model"], [8, 2, 1, "", "set_provider"]], "unify.clients.Unify": [[8, 2, 1, "", "__init__"], [8, 3, 1, "", "endpoint"], [8, 2, 1, "", "generate"], [8, 2, 1, "", "get_credit_balance"], [8, 3, 1, "", "model"], [8, 3, 1, "", "provider"], [8, 2, 1, "", "set_endpoint"], [8, 2, 1, "", "set_model"], [8, 2, 1, "", "set_provider"]], "unify.exceptions": [[9, 4, 1, "", "AuthenticationError"], [9, 4, 1, "", "BadRequestError"], [9, 4, 1, "", "ConflictError"], [9, 4, 1, "", "InternalServerError"], [9, 4, 1, "", "NotFoundError"], [9, 4, 1, "", "PermissionDeniedError"], [9, 4, 1, "", "RateLimitError"], [9, 4, 1, "", "UnifyError"], [9, 4, 1, "", "UnprocessableEntityError"]], "unify.utils": [[10, 5, 1, "", "delete_dataset"], [10, 5, 1, "", "delete_evaluation"], [10, 5, 1, "", "download_dataset"], [10, 5, 1, "", "evaluate"], [10, 5, 1, "", "get_credits"], [10, 5, 1, "", "list_datasets"], [10, 5, 1, "", "list_endpoints"], [10, 5, 1, "", "list_evaluations"], [10, 5, 1, "", "list_models"], [10, 5, 1, "", "list_providers"], [10, 5, 1, "", "upload_dataset_from_dictionary"], [10, 5, 1, "", "upload_dataset_from_file"]]}, "objtypes": {"0": "py:module", "1": "py:class", "2": "py:method", "3": "py:property", "4": "py:exception", "5": "py:function"}, "objnames": {"0": ["py", "module", "Python module"], "1": ["py", "class", "Python class"], "2": ["py", "method", "Python method"], "3": ["py", "property", "Python property"], "4": ["py", "exception", "Python exception"], "5": ["py", "function", "Python function"]}, "titleterms": {"benchmark": [0, 5], "prepar": [0, 1], "your": [0, 3, 4, 5], "dataset": [0, 1, 4], "how": 0, "doe": 0, "work": 0, "rout": 1, "what": 1, "i": 1, "llm": [1, 2], "foundat": 1, "router": [1, 3], "filter": 1, "endpoint": [1, 2, 4, 5], "maximis": 1, "throughput": 1, "fine": 1, "tune": 1, "custom": [1, 2, 3, 4], "train": [1, 3], "beta": [1, 3, 4, 5], "univers": 2, "api": 2, "queri": 2, "http": 2, "request": 2, "unifi": [2, 11, 12], "python": 2, "packag": 2, "openai": 2, "nodej": 2, "bill": 2, "advanc": 2, "featur": 2, "fallback": 2, "build": 3, "round": [3, 4, 5], "up": [3, 4, 5], "connect": 4, "stack": 4, "prerequisit": 4, "ad": 4, "qualiti": 5, "runtim": 5, "run": 5, "own": 5, "chat": 7, "client": 8, "except": 9, "util": 10, "welcom": [11, 12], "quick": [11, 12], "start": [11, 12], "On": [13, 14], "prem": 13, "access": 13, "sso": 14, "singl": 14, "sign": 14}, "envversion": {"sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "nbsphinx": 4, "sphinx": 60}, "alltitles": {"Routing": [[1, "routing"]], "What is LLM routing?": [[1, "what-is-llm-routing"]], "Foundation router": [[1, "foundation-router"]], "Filtering endpoints": [[1, "filtering-endpoints"]], "Maximising throughput": [[1, "maximising-throughput"]], "Fine-tuned custom router": [[1, "fine-tuned-custom-router"]], "Preparing a dataset": [[1, "preparing-a-dataset"]], "Training a router (beta)": [[1, "training-a-router-beta"]], "Benchmarking": [[0, "benchmarking"]], "Preparing your dataset": [[0, "preparing-your-dataset"]], "Benchmarking your dataset": [[0, "benchmarking-your-dataset"]], "How does it work?": [[0, "how-does-it-work"]], "Universal API": [[2, "universal-api"]], "Querying the API": [[2, "querying-the-api"]], "HTTP Requests": [[2, "http-requests"]], "Unify Python Package": [[2, "unify-python-package"]], "OpenAI Python Package": [[2, "openai-python-package"]], "OpenAI NodeJS Package": [[2, "openai-nodejs-package"]], "Billing": [[2, "billing"]], "Advanced features": [[2, "advanced-features"]], "Custom endpoints": [[2, "custom-endpoints"]], "LLM Fallbacks": [[2, "llm-fallbacks"]], "Connecting your stack": [[4, "connecting-your-stack"]], "Custom endpoints (Beta)": [[4, "custom-endpoints-beta"]], "Prerequisite": [[4, "prerequisite"]], "Adding the endpoints": [[4, "adding-the-endpoints"]], "Custom datasets": [[4, "custom-datasets"]], "Round Up": [[4, "round-up"], [3, "round-up"], [5, "round-up"]], "Building a custom router": [[3, "building-a-custom-router"]], "Training a custom router (Beta)": [[3, "training-a-custom-router-beta"]], "Customizing your router (Beta)": [[3, "customizing-your-router-beta"]], "Benchmarking endpoints": [[5, "benchmarking-endpoints"]], "Quality benchmarks": [[5, "quality-benchmarks"], [5, "id4"]], "Runtime benchmarks": [[5, "runtime-benchmarks"], [5, "id2"]], "Running your own benchmarks (Beta)": [[5, "running-your-own-benchmarks-beta"]], "SSO (Single Sign On)": [[14, "sso-single-sign-on"]], "On-Prem access": [[13, "on-prem-access"]], "Welcome to Unify!": [[12, "welcome-to-unify"], [11, "welcome-to-unify"]], "Quick Start": [[12, "quick-start"], [11, "quick-start"]], "Exceptions": [[9, "module-unify.exceptions"]], "Utils": [[10, "module-unify.utils"]], "Clients": [[8, "module-unify.clients"]], "Chat": [[7, "module-unify.chat"]]}, "indexentries": {"chatbot (class in unify.chat)": [[7, "unify.chat.ChatBot"]], "__init__() (unify.chat.chatbot method)": [[7, "unify.chat.ChatBot.__init__"]], "clear_chat_history() (unify.chat.chatbot method)": [[7, "unify.chat.ChatBot.clear_chat_history"]], "client (unify.chat.chatbot property)": [[7, "unify.chat.ChatBot.client"]], "endpoint (unify.chat.chatbot property)": [[7, "unify.chat.ChatBot.endpoint"]], "model (unify.chat.chatbot property)": [[7, "unify.chat.ChatBot.model"]], "module": [[7, "module-unify.chat"], [8, "module-unify.clients"], [9, "module-unify.exceptions"], [10, "module-unify.utils"]], "provider (unify.chat.chatbot property)": [[7, "unify.chat.ChatBot.provider"]], "run() (unify.chat.chatbot method)": [[7, "unify.chat.ChatBot.run"]], "set_client() (unify.chat.chatbot method)": [[7, "unify.chat.ChatBot.set_client"]], "set_endpoint() (unify.chat.chatbot method)": [[7, "unify.chat.ChatBot.set_endpoint"]], "set_model() (unify.chat.chatbot method)": [[7, "unify.chat.ChatBot.set_model"]], "set_provider() (unify.chat.chatbot method)": [[7, "unify.chat.ChatBot.set_provider"]], "unify.chat": [[7, "module-unify.chat"]], "asyncunify (class in unify.clients)": [[8, "unify.clients.AsyncUnify"]], "unify (class in unify.clients)": [[8, "unify.clients.Unify"]], "__init__() (unify.clients.asyncunify method)": [[8, "unify.clients.AsyncUnify.__init__"]], "__init__() (unify.clients.unify method)": [[8, "unify.clients.Unify.__init__"]], "endpoint (unify.clients.asyncunify property)": [[8, "unify.clients.AsyncUnify.endpoint"]], "endpoint (unify.clients.unify property)": [[8, "unify.clients.Unify.endpoint"]], "generate() (unify.clients.asyncunify method)": [[8, "unify.clients.AsyncUnify.generate"]], "generate() (unify.clients.unify method)": [[8, "unify.clients.Unify.generate"]], "get_credit_balance() (unify.clients.asyncunify method)": [[8, "unify.clients.AsyncUnify.get_credit_balance"]], "get_credit_balance() (unify.clients.unify method)": [[8, "unify.clients.Unify.get_credit_balance"]], "model (unify.clients.asyncunify property)": [[8, "unify.clients.AsyncUnify.model"]], "model (unify.clients.unify property)": [[8, "unify.clients.Unify.model"]], "provider (unify.clients.asyncunify property)": [[8, "unify.clients.AsyncUnify.provider"]], "provider (unify.clients.unify property)": [[8, "unify.clients.Unify.provider"]], "set_endpoint() (unify.clients.asyncunify method)": [[8, "unify.clients.AsyncUnify.set_endpoint"]], "set_endpoint() (unify.clients.unify method)": [[8, "unify.clients.Unify.set_endpoint"]], "set_model() (unify.clients.asyncunify method)": [[8, "unify.clients.AsyncUnify.set_model"]], "set_model() (unify.clients.unify method)": [[8, "unify.clients.Unify.set_model"]], "set_provider() (unify.clients.asyncunify method)": [[8, "unify.clients.AsyncUnify.set_provider"]], "set_provider() (unify.clients.unify method)": [[8, "unify.clients.Unify.set_provider"]], "unify.clients": [[8, "module-unify.clients"]], "authenticationerror": [[9, "unify.exceptions.AuthenticationError"]], "badrequesterror": [[9, "unify.exceptions.BadRequestError"]], "conflicterror": [[9, "unify.exceptions.ConflictError"]], "internalservererror": [[9, "unify.exceptions.InternalServerError"]], "notfounderror": [[9, "unify.exceptions.NotFoundError"]], "permissiondeniederror": [[9, "unify.exceptions.PermissionDeniedError"]], "ratelimiterror": [[9, "unify.exceptions.RateLimitError"]], "unifyerror": [[9, "unify.exceptions.UnifyError"]], "unprocessableentityerror": [[9, "unify.exceptions.UnprocessableEntityError"]], "unify.exceptions": [[9, "module-unify.exceptions"]], "delete_dataset() (in module unify.utils)": [[10, "unify.utils.delete_dataset"]], "delete_evaluation() (in module unify.utils)": [[10, "unify.utils.delete_evaluation"]], "download_dataset() (in module unify.utils)": [[10, "unify.utils.download_dataset"]], "evaluate() (in module unify.utils)": [[10, "unify.utils.evaluate"]], "get_credits() (in module unify.utils)": [[10, "unify.utils.get_credits"]], "list_datasets() (in module unify.utils)": [[10, "unify.utils.list_datasets"]], "list_endpoints() (in module unify.utils)": [[10, "unify.utils.list_endpoints"]], "list_evaluations() (in module unify.utils)": [[10, "unify.utils.list_evaluations"]], "list_models() (in module unify.utils)": [[10, "unify.utils.list_models"]], "list_providers() (in module unify.utils)": [[10, "unify.utils.list_providers"]], "unify.utils": [[10, "module-unify.utils"]], "upload_dataset_from_dictionary() (in module unify.utils)": [[10, "unify.utils.upload_dataset_from_dictionary"]], "upload_dataset_from_file() (in module unify.utils)": [[10, "unify.utils.upload_dataset_from_file"]]}})
\ No newline at end of file