diff --git a/hub/.doctrees/environment.pickle b/hub/.doctrees/environment.pickle index f91ca987b3..d979ad9456 100644 Binary files a/hub/.doctrees/environment.pickle and b/hub/.doctrees/environment.pickle differ diff --git a/hub/.doctrees/home/home.doctree b/hub/.doctrees/home/home.doctree index 9831307bd7..7ee8dad491 100644 Binary files a/hub/.doctrees/home/home.doctree and b/hub/.doctrees/home/home.doctree differ diff --git a/hub/.doctrees/index.doctree b/hub/.doctrees/index.doctree index c53c1f420b..30f9e047e7 100644 Binary files a/hub/.doctrees/index.doctree and b/hub/.doctrees/index.doctree differ diff --git a/hub/_sources/home/home.rst.txt b/hub/_sources/home/home.rst.txt index 32369b9ca4..7b301ae627 100644 --- a/hub/_sources/home/home.rst.txt +++ b/hub/_sources/home/home.rst.txt @@ -3,22 +3,40 @@ Welcome to Unify! We're on a mission to unify and simplify the LLM landscape. Unify lets you: -* **🔑 Use any LLM from any Provider**: With a single interface, you can use all LLMs from all providers by simply changing one string. -No need to manage several API keys or handle different input-output formats. Unify handles all of that for you! +* **🔑 Use any LLM from any Provider**: With a single interface, you can use all LLMs from all providers by simply changing one string. No need to manage several API keys or handle different input-output formats. Unify handles all of that for you! -* **📊 Improve LLM Performance**: Add your own custom tests and evals, and benchmark your own prompts on all models and providers. -Comparing quality, cost and speed, and iterate on your system prompt until all test cases pass, and you can deploy your app! +* **📊 Improve LLM Performance**: Add your own custom tests and evals, and benchmark your own prompts on all models and providers. Comparing quality, cost and speed, and iterate on your system prompt until all test cases pass, and you can deploy your app! * **🔀 Route to the Best LLM**: Improve quality, cost and speed by routing to the perfect model and provider for each individual prompt. Getting Started --------------- -We recommend you give the concepts section a quick read to get familiar with routing and benchmarking. Once you're ready to go, start by `Signing In `_. From this point you can either learn how to: +It's easiest to get started using our Python client. Simply install the package: -* **Use our interfaces (Recommended)**: The interfaces guides explain how to interact with endpoints and deploy your custom router in a no-code environment. +.. code-block:: bash -* **Make your first request**: The api guides explains how to start querying endpoints and using the router with our API. + pip install unifyai -.. warning:: - Throughout the guides, you'll notice some sections marked as (Beta). **Any section marked as Beta is currently not available** and only illustrate planned features we are currently working on. We're constantly iterating on our roadmap so if you'd like to leave some feedback or suggestion on features you'd like to see, `we'd love to discuss `_ this with you! +`Sign up `_ to get your API key. Then you're ready to go! 🚀 + +.. code-block:: python + + import unify + client = unify.Unify("gpt-4o@openai", api_key=) + client.generate("hello world!") + +You can also save your key as :code:`UNIFY_KEY` in your environment variables, avoiding the need to use the :code:`api_key` argument. + +You can list all available endpoints like so, any of which can be passed into the client: + +.. code-block:: python + + import unify + import random + endpoints = unify.utils.list_endpoints() + endpoint = random.choice(endpoints) + client = unify.Unify(endpoint) + client.generate("hello world!") + +That's it! You now have all models and providers at your fingertips ✨ \ No newline at end of file diff --git a/hub/home/home.html b/hub/home/home.html index 5deadf5819..7165e75b80 100644 --- a/hub/home/home.html +++ b/hub/home/home.html @@ -329,27 +329,33 @@

Welcome to Unify!#

We’re on a mission to unify and simplify the LLM landscape. Unify lets you:

    -

  • 🔑 Use any LLM from any Provider: With a single interface, you can use all LLMs from all providers by simply changing one string.

    • -
-

No need to manage several API keys or handle different input-output formats. Unify handles all of that for you!

-
    -

  • 📊 Improve LLM Performance: Add your own custom tests and evals, and benchmark your own prompts on all models and providers.

    • -
-

Comparing quality, cost and speed, and iterate on your system prompt until all test cases pass, and you can deploy your app!

-
    +

  • 🔑 Use any LLM from any Provider: With a single interface, you can use all LLMs from all providers by simply changing one string. No need to manage several API keys or handle different input-output formats. Unify handles all of that for you!

    • +

  • 📊 Improve LLM Performance: Add your own custom tests and evals, and benchmark your own prompts on all models and providers. Comparing quality, cost and speed, and iterate on your system prompt until all test cases pass, and you can deploy your app!

  • 🔀 Route to the Best LLM: Improve quality, cost and speed by routing to the perfect model and provider for each individual prompt.

Getting Started#

-

We recommend you give the concepts section a quick read to get familiar with routing and benchmarking. Once you’re ready to go, start by Signing In. From this point you can either learn how to:

-
    -

  • Use our interfaces (Recommended): The interfaces guides explain how to interact with endpoints and deploy your custom router in a no-code environment.

    • -

  • Make your first request: The api guides explains how to start querying endpoints and using the router with our API.

    • -
-
-

Warning

-

Throughout the guides, you’ll notice some sections marked as (Beta). Any section marked as Beta is currently not available and only illustrate planned features we are currently working on. We’re constantly iterating on our roadmap so if you’d like to leave some feedback or suggestion on features you’d like to see, we’d love to discuss this with you!

+

It’s easiest to get started using our Python client. Simply install the package:

+
pip install unifyai
+
+
+

Sign up to get your API key. Then you’re ready to go! 🚀

+
import unify
+client = unify.Unify("gpt-4o@openai", api_key=<your_key>)
+client.generate("hello world!")
+
+
+

You can also save your key as UNIFY_KEY in your environment variables, avoiding the need to use the api_key argument.

+

You can list all available endpoints like so, any of which can be passed into the client:

+
import unify
+import random
+endpoints = unify.utils.list_endpoints()
+endpoint = random.choice(endpoints)
+client = unify.Unify(endpoint)
+client.generate("hello world!")
+
+

That’s it! You now have all models and providers at your fingertips ✨

diff --git a/hub/index.html b/hub/index.html index 6f9bf201e8..34c8bb753a 100644 --- a/hub/index.html +++ b/hub/index.html @@ -304,27 +304,33 @@

Welcome to Unify!#

We’re on a mission to unify and simplify the LLM landscape. Unify lets you:

    -

  • 🔑 Use any LLM from any Provider: With a single interface, you can use all LLMs from all providers by simply changing one string.

    • -
-

No need to manage several API keys or handle different input-output formats. Unify handles all of that for you!

-
    -

  • 📊 Improve LLM Performance: Add your own custom tests and evals, and benchmark your own prompts on all models and providers.

    • -
-

Comparing quality, cost and speed, and iterate on your system prompt until all test cases pass, and you can deploy your app!

-
    +

  • 🔑 Use any LLM from any Provider: With a single interface, you can use all LLMs from all providers by simply changing one string. No need to manage several API keys or handle different input-output formats. Unify handles all of that for you!

    • +

  • 📊 Improve LLM Performance: Add your own custom tests and evals, and benchmark your own prompts on all models and providers. Comparing quality, cost and speed, and iterate on your system prompt until all test cases pass, and you can deploy your app!

  • 🔀 Route to the Best LLM: Improve quality, cost and speed by routing to the perfect model and provider for each individual prompt.

Getting Started#

-

We recommend you give the concepts section a quick read to get familiar with routing and benchmarking. Once you’re ready to go, start by Signing In. From this point you can either learn how to:

-
    -

  • Use our interfaces (Recommended): The interfaces guides explain how to interact with endpoints and deploy your custom router in a no-code environment.

    • -

  • Make your first request: The api guides explains how to start querying endpoints and using the router with our API.

    • -
-
-

Warning

-

Throughout the guides, you’ll notice some sections marked as (Beta). Any section marked as Beta is currently not available and only illustrate planned features we are currently working on. We’re constantly iterating on our roadmap so if you’d like to leave some feedback or suggestion on features you’d like to see, we’d love to discuss this with you!

+

It’s easiest to get started using our Python client. Simply install the package:

+
pip install unifyai
+
+
+

Sign up to get your API key. Then you’re ready to go! 🚀

+
import unify
+client = unify.Unify("gpt-4o@openai", api_key=<your_key>)
+client.generate("hello world!")
+
+
+

You can also save your key as UNIFY_KEY in your environment variables, avoiding the need to use the api_key argument.

+

You can list all available endpoints like so, any of which can be passed into the client:

+
import unify
+import random
+endpoints = unify.utils.list_endpoints()
+endpoint = random.choice(endpoints)
+client = unify.Unify(endpoint)
+client.generate("hello world!")
+
+

That’s it! You now have all models and providers at your fingertips ✨

diff --git a/hub/searchindex.js b/hub/searchindex.js index 22a764189d..3fe0391357 100644 --- a/hub/searchindex.js +++ b/hub/searchindex.js @@ -1 +1 @@ -Search.setIndex({"docnames": ["concepts/benchmarks", "concepts/deploy_router", "concepts/first_request", "concepts/images", "concepts/reference", "concepts/router", "concepts/unify_api", "console/building_router", "console/connecting_stack", "console/running_benchmarks", "docs/unify/unify", "docs/unify/unify/unify.chat", "docs/unify/unify/unify.clients", "docs/unify/unify/unify.exceptions", "docs/unify/unify/unify.utils", "home/home", "index", "on_prem/on_prem_access", "on_prem/sso", "tools/openapi", "tools/python_library"], "filenames": ["concepts/benchmarks.rst", "concepts/deploy_router.rst", "concepts/first_request.rst", "concepts/images.rst", "concepts/reference.rst", "concepts/router.rst", "concepts/unify_api.rst", "console/building_router.rst", "console/connecting_stack.rst", "console/running_benchmarks.rst", "docs/unify/unify.rst", "docs/unify/unify/unify.chat.rst", "docs/unify/unify/unify.clients.rst", "docs/unify/unify/unify.exceptions.rst", "docs/unify/unify/unify.utils.rst", "home/home.rst", "index.rst", "on_prem/on_prem_access.rst", "on_prem/sso.rst", "tools/openapi.rst", "tools/python_library.rst"], "titles": ["Benchmarking", "Deploying a router", "Making your first request", "On-Prem Images", "API Reference", "Routing", "Universal API", "Building a custom router", "Connecting your stack", "Benchmarking endpoints", "<no title>", "Chat", "Clients", "Exceptions", "Utils", "Welcome to Unify!", "Home", "On-Prem access", "SSO (Single Sign On)", "OpenAPI Specification", "Python Library"], "terms": {"when": [0, 1, 2, 4, 5, 7, 12], "compar": [0, 9, 15, 16], "llm": [0, 1, 2, 8, 9, 11, 15, 16], "i": [0, 1, 2, 4, 6, 7, 8, 9, 11, 12, 14, 15, 16, 18], "constant": 0, "tradeoff": 0, "make": [0, 1, 4, 7, 12, 15, 16], "between": [0, 2, 5, 6, 7, 12], "qualiti": [0, 5, 7, 15, 16], "cost": [0, 1, 5, 6, 7, 9, 15, 16], "latenc": [0, 1, 5, 7, 9], "stronger": 0, "model": [0, 1, 4, 5, 6, 7, 8, 9, 11, 12, 14, 15, 16, 18], "ar": [0, 1, 2, 5, 6, 7, 8, 9, 15, 16], "gener": [0, 1, 2, 4, 5, 6, 12, 18], "slower": 0, "more": [0, 2, 5, 9, 12], "expens": 0, "sometim": [0, 6], "overkil": 0, "task": [0, 1, 5, 7, 9], "hand": [0, 9], "complic": 0, "matter": 0, "further": [0, 7, 12], "new": [0, 2, 5, 6, 7, 8, 9], "releas": [0, 5], "weekli": 0, "each": [0, 1, 5, 6, 7, 9, 14, 15, 16], "claim": 0, "state": 0, "art": 0, "data": [0, 1, 7], "let": [0, 1, 2, 7, 8, 9, 15, 16], "you": [0, 1, 2, 4, 5, 6, 7, 8, 9, 15, 16, 18], "see": [0, 6, 7, 9, 15, 16], "differ": [0, 4, 5, 7, 9, 15, 16], "perform": [0, 1, 7, 9, 15, 16], "can": [0, 1, 2, 4, 5, 6, 7, 8, 9, 15, 16, 18], "relat": 0, "live": [0, 5, 6], "stat": 0, "pull": [0, 18], "from": [0, 1, 2, 4, 5, 6, 7, 9, 11, 12, 14, 15, 16, 18], "our": [0, 1, 2, 6, 7, 8, 9, 15, 16, 17, 18], "runtim": [0, 5, 6], "come": [0, 4, 5, 6], "out": [0, 2, 5, 6], "simpli": [0, 2, 9, 15, 16], "re": [0, 7, 9, 15, 16, 18], "run": [0, 2, 6, 7, 8, 11, 18], "thei": [0, 5, 6], "first": [0, 1, 4, 5, 6, 7, 9, 15, 16], "creat": [0, 2, 4, 6, 7, 8], "which": [0, 4, 5, 6, 7, 9, 18], "repres": [0, 11], "want": [0, 1, 2, 7, 8, 9], "evalu": [0, 9, 14], "need": [0, 1, 2, 5, 6, 7, 8, 9, 15, 16], "list": [0, 2, 6, 7, 9, 12, 14], "prompt": [0, 1, 5, 6, 8, 9, 12, 14, 15, 16], "option": [0, 1, 5, 7, 8, 11, 12, 14], "includ": [0, 1, 7, 8, 9], "refer": [0, 5, 8], "gold": 0, "standard": [0, 6, 8], "answer": [0, 5, 8], "contain": [0, 2, 5, 6, 8, 9, 12, 14], "tend": [0, 9], "get": [0, 1, 5, 6, 11, 12, 14, 17], "accur": 0, "The": [0, 2, 4, 5, 6, 7, 8, 9, 11, 12, 15, 16, 18], "file": [0, 2, 5, 8, 14, 18], "itself": 0, "should": [0, 2, 6, 8, 9, 18], "jsonl": [0, 5, 8], "format": [0, 4, 5, 6, 11, 12, 15, 16], "one": [0, 1, 2, 4, 5, 6, 8, 9, 15, 16], "entri": [0, 5], "per": [0, 1, 7], "line": [0, 2], "exampl": [0, 1, 2, 4, 5, 6, 7, 8], "below": [0, 1, 2, 7, 8, 9], "thi": [0, 1, 2, 4, 5, 6, 7, 8, 9, 14, 15, 16, 18], "ref_answ": [0, 5], "second": [0, 1, 7], "us": [0, 4, 5, 6, 7, 8, 9, 11, 12, 14, 15, 16, 18], "least": [0, 9], "50": 0, "most": [0, 5, 8], "result": [0, 5, 7, 8, 9], "current": [0, 1, 4, 6, 7, 9, 15, 16], "an": [0, 1, 5, 6, 7, 8, 11, 12, 14], "maximum": 0, "limit": [0, 2, 5, 6], "500": [0, 13], "we": [0, 1, 2, 5, 6, 7, 8, 9, 15, 16], "don": [0, 4, 7], "t": [0, 1, 2, 4, 5, 7, 9], "much": [0, 1, 5], "extra": 0, "detail": [0, 1, 2, 4, 6, 7, 18], "past": 0, "250": 0, "In": [0, 1, 2, 5, 6, 7, 8, 9, 15, 16], "dashboard": [0, 5, 7, 9], "click": [0, 5, 7, 8, 9], "select": [0, 5, 7, 9], "open": [0, 7, 9], "interfac": [0, 1, 2, 5, 6, 8, 9, 15, 16], "upload": [0, 2, 5, 7, 8, 9, 14], "finish": [0, 5], "ll": [0, 1, 2, 5, 7, 8, 9, 15, 16], "receiv": [0, 5, 7], "email": [0, 5, 7], "graph": [0, 5, 7, 9], "displai": [0, 5, 7, 9], "x": [0, 2, 4, 6, 7, 9, 18], "axi": [0, 9], "set": [0, 1, 2, 6, 7, 8, 11, 12], "time": [0, 1, 5, 6, 7, 9], "token": [0, 1, 5, 6, 7, 9, 12], "inter": [0, 1, 5, 7], "either": [0, 6, 7, 14, 15, 16], "linear": 0, "log": 0, "scale": 0, "gpt4o": 0, "judg": 0, "cf": 0, "http": [0, 2, 4, 12, 13, 14, 18], "arxiv": 0, "org": 0, "ab": 0, "2306": 0, "05685": 0, "": [0, 1, 2, 5, 6, 7, 8, 9, 12, 18], "respons": [0, 1, 2, 4, 5, 6, 12, 14, 18], "section": [1, 2, 7, 8, 9, 15, 16], "learn": [1, 2, 4, 5, 7, 8, 9, 15, 16], "how": [1, 2, 5, 6, 7, 8, 9, 15, 16], "unifi": [1, 2, 4, 7, 8, 11, 12, 13, 14, 18], "through": [1, 2, 6, 7, 8, 9], "api": [1, 5, 7, 8, 11, 12, 14, 15, 16, 18], "If": [1, 2, 4, 6, 7, 8, 9, 11, 12, 14], "haven": [1, 2, 9], "done": [1, 2, 7, 9], "so": [1, 2, 6, 7, 8, 9, 15, 16], "recommend": [1, 2, 9, 15, 16], "request": [1, 4, 7, 9, 13, 15, 16, 18], "familiar": [1, 15, 16], "leverag": 1, "inform": [1, 7, 9], "benchmark": [1, 2, 5, 6, 7, 8, 15, 16], "automat": [1, 6, 7], "rout": [1, 2, 7, 8, 15, 16], "best": [1, 5, 7, 15, 16], "provid": [1, 4, 5, 6, 7, 8, 9, 11, 12, 14, 15, 16], "choos": [1, 2, 5, 7], "valu": [1, 9, 11, 12, 14], "chang": [1, 5, 7, 9, 15, 16], "over": [1, 5, 9], "dynam": 1, "ensur": [1, 7, 9], "alwai": [1, 9], "without": 1, "have": [1, 2, 4, 5, 6, 7, 8, 9, 18], "monitor": 1, "yourself": 1, "To": [1, 2, 4, 5, 6, 9, 17], "onli": [1, 2, 6, 7, 9, 15, 16], "provier": 1, "name": [1, 2, 4, 6, 7, 8, 9, 11, 12, 14], "support": [1, 2, 6, 14], "configur": [1, 2, 5, 7, 9], "follow": [1, 4, 5, 6], "config": 1, "lowest": 1, "input": [1, 4, 15, 16], "output": [1, 6, 9, 12, 15, 16], "itl": 1, "ttft": [1, 9], "highest": 1, "tk": 1, "sec": [1, 9], "For": [1, 2, 5, 7, 8, 9], "e": [1, 2, 5, 7, 9], "g": [1, 2, 7, 9], "python": [1, 18], "packag": 1, "endpoint": [1, 4, 7, 11, 12, 14, 15, 16], "import": [1, 2, 6, 18], "o": [1, 2], "assum": [1, 2, 7, 8], "ad": [1, 2, 9], "unify_kei": [1, 2, 11, 12, 14], "your": [1, 4, 5, 6, 12, 15, 16, 18], "environ": [1, 2, 7, 11, 12, 14, 15, 16], "variabl": [1, 2, 11, 12, 14], "otherwis": [1, 2, 12, 14], "would": [1, 2, 6, 7, 8, 9], "specifi": [1, 2, 4, 5, 6, 7, 8, 9, 14], "api_kei": [1, 2, 6, 11, 12, 14, 18], "argument": [1, 2, 4], "mistral": [1, 2, 6, 9], "7b": [1, 2, 6, 9], "instruct": [1, 2, 6, 9], "v0": [1, 2, 4, 6, 9], "3": [1, 2, 4, 6, 7, 9, 18], "explain": [1, 2, 4, 9, 15, 16, 18], "who": [1, 2, 4, 18], "newton": [1, 2, 4, 18], "wa": [1, 2, 4, 12, 14, 18], "hi": [1, 2, 4, 6, 18], "entir": [1, 2, 4, 9, 18], "theori": [1, 2, 4, 18], "gravit": [1, 2, 4, 18], "give": [1, 2, 4, 15, 16, 18], "long": [1, 2, 4, 18], "pleas": [1, 2, 4, 18], "all": [1, 2, 4, 6, 7, 8, 9, 13, 14, 15, 16, 18], "achiev": [1, 2, 4, 18], "addition": 1, "multipl": 1, "other": [1, 2, 6, 7, 9], "featur": [1, 15, 16], "enabl": [1, 7], "ani": [1, 2, 6, 7, 9, 15, 16], "whose": [1, 14], "lower": [1, 12], "than": [1, 5, 9], "specif": [1, 2, 4, 5, 9, 14], "just": [1, 2, 9], "append": [1, 4], "float": [1, 12], "prefer": 1, "mode": 1, "keep": 1, "thing": 1, "simpl": 1, "alias": 1, "oc": 1, "ic": 1, "ot": 1, "illustr": [1, 15, 16], "some": [1, 2, 6, 7, 9, 15, 16], "0": [1, 2, 4, 6, 7, 12], "5input": 1, "case": [1, 2, 7, 9, 15, 16], "ha": [1, 2, 6, 7], "smaller": [1, 5], "5": 1, "credit": [1, 4, 6, 11, 12], "million": 1, "1output": 1, "likewis": [1, 2, 6], "scenario": 1, "direct": 1, "offer": [1, 17], "1": [1, 5, 12], "5ic": 1, "15itl": 1, "now": [1, 2, 7, 8, 9, 18], "someth": [1, 6, 9], "similar": [1, 5], "alia": 1, "also": [1, 2, 7, 8, 9], "consid": 1, "less": 1, "15": 1, "m": 1, "depend": [1, 7], "might": 1, "where": [1, 2, 7, 9, 12], "meet": [1, 6, 7], "criteria": 1, "render": 1, "unfulfil": 1, "404": [1, 13], "error": [1, 4, 12, 13, 14], "correspond": [1, 2, 4, 7, 9], "explan": 1, "detect": 1, "polici": 1, "do": [1, 2, 8], "like": [1, 2, 6, 7, 8, 9, 12, 15, 16, 18], "won": 1, "work": [1, 7, 15, 16], "sinc": 1, "price": 1, "yet": [1, 4, 7], "001ic": 1, "status_cod": [1, 2], "cheapest": 1, "fallback": 1, "payload": [1, 2, 6], "train": [1, 9], "befor": [1, 2, 7, 9], "id": [1, 2, 4, 6, 7], "same": [1, 2, 5, 7], "call": [1, 7, 17], "code": [1, 2, 6, 15, 16], "send": [1, 4, 9], "gpt": [1, 7], "claud": [1, 7, 18], "llama3": [1, 7], "anthropic_8": [1, 7], "28e": [1, 7], "03_4": [1, 7], "66e": [1, 7], "4_1": [1, 7], "00e": [1, 7], "06": [1, 6, 7], "queri": [1, 4, 5, 7, 8, 15, 16], "curl": [1, 2, 4, 6, 18], "among": 1, "page": [1, 2, 4, 6, 7, 9], "That": [1, 7, 8, 9], "know": [1, 2, 8, 9], "care": 1, "about": [1, 4, 5, 9], "start": [1, 2, 11], "applic": [1, 2, 4, 6, 7, 13, 18], "across": [2, 7, 8, 9], "alreadi": [2, 7, 8], "sign": [2, 4, 6, 15, 16], "consol": [2, 6, 7, 8, 9], "greet": 2, "There": [2, 6, 8, 9], "link": 2, "interact": [2, 4, 6, 11, 12, 15, 16], "suspect": 2, "leak": 2, "wai": [2, 6, 7, 8], "safe": 2, "regener": 2, "replac": [2, 6], "old": 2, "workflow": [2, 6], "balanc": [2, 4, 6, 7], "account": [2, 4, 6, 7, 9, 12], "both": [2, 6, 7], "identifi": 2, "given": [2, 4, 14], "go": [2, 4, 5, 6, 7, 9, 15, 16], "copi": [2, 5, 7], "top": [2, 6, 9], "row": 2, "tabl": [2, 9], "v2": 2, "show": [2, 7, 8, 9, 11], "firework": [2, 6], "ai": [2, 4, 6, 8], "custom": [2, 9, 13, 15, 16], "abl": [2, 5], "easiest": 2, "unifyai": [2, 6, 18], "instal": [2, 6], "pip": [2, 6], "script": 2, "insert": 2, "your_api_kei": [2, 4, 6], "env": [2, 18], "project": 2, "pass": [2, 15, 16], "client": [2, 6, 11, 18], "store": 2, "safeti": 2, "remain": [2, 9, 12], "readi": [2, 7, 8, 15, 16], "method": 2, "abov": 2, "return": [2, 6, 11, 12, 14], "string": [2, 5, 6, 12, 15, 16], "access": [2, 6, 7, 9, 11, 12, 18], "coupl": 2, "list_model": [2, 14], "list_provid": [2, 14], "addit": 2, "synchron": 2, "asynchron": [2, 12], "well": [2, 6, 7, 8], "stream": [2, 4, 6, 12], "check": [2, 9], "repo": 2, "text": [2, 4, 6], "chat": [2, 5, 6, 7, 18], "complet": [2, 5, 6, 7, 9, 18], "wouldn": 2, "normal": 2, "allow": [2, 5, 7, 9], "bypass": 2, "model_nam": [2, 11, 12], "provider_nam": [2, 11, 12], "been": [2, 7], "deploi": [2, 6, 7, 9, 15, 16], "languag": [2, 6], "post": [2, 6, 18], "h": [2, 4, 6, 18], "accept": [2, 4, 6, 18], "json": [2, 4, 6, 12, 14, 18], "author": [2, 4, 6], "bearer": [2, 4, 6], "your_unify_kei": [2, 6], "content": [2, 4, 6, 12, 14, 18], "type": [2, 4, 6, 11, 12, 14, 18], "d": [2, 4, 6, 7, 15, 16, 18], "messag": [2, 4, 6, 12, 18], "role": [2, 4, 6, 18], "user": [2, 4, 6, 7, 12, 18], "true": [2, 6, 12], "Or": 2, "url": [2, 6], "header": [2, 6], "print": [2, 6], "200": [2, 4], "chunk": [2, 6, 12], "iter_cont": 2, "chunk_siz": 2, "1024": [2, 12], "decod": 2, "utf": 2, "8": [2, 12], "els": [2, 9], "doc": 2, "avail": [2, 4, 5, 8, 14, 15, 16], "here": [2, 4, 5, 7], "thank": 2, "easili": 2, "integr": [2, 8], "lot": 2, "switch": 2, "base_url": [2, 6, 18], "sai": [2, 6], "test": [2, 7, 9, 15, 16], "word": 2, "showcas": 2, "function": 2, "choic": [2, 4, 6], "delta": [2, 6], "end": [2, 6], "take": [2, 7], "look": [2, 6, 7, 18], "snippet": 2, "offlin": 2, "api_bas": 2, "order": [2, 9], "insert_model": 2, "next": [2, 5, 6, 7, 8, 9], "lorem": [3, 19, 20], "ipsum": [3, 19, 20], "welcom": 4, "resourc": 4, "up": [4, 6, 12], "kei": [4, 6, 8, 11, 12, 14, 15, 16], "retriev": [4, 11, 12], "authent": 4, "ok": 4, "success": [4, 12, 14], "oper": 4, "associ": [4, 14], "corresponding_user_id": 4, "232": [4, 6], "32": [4, 6], "401": [4, 13], "unauthor": [4, 13], "invalid": 4, "403": [4, 13], "forbidden": [4, 13], "Not": [4, 13], "host": [4, 8], "openai": [4, 8, 11, 12], "sure": 4, "its": [4, 6, 7], "after": [4, 5, 7], "llama": [4, 6, 7, 9], "8b": [4, 6], "anyscal": 4, "fals": [4, 11, 12], "schema": 4, "object": [4, 6, 11, 12], "defin": 4, "1704999905": 4, "meta": 4, "hf": 4, "xr868c": 4, "t4z": 4, "tkltfxxzsvq57wmhxb34el5zuuxsatfu": 4, "usag": [4, 6], "completion_token": [4, 6], "512": 4, "prompt_token": [4, 6], "34": 4, "total_token": [4, 6], "546": 4, "finish_reason": [4, 6], "length": [4, 9], "index": [4, 6], "isaac": 4, "1643": 4, "1727": 4, "assist": [4, 6], "422": [4, 13], "unprocess": [4, 13], "entiti": [4, 13], "flexibl": 5, "handl": [5, 6, 15, 16], "advantag": 5, "sever": [5, 15, 16], "reason": [5, 6], "small": 5, "faster": 5, "cheaper": 5, "wherea": 5, "bigger": 5, "capabl": 5, "often": 5, "rang": 5, "difficulti": 5, "everi": [5, 6], "week": 5, "strength": 5, "weak": 5, "continu": 5, "improv": [5, 15, 16], "ride": 5, "wave": 5, "abil": 5, "minimis": 5, "base": [5, 7, 11, 12, 13], "statist": 5, "purpos": 5, "wide": 5, "varieti": [5, 8], "green": 5, "star": [5, 9], "q": 5, "c": 5, "4": [5, 7, 12], "65e": 5, "03": 5, "2": [5, 12], "08e": 5, "05": 5, "07e": 5, "paramet": [5, 6], "stand": 5, "rel": 5, "weigh": 5, "customiz": 5, "meaning": 5, "directli": 5, "restrict": 5, "them": [5, 7, 9], "model1": 5, "model2": 5, "model3": 5, "provider1": 5, "provider2": 5, "rate": 5, "hit": 5, "move": [5, 7], "mean": [5, 18], "higher": [5, 12], "singl": [5, 6, 12, 15, 16], "ve": [5, 8, 9], "found": [5, 13], "those": [5, 8], "product": [5, 6], "visualis": 5, "form": [5, 6], "xxx": 5, "yyi": 5, "accuraci": 5, "final": [5, 7], "system": [5, 12, 15, 16], "onc": [5, 7, 8, 9, 15, 16, 18], "sea": 6, "own": [6, 8, 15, 16], "subtl": 6, "nuanc": 6, "A": [6, 7, 12, 14], "common": [6, 8], "One": 6, "program": 6, "three": 6, "command": 6, "termin": 6, "unify_api_kei": 6, "hello": 6, "field": 6, "find": [6, 9], "made": 6, "togeth": 6, "1718888877": 6, "896bfc1ae84271aa": 6, "lhr": 6, "25": 6, "13": 6, "38": 6, "7": 6, "6e": 6, "stop": [6, 12], "It": 6, "nice": 6, "help": 6, "seed": 6, "11563975138181362140": 6, "via": [6, 8], "number": [6, 12], "total": [6, 14], "u": [6, 9, 17], "temperatur": [6, 12], "max_token": [6, 12], "mai": [6, 7], "compat": 6, "download": [6, 14], "document": 6, "readm": 6, "sampl": [6, 12], "infer": 6, "design": [6, 9], "exist": 6, "straightforward": 6, "try": 6, "rememb": 6, "const": 6, "baseurl": 6, "apikei": 6, "await": 6, "again": [6, 8], "manag": [6, 15, 16], "spend": 6, "behind": [6, 9], "scene": 6, "refil": 6, "get_credit": 6, "your_user_id": 6, "fine": [6, 7], "tune": [6, 7, 9], "add": [6, 7, 8, 9, 15, 16], "relev": 6, "individu": [6, 15, 16], "outag": 6, "disrupt": 6, "combat": 6, "down": 6, "fail": 6, "until": [6, 15, 16], "succe": 6, "reach": 6, "tag": 6, "becom": 6, "model_a": 6, "provider_a": 6, "model_b": 6, "provider_b": 6, "actual": 6, "went": 6, "window": 7, "dataset": [7, 9, 14], "intend": 7, "earlier": [7, 9], "variant": 7, "dropdown": [7, 9], "notic": [7, 15, 16], "becaus": 7, "doe": 7, "button": [7, 8], "submit": [7, 9], "job": [7, 9], "schedul": 7, "behalf": [7, 8], "ahead": 7, "duplic": 7, "previous": 7, "calls1": [7, 9], "call2": 7, "q1": 7, "q2": 7, "repeat": 7, "howev": 7, "70b": 7, "trigger": [7, 9], "manual": 7, "track": [7, 9], "progress": [7, 9], "With": [7, 15, 16], "stage": 7, "accordingli": 7, "visual": [7, 9], "step": [7, 8, 18], "explor": [7, 9], "variou": [7, 9], "possibl": 7, "trade": 7, "off": [7, 8], "speed": [7, 9, 15, 16], "variat": 7, "These": 7, "As": [7, 9], "point": [7, 9, 15, 16], "plot": [7, 9], "view": [7, 9], "constrain": 7, "search": 7, "space": 7, "certain": 7, "deploy": 7, "guarante": 7, "requir": 7, "Of": 7, "cours": 7, "visibl": 7, "remov": [7, 9, 14], "presum": 7, "anthrop": [7, 18], "properli": 7, "save": [7, 14], "moment": 7, "therefor": 7, "appli": 7, "altern": [7, 8], "had": 7, "overwritten": 7, "clauc": 7, "place": 7, "legend": [7, 9], "underneath": 7, "parent": 7, "slightli": 7, "reduc": 7, "amount": 7, "decid": 7, "stick": 7, "decis": 7, "avoid": 7, "immedi": 7, "futur": 7, "appear": [7, 9], "full": 7, "frequenc": 7, "gpt4": 7, "42": 7, "29": 7, "calls2": [7, 9], "18": 7, "11": 7, "metric": [7, 9], "ax": 7, "verifi": 7, "session": 7, "perman": 7, "veri": 7, "happi": [7, 9], "forget": 7, "delet": [7, 14], "anytim": 7, "press": [7, 9], "besid": 7, "clipboard": 7, "along": 7, "default": [7, 9, 11, 12, 14], "shelf": 8, "azur": 8, "ml": 8, "catalog": 8, "vertex": 8, "garden": 8, "aw": 8, "bedrock": 8, "whole": 8, "major": 8, "cloud": 8, "regardless": 8, "expos": [8, 9], "adher": 8, "got": 8, "said": 8, "screen": 8, "local": [8, 14, 18], "build": 8, "label": [8, 9], "preview": 8, "imag": [8, 9], "router": [8, 9, 15, 16], "navig": 9, "ones": 9, "methodologi": 9, "head": 9, "By": 9, "six": 9, "openherm": 9, "shown": 9, "On": 9, "against": 9, "scatter": 9, "replot": 9, "similarili": 9, "score": 9, "updat": 9, "averag": 9, "granular": 9, "outsid": 9, "popular": 9, "period": 9, "suit": 9, "dedic": 9, "region": 9, "sequenc": [9, 12], "latest": [9, 18], "sort": 9, "icon": 9, "column": 9, "reveal": 9, "pattern": 9, "figur": 9, "fit": 9, "resum": 9, "previou": 9, "privat": 9, "server": [9, 13], "vari": 9, "fastest": 9, "under": 9, "ip": 9, "address": 9, "lock": 9, "indic": 9, "necessari": 9, "subsect": 9, "separ": 9, "expand": 9, "pair": 9, "histori": [9, 11, 12], "mixtral": 9, "financ": 9, "alongsid": 9, "foundat": 9, "class": [11, 12, 13], "chatbot": 11, "none": [11, 12, 14], "agent": 11, "__init__": [11, 12], "initi": [11, 12], "arg": [11, 12, 14], "str": [11, 12, 14], "uploaded_bi": [11, 12], "must": [11, 12, 14], "attempt": [11, 12], "rais": [11, 12, 13, 14], "unifyerror": [11, 12, 13], "miss": [11, 12], "clear_chat_histori": 11, "clear": 11, "properti": [11, 12], "dar201": [11, 12], "noqa": [11, 12], "show_credit": 11, "show_provid": 11, "loop": 11, "bool": [11, 12], "whether": 11, "consumpt": 11, "set_client": 11, "dar101": [11, 12], "set_endpoint": [11, 12], "set_model": [11, 12], "set_provid": [11, 12], "asyncunifi": 12, "async": 12, "user_prompt": 12, "system_prompt": 12, "union": 12, "asyncgener": 12, "dict": [12, 14], "dictionari": [12, 14], "convers": 12, "int": 12, "max": 12, "what": [12, 18], "random": 12, "while": 12, "focus": 12, "determinist": 12, "yield": 12, "occur": 12, "dure": 12, "get_credit_bal": 12, "left": 12, "badrequesterror": [12, 13, 14], "valueerror": [12, 14], "pars": [12, 14], "authenticationerror": 13, "400": 13, "bad": 13, "conflicterror": 13, "409": 13, "conflict": 13, "internalservererror": 13, "intern": 13, "notfounderror": 13, "permissiondeniederror": 13, "ratelimiterror": 13, "429": 13, "too": 13, "mani": 13, "unprocessableentityerror": 13, "delete_dataset": 14, "platform": 14, "path": 14, "info": 14, "msg": 14, "delete_evalu": 14, "download_dataset": 14, "list_dataset": 14, "fetch": 14, "list_endpoint": 14, "empti": 14, "list_evalu": 14, "upload_dataset_from_dictionari": 14, "upload_dataset_from_fil": 14, "mission": [15, 16], "simplifi": [15, 16], "landscap": [15, 16], "No": [15, 16], "eval": [15, 16], "iter": [15, 16], "app": [15, 16], "perfect": [15, 16], "concept": [15, 16], "quick": [15, 16], "read": [15, 16], "guid": [15, 16], "throughout": [15, 16], "mark": [15, 16], "beta": [15, 16], "plan": [15, 16], "constantli": [15, 16], "roadmap": [15, 16], "leav": [15, 16], "feedback": [15, 16], "suggest": [15, 16], "love": [15, 16], "discuss": [15, 16], "book": 17, "prem": 18, "microservic": 18, "grant": 18, "docker": 18, "Then": 18, "dt": 18, "p": 18, "8000": 18, "env_file_path": 18, "deepinfra_api_kei": 18, "stability_api_kei": 18, "mistral_ai_api_kei": 18, "perplexity_ai_api_kei": 18, "anyscale_api_kei": 18, "octoai_api_kei": 18, "openai_api_kei": 18, "together_ai_api_kei": 18, "replicate_api_kei": 18, "lepton_ai_api_kei": 18, "fireworks_ai_api_kei": 18, "anthropic_api_kei": 18, "vertexai_service_acc_json": 18, "vertexai_gcloud_path": 18, "vertexai_project": 18, "vertexai_loc": 18, "servic": 18, "weather": 18, "san": 18, "francisco": 18, "tokyo": 18, "pari": 18, "localhost": 18, "haiku": 18, "random_key_wont_be_us": 18, "life": 18}, "objects": {"unify": [[11, 0, 0, "-", "chat"], [12, 0, 0, "-", "clients"], [13, 0, 0, "-", "exceptions"], [14, 0, 0, "-", "utils"]], "unify.chat": [[11, 1, 1, "", "ChatBot"]], "unify.chat.ChatBot": [[11, 2, 1, "", "__init__"], [11, 2, 1, "", "clear_chat_history"], [11, 3, 1, "", "client"], [11, 3, 1, "", "endpoint"], [11, 3, 1, "", "model"], [11, 3, 1, "", "provider"], [11, 2, 1, "", "run"], [11, 2, 1, "", "set_client"], [11, 2, 1, "", "set_endpoint"], [11, 2, 1, "", "set_model"], [11, 2, 1, "", "set_provider"]], "unify.clients": [[12, 1, 1, "", "AsyncUnify"], [12, 1, 1, "", "Unify"]], "unify.clients.AsyncUnify": [[12, 2, 1, "", "__init__"], [12, 3, 1, "", "endpoint"], [12, 2, 1, "", "generate"], [12, 2, 1, "", "get_credit_balance"], [12, 3, 1, "", "model"], [12, 3, 1, "", "provider"], [12, 2, 1, "", "set_endpoint"], [12, 2, 1, "", "set_model"], [12, 2, 1, "", "set_provider"]], "unify.clients.Unify": [[12, 2, 1, "", "__init__"], [12, 3, 1, "", "endpoint"], [12, 2, 1, "", "generate"], [12, 2, 1, "", "get_credit_balance"], [12, 3, 1, "", "model"], [12, 3, 1, "", "provider"], [12, 2, 1, "", "set_endpoint"], [12, 2, 1, "", "set_model"], [12, 2, 1, "", "set_provider"]], "unify.exceptions": [[13, 4, 1, "", "AuthenticationError"], [13, 4, 1, "", "BadRequestError"], [13, 4, 1, "", "ConflictError"], [13, 4, 1, "", "InternalServerError"], [13, 4, 1, "", "NotFoundError"], [13, 4, 1, "", "PermissionDeniedError"], [13, 4, 1, "", "RateLimitError"], [13, 4, 1, "", "UnifyError"], [13, 4, 1, "", "UnprocessableEntityError"]], "unify.utils": [[14, 5, 1, "", "delete_dataset"], [14, 5, 1, "", "delete_evaluation"], [14, 5, 1, "", "download_dataset"], [14, 5, 1, "", "evaluate"], [14, 5, 1, "", "list_datasets"], [14, 5, 1, "", "list_endpoints"], [14, 5, 1, "", "list_evaluations"], [14, 5, 1, "", "list_models"], [14, 5, 1, "", "list_providers"], [14, 5, 1, "", "upload_dataset_from_dictionary"], [14, 5, 1, "", "upload_dataset_from_file"]]}, "objtypes": {"0": "py:module", "1": "py:class", "2": "py:method", "3": "py:property", "4": "py:exception", "5": "py:function"}, "objnames": {"0": ["py", "module", "Python module"], "1": ["py", "class", "Python class"], "2": ["py", "method", "Python method"], "3": ["py", "property", "Python property"], "4": ["py", "exception", "Python exception"], "5": ["py", "function", "Python function"]}, "titleterms": {"benchmark": [0, 9], "prepar": [0, 5], "your": [0, 2, 7, 8, 9], "dataset": [0, 5, 8], "how": 0, "doe": 0, "work": 0, "deploi": 1, "router": [1, 5, 7], "us": [1, 2], "base": 1, "optim": 1, "metric": 1, "defin": 1, "threshold": 1, "custom": [1, 5, 6, 7, 8], "round": [1, 2, 7, 8, 9], "up": [1, 2, 7, 8, 9], "make": 2, "first": 2, "request": [2, 6], "get": [2, 4, 15, 16], "kei": 2, "find": 2, "model": 2, "provid": 2, "queri": [2, 6], "an": 2, "endpoint": [2, 5, 6, 8, 9], "python": [2, 6, 20], "packag": [2, 6], "openai": [2, 6], "api": [2, 4, 6], "format": 2, "compat": 2, "tool": 2, "sdk": 2, "open": 2, "interpret": 2, "On": [3, 17, 18], "prem": [3, 17], "imag": 3, "refer": 4, "get_credit": 4, "post": 4, "chat": [4, 11], "complet": 4, "rout": 5, "what": 5, "i": 5, "llm": [5, 6], "foundat": 5, "filter": 5, "maximis": 5, "throughput": 5, "fine": 5, "tune": 5, "train": [5, 7], "beta": [5, 7, 8, 9], "univers": 6, "authent": 6, "http": 6, "unifi": [6, 15, 16], "nodej": 6, "bill": 6, "advanc": 6, "featur": 6, "fallback": 6, "build": 7, "connect": 8, "stack": 8, "prerequisit": 8, "ad": 8, "qualiti": 9, "runtim": 9, "run": 9, "own": 9, "client": 12, "except": 13, "util": 14, "welcom": [15, 16], "start": [15, 16], "access": 17, "sso": 18, "singl": 18, "sign": 18, "openapi": 19, "specif": 19, "librari": 20}, "envversion": {"sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "nbsphinx": 4, "sphinx": 60}, "alltitles": {"On-Prem access": [[17, "on-prem-access"]], "OpenAPI Specification": [[19, "openapi-specification"]], "SSO (Single Sign On)": [[18, "sso-single-sign-on"]], "Welcome to Unify!": [[15, "welcome-to-unify"], [16, "welcome-to-unify"]], "Getting Started": [[15, "getting-started"], [16, "getting-started"]], "API Reference": [[4, "api-reference"]], "GET /get_credits": [[4, "get-get-credits"]], "POST /chat/completions": [[4, "post-chat-completions"]], "Deploying a router": [[1, "deploying-a-router"]], "Using the base router": [[1, "using-the-base-router"]], "Optimizing a metric": [[1, "optimizing-a-metric"]], "Defining thresholds": [[1, "defining-thresholds"]], "Using a custom router": [[1, "using-a-custom-router"]], "Round Up": [[1, "round-up"], [2, "round-up"], [9, "round-up"], [8, "round-up"], [7, "round-up"]], "Making your first request": [[2, "making-your-first-request"]], "Getting a key": [[2, "getting-a-key"]], "Finding a model and provider": [[2, "finding-a-model-and-provider"]], "Querying an endpoint": [[2, "querying-an-endpoint"]], "Using the Python Package": [[2, "using-the-python-package"]], "Using the OpenAI API Format": [[2, "using-the-openai-api-format"]], "Compatible Tools": [[2, "compatible-tools"]], "OpenAI SDK": [[2, "openai-sdk"]], "Open Interpreter": [[2, "open-interpreter"]], "On-Prem Images": [[3, "on-prem-images"]], "Benchmarking": [[0, "benchmarking"]], "Preparing your dataset": [[0, "preparing-your-dataset"]], "Benchmarking your dataset": [[0, "benchmarking-your-dataset"]], "How does it work?": [[0, "how-does-it-work"]], "Benchmarking endpoints": [[9, "benchmarking-endpoints"]], "Quality benchmarks": [[9, "quality-benchmarks"], [9, "id4"]], "Runtime benchmarks": [[9, "runtime-benchmarks"], [9, "id2"]], "Running your own benchmarks (Beta)": [[9, "running-your-own-benchmarks-beta"]], "Connecting your stack": [[8, "connecting-your-stack"]], "Custom endpoints (Beta)": [[8, "custom-endpoints-beta"]], "Prerequisite": [[8, "prerequisite"]], "Adding the endpoints": [[8, "adding-the-endpoints"]], "Custom datasets": [[8, "custom-datasets"]], "Building a custom router": [[7, "building-a-custom-router"]], "Training a custom router (Beta)": [[7, "training-a-custom-router-beta"]], "Customizing your router (Beta)": [[7, "customizing-your-router-beta"]], "Universal API": [[6, "universal-api"]], "Authentication": [[6, "authentication"]], "Querying the API": [[6, "querying-the-api"]], "HTTP Request": [[6, "http-request"]], "Unify Python Package": [[6, "unify-python-package"]], "OpenAI Python Package": [[6, "openai-python-package"]], "OpenAI NodeJS Package": [[6, "openai-nodejs-package"]], "Billing": [[6, "billing"]], "Advanced features": [[6, "advanced-features"]], "Custom endpoints": [[6, "custom-endpoints"]], "LLM Fallbacks": [[6, "llm-fallbacks"]], "Routing": [[5, "routing"]], "What is LLM routing?": [[5, "what-is-llm-routing"]], "Foundation router": [[5, "foundation-router"]], "Filtering endpoints": [[5, "filtering-endpoints"]], "Maximising throughput": [[5, "maximising-throughput"]], "Fine-tuned custom router": [[5, "fine-tuned-custom-router"]], "Preparing a dataset": [[5, "preparing-a-dataset"]], "Training a router (beta)": [[5, "training-a-router-beta"]], "Python Library": [[20, "python-library"]], "Exceptions": [[13, "module-unify.exceptions"]], "Utils": [[14, "module-unify.utils"]], "Chat": [[11, "module-unify.chat"]], "Clients": [[12, "module-unify.clients"]]}, "indexentries": {"chatbot (class in unify.chat)": [[11, "unify.chat.ChatBot"]], "__init__() (unify.chat.chatbot method)": [[11, "unify.chat.ChatBot.__init__"]], "clear_chat_history() (unify.chat.chatbot method)": [[11, "unify.chat.ChatBot.clear_chat_history"]], "client (unify.chat.chatbot property)": [[11, "unify.chat.ChatBot.client"]], "endpoint (unify.chat.chatbot property)": [[11, "unify.chat.ChatBot.endpoint"]], "model (unify.chat.chatbot property)": [[11, "unify.chat.ChatBot.model"]], "module": [[11, "module-unify.chat"], [12, "module-unify.clients"], [13, "module-unify.exceptions"], [14, "module-unify.utils"]], "provider (unify.chat.chatbot property)": [[11, "unify.chat.ChatBot.provider"]], "run() (unify.chat.chatbot method)": [[11, "unify.chat.ChatBot.run"]], "set_client() (unify.chat.chatbot method)": [[11, "unify.chat.ChatBot.set_client"]], "set_endpoint() (unify.chat.chatbot method)": [[11, "unify.chat.ChatBot.set_endpoint"]], "set_model() (unify.chat.chatbot method)": [[11, "unify.chat.ChatBot.set_model"]], "set_provider() (unify.chat.chatbot method)": [[11, "unify.chat.ChatBot.set_provider"]], "unify.chat": [[11, "module-unify.chat"]], "asyncunify (class in unify.clients)": [[12, "unify.clients.AsyncUnify"]], "unify (class in unify.clients)": [[12, "unify.clients.Unify"]], "__init__() (unify.clients.asyncunify method)": [[12, "unify.clients.AsyncUnify.__init__"]], "__init__() (unify.clients.unify method)": [[12, "unify.clients.Unify.__init__"]], "endpoint (unify.clients.asyncunify property)": [[12, "unify.clients.AsyncUnify.endpoint"]], "endpoint (unify.clients.unify property)": [[12, "unify.clients.Unify.endpoint"]], "generate() (unify.clients.asyncunify method)": [[12, "unify.clients.AsyncUnify.generate"]], "generate() (unify.clients.unify method)": [[12, "unify.clients.Unify.generate"]], "get_credit_balance() (unify.clients.asyncunify method)": [[12, "unify.clients.AsyncUnify.get_credit_balance"]], "get_credit_balance() (unify.clients.unify method)": [[12, "unify.clients.Unify.get_credit_balance"]], "model (unify.clients.asyncunify property)": [[12, "unify.clients.AsyncUnify.model"]], "model (unify.clients.unify property)": [[12, "unify.clients.Unify.model"]], "provider (unify.clients.asyncunify property)": [[12, "unify.clients.AsyncUnify.provider"]], "provider (unify.clients.unify property)": [[12, "unify.clients.Unify.provider"]], "set_endpoint() (unify.clients.asyncunify method)": [[12, "unify.clients.AsyncUnify.set_endpoint"]], "set_endpoint() (unify.clients.unify method)": [[12, "unify.clients.Unify.set_endpoint"]], "set_model() (unify.clients.asyncunify method)": [[12, "unify.clients.AsyncUnify.set_model"]], "set_model() (unify.clients.unify method)": [[12, "unify.clients.Unify.set_model"]], "set_provider() (unify.clients.asyncunify method)": [[12, "unify.clients.AsyncUnify.set_provider"]], "set_provider() (unify.clients.unify method)": [[12, "unify.clients.Unify.set_provider"]], "unify.clients": [[12, "module-unify.clients"]], "authenticationerror": [[13, "unify.exceptions.AuthenticationError"]], "badrequesterror": [[13, "unify.exceptions.BadRequestError"]], "conflicterror": [[13, "unify.exceptions.ConflictError"]], "internalservererror": [[13, "unify.exceptions.InternalServerError"]], "notfounderror": [[13, "unify.exceptions.NotFoundError"]], "permissiondeniederror": [[13, "unify.exceptions.PermissionDeniedError"]], "ratelimiterror": [[13, "unify.exceptions.RateLimitError"]], "unifyerror": [[13, "unify.exceptions.UnifyError"]], "unprocessableentityerror": [[13, "unify.exceptions.UnprocessableEntityError"]], "unify.exceptions": [[13, "module-unify.exceptions"]], "delete_dataset() (in module unify.utils)": [[14, "unify.utils.delete_dataset"]], "delete_evaluation() (in module unify.utils)": [[14, "unify.utils.delete_evaluation"]], "download_dataset() (in module unify.utils)": [[14, "unify.utils.download_dataset"]], "evaluate() (in module unify.utils)": [[14, "unify.utils.evaluate"]], "list_datasets() (in module unify.utils)": [[14, "unify.utils.list_datasets"]], "list_endpoints() (in module unify.utils)": [[14, "unify.utils.list_endpoints"]], "list_evaluations() (in module unify.utils)": [[14, "unify.utils.list_evaluations"]], "list_models() (in module unify.utils)": [[14, "unify.utils.list_models"]], "list_providers() (in module unify.utils)": [[14, "unify.utils.list_providers"]], "unify.utils": [[14, "module-unify.utils"]], "upload_dataset_from_dictionary() (in module unify.utils)": [[14, "unify.utils.upload_dataset_from_dictionary"]], "upload_dataset_from_file() (in module unify.utils)": [[14, "unify.utils.upload_dataset_from_file"]]}}) \ No newline at end of file +Search.setIndex({"docnames": ["concepts/benchmarks", "concepts/deploy_router", "concepts/first_request", "concepts/images", "concepts/reference", "concepts/router", "concepts/unify_api", "console/building_router", "console/connecting_stack", "console/running_benchmarks", "docs/unify/unify", "docs/unify/unify/unify.chat", "docs/unify/unify/unify.clients", "docs/unify/unify/unify.exceptions", "docs/unify/unify/unify.utils", "home/home", "index", "on_prem/on_prem_access", "on_prem/sso", "tools/openapi", "tools/python_library"], "filenames": ["concepts/benchmarks.rst", "concepts/deploy_router.rst", "concepts/first_request.rst", "concepts/images.rst", "concepts/reference.rst", "concepts/router.rst", "concepts/unify_api.rst", "console/building_router.rst", "console/connecting_stack.rst", "console/running_benchmarks.rst", "docs/unify/unify.rst", "docs/unify/unify/unify.chat.rst", "docs/unify/unify/unify.clients.rst", "docs/unify/unify/unify.exceptions.rst", "docs/unify/unify/unify.utils.rst", "home/home.rst", "index.rst", "on_prem/on_prem_access.rst", "on_prem/sso.rst", "tools/openapi.rst", "tools/python_library.rst"], "titles": ["Benchmarking", "Deploying a router", "Making your first request", "On-Prem Images", "API Reference", "Routing", "Universal API", "Building a custom router", "Connecting your stack", "Benchmarking endpoints", "<no title>", "Chat", "Clients", "Exceptions", "Utils", "Welcome to Unify!", "Home", "On-Prem access", "SSO (Single Sign On)", "OpenAPI Specification", "Python Library"], "terms": {"when": [0, 1, 2, 4, 5, 7, 12], "compar": [0, 9, 15, 16], "llm": [0, 1, 2, 8, 9, 11, 15, 16], "i": [0, 1, 2, 4, 6, 7, 8, 9, 11, 12, 14, 18], "constant": 0, "tradeoff": 0, "make": [0, 1, 4, 7, 12], "between": [0, 2, 5, 6, 7, 12], "qualiti": [0, 5, 7, 15, 16], "cost": [0, 1, 5, 6, 7, 9, 15, 16], "latenc": [0, 1, 5, 7, 9], "stronger": 0, "model": [0, 1, 4, 5, 6, 7, 8, 9, 11, 12, 14, 15, 16, 18], "ar": [0, 1, 2, 5, 6, 7, 8, 9], "gener": [0, 1, 2, 4, 5, 6, 12, 15, 16, 18], "slower": 0, "more": [0, 2, 5, 9, 12], "expens": 0, "sometim": [0, 6], "overkil": 0, "task": [0, 1, 5, 7, 9], "hand": [0, 9], "complic": 0, "matter": 0, "further": [0, 7, 12], "new": [0, 2, 5, 6, 7, 8, 9], "releas": [0, 5], "weekli": 0, "each": [0, 1, 5, 6, 7, 9, 14, 15, 16], "claim": 0, "state": 0, "art": 0, "data": [0, 1, 7], "let": [0, 1, 2, 7, 8, 9, 15, 16], "you": [0, 1, 2, 4, 5, 6, 7, 8, 9, 15, 16, 18], "see": [0, 6, 7, 9], "differ": [0, 4, 5, 7, 9, 15, 16], "perform": [0, 1, 7, 9, 15, 16], "can": [0, 1, 2, 4, 5, 6, 7, 8, 9, 15, 16, 18], "relat": 0, "live": [0, 5, 6], "stat": 0, "pull": [0, 18], "from": [0, 1, 2, 4, 5, 6, 7, 9, 11, 12, 14, 15, 16, 18], "our": [0, 1, 2, 6, 7, 8, 9, 15, 16, 17, 18], "runtim": [0, 5, 6], "come": [0, 4, 5, 6], "out": [0, 2, 5, 6], "simpli": [0, 2, 9, 15, 16], "re": [0, 7, 9, 15, 16, 18], "run": [0, 2, 6, 7, 8, 11, 18], "thei": [0, 5, 6], "first": [0, 1, 4, 5, 6, 7, 9], "creat": [0, 2, 4, 6, 7, 8], "which": [0, 4, 5, 6, 7, 9, 15, 16, 18], "repres": [0, 11], "want": [0, 1, 2, 7, 8, 9], "evalu": [0, 9, 14], "need": [0, 1, 2, 5, 6, 7, 8, 9, 15, 16], "list": [0, 2, 6, 7, 9, 12, 14, 15, 16], "prompt": [0, 1, 5, 6, 8, 9, 12, 14, 15, 16], "option": [0, 1, 5, 7, 8, 11, 12, 14], "includ": [0, 1, 7, 8, 9], "refer": [0, 5, 8], "gold": 0, "standard": [0, 6, 8], "answer": [0, 5, 8], "contain": [0, 2, 5, 6, 8, 9, 12, 14], "tend": [0, 9], "get": [0, 1, 5, 6, 11, 12, 14, 17], "accur": 0, "The": [0, 2, 4, 5, 6, 7, 8, 9, 11, 12, 18], "file": [0, 2, 5, 8, 14, 18], "itself": 0, "should": [0, 2, 6, 8, 9, 18], "jsonl": [0, 5, 8], "format": [0, 4, 5, 6, 11, 12, 15, 16], "one": [0, 1, 2, 4, 5, 6, 8, 9, 15, 16], "entri": [0, 5], "per": [0, 1, 7], "line": [0, 2], "exampl": [0, 1, 2, 4, 5, 6, 7, 8], "below": [0, 1, 2, 7, 8, 9], "thi": [0, 1, 2, 4, 5, 6, 7, 8, 9, 14, 18], "ref_answ": [0, 5], "second": [0, 1, 7], "us": [0, 4, 5, 6, 7, 8, 9, 11, 12, 14, 15, 16, 18], "least": [0, 9], "50": 0, "most": [0, 5, 8], "result": [0, 5, 7, 8, 9], "current": [0, 1, 4, 6, 7, 9], "an": [0, 1, 5, 6, 7, 8, 11, 12, 14], "maximum": 0, "limit": [0, 2, 5, 6], "500": [0, 13], "we": [0, 1, 2, 5, 6, 7, 8, 9, 15, 16], "don": [0, 4, 7], "t": [0, 1, 2, 4, 5, 7, 9], "much": [0, 1, 5], "extra": 0, "detail": [0, 1, 2, 4, 6, 7, 18], "past": 0, "250": 0, "In": [0, 1, 2, 5, 6, 7, 8, 9], "dashboard": [0, 5, 7, 9], "click": [0, 5, 7, 8, 9], "select": [0, 5, 7, 9], "open": [0, 7, 9], "interfac": [0, 1, 2, 5, 6, 8, 9, 15, 16], "upload": [0, 2, 5, 7, 8, 9, 14], "finish": [0, 5], "ll": [0, 1, 2, 5, 7, 8, 9], "receiv": [0, 5, 7], "email": [0, 5, 7], "graph": [0, 5, 7, 9], "displai": [0, 5, 7, 9], "x": [0, 2, 4, 6, 7, 9, 18], "axi": [0, 9], "set": [0, 1, 2, 6, 7, 8, 11, 12], "time": [0, 1, 5, 6, 7, 9], "token": [0, 1, 5, 6, 7, 9, 12], "inter": [0, 1, 5, 7], "either": [0, 6, 7, 14], "linear": 0, "log": 0, "scale": 0, "gpt4o": 0, "judg": 0, "cf": 0, "http": [0, 2, 4, 12, 13, 14, 18], "arxiv": 0, "org": 0, "ab": 0, "2306": 0, "05685": 0, "": [0, 1, 2, 5, 6, 7, 8, 9, 12, 15, 16, 18], "respons": [0, 1, 2, 4, 5, 6, 12, 14, 18], "section": [1, 2, 7, 8, 9], "learn": [1, 2, 4, 5, 7, 8, 9], "how": [1, 2, 5, 6, 7, 8, 9], "unifi": [1, 2, 4, 7, 8, 11, 12, 13, 14, 18], "through": [1, 2, 6, 7, 8, 9], "api": [1, 5, 7, 8, 11, 12, 14, 15, 16, 18], "If": [1, 2, 4, 6, 7, 8, 9, 11, 12, 14], "haven": [1, 2, 9], "done": [1, 2, 7, 9], "so": [1, 2, 6, 7, 8, 9, 15, 16], "recommend": [1, 2, 9], "request": [1, 4, 7, 9, 13, 18], "familiar": 1, "leverag": 1, "inform": [1, 7, 9], "benchmark": [1, 2, 5, 6, 7, 8, 15, 16], "automat": [1, 6, 7], "rout": [1, 2, 7, 8, 15, 16], "best": [1, 5, 7, 15, 16], "provid": [1, 4, 5, 6, 7, 8, 9, 11, 12, 14, 15, 16], "choos": [1, 2, 5, 7], "valu": [1, 9, 11, 12, 14], "chang": [1, 5, 7, 9, 15, 16], "over": [1, 5, 9], "dynam": 1, "ensur": [1, 7, 9], "alwai": [1, 9], "without": 1, "have": [1, 2, 4, 5, 6, 7, 8, 9, 15, 16, 18], "monitor": 1, "yourself": 1, "To": [1, 2, 4, 5, 6, 9, 17], "onli": [1, 2, 6, 7, 9], "provier": 1, "name": [1, 2, 4, 6, 7, 8, 9, 11, 12, 14], "support": [1, 2, 6, 14], "configur": [1, 2, 5, 7, 9], "follow": [1, 4, 5, 6], "config": 1, "lowest": 1, "input": [1, 4, 15, 16], "output": [1, 6, 9, 12, 15, 16], "itl": 1, "ttft": [1, 9], "highest": 1, "tk": 1, "sec": [1, 9], "For": [1, 2, 5, 7, 8, 9], "e": [1, 2, 5, 7, 9], "g": [1, 2, 7, 9], "python": [1, 15, 16, 18], "packag": [1, 15, 16], "endpoint": [1, 4, 7, 11, 12, 14, 15, 16], "import": [1, 2, 6, 15, 16, 18], "o": [1, 2], "assum": [1, 2, 7, 8], "ad": [1, 2, 9], "unify_kei": [1, 2, 11, 12, 14, 15, 16], "your": [1, 4, 5, 6, 12, 15, 16, 18], "environ": [1, 2, 7, 11, 12, 14, 15, 16], "variabl": [1, 2, 11, 12, 14, 15, 16], "otherwis": [1, 2, 12, 14], "would": [1, 2, 6, 7, 8, 9], "specifi": [1, 2, 4, 5, 6, 7, 8, 9, 14], "api_kei": [1, 2, 6, 11, 12, 14, 15, 16, 18], "argument": [1, 2, 4, 15, 16], "mistral": [1, 2, 6, 9], "7b": [1, 2, 6, 9], "instruct": [1, 2, 6, 9], "v0": [1, 2, 4, 6, 9], "3": [1, 2, 4, 6, 7, 9, 18], "explain": [1, 2, 4, 9, 18], "who": [1, 2, 4, 18], "newton": [1, 2, 4, 18], "wa": [1, 2, 4, 12, 14, 18], "hi": [1, 2, 4, 6, 18], "entir": [1, 2, 4, 9, 18], "theori": [1, 2, 4, 18], "gravit": [1, 2, 4, 18], "give": [1, 2, 4, 18], "long": [1, 2, 4, 18], "pleas": [1, 2, 4, 18], "all": [1, 2, 4, 6, 7, 8, 9, 13, 14, 15, 16, 18], "achiev": [1, 2, 4, 18], "addition": 1, "multipl": 1, "other": [1, 2, 6, 7, 9], "featur": 1, "enabl": [1, 7], "ani": [1, 2, 6, 7, 9, 15, 16], "whose": [1, 14], "lower": [1, 12], "than": [1, 5, 9], "specif": [1, 2, 4, 5, 9, 14], "just": [1, 2, 9], "append": [1, 4], "float": [1, 12], "prefer": 1, "mode": 1, "keep": 1, "thing": 1, "simpl": 1, "alias": 1, "oc": 1, "ic": 1, "ot": 1, "illustr": 1, "some": [1, 2, 6, 7, 9], "0": [1, 2, 4, 6, 7, 12], "5input": 1, "case": [1, 2, 7, 9, 15, 16], "ha": [1, 2, 6, 7], "smaller": [1, 5], "5": 1, "credit": [1, 4, 6, 11, 12], "million": 1, "1output": 1, "likewis": [1, 2, 6], "scenario": 1, "direct": 1, "offer": [1, 17], "1": [1, 5, 12], "5ic": 1, "15itl": 1, "now": [1, 2, 7, 8, 9, 15, 16, 18], "someth": [1, 6, 9], "similar": [1, 5], "alia": 1, "also": [1, 2, 7, 8, 9, 15, 16], "consid": 1, "less": 1, "15": 1, "m": 1, "depend": [1, 7], "might": 1, "where": [1, 2, 7, 9, 12], "meet": [1, 6, 7], "criteria": 1, "render": 1, "unfulfil": 1, "404": [1, 13], "error": [1, 4, 12, 13, 14], "correspond": [1, 2, 4, 7, 9], "explan": 1, "detect": 1, "polici": 1, "do": [1, 2, 8], "like": [1, 2, 6, 7, 8, 9, 12, 15, 16, 18], "won": 1, "work": [1, 7], "sinc": 1, "price": 1, "yet": [1, 4, 7], "001ic": 1, "status_cod": [1, 2], "cheapest": 1, "fallback": 1, "payload": [1, 2, 6], "train": [1, 9], "befor": [1, 2, 7, 9], "id": [1, 2, 4, 6, 7], "same": [1, 2, 5, 7], "call": [1, 7, 17], "code": [1, 2, 6], "send": [1, 4, 9], "gpt": [1, 7, 15, 16], "claud": [1, 7, 18], "llama3": [1, 7], "anthropic_8": [1, 7], "28e": [1, 7], "03_4": [1, 7], "66e": [1, 7], "4_1": [1, 7], "00e": [1, 7], "06": [1, 6, 7], "queri": [1, 4, 5, 7, 8], "curl": [1, 2, 4, 6, 18], "among": 1, "page": [1, 2, 4, 6, 7, 9], "That": [1, 7, 8, 9, 15, 16], "know": [1, 2, 8, 9], "care": 1, "about": [1, 4, 5, 9], "start": [1, 2, 11], "applic": [1, 2, 4, 6, 7, 13, 18], "across": [2, 7, 8, 9], "alreadi": [2, 7, 8], "sign": [2, 4, 6, 15, 16], "consol": [2, 6, 7, 8, 9], "greet": 2, "There": [2, 6, 8, 9], "link": 2, "interact": [2, 4, 6, 11, 12], "suspect": 2, "leak": 2, "wai": [2, 6, 7, 8], "safe": 2, "regener": 2, "replac": [2, 6], "old": 2, "workflow": [2, 6], "balanc": [2, 4, 6, 7], "account": [2, 4, 6, 7, 9, 12], "both": [2, 6, 7], "identifi": 2, "given": [2, 4, 14], "go": [2, 4, 5, 6, 7, 9, 15, 16], "copi": [2, 5, 7], "top": [2, 6, 9], "row": 2, "tabl": [2, 9], "v2": 2, "show": [2, 7, 8, 9, 11], "firework": [2, 6], "ai": [2, 4, 6, 8], "custom": [2, 9, 13, 15, 16], "abl": [2, 5], "easiest": [2, 15, 16], "unifyai": [2, 6, 15, 16, 18], "instal": [2, 6, 15, 16], "pip": [2, 6, 15, 16], "script": 2, "insert": 2, "your_api_kei": [2, 4, 6], "env": [2, 18], "project": 2, "pass": [2, 15, 16], "client": [2, 6, 11, 15, 16, 18], "store": 2, "safeti": 2, "remain": [2, 9, 12], "readi": [2, 7, 8, 15, 16], "method": 2, "abov": 2, "return": [2, 6, 11, 12, 14], "string": [2, 5, 6, 12, 15, 16], "access": [2, 6, 7, 9, 11, 12, 18], "coupl": 2, "list_model": [2, 14], "list_provid": [2, 14], "addit": 2, "synchron": 2, "asynchron": [2, 12], "well": [2, 6, 7, 8], "stream": [2, 4, 6, 12], "check": [2, 9], "repo": 2, "text": [2, 4, 6], "chat": [2, 5, 6, 7, 18], "complet": [2, 5, 6, 7, 9, 18], "wouldn": 2, "normal": 2, "allow": [2, 5, 7, 9], "bypass": 2, "model_nam": [2, 11, 12], "provider_nam": [2, 11, 12], "been": [2, 7], "deploi": [2, 6, 7, 9, 15, 16], "languag": [2, 6], "post": [2, 6, 18], "h": [2, 4, 6, 18], "accept": [2, 4, 6, 18], "json": [2, 4, 6, 12, 14, 18], "author": [2, 4, 6], "bearer": [2, 4, 6], "your_unify_kei": [2, 6], "content": [2, 4, 6, 12, 14, 18], "type": [2, 4, 6, 11, 12, 14, 18], "d": [2, 4, 6, 7, 18], "messag": [2, 4, 6, 12, 18], "role": [2, 4, 6, 18], "user": [2, 4, 6, 7, 12, 18], "true": [2, 6, 12], "Or": 2, "url": [2, 6], "header": [2, 6], "print": [2, 6], "200": [2, 4], "chunk": [2, 6, 12], "iter_cont": 2, "chunk_siz": 2, "1024": [2, 12], "decod": 2, "utf": 2, "8": [2, 12], "els": [2, 9], "doc": 2, "avail": [2, 4, 5, 8, 14, 15, 16], "here": [2, 4, 5, 7], "thank": 2, "easili": 2, "integr": [2, 8], "lot": 2, "switch": 2, "base_url": [2, 6, 18], "sai": [2, 6], "test": [2, 7, 9, 15, 16], "word": 2, "showcas": 2, "function": 2, "choic": [2, 4, 6, 15, 16], "delta": [2, 6], "end": [2, 6], "take": [2, 7], "look": [2, 6, 7, 18], "snippet": 2, "offlin": 2, "api_bas": 2, "order": [2, 9], "insert_model": 2, "next": [2, 5, 6, 7, 8, 9], "lorem": [3, 19, 20], "ipsum": [3, 19, 20], "welcom": 4, "resourc": 4, "up": [4, 6, 12, 15, 16], "kei": [4, 6, 8, 11, 12, 14, 15, 16], "retriev": [4, 11, 12], "authent": 4, "ok": 4, "success": [4, 12, 14], "oper": 4, "associ": [4, 14], "corresponding_user_id": 4, "232": [4, 6], "32": [4, 6], "401": [4, 13], "unauthor": [4, 13], "invalid": 4, "403": [4, 13], "forbidden": [4, 13], "Not": [4, 13], "host": [4, 8], "openai": [4, 8, 11, 12, 15, 16], "sure": 4, "its": [4, 6, 7], "after": [4, 5, 7], "llama": [4, 6, 7, 9], "8b": [4, 6], "anyscal": 4, "fals": [4, 11, 12], "schema": 4, "object": [4, 6, 11, 12], "defin": 4, "1704999905": 4, "meta": 4, "hf": 4, "xr868c": 4, "t4z": 4, "tkltfxxzsvq57wmhxb34el5zuuxsatfu": 4, "usag": [4, 6], "completion_token": [4, 6], "512": 4, "prompt_token": [4, 6], "34": 4, "total_token": [4, 6], "546": 4, "finish_reason": [4, 6], "length": [4, 9], "index": [4, 6], "isaac": 4, "1643": 4, "1727": 4, "assist": [4, 6], "422": [4, 13], "unprocess": [4, 13], "entiti": [4, 13], "flexibl": 5, "handl": [5, 6, 15, 16], "advantag": 5, "sever": [5, 15, 16], "reason": [5, 6], "small": 5, "faster": 5, "cheaper": 5, "wherea": 5, "bigger": 5, "capabl": 5, "often": 5, "rang": 5, "difficulti": 5, "everi": [5, 6], "week": 5, "strength": 5, "weak": 5, "continu": 5, "improv": [5, 15, 16], "ride": 5, "wave": 5, "abil": 5, "minimis": 5, "base": [5, 7, 11, 12, 13], "statist": 5, "purpos": 5, "wide": 5, "varieti": [5, 8], "green": 5, "star": [5, 9], "q": 5, "c": 5, "4": [5, 7, 12], "65e": 5, "03": 5, "2": [5, 12], "08e": 5, "05": 5, "07e": 5, "paramet": [5, 6], "stand": 5, "rel": 5, "weigh": 5, "customiz": 5, "meaning": 5, "directli": 5, "restrict": 5, "them": [5, 7, 9], "model1": 5, "model2": 5, "model3": 5, "provider1": 5, "provider2": 5, "rate": 5, "hit": 5, "move": [5, 7], "mean": [5, 18], "higher": [5, 12], "singl": [5, 6, 12, 15, 16], "ve": [5, 8, 9], "found": [5, 13], "those": [5, 8], "product": [5, 6], "visualis": 5, "form": [5, 6], "xxx": 5, "yyi": 5, "accuraci": 5, "final": [5, 7], "system": [5, 12, 15, 16], "onc": [5, 7, 8, 9, 18], "sea": 6, "own": [6, 8, 15, 16], "subtl": 6, "nuanc": 6, "A": [6, 7, 12, 14], "common": [6, 8], "One": 6, "program": 6, "three": 6, "command": 6, "termin": 6, "unify_api_kei": 6, "hello": [6, 15, 16], "field": 6, "find": [6, 9], "made": 6, "togeth": 6, "1718888877": 6, "896bfc1ae84271aa": 6, "lhr": 6, "25": 6, "13": 6, "38": 6, "7": 6, "6e": 6, "stop": [6, 12], "It": [6, 15, 16], "nice": 6, "help": 6, "seed": 6, "11563975138181362140": 6, "via": [6, 8], "number": [6, 12], "total": [6, 14], "u": [6, 9, 17], "temperatur": [6, 12], "max_token": [6, 12], "mai": [6, 7], "compat": 6, "download": [6, 14], "document": 6, "readm": 6, "sampl": [6, 12], "infer": 6, "design": [6, 9], "exist": 6, "straightforward": 6, "try": 6, "rememb": 6, "const": 6, "baseurl": 6, "apikei": 6, "await": 6, "again": [6, 8], "manag": [6, 15, 16], "spend": 6, "behind": [6, 9], "scene": 6, "refil": 6, "get_credit": 6, "your_user_id": 6, "fine": [6, 7], "tune": [6, 7, 9], "add": [6, 7, 8, 9, 15, 16], "relev": 6, "individu": [6, 15, 16], "outag": 6, "disrupt": 6, "combat": 6, "down": 6, "fail": 6, "until": [6, 15, 16], "succe": 6, "reach": 6, "tag": 6, "becom": 6, "model_a": 6, "provider_a": 6, "model_b": 6, "provider_b": 6, "actual": 6, "went": 6, "window": 7, "dataset": [7, 9, 14], "intend": 7, "earlier": [7, 9], "variant": 7, "dropdown": [7, 9], "notic": 7, "becaus": 7, "doe": 7, "button": [7, 8], "submit": [7, 9], "job": [7, 9], "schedul": 7, "behalf": [7, 8], "ahead": 7, "duplic": 7, "previous": 7, "calls1": [7, 9], "call2": 7, "q1": 7, "q2": 7, "repeat": 7, "howev": 7, "70b": 7, "trigger": [7, 9], "manual": 7, "track": [7, 9], "progress": [7, 9], "With": [7, 15, 16], "stage": 7, "accordingli": 7, "visual": [7, 9], "step": [7, 8, 18], "explor": [7, 9], "variou": [7, 9], "possibl": 7, "trade": 7, "off": [7, 8], "speed": [7, 9, 15, 16], "variat": 7, "These": 7, "As": [7, 9], "point": [7, 9], "plot": [7, 9], "view": [7, 9], "constrain": 7, "search": 7, "space": 7, "certain": 7, "deploy": 7, "guarante": 7, "requir": 7, "Of": 7, "cours": 7, "visibl": 7, "remov": [7, 9, 14], "presum": 7, "anthrop": [7, 18], "properli": 7, "save": [7, 14, 15, 16], "moment": 7, "therefor": 7, "appli": 7, "altern": [7, 8], "had": 7, "overwritten": 7, "clauc": 7, "place": 7, "legend": [7, 9], "underneath": 7, "parent": 7, "slightli": 7, "reduc": 7, "amount": 7, "decid": 7, "stick": 7, "decis": 7, "avoid": [7, 15, 16], "immedi": 7, "futur": 7, "appear": [7, 9], "full": 7, "frequenc": 7, "gpt4": 7, "42": 7, "29": 7, "calls2": [7, 9], "18": 7, "11": 7, "metric": [7, 9], "ax": 7, "verifi": 7, "session": 7, "perman": 7, "veri": 7, "happi": [7, 9], "forget": 7, "delet": [7, 14], "anytim": 7, "press": [7, 9], "besid": 7, "clipboard": 7, "along": 7, "default": [7, 9, 11, 12, 14], "shelf": 8, "azur": 8, "ml": 8, "catalog": 8, "vertex": 8, "garden": 8, "aw": 8, "bedrock": 8, "whole": 8, "major": 8, "cloud": 8, "regardless": 8, "expos": [8, 9], "adher": 8, "got": 8, "said": 8, "screen": 8, "local": [8, 14, 18], "build": 8, "label": [8, 9], "preview": 8, "imag": [8, 9], "router": [8, 9], "navig": 9, "ones": 9, "methodologi": 9, "head": 9, "By": 9, "six": 9, "openherm": 9, "shown": 9, "On": 9, "against": 9, "scatter": 9, "replot": 9, "similarili": 9, "score": 9, "updat": 9, "averag": 9, "granular": 9, "outsid": 9, "popular": 9, "period": 9, "suit": 9, "dedic": 9, "region": 9, "sequenc": [9, 12], "latest": [9, 18], "sort": 9, "icon": 9, "column": 9, "reveal": 9, "pattern": 9, "figur": 9, "fit": 9, "resum": 9, "previou": 9, "privat": 9, "server": [9, 13], "vari": 9, "fastest": 9, "under": 9, "ip": 9, "address": 9, "lock": 9, "indic": 9, "necessari": 9, "subsect": 9, "separ": 9, "expand": 9, "pair": 9, "histori": [9, 11, 12], "mixtral": 9, "financ": 9, "alongsid": 9, "foundat": 9, "class": [11, 12, 13], "chatbot": 11, "none": [11, 12, 14], "agent": 11, "__init__": [11, 12], "initi": [11, 12], "arg": [11, 12, 14], "str": [11, 12, 14], "uploaded_bi": [11, 12], "must": [11, 12, 14], "attempt": [11, 12], "rais": [11, 12, 13, 14], "unifyerror": [11, 12, 13], "miss": [11, 12], "clear_chat_histori": 11, "clear": 11, "properti": [11, 12], "dar201": [11, 12], "noqa": [11, 12], "show_credit": 11, "show_provid": 11, "loop": 11, "bool": [11, 12], "whether": 11, "consumpt": 11, "set_client": 11, "dar101": [11, 12], "set_endpoint": [11, 12], "set_model": [11, 12], "set_provid": [11, 12], "asyncunifi": 12, "async": 12, "user_prompt": 12, "system_prompt": 12, "union": 12, "asyncgener": 12, "dict": [12, 14], "dictionari": [12, 14], "convers": 12, "int": 12, "max": 12, "what": [12, 18], "random": [12, 15, 16], "while": 12, "focus": 12, "determinist": 12, "yield": 12, "occur": 12, "dure": 12, "get_credit_bal": 12, "left": 12, "badrequesterror": [12, 13, 14], "valueerror": [12, 14], "pars": [12, 14], "authenticationerror": 13, "400": 13, "bad": 13, "conflicterror": 13, "409": 13, "conflict": 13, "internalservererror": 13, "intern": 13, "notfounderror": 13, "permissiondeniederror": 13, "ratelimiterror": 13, "429": 13, "too": 13, "mani": 13, "unprocessableentityerror": 13, "delete_dataset": 14, "platform": 14, "path": 14, "info": 14, "msg": 14, "delete_evalu": 14, "download_dataset": 14, "list_dataset": 14, "fetch": 14, "list_endpoint": [14, 15, 16], "empti": 14, "list_evalu": 14, "upload_dataset_from_dictionari": 14, "upload_dataset_from_fil": 14, "mission": [15, 16], "simplifi": [15, 16], "landscap": [15, 16], "No": [15, 16], "eval": [15, 16], "iter": [15, 16], "app": [15, 16], "perfect": [15, 16], "Then": [15, 16, 18], "4o": [15, 16], "your_kei": [15, 16], "world": [15, 16], "util": [15, 16], "fingertip": [15, 16], "book": 17, "prem": 18, "microservic": 18, "grant": 18, "docker": 18, "dt": 18, "p": 18, "8000": 18, "env_file_path": 18, "deepinfra_api_kei": 18, "stability_api_kei": 18, "mistral_ai_api_kei": 18, "perplexity_ai_api_kei": 18, "anyscale_api_kei": 18, "octoai_api_kei": 18, "openai_api_kei": 18, "together_ai_api_kei": 18, "replicate_api_kei": 18, "lepton_ai_api_kei": 18, "fireworks_ai_api_kei": 18, "anthropic_api_kei": 18, "vertexai_service_acc_json": 18, "vertexai_gcloud_path": 18, "vertexai_project": 18, "vertexai_loc": 18, "servic": 18, "weather": 18, "san": 18, "francisco": 18, "tokyo": 18, "pari": 18, "localhost": 18, "haiku": 18, "random_key_wont_be_us": 18, "life": 18}, "objects": {"unify": [[11, 0, 0, "-", "chat"], [12, 0, 0, "-", "clients"], [13, 0, 0, "-", "exceptions"], [14, 0, 0, "-", "utils"]], "unify.chat": [[11, 1, 1, "", "ChatBot"]], "unify.chat.ChatBot": [[11, 2, 1, "", "__init__"], [11, 2, 1, "", "clear_chat_history"], [11, 3, 1, "", "client"], [11, 3, 1, "", "endpoint"], [11, 3, 1, "", "model"], [11, 3, 1, "", "provider"], [11, 2, 1, "", "run"], [11, 2, 1, "", "set_client"], [11, 2, 1, "", "set_endpoint"], [11, 2, 1, "", "set_model"], [11, 2, 1, "", "set_provider"]], "unify.clients": [[12, 1, 1, "", "AsyncUnify"], [12, 1, 1, "", "Unify"]], "unify.clients.AsyncUnify": [[12, 2, 1, "", "__init__"], [12, 3, 1, "", "endpoint"], [12, 2, 1, "", "generate"], [12, 2, 1, "", "get_credit_balance"], [12, 3, 1, "", "model"], [12, 3, 1, "", "provider"], [12, 2, 1, "", "set_endpoint"], [12, 2, 1, "", "set_model"], [12, 2, 1, "", "set_provider"]], "unify.clients.Unify": [[12, 2, 1, "", "__init__"], [12, 3, 1, "", "endpoint"], [12, 2, 1, "", "generate"], [12, 2, 1, "", "get_credit_balance"], [12, 3, 1, "", "model"], [12, 3, 1, "", "provider"], [12, 2, 1, "", "set_endpoint"], [12, 2, 1, "", "set_model"], [12, 2, 1, "", "set_provider"]], "unify.exceptions": [[13, 4, 1, "", "AuthenticationError"], [13, 4, 1, "", "BadRequestError"], [13, 4, 1, "", "ConflictError"], [13, 4, 1, "", "InternalServerError"], [13, 4, 1, "", "NotFoundError"], [13, 4, 1, "", "PermissionDeniedError"], [13, 4, 1, "", "RateLimitError"], [13, 4, 1, "", "UnifyError"], [13, 4, 1, "", "UnprocessableEntityError"]], "unify.utils": [[14, 5, 1, "", "delete_dataset"], [14, 5, 1, "", "delete_evaluation"], [14, 5, 1, "", "download_dataset"], [14, 5, 1, "", "evaluate"], [14, 5, 1, "", "list_datasets"], [14, 5, 1, "", "list_endpoints"], [14, 5, 1, "", "list_evaluations"], [14, 5, 1, "", "list_models"], [14, 5, 1, "", "list_providers"], [14, 5, 1, "", "upload_dataset_from_dictionary"], [14, 5, 1, "", "upload_dataset_from_file"]]}, "objtypes": {"0": "py:module", "1": "py:class", "2": "py:method", "3": "py:property", "4": "py:exception", "5": "py:function"}, "objnames": {"0": ["py", "module", "Python module"], "1": ["py", "class", "Python class"], "2": ["py", "method", "Python method"], "3": ["py", "property", "Python property"], "4": ["py", "exception", "Python exception"], "5": ["py", "function", "Python function"]}, "titleterms": {"benchmark": [0, 9], "prepar": [0, 5], "your": [0, 2, 7, 8, 9], "dataset": [0, 5, 8], "how": 0, "doe": 0, "work": 0, "deploi": 1, "router": [1, 5, 7], "us": [1, 2], "base": 1, "optim": 1, "metric": 1, "defin": 1, "threshold": 1, "custom": [1, 5, 6, 7, 8], "round": [1, 2, 7, 8, 9], "up": [1, 2, 7, 8, 9], "make": 2, "first": 2, "request": [2, 6], "get": [2, 4, 15, 16], "kei": 2, "find": 2, "model": 2, "provid": 2, "queri": [2, 6], "an": 2, "endpoint": [2, 5, 6, 8, 9], "python": [2, 6, 20], "packag": [2, 6], "openai": [2, 6], "api": [2, 4, 6], "format": 2, "compat": 2, "tool": 2, "sdk": 2, "open": 2, "interpret": 2, "On": [3, 17, 18], "prem": [3, 17], "imag": 3, "refer": 4, "get_credit": 4, "post": 4, "chat": [4, 11], "complet": 4, "rout": 5, "what": 5, "i": 5, "llm": [5, 6], "foundat": 5, "filter": 5, "maximis": 5, "throughput": 5, "fine": 5, "tune": 5, "train": [5, 7], "beta": [5, 7, 8, 9], "univers": 6, "authent": 6, "http": 6, "unifi": [6, 15, 16], "nodej": 6, "bill": 6, "advanc": 6, "featur": 6, "fallback": 6, "build": 7, "connect": 8, "stack": 8, "prerequisit": 8, "ad": 8, "qualiti": 9, "runtim": 9, "run": 9, "own": 9, "client": 12, "except": 13, "util": 14, "welcom": [15, 16], "start": [15, 16], "access": 17, "sso": 18, "singl": 18, "sign": 18, "openapi": 19, "specif": 19, "librari": 20}, "envversion": {"sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "nbsphinx": 4, "sphinx": 60}, "alltitles": {"Welcome to Unify!": [[16, "welcome-to-unify"], [15, "welcome-to-unify"]], "Getting Started": [[16, "getting-started"], [15, "getting-started"]], "OpenAPI Specification": [[19, "openapi-specification"]], "SSO (Single Sign On)": [[18, "sso-single-sign-on"]], "On-Prem access": [[17, "on-prem-access"]], "API Reference": [[4, "api-reference"]], "GET /get_credits": [[4, "get-get-credits"]], "POST /chat/completions": [[4, "post-chat-completions"]], "Benchmarking": [[0, "benchmarking"]], "Preparing your dataset": [[0, "preparing-your-dataset"]], "Benchmarking your dataset": [[0, "benchmarking-your-dataset"]], "How does it work?": [[0, "how-does-it-work"]], "Deploying a router": [[1, "deploying-a-router"]], "Using the base router": [[1, "using-the-base-router"]], "Optimizing a metric": [[1, "optimizing-a-metric"]], "Defining thresholds": [[1, "defining-thresholds"]], "Using a custom router": [[1, "using-a-custom-router"]], "Round Up": [[1, "round-up"], [2, "round-up"], [7, "round-up"], [9, "round-up"], [8, "round-up"]], "On-Prem Images": [[3, "on-prem-images"]], "Making your first request": [[2, "making-your-first-request"]], "Getting a key": [[2, "getting-a-key"]], "Finding a model and provider": [[2, "finding-a-model-and-provider"]], "Querying an endpoint": [[2, "querying-an-endpoint"]], "Using the Python Package": [[2, "using-the-python-package"]], "Using the OpenAI API Format": [[2, "using-the-openai-api-format"]], "Compatible Tools": [[2, "compatible-tools"]], "OpenAI SDK": [[2, "openai-sdk"]], "Open Interpreter": [[2, "open-interpreter"]], "Python Library": [[20, "python-library"]], "Building a custom router": [[7, "building-a-custom-router"]], "Training a custom router (Beta)": [[7, "training-a-custom-router-beta"]], "Customizing your router (Beta)": [[7, "customizing-your-router-beta"]], "Universal API": [[6, "universal-api"]], "Authentication": [[6, "authentication"]], "Querying the API": [[6, "querying-the-api"]], "HTTP Request": [[6, "http-request"]], "Unify Python Package": [[6, "unify-python-package"]], "OpenAI Python Package": [[6, "openai-python-package"]], "OpenAI NodeJS Package": [[6, "openai-nodejs-package"]], "Billing": [[6, "billing"]], "Advanced features": [[6, "advanced-features"]], "Custom endpoints": [[6, "custom-endpoints"]], "LLM Fallbacks": [[6, "llm-fallbacks"]], "Routing": [[5, "routing"]], "What is LLM routing?": [[5, "what-is-llm-routing"]], "Foundation router": [[5, "foundation-router"]], "Filtering endpoints": [[5, "filtering-endpoints"]], "Maximising throughput": [[5, "maximising-throughput"]], "Fine-tuned custom router": [[5, "fine-tuned-custom-router"]], "Preparing a dataset": [[5, "preparing-a-dataset"]], "Training a router (beta)": [[5, "training-a-router-beta"]], "Benchmarking endpoints": [[9, "benchmarking-endpoints"]], "Quality benchmarks": [[9, "quality-benchmarks"], [9, "id4"]], "Runtime benchmarks": [[9, "runtime-benchmarks"], [9, "id2"]], "Running your own benchmarks (Beta)": [[9, "running-your-own-benchmarks-beta"]], "Connecting your stack": [[8, "connecting-your-stack"]], "Custom endpoints (Beta)": [[8, "custom-endpoints-beta"]], "Prerequisite": [[8, "prerequisite"]], "Adding the endpoints": [[8, "adding-the-endpoints"]], "Custom datasets": [[8, "custom-datasets"]], "Exceptions": [[13, "module-unify.exceptions"]], "Utils": [[14, "module-unify.utils"]], "Chat": [[11, "module-unify.chat"]], "Clients": [[12, "module-unify.clients"]]}, "indexentries": {"chatbot (class in unify.chat)": [[11, "unify.chat.ChatBot"]], "__init__() (unify.chat.chatbot method)": [[11, "unify.chat.ChatBot.__init__"]], "clear_chat_history() (unify.chat.chatbot method)": [[11, "unify.chat.ChatBot.clear_chat_history"]], "client (unify.chat.chatbot property)": [[11, "unify.chat.ChatBot.client"]], "endpoint (unify.chat.chatbot property)": [[11, "unify.chat.ChatBot.endpoint"]], "model (unify.chat.chatbot property)": [[11, "unify.chat.ChatBot.model"]], "module": [[11, "module-unify.chat"], [12, "module-unify.clients"], [13, "module-unify.exceptions"], [14, "module-unify.utils"]], "provider (unify.chat.chatbot property)": [[11, "unify.chat.ChatBot.provider"]], "run() (unify.chat.chatbot method)": [[11, "unify.chat.ChatBot.run"]], "set_client() (unify.chat.chatbot method)": [[11, "unify.chat.ChatBot.set_client"]], "set_endpoint() (unify.chat.chatbot method)": [[11, "unify.chat.ChatBot.set_endpoint"]], "set_model() (unify.chat.chatbot method)": [[11, "unify.chat.ChatBot.set_model"]], "set_provider() (unify.chat.chatbot method)": [[11, "unify.chat.ChatBot.set_provider"]], "unify.chat": [[11, "module-unify.chat"]], "asyncunify (class in unify.clients)": [[12, "unify.clients.AsyncUnify"]], "unify (class in unify.clients)": [[12, "unify.clients.Unify"]], "__init__() (unify.clients.asyncunify method)": [[12, "unify.clients.AsyncUnify.__init__"]], "__init__() (unify.clients.unify method)": [[12, "unify.clients.Unify.__init__"]], "endpoint (unify.clients.asyncunify property)": [[12, "unify.clients.AsyncUnify.endpoint"]], "endpoint (unify.clients.unify property)": [[12, "unify.clients.Unify.endpoint"]], "generate() (unify.clients.asyncunify method)": [[12, "unify.clients.AsyncUnify.generate"]], "generate() (unify.clients.unify method)": [[12, "unify.clients.Unify.generate"]], "get_credit_balance() (unify.clients.asyncunify method)": [[12, "unify.clients.AsyncUnify.get_credit_balance"]], "get_credit_balance() (unify.clients.unify method)": [[12, "unify.clients.Unify.get_credit_balance"]], "model (unify.clients.asyncunify property)": [[12, "unify.clients.AsyncUnify.model"]], "model (unify.clients.unify property)": [[12, "unify.clients.Unify.model"]], "provider (unify.clients.asyncunify property)": [[12, "unify.clients.AsyncUnify.provider"]], "provider (unify.clients.unify property)": [[12, "unify.clients.Unify.provider"]], "set_endpoint() (unify.clients.asyncunify method)": [[12, "unify.clients.AsyncUnify.set_endpoint"]], "set_endpoint() (unify.clients.unify method)": [[12, "unify.clients.Unify.set_endpoint"]], "set_model() (unify.clients.asyncunify method)": [[12, "unify.clients.AsyncUnify.set_model"]], "set_model() (unify.clients.unify method)": [[12, "unify.clients.Unify.set_model"]], "set_provider() (unify.clients.asyncunify method)": [[12, "unify.clients.AsyncUnify.set_provider"]], "set_provider() (unify.clients.unify method)": [[12, "unify.clients.Unify.set_provider"]], "unify.clients": [[12, "module-unify.clients"]], "authenticationerror": [[13, "unify.exceptions.AuthenticationError"]], "badrequesterror": [[13, "unify.exceptions.BadRequestError"]], "conflicterror": [[13, "unify.exceptions.ConflictError"]], "internalservererror": [[13, "unify.exceptions.InternalServerError"]], "notfounderror": [[13, "unify.exceptions.NotFoundError"]], "permissiondeniederror": [[13, "unify.exceptions.PermissionDeniedError"]], "ratelimiterror": [[13, "unify.exceptions.RateLimitError"]], "unifyerror": [[13, "unify.exceptions.UnifyError"]], "unprocessableentityerror": [[13, "unify.exceptions.UnprocessableEntityError"]], "unify.exceptions": [[13, "module-unify.exceptions"]], "delete_dataset() (in module unify.utils)": [[14, "unify.utils.delete_dataset"]], "delete_evaluation() (in module unify.utils)": [[14, "unify.utils.delete_evaluation"]], "download_dataset() (in module unify.utils)": [[14, "unify.utils.download_dataset"]], "evaluate() (in module unify.utils)": [[14, "unify.utils.evaluate"]], "list_datasets() (in module unify.utils)": [[14, "unify.utils.list_datasets"]], "list_endpoints() (in module unify.utils)": [[14, "unify.utils.list_endpoints"]], "list_evaluations() (in module unify.utils)": [[14, "unify.utils.list_evaluations"]], "list_models() (in module unify.utils)": [[14, "unify.utils.list_models"]], "list_providers() (in module unify.utils)": [[14, "unify.utils.list_providers"]], "unify.utils": [[14, "module-unify.utils"]], "upload_dataset_from_dictionary() (in module unify.utils)": [[14, "unify.utils.upload_dataset_from_dictionary"]], "upload_dataset_from_file() (in module unify.utils)": [[14, "unify.utils.upload_dataset_from_file"]]}}) \ No newline at end of file