Skip to content

Commit

Permalink
added line six groups gif.
Browse files Browse the repository at this point in the history
  • Loading branch information
djl11 committed Feb 22, 2025
1 parent b669a8d commit 1567bba
Show file tree
Hide file tree
Showing 4 changed files with 61 additions and 19 deletions.
19 changes: 19 additions & 0 deletions demo_testing/bar_chart_grouped.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import unify
unify
import random
random.seed(0)

base_msg = "You are a helpful assistant."
for tool_use in [True, False]:
for sys_msg in [
base_msg,
base_msg + " You can use tools.",
base_msg + " You can use tools and search the web."
]:
for example_idx in range(10):
unify.log(
tool_use=tool_use,
sys_msg=sys_msg,
example_idx=example_idx,
score=random.uniform(0, 10)
)
8 changes: 4 additions & 4 deletions demo_testing/derived_entry.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@
unify.activate("derived_entry", overwrite=True)

for _ in range(20):
x = random.randint(0, 100)
y = random.randint(0, 100)
x = random.random()
y = random.random()
unify.log(
x=x,
y=y,
x0=x,
x1=y,
# length=(x**2 + y**2)**0.5
)
28 changes: 28 additions & 0 deletions demo_testing/line_graph_grouped.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import unify
import random
random.seed(0)
unify.activate("line_graph_grouped", overwrite=True)

model_speeds = {
"llama-3.1-8b-chat": 12,
"llama-3.1-70b-chat": 7,
"llama-3.1-405b-chat": 4
}
provider_speeds = {
"fireworks-ai": 10,
"together-ai": 9,
"aws-bedrock": 8
}
for i in range(25):
for model, model_speed in model_speeds.items():
for provider, provider_speed in provider_speeds.items():
unify.log(
model=model,
provider=provider,
time=i,
speed=(
model_speed +
provider_speed +
random.uniform(-2, 2)
)
)
25 changes: 10 additions & 15 deletions interfaces/plots.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -131,23 +131,22 @@ and then limit the time horizon to the last 5 minutes, for example.

<img class="dark-light" width="100%" src="https://raw.githubusercontent.com/unifyai/unifyai.github.io/refs/heads/main/img/externally_linked/line_group_stream_dark.gif"/>

{/* ToDo: Add support for multi-grouping (https://app.clickup.com/t/86c1xw4bf) */}

{/* As with scatter graphs, we can also group by multiple columns.
Let's say we also want to group by provider.
Similar to the derived "identity" column before,
we can create a derived "endpoint" column to plot a unique line for each model + provider combination.
Let's create some dummy data for this scenario:

```python
model_speeds = {
"llama-3.1-70b": 10,
"llama-3.1-405b": 5,
"llama-3.1-8k": 1
"llama-3.1-8b-chat": 12,
"llama-3.1-70b-chat": 7,
"llama-3.1-405b-chat": 4
}
provider_speeds = {
"fireworks-ai": 10,
"together-ai": 9,
"aws-bedrock": 8
}
for i in range(10):
for i in range(25):
for model, model_speed in model_speeds.items():
for provider, provider_speed in provider_speeds.items():
unify.log(
Expand All @@ -157,14 +156,14 @@ for i in range(10):
speed=(
model_speed +
provider_speed +
random.uniform(-5, 5)
random.uniform(-2, 2)
)
)
```

We could then create a plot which represents each unique **endpoint**, by grouping by the model *and* provider.
Let's then create the derived column and plot results for each unique **endpoint**.

IMG */}
<img class="dark-light" width="100%" src="https://raw.githubusercontent.com/unifyai/unifyai.github.io/refs/heads/main/img/externally_linked/line_six_groups_dark.gif"/>

## Bar Charts

Expand Down Expand Up @@ -198,10 +197,6 @@ to see how each of our experiments performed across the 10 examples.

IMG

If we want to see the extremes, we could also plot the *min* or *max* score, and add plots for these.

IMG

## Histograms

Finally, we can also plot histograms, which take a *single* numeric column,
Expand Down

0 comments on commit 1567bba

Please sign in to comment.