Skip to content

Commit

Permalink
finished experiments section.
Browse files Browse the repository at this point in the history
  • Loading branch information
djl11 committed Feb 22, 2025
1 parent 47f80be commit 2132d9a
Show file tree
Hide file tree
Showing 2 changed files with 49 additions and 27 deletions.
11 changes: 11 additions & 0 deletions demo_testing/experiment_demo.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import unify
import random
unify.activate("experiments-demo")
with unify.Experiment(-1, overwrite=True), unify.Params(
sys_msg="You are a helpful assistant."
):
for _ in range(5):
unify.log(
question_number=random.randint(1, 20),
score=random.random()
)
65 changes: 38 additions & 27 deletions logging/experiments.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,47 +3,58 @@ title: 'Experiments'
---

The `unify.Experiment` class is a very simple convenience class for logging experiments.
It simply creates a parameter with the name `"experiment"`, and automatically increments the value as an ascending integer for each new experiment, if an explicit name is not provided.
It simply creates a parameter with the name `"experiment"`,
and automatically increments the value as an ascending integer for each new experiment,
if an explicit name is not provided.

```python
with unify.Experiment():
for _ in range(10):
unify.log(score=random.random())
with unify.Experiment(), unify.Params(
sys_msg="You are a helpful assistant."
):
for _ in range(5):
unify.log(
question_number=random.randint(1, 20),
score=random.random()
)
```

IMG

Let's run another experiment, but give it an explicit name.
This will show in the table like so:

```python
with unify.Experiment("my_experiment"):
for _ in range(10):
unify.log(score=random.random())
```
<img class="dark-light" width="100%" src="https://raw.githubusercontent.com/unifyai/unifyai.github.io/refs/heads/main/img/externally_linked/experiment_0_dark.png"/>

IMG

it's often useful to use the `with unify.Experiment()` alongside a `unify.Parameter()` to represent the settable parameters of an experiment.
Let's run another experiment, but give it an explicit name.

```python
with unify.Experiment("final_experiment"), unify.Parameter(
system_message="You are a helpful assistant."
with unify.Experiment("my_experiment"), unify.Params(
sys_msg="You are a helpful assistant."
):
for _ in range(10):
unify.log(score=random.random())
for _ in range(5):
unify.log(
question_number=random.randint(1, 20),
score=random.random()
)
```

IMG
This will show in the table like so:

You can also use negative indexing, and the `overwrite` argument. For example, the following will overwrite the previous experiment.
Integers are interpreted as the experiment version, and strings are interpreted as the experiment name.
<img class="dark-light" width="100%" src="https://raw.githubusercontent.com/unifyai/unifyai.github.io/refs/heads/main/img/externally_linked/named_experiment.png"/>

You can also use negative indexing, and the `overwrite` argument.
For example, the following will overwrite the previous experiment.
Integers are interpreted as the experiment version,
and strings are interpreted as the experiment name.

```python
with unify.Experiment(-1, overwrite=True), unify.Parameter(
system_message="You are a *great* assistant."
with unify.Experiment(-1, overwrite=True), unify.Params(
sys_msg="You are a helpful assistant."
):
for _ in range(10):
unify.log(score=random.random())
for _ in range(5):
unify.log(
question_number=random.randint(1, 20),
score=random.random()
)
```

IMG
The overwrite can be seen in the table, after pressing the `refresh` button:

<img class="dark-light" width="100%" src="https://raw.githubusercontent.com/unifyai/unifyai.github.io/refs/heads/main/img/externally_linked/experiment_overwrite_dark.gif"/>

0 comments on commit 2132d9a

Please sign in to comment.