You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A useful 3rd party tool has been developed by @naimenz for visualizing
openai/eval logs: https://github.com/naimenz/logviz
Adding a link to it from our README seems good as it is probably useful
for users. :)
Copy file name to clipboardExpand all lines: docs/run-evals.md
+4
Original file line number
Diff line number
Diff line change
@@ -38,3 +38,7 @@ Running with more threads will make the eval faster, though keep in mind the cos
38
38
If you have to stop your run or your run crashes, we've got you covered! `oaievalset` records the evals that finished in `/tmp/oaievalset/{model}.{eval_set}.progress.txt`. You can simply rerun the command to pick up where you left off. If you want to run the eval set starting from the beginning, delete this progress file.
39
39
40
40
Unfortunately, you can't resume a single eval from the middle. You'll have to restart from the beginning, so try to keep your individual evals quick to run.
41
+
42
+
## Logging
43
+
44
+
By default, `oaieval`[records events](/evals/record.py) into local JSONL logs which can be inspected using a text editor or analyzed programmatically. 3rd-party tools such as [naimenz/logviz](https://github.com/naimenz/logviz) may be helpful to visualize the logs, though we don't provide support or guarantees for their use.
0 commit comments