Skip to content

Commit 23bd266

Browse files
authored
Update How_to_count_tokens_with_tiktoken.ipynb (openai#368)
* Update How_to_count_tokens_with_tiktoken.ipynb add c# tiktoken lib. * Update How_to_count_tokens_with_tiktoken.ipynb fix format. * Update How_to_count_tokens_with_tiktoken.ipynb fix format.
1 parent 5008f3a commit 23bd266

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

examples/How_to_count_tokens_with_tiktoken.ipynb

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
{
22
"cells": [
33
{
4+
"attachments": {},
45
"cell_type": "markdown",
56
"metadata": {},
67
"source": [
@@ -36,7 +37,7 @@
3637
"\n",
3738
"For `cl100k_base` and `p50k_base` encodings:\n",
3839
"- Python: [tiktoken](https://github.com/openai/tiktoken/blob/main/README.md)\n",
39-
"- .NET / C#: [SharpToken](https://github.com/dmitry-brazhenko/SharpToken)\n",
40+
"- .NET / C#: [SharpToken](https://github.com/dmitry-brazhenko/SharpToken), [TiktokenSharp](https://github.com/aiqinxuancai/TiktokenSharp)\n",
4041
"- Java: [jtokkit](https://github.com/knuddelsgmbh/jtokkit)\n",
4142
"\n",
4243
"For `r50k_base` (`gpt2`) encodings, tokenizers are available in many languages.\n",

0 commit comments

Comments
 (0)