Skip to content

Commit 88fd65a

Browse files
committed
docs: update README
1 parent 612123a commit 88fd65a

File tree

1 file changed

+22
-9
lines changed

1 file changed

+22
-9
lines changed

README.md

Lines changed: 22 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -244,8 +244,7 @@ Each type of node owns a set of properties which affect the computational perfor
244244
* `Input Prefix`: append before every user input
245245
* `Input Suffix`: append after every user input
246246
* `Should Output prompt`: whether the input prompt should be included in the output
247-
* `Should Output Bos`: whether the special bos (beginning of sequence) token should be included in the output
248-
* `Should Output Eos`: whether the special eos (ending of sequence) token should be included in the output
247+
* `Should Output Special`: whether the special (e.g., beginning of sequence and ending of sequence) token should be included in the output
249248
* `Context Size`: number of tokens the model can process at a time
250249
* `N Predict`: number of new tokens to generate, generate infinite sequence if -1
251250
* `N Keep`: when the model run out of `context size`, it starts to forget about earlier context, set this variable to force the model to keep a number of the earliest tokens to keep the conversation relevant
@@ -258,6 +257,8 @@ Each type of node owns a set of properties which affect the computational perfor
258257
* `Min P`: only sample from tokens with at least this probability, disabledd if 0.0
259258
* `N Thread`: number of cpu threads to use
260259
* `N GPU Layer`: number of layer offloaded to GPU
260+
* `Main GPU`: the main GPU for computation
261+
* `Split Mode`: how the computation will be distributed if there are multiple GPU in your systemm (0: None, 1: Layer, 2: Row)
261262
* `Escape`: process escape character in input prompt
262263
* `N Batch`: maximum number of tokens per iteration during continuous batching
263264
* `N Ubatch`: maximum batch size for computation
@@ -315,11 +316,10 @@ Suppose you want to generate a character with:
315316
* `weapon`: either "sword", "bow", or "wand
316317
* `description`: a text with minimum 10 character
317318

318-
You should first create a GDLlama node, and turn `Should Output prompt`, `Should Output Bos`, and `Should Output Eos` off either by inspector or by script:
319+
You should first create a GDLlama node, and turn `Should Output prompt` and `Should Output Special` off either by inspector or by script:
319320
```
320321
should_output_prompt = false
321-
should_output_bos = false
322-
should_output_eos = false
322+
should_output_special = false
323323
```
324324

325325

@@ -408,15 +408,16 @@ Besides the functions and signals from GDEmbedding, LlmDB has a few more functio
408408
* `has_id(id: String, p_table_name: String) -> bool`: whether the table has a specific id stored
409409
* `split_text(text: String) -> PackedStringArray`: split a piece of text first by all `Absolute Separators`, then by one of the appropiate `Chunk Separators`, such that any text chunk is shorter than `Chunk Size` (measured in character), and the overlap is close to but not greater than `Chunk Overlap`. If the algorithm failed to satisfy the contraints, there will be an error message printed out and the returned chunk will be greater than the `Chunk Size`
410410
* `store_text_by_id(id: String, text: String)`: split the text and store the chunks in the database, be aware that `store_meta` should have been called previously such that the `id` with the corresponding meta is already in the database
411-
* `run_store_text_by_id(id: String, text: String) -> Error`: run `store_text_by_id` in background
411+
* `run_store_text_by_id(id: String, text: String) -> Error`: run `store_text_by_id` in background, emits `store_text_finished` signal when finished
412412
* `store_text_by_meta(meta_dict: Dictionary, text: String)`: split the text and store the chunks in the database with the metadata defined in `meta_dict`, be aware that the metadata should be valid, every key should be a name stored in the `.meta` property and the corresponding type should be correct
413-
* `run_store_text_by_meta(meta_dict: Dictionary, text: String) -> Error` run `store_text_by_meta` in background
413+
* `run_store_text_by_meta(meta_dict: Dictionary, text: String) -> Error` run `store_text_by_meta` in background, emits `store_text_finished` signal when finished
414414
* `retrieve_similar_texts(text: String, where: String, n_results: int) -> PackedStringArray`: retrieve `n_results` most similar text chunks to `text`, `where` should be empty or an sql WHERE clause to filter the chunks by metadata
415415
* `run_retrieve_similar_texts(text: String, where: String, n_results: int) -> Error`:
416416
run `retrieve_similar_texts` in background, and emits a `retrieve_similar_texts_finished` signal once it is done
417417

418418
### Signals
419419

420+
* `store_text_finished`: emitted when `run_store_text_by_id` or `run_store_text_by_meta` is finished
420421
* `retrieve_similar_texts_finished(array: PackedStringArray)`: contains an array of `String`, emitted when `run_retrieve_similar_texts` is finished
421422

422423
## LlmDBMetaData
@@ -436,14 +437,26 @@ There are 4 static functions to create LlmDBMetaData
436437

437438
# FAQ
438439

439-
1. Does it support languages other than English?
440+
1. You have an issue, how to get more debug message?
441+
442+
Turn on `Verbose stdout` in `Project Settings`, consider running Godot from a terminal to get additional logging messages.
443+
444+
2. Does it support languages other than English?
440445

441446
Yes, the plugin uses utf8 encoding so it has multilingual support naturally. However, a language model may be trained with English data only and it won't be able to generate text other than English, choose the language model based on your need.
442447

443-
2. Observing strange tokens in generated text, such as `<eot_id>` when `Should Output Eos` is off.
448+
3. Observing strange tokens in generated text, such as `<eot_id>` when `Should Output Special` is off.
444449

445450
You are always welcome to open an issue. However, be aware that the standard of GGUF format can be changed to support new features and models, such that the bug can come from the model side instead of within this plugin. For example, some older llama 3 GGUF model may not be compatible with the latest format, you may try to search for a newer model with fixes such as [this](https://huggingface.co/NikolayKozloff/Meta-Llama-3-8B-Instruct-bf16-correct-pre-tokenizer-and-EOS-token-Q8_0-Q6_k-Q4_K_M-GGUF/tree/main).
446451

452+
4. You are runningg Arch linux (or its derivatives such as Manjaro) and your Godot Editor crash.
453+
454+
The Arch build of Godot is bugged, download Godot from the official website instead.
455+
456+
5. You have a discrete GPU and you see `unable to load model` error, you have make sure that the model parameters are correctly set.
457+
458+
There is currently a bug on vulkan backend if you have multiple drivers installed for the same GPU, try to turn `Split Mode` to `NONE` (0) and set your `Main GPU` manually (starting from 0) to see if it works.
459+
447460
# Compile from source
448461
Install build tools and Vulkan SDK for your operating system, then clone this repository
449462
```

0 commit comments

Comments
 (0)