Skip to content

Issues: turboderp-org/exllamav2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

101 Open 410 Closed
101 Open 410 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[BUG] When trying inference with Qwen2.5-VL-72B with Qwen2.5-VL-7B as a draft model, I get "IndexError: index out of range in self" (both models have identical vocab.json) bug Something isn't working
#733 opened Feb 6, 2025 by Lissanro
3 tasks done
[BUG] Exception in ASGI application when trying inference with an image wit h Qwen2.5-VL-72B bug Something isn't working
#732 opened Feb 5, 2025 by Lissanro
3 tasks done
2
[BUG] bug Something isn't working
#730 opened Feb 2, 2025 by lugangqi
3 tasks done
1
Does exllamav2 support p104-100 8g graphics card
#729 opened Feb 2, 2025 by lugangqi
3 tasks done
3
[BUG] Mistral-Small-24B-Instruct-2501 - Tensor Parallel outputs garbled text. bug Something isn't working
#728 opened Jan 31, 2025 by mindkrypted
3 tasks done
[BUG] 2080ti can't quant a 12B model bug Something isn't working
#725 opened Jan 28, 2025 by frenzybiscuit
3 tasks done
1
[REQUEST] Support new SOTA vision model: Qwen 2.5 VL (3B, 7B, 72B)
#724 opened Jan 27, 2025 by ThomasBaruzier
3 tasks done
1
[REQUEST] Support x-grammar structured output framework integration
#723 opened Jan 27, 2025 by debasish-mihup
3 tasks done
1
[REQUEST] GraniteMoeForCausalLM architecture support
#722 opened Jan 27, 2025 by cal066
3 tasks done
[REQUEST] Add support for Blackwell B200 with Cuda 12.8
#721 opened Jan 25, 2025 by ofirkris
3 tasks done
[BUG] LORA fail load/inference for tensor parallel. bug Something isn't working
#719 opened Jan 22, 2025 by Ph0rk0z
3 tasks done
Question - Batch Processing
#718 opened Jan 22, 2025 by virentakia
3 tasks done
4
[REQUEST]Capture the count of output tokens
#717 opened Jan 21, 2025 by UTSAV-44
3 tasks done
1
[REQUEST] Multi gpu conversion
#715 opened Jan 14, 2025 by IMbackK
3 tasks done
2
[REQUEST] MiniCPM3-4B Support
#714 opened Jan 14, 2025 by meigami0
3 tasks done
[REQUEST] Tensor Parallelism For Qwen2VL
#713 opened Jan 10, 2025 by iamthemulti
3 tasks done
[REQUEST] graceful fallback when gpu_split is misspecified
#711 opened Jan 4, 2025 by jwlockhart
3 tasks done
2
[REQUEST] PerChannel Setting
#707 opened Dec 27, 2024 by Coco58323
3 tasks done
1
[REQUEST] Sage Attention? Anyone tried it with exllama?
#702 opened Dec 21, 2024 by Ph0rk0z
3 tasks done
2
[BUG] Qwen2.5-72B-2.xxbpw/Llama-70B-2.4bpw/even-4.xxbpw(maybe related to KV caching code) garbage output on some specific prompts. bug Something isn't working
#697 opened Dec 14, 2024 by Originalimoc
3 tasks done
15
[BUG] lmformatenforcer integration seems to be broken on new versions bug Something isn't working
#696 opened Dec 11, 2024 by hvico
3 tasks done
2
[REQUEST] EXAONE 3.5 Support
#695 opened Dec 9, 2024 by emzaedu
3 tasks done
2
[BUG] ExLlamaV2DynamicGenerator class is not multiple threads supported bug Something isn't working
#690 opened Nov 29, 2024 by UTSAV-44
3 tasks done
4
[BUG] generator.iterate() returns corrupted result objects in some cases bug Something isn't working
#689 opened Nov 29, 2024 by p-e-w
3 tasks done
8
[REQUEST] High throughput with large batch size
#686 opened Nov 26, 2024 by fzyzcjy
3 tasks done
5
ProTip! Add no:assignee to see everything that’s not assigned.