[Model] Add use_qk_norm option for Cohere model#2877
[Model] Add use_qk_norm option for Cohere model#2877tlopex wants to merge 4 commits intomlc-ai:mainfrom
Conversation
|
cc @MasterJH5574 |
|
@tlopex Sorry for the delayed response. Split-and-combine is okay for now. Could you try the 4b quantized version on your end? If that's not possible I can find a way to test on my side. |
|
@MasterJH5574 |
|
@MasterJH5574 Hi! Could you find a way to test this, so we can determine if the PR is ready for merging? |
This pr updates
use_qk_normoption for Cohere series models like Command-R-Plus.