[feature request] enable the newer cpu instructions

[maxvaneck]

greetings,

i recently bought a new system and the cpu has some very nice newfangled instructions(avx512 and variants), looking at cmakelists.txt those are not turned on so perhaps an additional new cpu executable is possible like the oldpc one

thx for looking at this,
Max

[LostRuins]

Its tricky cause those are not well supported and I have no way to test it

[JohnLoveJoy]

ggml-org/ggml#1435

You should open a request in GGML git, from what I understand, KoboldCPP is simply a UI that brings together all these projects running in the background.

[LostRuins]

Avx512 is already supporter in llama.cpp, but it's not enabled in koobldcpp builds because it's not testable for me

[maxvaneck]

ah ok got it, that's reasonable

[Nesaliti]

If it's a question of an .exe with the function enabled, can't the test be made by users? Like, non-cuda AVX512 pushed separately like a beta-test, or a discussion with build downloaded through mediaget. Or is it more complicated and requires a lot of iterations?

[GreenShadows]

Someone with the necessary expertise could simply fork the code, modify it, and compile it to support AVX512.

[maxvaneck]

huh i guess i could give it a try