quantize TODO Llama 2 7B Quantization Bits per Weight (BPW) Q2_K 3.35 Q3_K_S 3.50 Q3_K_M 3.91 Q3_K_L 4.27 Q4_K_S 4.58 Q4_K_M 4.84 Q5_K_S 5.52 Q5_K_M 5.68 Q6_K 6.56 Llama 2 13B Quantization Bits per Weight (BPW) Q2_K 3.34 Q3_K_S 3.48 Q3_K_M 3.89 Q3_K_L 4.26 Q4_K_S 4.56 Q4_K_M 4.83 Q5_K_S 5.51 Q5_K_M 5.67 Q6_K 6.56 Llama 2 70B Quantization Bits per Weight (BPW) Q2_K 3.40 Q3_K_S 3.47 Q3_K_M 3.85 Q3_K_L 4.19 Q4_K_S 4.53 Q4_K_M 4.80 Q5_K_S 5.50 Q5_K_M 5.65 Q6_K 6.56
