Skip to content

Commit 63b4547

Browse files
Artefact2hodlen
authored andcommitted
convert : use f32 outtype for bf16 tensors (ggml-org#6106)
The old behaviour is to use f16, but bf16 to f16 is not a lossless conversion. Change the outtype to f32 to default to a lossless conversion.
1 parent f68d6a2 commit 63b4547

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

convert.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -1167,9 +1167,9 @@ def write_all(
11671167
def pick_output_type(model: LazyModel, output_type_str: str | None) -> GGMLFileType:
11681168
wq_type = model[gguf.TENSOR_NAMES[gguf.MODEL_TENSOR.ATTN_Q].format(bid=0) + ".weight"].data_type
11691169

1170-
if output_type_str == "f32" or (output_type_str is None and wq_type == DT_F32):
1170+
if output_type_str == "f32" or (output_type_str is None and wq_type in (DT_F32, DT_BF16)):
11711171
return GGMLFileType.AllF32
1172-
if output_type_str == "f16" or (output_type_str is None and wq_type in (DT_F16, DT_BF16)):
1172+
if output_type_str == "f16" or (output_type_str is None and wq_type == DT_F16):
11731173
return GGMLFileType.MostlyF16
11741174
if output_type_str == "q8_0":
11751175
return GGMLFileType.MostlyQ8_0

0 commit comments

Comments
 (0)