Skip to content

Commit 6b73184

Browse files
Use specified max context.py
Instead of original_max_position_embeddings. This appears to be what transformers intended, and does not update dynamically with sequence leng there.
1 parent 8dca1ab commit 6b73184

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

exllamav2/device.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -187,7 +187,7 @@ def prepare_sincos(self):
187187

188188
elif cfg.alt_rope_method == "yarn":
189189

190-
yarn_max_position_embeddings = cfg.yarn_rope_original_max_position_embeddings
190+
yarn_max_position_embeddings = cfg.max_seq_len
191191

192192
# Only activate if longer than original ctx
193193
if cfg.max_seq_len > cfg.yarn_rope_original_max_position_embeddings:

0 commit comments

Comments
 (0)