Skip to content

Commit 58f723a

Browse files
authored
fix(moe): fix module prefetch for moe (#421)
1 parent 10ede60 commit 58f723a

File tree

1 file changed

+1
-0
lines changed
  • internlm/core/parallel/comm

1 file changed

+1
-0
lines changed

internlm/core/parallel/comm/isp.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -892,6 +892,7 @@ def is_allgather_launch_module(name, module):
892892
for name, child in block.named_modules():
893893
if is_allgather_launch_module(name, child):
894894
self._overlap_states[cid].isp_prefetch_launch_module.append(child)
895+
self._overlap_states[cid].module_to_index[child] = idx
895896
if isinstance(child, (ParallelLinearWithCommExt)):
896897
if is_moe_param(child.weight) != self.is_moe:
897898
continue

0 commit comments

Comments
 (0)