Skip to content

Commit 3f6e838

Browse files
committed
fix deepseek mla
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
1 parent 88a689d commit 3f6e838

File tree

1 file changed

+1
-1
lines changed
  • vllm_ascend/models/layers

1 file changed

+1
-1
lines changed

vllm_ascend/models/layers/mla.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -133,7 +133,7 @@ def forward(
133133
dtype=hidden_states.dtype,
134134
device=hidden_states.device)
135135
output = self.mla_attn.impl.forward(hidden_states, kv_cache,
136-
forward_context.attn_metadata,
136+
forward_context.attn_metadata[self.mla_attn.layer_name],
137137
need_gather_q_kv, output)
138138
output = output.view(-1, output_shape[-1])
139139
return output

0 commit comments

Comments
 (0)