We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 88a689d commit 3f6e838Copy full SHA for 3f6e838
vllm_ascend/models/layers/mla.py
@@ -133,7 +133,7 @@ def forward(
133
dtype=hidden_states.dtype,
134
device=hidden_states.device)
135
output = self.mla_attn.impl.forward(hidden_states, kv_cache,
136
- forward_context.attn_metadata,
+ forward_context.attn_metadata[self.mla_attn.layer_name],
137
need_gather_q_kv, output)
138
output = output.view(-1, output_shape[-1])
139
return output
0 commit comments