You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
INFO 09-03 08:14:32 [default_loader.py:262] Loading weights took 3.10 seconds
INFO 09-03 08:14:33 [model_runner_v1.py:2114] Loading model weights took 14.2488 GB
..[ERROR] the socversion Ascend910B1 of bin package does not match the current device socverison Ascend310P3. Please modify default socversion in run.sh or execute run.sh with socversion parameter.
..
How would you like to use vllm on ascend
I want to run inference of a [specific model](put link here). I don't know how to integrate it with vllm.