Skip to content

Commit 0df4d22

Browse files
committed
lora cpu fix
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
1 parent d31bdfd commit 0df4d22

File tree

1 file changed

+9
-6
lines changed
  • server/text_generation_server/layers

1 file changed

+9
-6
lines changed

server/text_generation_server/layers/lora.py

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,15 @@
1616
punica_sgmv = None
1717

1818
if SYSTEM == "ipex":
19-
from intel_extension_for_pytorch.llm.functional import (
20-
bgmv_expand,
21-
bgmv_shrink,
22-
sgmv_expand,
23-
sgmv_shrink,
24-
)
19+
try:
20+
from intel_extension_for_pytorch.llm.functional import (
21+
bgmv_expand,
22+
bgmv_shrink,
23+
sgmv_expand,
24+
sgmv_shrink,
25+
)
26+
except ImportError:
27+
pass
2528

2629

2730
if TYPE_CHECKING:

0 commit comments

Comments
 (0)