vllm with lora adapter
#3302
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
{{ message }}
-
--infer_backend vllm to enjoy 270% inference speed. (LoRA is not yet supported, merge it first.)
Beta Was this translation helpful? Give feedback.
All reactions