Serve with vLLM

by faheemraza1 - opened Sep 7

Sep 7

Can this model be served with vllm on an RTX 3090? if so, please share the command and values of required parameters.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment