you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 3 months ago

vLLM.

It's slower to start, but once it gets going, pretty solid.

  • source