Nine days ago I opened a small vLLM pull request that made int8 matrix multiplies about 13% faster on an RTX 4090. The next day a reviewer ran it on a smaller Ada card and one layer got up to 81% slow

Markdown supported · ⌘↵ to post
No responses yet.