FlexBench: An Open-Source, Modular MLPerf Benchmark for LLM Inference

TL;DR AI system benchmarks like MLPerf struggle to keep pace with the rapidly evolving model landscape, making it difficult for organizations to make informed deployment decisions. We believe benchmar