InferenceMax AI benchmark tests software stacks, efficiency, and TCO — vendor-neutral suite runs nightly and tracks performance changes over time

SemiAnalysis introduces InferenceMax, an open-source AI benchmarking suite focusing on the efficiency of AI software stacks in real-world inference scenarios. The suite, released under the Apache 2.0 license, measures the performance of various AI accelerator hardware and software combinations nightly, emphasizing TCO (total cost of ownership) in dollars per million tokens. InferenceMax aims to provide a vendor-neutral, real-world application benchmark that considers factors like throughput and interactivity, highlighting the importance of finding an optimal balance between the two for efficiency. The project collaborates with major vendors like AMD and Nvidia to uncover bugs and improve default configurations for better performance.