epoch ai

OpenAI's O3 Model Falls Short on the FrontierMath Benchmark: What's the Real Score?

OpenAI's O3 model, which initially claimed to solve over 25% of complex math problems on the FrontierMath benchmark, was found to have closer to a 10% success rate according to independent tests by Epoch AI. This discrepancy highlights the evolving nature of both the AI models and the benchmarks themselves. The incident underscores the importance of critically evaluating AI performance claims, as newer models like O4 and O3 mini have since outperformed O3 under the updated benchmark conditions.