In its December 6, 2023 launch announcement, Google stated that “Gemini Ultra is the first model to outperform human experts on MMLU (massive multitask language understanding),” a benchmark spanning 57 subjects including math, history, law, and medicine. The claim is Google’s own framing of its top-tier Gemini model’s benchmark result, drawn directly from the launch post.