In February 2024, Google’s Gemini model produced widely shared images in which prompts for historical figures and groups returned people whose depicted race or gender did not match the historical record. On February 23, 2024, Google published a post by Senior Vice President Prabhakar Raghavan titled “Gemini image generation got it wrong. We’ll do better,” explaining the company’s response and what had gone wrong.
In the post, Google stated that “we temporarily paused image generation of people in Gemini while we work on an improved version.” It described two underlying problems. First, “our tuning to ensure that Gemini showed a range of people failed to account for cases that should clearly not show a range.” Second, “the model became way more cautious than we intended and refused to answer certain prompts entirely.” The company acknowledged that “some of the images generated are inaccurate or even offensive.”
Google committed to fixing the feature before re-enabling it, writing that “we turned the image generation of people off and will work to improve it significantly before turning it back on. This process will include extensive testing.” The episode is a primary-sourced example of a company publicly documenting one of its own product failures, and it illustrates how interventions intended to reduce bias in model output can themselves produce inaccurate results.