On Thursday, weeks after launching its strongest AI mannequin but, Gemini 2.5 Professional, Google revealed a technical report exhibiting the outcomes of its inside security evaluations. Nevertheless, the report is mild on the small print, specialists say, making it tough to find out which dangers the mannequin would possibly pose.
Technical studies present helpful — and unflattering, at occasions — information that firms don’t all the time broadly promote about their AI. By and enormous, the AI neighborhood sees these studies as good-faith efforts to assist impartial analysis and security evaluations.
Google takes a special security reporting method than a few of its AI rivals, publishing technical studies solely as soon as it considers a mannequin to have graduated from the “experimental” stage. The corporate additionally doesn’t embody findings from all of its “harmful functionality” evaluations in these write-ups; it reserves these for a separate audit.
A number of specialists TechCrunch spoke with had been nonetheless dissatisfied by the sparsity of the Gemini 2.5 Professional report, nevertheless, which they famous doesn’t point out Google’s Frontier Security Framework (FSF). Google launched the FSF final 12 months in what it described as an effort to establish future AI capabilities that would trigger “extreme hurt.”
“This [report] may be very sparse, accommodates minimal info, and got here out weeks after the mannequin was already made out there to the general public,” Peter Wildeford, co-founder of the Institute for AI Coverage and Technique, advised TechCrunch. “It’s unimaginable to confirm if Google resides as much as its public commitments and thus unimaginable to evaluate the security and safety of their fashions.”
Thomas Woodside, co-founder of the Safe AI Mission, stated that whereas he’s glad Google launched a report for Gemini 2.5 Professional, he’s not satisfied of the corporate’s dedication to delivering well timed supplemental security evaluations. Woodside identified that the final time Google revealed the outcomes of harmful functionality exams was in June 2024 — for a mannequin introduced in February that very same 12 months.
Not inspiring a lot confidence, Google hasn’t made out there a report for Gemini 2.5 Flash, a smaller, extra environment friendly mannequin the corporate introduced final week. A spokesperson advised TechCrunch a report for Flash is “coming quickly.”
“I hope it is a promise from Google to start out publishing extra frequent updates,” Woodside advised TechCrunch. “These updates ought to embody the outcomes of evaluations for fashions that haven’t been publicly deployed but, since these fashions might additionally pose severe dangers.”
Google could have been one of many first AI labs to suggest standardized studies for fashions, but it surely’s not the one one which’s been accused of underdelivering on transparency recently. Meta launched a equally skimpy security analysis of its new Llama 4 open fashions, and OpenAI opted to not publish any report for its GPT-4.1 sequence.
Hanging over Google’s head are assurances the tech big made to regulators to take care of a excessive commonplace of AI security testing and reporting. Two years in the past, Google advised the U.S. authorities it could publish security studies for all “important” public AI fashions “inside scope.” The corporate adopted up that promise with comparable commitments to different nations, pledging to “present public transparency” round AI merchandise.
Kevin Bankston, a senior adviser on AI governance on the Middle for Democracy and Expertise, referred to as the development of sporadic and imprecise studies a “race to the underside” on AI security.
“Mixed with studies that competing labs like OpenAI have shaved their security testing time earlier than launch from months to days, this meager documentation for Google’s high AI mannequin tells a troubling story of a race to the underside on AI security and transparency as firms rush their fashions to market,” he advised TechCrunch.
Google has stated in statements that, whereas not detailed in its technical studies, it conducts security testing and “adversarial crimson teaming” for fashions forward of launch.