Study finds LLMs mostly accurate on climate facts, but pushback tests reveal gaps
Researchers from University College London and UNESCO tested how large language models handle climate science questions, examining 33 models across a range of benchmarks.

Researchers from University College London and UNESCO tested how large language models handle climate science questions, examining 33 models across a range of benchmarks.
The study scored models on 600 questions covering textbook multiple-choice items, reading comprehension drawn from scientific papers, and questions about how confident climate scientists are in conclusions from IPCC reports. The last category proved hardest: flagship models answered about 78 percent correctly, while the smallest open-weight models managed around 47 percent. On the other benchmarks, frontier and large open-weight models generally scored near 90 percent, with small open-weight models in the 70s.
The research also examined more complex interactions, including cases where users push back on an answer because they believe they are right. Because LLM responses are served ephemerally to individual users and confined to their chat sessions, there is no direct way for researchers to observe what information people actually receive, so the team tested models with varied prompts and evaluated their outputs from multiple angles.
The findings suggest strong performance on straightforward factual questions but more uneven results as tasks grow harder and interactions more adversarial.
Source: Ars Technica
Ready to Modernize Your Business?
Get your AI automation roadmap in minutes, not months.
Analyze Your Workflows →