Almost half (44%) of our tested claims are now expired in today’s models. Another 34% have weakened significantly, 16% persist, and 6% have strengthened relative to the original claim. Persistence varies by topic. Five of nine reasoning claims and six of eleven sycophancy-and-deception claims are classified as expired. Three of ten claims about models as judges, reviewers, and raters persist or strengthen.
Stanford researchers retest 50 widely cited LLM findings - most fade within 15 months via x.com
Discussion