GPT-5.6 Sol API and credit pricing cut by over 20%Claude Security now scans GitHub repos with Mythos 5 Zero Data Retention continues with Private Safety Processing preview for stronger safety Claude can now send Gmail emails and manage Google Drive files Strengthen monitoring for high-risk training and pause RL runs Approved defenders can advance advanced vulnerability research with GPT-5.6-Cyber Cyclone forecasts now provide over a day of extra lead time Talk while reasoning or using tools without conversation breaks AI delivers new results on 10 long-standing math problems with proofs released GPT-5.6 Luna and Terra prices drop 80% and 20%, with faster Sol option added Opus 5 now available on all paid plans and API Securely link health records to understand symptom changes and test results in context Run code inside notes for deeper analysis GPT-Red boosts prompt injection resistance significantly Cut lesson prep time with AI You can move from conversation to documents faster Run AI inference in the browser and cut wait time Review how you use Claude and cut waste Long tasks can move from draft to presentation more easily Track the latest safety rules for bigger models GPT-5.6 Sol API and credit pricing cut by over 20%Claude Security now scans GitHub repos with Mythos 5 Zero Data Retention continues with Private Safety Processing preview for stronger safety Claude can now send Gmail emails and manage Google Drive files Strengthen monitoring for high-risk training and pause RL runs Approved defenders can advance advanced vulnerability research with GPT-5.6-Cyber Cyclone forecasts now provide over a day of extra lead time Talk while reasoning or using tools without conversation breaks AI delivers new results on 10 long-standing math problems with proofs released GPT-5.6 Luna and Terra prices drop 80% and 20%, with faster Sol option added Opus 5 now available on all paid plans and API Securely link health records to understand symptom changes and test results in context Run code inside notes for deeper analysis GPT-Red boosts prompt injection resistance significantly Cut lesson prep time with AI You can move from conversation to documents faster Run AI inference in the browser and cut wait time Review how you use Claude and cut waste Long tasks can move from draft to presentation more easily Track the latest safety rules for bigger models

Official sources only. Rumors, leaks, and get-rich schemes are excluded.

← Back to top

AI BriefingOpenAIPolicy00:00

AI summarized from verified sources

OpenAI Plans to Retire SWE-bench Verified Evaluation

Users can choose more trustworthy model performance metrics without relying on potentially misleading benchmark scores.

SOURCE CHECK

1 sources

VERIFIED

Sources

Primary / openai.com

Official Blog

Key Points

1Verified scores distorted by contamination, less reflecting true skill
2Testing excludes many correct solutions
3OpenAI encourages adoption of SWE-bench Pro reports

OpenAI announced they will discontinue SWE-bench Verified due to its diminished ability to measure cutting-edge coding skills. Test flaws and data contamination reduce its reliability. They recommend using SWE-bench Pro instead, signaling forthcoming revisions to evaluation methods.

What changed

OpenAI announced they will discontinue SWE-bench Verified due to its diminished ability to measure cutting-edge coding skills. Test flaws and data contamination reduce its reliability. They recommend using SWE-bench Pro instead, signaling forthcoming revisions to evaluation methods.

Why it matters

Users can choose more trustworthy model performance metrics without relying on potentially misleading benchmark scores.

What to watch

Users can choose more trustworthy model performance metrics without relying on potentially misleading benchmark scores. Key checks: Verified scores distorted by contamination, less reflecting true skill / Testing excludes many correct solutions / OpenAI encourages adoption of SWE-bench Pro reports.