AI Alignment

OpenAI presents a framework and 13 tests across 24 environments showing that monitoring a model’s internal reasoning works better than checking outputs alone for scalable AI control.

OpenAI says it will keep nonprofit leadership while giving its PBC equity to unlock over $100B to build safe, beneficial AI for humanity.

This work shows that training language models on wrong answers can spread misalignment, pinpoints an internal feature causing it, and demonstrates it can be reversed with minimal fine-tuning.

OpenAI has appointed Zico Kolter to its board to strengthen governance with AI safety and alignment expertise.

This introduces research on using deep learning generalization to let weak supervisors reliably control much stronger AI models for superalignment.

It explains how ChatGPT’s behavior is set today and how OpenAI plans to improve it with more user customization and public input.

InstructGPT uses human-in-the-loop alignment training to better follow instructions than GPT-3 while being more truthful and less toxic, and it’s now the default model on the API.

OpenAI is testing a “confessions” training method to get language models to admit mistakes or bad behavior, improving honesty and trust.

OpenAI surveyed 1,000+ people worldwide to compare their views on how AI should behave with its Model Spec, using the results to shape AI defaults around diverse human values.

A deeper look at what we missed about sycophancy, what went wrong, and what we’ll change next.

A new rule-based reward method helps align AI models to behave safely without needing lots of human-labeled data.

A new math-solving model improves accuracy and alignment by rewarding each correct reasoning step instead of only the final answer.

We’re developing AI that better learns from human feedback and helps humans evaluate AI, aiming to build an aligned system that can solve remaining alignment challenges.

It explains how to use human feedback to scale oversight and improve AI at hard-to-judge tasks like summarizing books.

Apollo Research and OpenAI tested frontier AI models for hidden misalignment (“scheming”), found signs of it in controlled experiments, and shared examples plus early stress tests for reducing it.

ChatGPT is built to be useful, trustworthy, and flexible so you can customize it to your needs.

A new alignment method teaches o1 language models safety rules and how to reason about them to behave more safely.

A $10M grant program funding technical research to align and make superhuman AI systems safe, including interpretability and scalable oversight.

It urges starting now to plan how to govern superintelligent AI systems far more capable than AGI.

AI critique-writing models help people spot errors in summaries, and bigger models critique better than they summarize, aiding human oversight of AI.