The model knew the answer was 12. It kept saying 36. Then it started writing in all caps that it was possessed by a demon.
Emmanuel Ameisen, Interpretability Researcher at Anthropic, on what they found when they looked inside, and why it’s so hard to catch a model thinking badly.
