Intelligent content that gives practitioners, innovators and leaders an inside look at the present and future of ML & AI technologies.

LATEST
Play Video
EPISODE 727  |  
April 14, 2025
In this episode, Emmanuel Ameisen, a research engineer at Anthropic, returns to discuss two recent papers: "Circuit Tracing: Revealing Language Model Computational Graphs" and "On the Biology of a Large Language Model." Emmanuel explains how his team developed mechanistic interpretability methods to understand the internal workings of Claude by replacing dense neural network components with sparse, interpretable alternatives. The conversation explores several fascinating discoveries about large language models, including how they plan ahead when writing poetry (selecting the rhyming word "rabbit" before crafting the sentence leading to it), perform mathematical calculations using unique algorithms, and process concepts across multiple languages using shared neural representations. Emmanuel details how the team can intervene in model behavior by manipulating specific neural pathways, revealing how concepts are distributed throughout the network's MLPs and attention mechanisms. The discussion highlights both capabilities and limitations of LLMs, showing how hallucinations occur through separate recognition and recall circuits, and demonstrates why chain-of-thought explanations aren't always faithful representations of the model's actual reasoning. This research ultimately supports Anthropic's safety strategy by providing a deeper understanding of how these AI systems actually work.
RECENT

INSIGHTS

LATEST REPORT

Retrieval-augmented generation promised to bring ChatGPT’s magic to enterprise data. But while organizations rushed to build chatbots, they often struggled to deliver real business value. This comprehensive guide reveals RAG’s full potential beyond conversational interfaces.

Community

The TWIML Community is a global network of machine learning, deep learning and AI practitioners and enthusiasts.

We organize ongoing educational programs including study groups for several popular ML/AI courses such as Fast.ai Deep Learning, Machine learning and NLP, Stanford CS224N, Deeplearning.ai and more. We also host several special interest groups focused on topics like Swift for Tensorflow, and competing in Kaggle competitions.

TWIML Community

Work with Us

TWIML creates and curates intelligent content that helps makers build better experiences for their users, and gives executives an inside look at the real-world application of intelligence technologies. We also build and support communities of innovators who are as excited about these technologies as we are. We advise a variety of leading organizations as well, helping to craft strategies for taking advantage of the vast opportunities created by ML and AI.