Mechanistic Interpretibility Resources

Mechanistic interpretability aims to reverse-engineer a neural network into human-understandable mechanisms. MI focuses on transformers (specifically LLMs) but is not limited to these neural network architectures

People

1. Neel Nanda
2. Alignment Forum

Primer on LLMs

1. Large language models, explained with a minimum of math and jargon

Transformers

1. Transformers Interpretibility
2. 200 Concrete Open Problems in MI

Quick Guides to MI

1. What is Mechanistic Interpretability and where did it come from?
2. Introduction to Mechanistic Interpretability
3. “Mechanistic interpretability” for LLMs, explained

How to get started with MI ?

1. Concrete Steps to Get Started in Transformer Mechanistic Interpretability

Relevant Papers

1. Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models
2. A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models
3. Mechanistic Interpretability for AI Safety : A Review

Straight from Anthropic

1. Mapping the mind of a Large Language model
2. Interpretibility Dreams
3. Golden Gate Claude
4. Toy Models of Superposition
5. Transformer Circuits Thread

Blogs

1. Neel Nanda’s case on why we need interpretibility research
2. A Microscope into the Dark Matter of Interpretability

Libraries

1. Transfomer Lens
2. Neuronpedia
3. Interpretable Neural Networks