The Beginning
The lab is named after Paul Erdős, who worked on whatever problem was in front of him with whoever was interested. It was started by students, and I joined as a founding AI researcher in February 2026. I worked there until July.
What I worked on
Three threads, and they turned out to be the same problem seen from different angles.
Knowledge distillation
You train a big teacher model, then try to pour what it knows into a smaller student. The student always stops short. I built pipelines to do the compression, but the part I actually cared about was why it stops - which pieces of what the teacher knows survive the transfer, and which ones quietly do not.
Representation learning
A model builds its own internal picture of the world, and that picture is what you are really moving when you distil or transfer anything. I looked at how those representations form, whether they hold up on data from a different domain, and why a model asked to learn something new so often forgets what it already knew.
Mechanistic interpretability
The same question again, from the inside: instead of measuring what a transformer outputs, open it up and read the attention patterns and circuits directly. Knowing which internal structure does the work is what tells you what distillation is failing to carry across.
Where it went
The distillation work became a paper on the minimum width a student network needs, and the interpretability threads fed into the SAE experiments. If any of it is useful to you, get in touch.
The research write-ups are on the Erdős AI Lab page.