Even babies have a basic understanding of physics: things don't just go through one another, or wink out of existence. But what's going on in there? In a recent NeurIPS paper we sketch a possible answer