How does a neural net represent language? See the visualizations and geometry in this PAIR team paper and blog post
The blog at explores the shape of these representations, with math and visualization. A mysterious “squared distance” effect they found turns out to be surprisingly natural!
Very cool exploration of the geometry of language embeddings, with some fun math I did not know.
Analyzing and visualizing syntax trees in the high-dimensional spaces of neural nets. Check out the new PAIR paper on BERT geometry And the blog post on “Language, trees, and geometry in neural networks”
We still don't know how neural networks represent linguistic information but PAIR team from Google adds some really nice insights to this discussion. #AI #ML
"Language, trees, and geometry in neural networks" this post is awesome. connects a number of things in ways I'd never thought of. can't wait to properly digest the paper :D