RJ Honicky Sure! I first read about it in the Quanta magazine last year: quantamagazine org a-new-approach-to-computation-reimagines-artificial-intelligence-20230413 There is actually a website dedicated to the topic (hd-computing com), and the first paper mentioned in the "Course: Computing with High-Dimensional Vectors" section gives a thorough explanation of the basic principles. I'm not so much a machine learning specialist/researcher as just someone who likes to read about interesting science, but this felt like the first intuitively understandable (even if incomplete) model of how/why the seemingly nebulous transformations of neural activations can result in meaningful manipulation of knowledge. (Sorry about not providing direct links -- I'm a new user, so they're not allowed)

Out of curiosity: have you heard of vector symbolic architectures (also known as hyperdimensional computing)? They are not LLMs, but in some ways seem to have very similar underlying dynamics, and tend to use 1-bit or ternary representations. Though, I think their biggest draw is the easy-to-understand mathematical framework for how arbitrarily complex knowledge structures can be meaningfully built and manipulated in a high-dimensional vector space. In case you decide to give them a closer look, it would be interesting to know your thoughts on whether VSAs and LLMs might be doing something fundamentally similar. :-)