The example given for inverting an embedding back to text doesn't help the idea that this effect is reflecting some "shared statistical model of reality": What would be the plausible whalesong mapping of "Mage (foaled April 18, 2020) is an American Thoroughbred racehorse who won the 2023 Kentucky Derby"?
There isn't anything core to reality about Kentucky, its Derby, the Gregorian calendar, America, horse breeds, etc. These are all cultural inventions that happen to have particular importance in global human culture because of accidents of history, and are well-attested in training sets. At best we are seeing some statistical convergence on training sets because everyone is training on the same pile and scraping the barrel for any differences.
I fail to see how that matters. You're implying that all reality is cultural, but that seems irrelevant. The same thing would apply to scientific facts, but whales not having a word for science doesn't make it not real.
If we somehow discover LLMs right after Newton discovered the theory of gravity, and then a while later Einstein discovers General Relativity, then GR would not be in the training set of the neural net. That doesn't make GR any less of a description of reality! You also can't convert General Relativity into whalesong!
But you CAN explain General Relativity in English, or in Chinese to a person in china. So the fact that we can create a mapping from the concept of General Relativity in the neural network of the brain of a human in the USA using english, to someone in china using chinese, to a ML model, is what makes it a shared statistical model of reality.
You also can't convert General Relativity to the language of "infant babble", does it make general relativity any less real?
> You're implying that all reality is cultural,
Let's look at two examples of cultural reality:
Fan death in South Korea. Where people believe that a fan running while you sleep can kill you.
The book "Pure, White and Deadly". Where we discredited the author and his findings and spent decades blaming fat, while packing on the pounds with high fructose corn syrup.
An LLM isn't going to find some intrinsic truth, that we are ignoring, in its data set. An LLM isn't going to find issues in the reproducibility / replication crisis. I have not seen one discredit a scientific paper with its own findings.
To be clear LLM's can be amazing tools, but garbage in garbage out still applies.
You also can't translate "Mage (foaled April 18, 2020) is an American Thoroughbred racehorse who won the 2023 Kentucky Derby" into Hellenistic Greek or some modern indigenous languages because there isn't enough shared context; you'd need to give humans speaking those languages a glossary for any of the translation to make sense, or allow them to interrogate an LLM to act as the glossary.
I'd say our current largest LLMs probably contain sufficient detail to explain a concept like a named race horse starting from QCD+gravity and ending up at cultural human events, given a foothold of some common ground to translate into a new unknown language. In a sense, that's what a model of reality is. I think it's possible because LLMs figure out translation between human languages by default with enough pretraining.
Why QCD? Quantum chromodynamics, the quantized theory of the nuclear strong force? There is also QED, quantum electrodynamics, which is the quantized field theory for electrodynamics, and then also QFD (quantum flavordynamics) for the weak force. Do you seriously mean to imply that the quantum field theory corresponding to ONLY the strong force, plus gravity, explains every emergent phenomena from there to culture? Fully half of the fundamental forces we account for, in two disparate theoretical frameworks?
> You also can't translate .. into Hellenistic Greek or some modern indigenous languages because there isn't enough shared context; you'd need to give humans speaking those languages a glossary for any of the translation to make sense
What? By substitution, this means you can translate it. As long as we're assuming a large enough basis of concept vectors of course it works.
> I'd say our current largest LLMs probably contain sufficient detail to explain a concept like a named race horse starting from QCD+gravity and ending up at cultural human events
What? I'm curious how you'd propose to move from gravity to culture. This is like TFAs assertion that the M+B game might be as expressive as 20 questions / universal. M or B is just (bad,sentient) or (good,object). Sure, entangling a few concepts is slightly more expressive than playing with a completely flattened world of 1/0 due to some increase in dimensionality. But trying to pinpoint anything like (neutral,concept) fails because the basis set isn't fundamentally large enough. Any explanation of how this could work will have to cheat, like when TFA smuggles new information in by communicating details about distance-from-basis. For example to really get to the word or concept of "neutral" from inferred good/bad dichotomy of bread/mussolini, you would have to answer "Hmmmmmm, closer to bread I guess" in one iteration and then "Umm.. closer to Mussolini I guess" when asked again, and then have the interrogator notice the uncertainty/hesitation/contradiction and then infer neutrality. This is just the simple case.. physics to culture seems much harder
I completely believe that physics to culture is intractable given our current corpus, to the degree it's probably a nonsense claim. There are so many emergent phenomena that introduce confounding effects at every level of abstraction.
Also, why QCD? Quantum chromodynamics, the quantized theory of the nuclear strong force? There is also QED, quantum electrodynamics, which is the quantized field theory for electrodynamics, and then also QFD (quantum flavordynamics) for the weak force. Does OP seriously mean to imply that the quantum field theory corresponding to ONLY the strong force, plus gravity, explains every emergent phenomena from there to culture? Fully half of the fundamental forces we account for, in two disparate theoretical frameworks?
OP's comment is not serious speculation.