Hi Matthijs, great column. I wonder if LLMs as they are conceived now can ever get to what you propose based only on the simple statistical models that drive them. I mean, it is easy to imagine scanning text and looking for word-order patterns, without any need to understand meaning. But what connections can we imagine that would link images, diagrams, 3D models, etc? To me, text-based LLMs have to incorporate meaning before we can ever hope to go to higher dimensionality.
Hi Rob, I agree: we may need another paradigm shift to rise from word-based 'reasoning' to something that is closer to 'concept-based reasoning'. I think multimodality needs to be part of that, but may not be sufficient. Still, I'm impressed by how much we can learn from 'emergent behavior,' whether from LLMs or from cardiac electrophysiology.
Hi Matthijs, great column. I wonder if LLMs as they are conceived now can ever get to what you propose based only on the simple statistical models that drive them. I mean, it is easy to imagine scanning text and looking for word-order patterns, without any need to understand meaning. But what connections can we imagine that would link images, diagrams, 3D models, etc? To me, text-based LLMs have to incorporate meaning before we can ever hope to go to higher dimensionality.
Cheers, Rob
Hi Rob, I agree: we may need another paradigm shift to rise from word-based 'reasoning' to something that is closer to 'concept-based reasoning'. I think multimodality needs to be part of that, but may not be sufficient. Still, I'm impressed by how much we can learn from 'emergent behavior,' whether from LLMs or from cardiac electrophysiology.