Show notes
Scot Morris, OD, and Rehan Ahmed, MD, on the limits of current AI and the role of data architecture in healthcare.
- The limits of LLMs: A Google DeepMind position paper titled “LLMs Can’t Jump.” AI has mastered pattern recognition (induction) and logical deduction, but lacks abduction: the ability to invent a new explanation when no existing pattern fits.
- Understanding data models: Dr. Morris walks through the four core data models (relational, document/NoSQL, graph and vector) and how patient care is becoming a structured data-modeling process.
- The EHR problem: Today’s EHRs are digitized 25-year-old billing tables that act as closed silos. The hosts call for open architecture so better tools can connect medical data.
Transcript
Welcome to Real Talk on AI in Eye Care with your host, Dr. Scot Morris. And I'm co-host, Dr. Rehan Ahmed. In each episode, we discuss current news, add in a little AI education, and debate innovative AI tools and topics that will change the industry. Hey, Scott. So for News of the Week, it's actually connected to a couple weeks ago. Remember we talked about the AI from Harvard that sort of reopened? Almost like 400 cases, kind of these unsolved mysteries, and it handed clinicians diagnosis with their wealth. So this week's story is kind of like, honestly, it's like a splash of cold water. And the title is really great. It's a, it's a position paper out of a scholar from Google DeepMind.
That's titled LLMs Can't Jump. It's like the title of a movie or something. LLMs can't jump. So the author is- I love the title. Yeah, yeah. It's a great paper. I highly encourage it. If you just Google LLMs Can't Jump, you're going to get to it. Because the author is Tom Zahavi. He, and he's not a nobody. He actually co-leads DeepMind's discovery team. So this is the same team that came up with AlphaGo, the protein folding thing, the model, and all the other tons of amazing stuff. So he starts the position paper by talking about Einstein and how Einstein came up with the theory of relativity. Einstein first, he wrote a letter to his friend, a guy named Maurice Solovine, sketching how discovery actually works.
It's a cycle that runs from an actual, physical, sensory experience through a quote, quote, intuitive metaphorical jump, sort of up to axioms, like rules, and then that down through logical deduction. So that's kind of like how, how we discover things, like simple things and also great things like relativity. So he comes up with, so Zahavi talked about three classical modes of inference, and I'm a philosophy major, so I'm kind of nerding out on this stuff. So the first rule is induction. And we sort of remember that when we were like, you know, going through high school and college, what induction is. That's deriving rules from patterns and data. Then there's deduction, deriving consequences from rules. And then here's this thing I actually haven't heard of called abduction.
Like abduction, like abducting. Like abducting your eye, yeah. Okay. Inventing an explanation for a surprising observation. So he, so this guy, Tom Sahabi, he argues that AI, and we can't disagree with this, AI is definitely mastered the first. It is rapidly conquering the second. So talking about deduction and induction, you know, that's kind of like what we've talked about. You know, and thinking about like how these AIs are sort of solving mathematical. So it's definitely done, you know, deduction, induction. But his point is it just fundamentally lacks the machinery for the third, and that's the job. So he studies the general relativity. This is the detail I find sort of really fascinating.
In 1907, there was no data crisis. So the Newton's gravity had been verified. They sort of figured it out. And what's interesting, if you give an LLM all the data up until 1907, it also doesn't come up with relativity, which is also an interesting thing, right? Like it doesn't just come up with it on its own, but why not? So Einstein, what he writes in this letter is that he imagined someone falling from a tree, just in his mind. He imagined someone falling from a tree, and this idea of weightlessness happened. Then he comes up through a number of different things, like this is how he comes up with relativity. And it's kind of interesting. Like it's like when you have commercial LLMs ask them to do some creative challenge, their outputs that they come up with are much more alike.
They're much more clustered than what humans come up with. So how does it work? There's also a lot of AI slop in that, I think. Tons of slop. So, you know, it's a real example. So we always talk about autonomous diabetic retinopathy screening. That's induction, mastered, billable, codable, and then the rare disease thing we talked about a couple weeks ago. That's not a counter example. Those diseases were already in the literature, just somewhere. And it took a machine to comb through the needle and the A-stuff, metaphorically, of course. So that's induction and deduction at supreme scale, and it's amazing. But as of now, LLMs, and this is an important distinction, LLMs, they can't jump.
They would never invent it. They can't create something that doesn't exist. But it's funny how the paper ends. And you could probably sort of figure out where it's going. So his proposed fix is, we've talked about it in the past, and we've talked about it in the future, is world models. Or I think what we called it physical AI, AI grounded in physical sensory simulation. So we will leave that for a separate topic, but I just wanted to talk about it. So that paper came out just a week ago, a week or so ago, sorry. And we will have a link for those who are interested. But Scott, what are your thoughts? Well, we've had this conversation all along, is it can create based on patterns, but if no pattern exists, an LLM is useless.
Because it has to have a pattern to follow, as you said, induction and deduction. But any time you ask it to do something creative that's in your mind that doesn't exist somewhere else, you get nothing. I'm trying to design, there's a future podcast I want to cover called The Innovative Practice of the Future. And so I'm trying to design a blueprint for it for another project I'm working on. And I kind of laid all this out, and I literally had three pages of prompt. I thought, okay, I cannot be any more detailed than this. And what it came up with was so far off base, I was like, man, where did I go wrong on that? And then it kind of hit me, kind of just what you said, is that I asked it to create something that was in my brain that doesn't exist, because there are no models for it, and it completely failed.
Yeah, what I do like is if you already have a creative idea, it's great to bounce that idea off of and sort of like pressure tests, certain assumptions. Coming up with a brand new idea. Now, I hope for our audience, we sort of, I don't call it proof, but you sort of get where the limits are. And maybe, I know Scott, you had a recent column on education, and I know you touched on this. Maybe this is where we steer education on the things that LLMs can't do. True. True. Good point. Something kind of brings me into the knowledge bite for the day. You talked about models towards the end. And I think that's something that, once again, and we kind of try to not beat a dead horse with this, but data is the new goal, the new oil, as you always say, Rayon.
But the challenge that is, is that where does that data come from? And like we were just discussing, if there is no data out there, LLMs have a hard time. But for the audience, remember, LLMs are only one type of AI. They're not all of AI. They're one type of AI. But the type of AI that's what we're all using right now, and it's kind of in the news and blah, blah, blah, but it's completely dependent upon data. And so I want to talk a little bit about what are data models. And so, you know, let's talk about, and I want to try to put it in perspective of your practice, right? Because we're getting to that point now where everybody's like, yeah, but how do I use it in my practice?
And so I want to talk about how your practice runs. And this is kind of a different mindset, if you will. We've never really thought, or at least been thinking the last three or four years, I've started to think like that way. What are we really creating? It's not just a patient experience. It's a data model about that individual. And then we do it 25, 30, 40 times a day. We're creating a data model. And when you stop looking at it as, I'm just taking care of patients, but I'm taking care of data, you start looking at things a little different. And, you know, in eye care, we generate huge amounts of data. I mean, whether it be OCT scans or visual fields or chart notes or, you know, our ICD-10 and CPT codes or what's your IOP trend or, you know, I mean, the data alone, there's a lot of data, but it's just noise.
And that's right now with the phase we're in. We're in the noise phase where we're collecting all this data, but we've not really given it any structure. And we're then we're going to feed it into an LM and say, okay, I want you to give it structure. And these were the LMs are starting to struggle because we haven't given it a good structure model. And so I want to talk a little bit about what is a data model and simply put, it is an architectural blueprint that defines how information is organized, stored. And the last part is the most important, connected. And I think that we're going to get in our fireside. We're going to talk about, you know, where information is gained, organized and stored.
But our key critical process in what we're struggling right now in our industry is connectedness. So, you know, I look at this one, your data model, when we start building them right and we use the right clinical workflow. And I think that's a really big piece of this puzzle. When we have the right data model and we can plug it into the right clinical workflow correctly, now is when we start connecting things and we start decreasing friction. Until that point, we're creating a lot of duplicate entry, a lot of disconnected devices and a lot of friction. So I want to break down really quick four different core kinds of data models. So one is what's called a relational model.
And this is by far what most of what we've done in the last few years is about, where we organize data in district tables with rows and columns. I think most people think spreadsheet. Now, obviously, we're way past Excel spreadsheets, but these tables are similar to rows and columns, each connected by some different calculation, formula, key, whatever the case may be. And, you know, if you think about this, your EHRs that every one of us use is a table. That's all EHRs is a big, huge data table that we input information into. Table A maybe scores, stores demographics and maybe table B stores appointment times. And table C stores, I don't know, CPT codes.
And then some ID links them together. And that's a very structured data table. But the problem is it's really hard to force what's happening now into those data tables. Like we collect them doing a project for one of my companies and there are 720 or more data points in an exam. Most EHRs collect about 200. So where does all the rest of that data go? Who knows? It doesn't go anywhere. And there might be a lot of gold or oil, as Rayon always says, in that lost information. So we have relational models. Think of them as structural tables. And then we have a second model, which is kind of document models, or you may hear of calls, no SQL, no SQL tables.
Here, there is no table. It's a flexible document and usually use something called JSON files rather than rows. It's almost like taking all the data and throw it into a big pot and saying, okay, go look for these sequences of information. And that's what we're kind of doing with our specialty exam templates or much of our diagnostic exports. We're just saying, create a file, throw it in there. And some day, somehow, we're going to be able to start accessing that data. We have a little way to go, but those are the no SQLs or the data pots, if you will. Third model is called a graph data. And this is something that's becoming more and more common.
We're starting to leave the first two models and get into these graph data models. Now we're working on relationships between the data points. So now we're moving more into, instead of dumping it all in the pot, we're throwing in and going, what's the relationship between these data points? We're now starting to make 3D and maybe even to some degree 4D models based on how these data points are connected. In other words, we talk about this a lot with oculomics. Imagine mapping that somebody has type 2 diabetes. And at the same time, you're mapping that patient's hemoglobin A1c history with when they switch medications with what their macular edema severity is and if they have any one of these days, their genetic risks.
So now we're starting to look at relationships of how things are connected across lots of different factors for a single patient and then soon hundreds or thousands or tens of thousands or hundreds of thousands of patients in terms of looking for trends. And then last but not least is the fourth model. And this is the model that, you know, I'm not sure, with exception, the most advanced models were really there yet, but these are called the vector models. And these are where we're taking all of this unstructured data, think of it like an OCT, and you're throwing it into a vector and then saying, okay, I want you to take these high res images or text and create these into coordinates on a grid, kind of somewhat related to the graph model.
It's like the next evolution of the graph model. So this is really looking at diagnostic pattern recognition where instead of taking an image, you're now creating a digital trail that then computers, machines can look at and go, oh, well, let me find what the similarities are, the differences are, the trends, the patterns. So we can start looking at instead of looking at it as an image, it's converting into true digital data. Well, why does any of this matter? Well, and I think we're going to talk a little bit about this in a few minutes when we get to the fireside, but why does that matter? Because all of us need to start thinking about what is the architecture for the data that we're inputting into our EHRs every day?
Because if you have the right data, but you put it in the wrong place or put it in the wrong format, it's useless. Just as useless as if you don't correct it at all, don't collect it at all. So it's great if you have data, but if we don't use it or don't store it in a way it can be used, it's really not that much, it's not that helpful. So I want you to think about, especially as we go on these next few weeks in terms of what we're going to talk about, about where's your model, where's your data going, and are you actually able to use the data? So how is that patient data being structured? Behind the curtain, and the curtain is right now, the universal databases we're using, which are EHRs, which leads us into, Rayon, let's talk about the problems with all today's technology.
Yeah, that was a great summary of data models and data is definitely the new oil. This comes up because startups will reach out to us and talk about their new idea. And one of the things that often comes up is we have a new mousetrap that can do something better, talk to patients in a better way, do diagnostics in a better way, some type of clinical workflow in a better way. The issue is often, and we love our EMR systems, but they are the ones closest to it.
It really is about who owns the data, who's collecting the data, who has access to data, and more importantly than who can block others from that. Ah, there you go.
And that's, I think, something we're struggling with and we really have to talk about whether we're stifling innovation here. And there may be a role in the future in which EMRs are really one of the data models you could talk about and just a data model and then you have other and as clinicians, we can have the choice of how we want to interact with it. This is the sort of future I would love to live in. I don't want my EMR doing everything, everything kind of mediocre, in a mediocre way. I'd rather have the best of a lot of different things doing something great. It would be great if it would be one ecosystem like if people are on their Google or their Apple and they're doing everything.
But right now, our EMRs, I've yet to hear one person, it's like one of those things, I love my EMR said no one ever. Never, ever. As of now, I mean, people do, they're getting better. I think there's goods and bads and all these sorts of things. But, you know, and they're evolving too and they're working on the same problem but I would just, my humble plea is that they open up their systems for others to try new things. Yeah, we had a conversation about that a few months ago about begging, pretty much begging for open architecture. And you brought up a great point, Rehan, is that, you know, when I was talking about data models, all the EHRs right now are pretty much just table models.
That's all they are, first version data models. And I do think, will that still exist? But will there be some interface or next level platform that takes the EHRs data and takes scan data and takes genetic data and takes laboratory data and access a vector and says, let me combine all those things, right? Because right now, as you said, and I was having this conversation yesterday morning with a company that has this really new cool product, can't wait for it to come to North America. It's amazing, but it's going to fail because they don't understand workflow, number one. And number two, their data doesn't coincide or isn't going to be something that can be put into an EHR.
So right now I'm like, so you want them to go open another dashboard with another username, with another password, in addition to the 20 that everybody has to use every day. I mean, we have document management systems, we have, you know, coverage and benefit engines, we have communication systems, we have our EHR, we have our e-prescribing, you know, I mean, the list goes on and on and on of all of the subscription model. And that's because we keep trying to tack things on to a data model that we built 25, 30 years ago. And not that those data models like EHRs or like communication systems or, you know, phone systems or document management, not that those are important, but we're going to soon need something to connect all those.
Yeah, because those EHRs are built and they serve a different purpose. And they're primarily a billing and coding thing. They're primarily billing and coding. They're ultimately billing and coding database. Well, actually, we digitized health records. That's all we did. We digitized what we used to write down on a piece of paper. But it doesn't help with clinical decision making in any way at all. Much talk about this in future episodes, Scott. I think we opened up another proverbial can of worms. But excited to hear everyone's feedback. Again, all of our podcasts, our articles, our columns. Scott, I know you're a great one this week. We'll talk about it soon.
I know. I know. We'll be is at aiinicare.com. You could reach me at Raycon, R-E-H-N at aiinicare.ai and Scott S-C-O-T at aiinicare.ai. Thanks for a great episode. We will talk to you next week.