Welcome to Real Talk on AI and I care with your host, Dr. Scott Morris. And I'm co host, Dr. Rehan Ahmed.
In each episode, we discuss current news, adding a little AI education, and debate innovative AI tools and topics that will change the industry. Hey, good evening, Scott. How are you doing? Hey, Rehan.
Good. But man, good to talk to you. I am looking forward to today's real talk. This is real talk number 51.
We have some interesting stuff we're going to cover. Rehan, talk to us a little bit about you and I were talking in our pre-meeting about some of the really interesting news of the day that come out in the last day or so. Yeah, Phil said. So yeah, this is sort of hot off the press.
This is an article that was published in nature and medicine. And it came out just just like two weeks ago, January, or sorry, two and 12. And it was basically what these researchers found. And it sort of caught me very surprised because I use open evidence a lot.
Basically that the punchline is the AI tools that are built and marketed specifically for healthcare providers. And here I've done like open evidence, which is very popular up to date, expert AI. The question was, well, are they actually better than general purpose frontier models like claw GBT 5.2 Gemini 3.1. And you would think, hey, if the AI is specialized in medical knowledge, it should be better, right?
That's what we're using. We're using this. I use open evidence all the time for medical questions. Surprise surprise, the specialized clinical tools.
And the reason we're talking about the big, they lost across knowledge alignment clinicians and real world questions they came in behind. The issue when you read the paper, you find that in sort of like multiple choice questions actually performed similarly, although they the special still certain loss, which is interesting. But when they actually had clinicians grade the answers and they're totally blinded on. You know claw Gemini open evidence, etc.
The specialist tools like open evidence up to date all performed worse. And so, but you know, what is this, you know, how did this happen, right? And the thing that was supposed to actually make these tools safer and better. And for those who haven't used open evidence, I highly, you know, I actually recommend people using just to see how it works in the citations that it curates and how it gets it.
The use is something that we've talked about before in our knowledge about using. It's called a rag or retrieval augmented generation, basically giving them sort of a library of journals and articles from top to your journals and saying you've got you focus on these journals. You focus on the jambas, the, you know, British journal, the blue journal or the BGR, the Bertram, the Mology, etc, etc. And use that when you were accessing knowledge, the problem with these researchers think happened why they're actually performing worse is that when you have these rag systems, it sort of confuses the model.
And there's so they're pulling sources and trying to integrate it with these frontier general purpose models and that degrades performance. Meanwhile, the frontier elements and what's the frontier, I mean like the GPT 5 2s, Claudo, bus, they're iterating faster, they're doing more sort of general alignment work. And especially it's just got past, it's kind of like this weird thing where the generalists are just doing better and better because they're trained on a on a wider variety of data and they don't have to deal with these rag systems. It's kind of like, you know, sky view, you know, if you give a resident.
Or someone like new to medicine like 10 journals and say you can only use these to answer questions, they're going to trip over themselves, they won't be able to reason in the same way. But let's talk about what this data sets to limited right that's the problem. I always thought with the rag is you're only pulling from. You know, a certain set of journals, but the journals don't cover everything right there's a lot of range that's missing there.
Yeah, where I think that the general purpose LLMs are looking at a wider range of stuff that was published in a lot of different places. Maybe maybe the general person elements can make connections in an easier way right. The more understandable way. Yeah, definitely for us as readers.
So a couple of caveats and I guess a caution my caveat would be, you know, this is still text only right. I think there's still says nothing about sort of what we talk about in I and I care like the autonomous DR screening or OCT. Those are like domain specific very image specific. So it doesn't say anything about that just sort of an FYI.
The thing that's the concern to me is that you know hospitals the American Academy of Technology, by the way, a couple weeks ago signed an agreement with open evidence. And it's being used in hospitals all the time. I know my buddies are like working emergency medicine. They love open evidence.
The interesting thing people just adopt these tools without actually testing them, you know beforehand. And we just sort of mean included. Sort of just suit what you're trained on these on these medical journals and on these proper references in a rag system and you sound that way in your fighting citations. You should be better, but no one actually tested it.
So I think that's a sort of a pause for reflection. I'm still using open evidence. I select it. I like the citations, but it definitely gives me pause.
And I think everyone in the field a little bit of pause. It's a super interesting article. But I go back to you know what we see to be the theme we talk about every two weeks is what's the data that all these models are using the answers they're using journal articles, but that's that's a very limited set. Very biased set of data to learn from.
And you know, I think I'll go back and preach the speech I always do is until we have a universal database where we can look across genotype phenotype. And all of this is going to have issues. It's going to have bias and you know, I mean, I know where a few years away from that, but you know, I think everybody's I think everybody whether you be the LM's or you be the super specialized kind of models like open evidence. Everybody's looking for a foothold.
But I still think we're we're not pulling from the right information because the right information doesn't exist yet. It's interesting. I mean, I I when you're talking about that is doing a little homework on that and I'm like, oh my gosh, so all the stuff that we've been doing. It's like going and listening to I'm not trying to pick on any professors, but going listen to five professors.
And thinking that that's all there is to know. Yeah, yeah, that's that's not the way that works. Yeah, I think you're telling me talking about cyber security in our in our knowledge by. Yeah, you know, I definitely need to learn a lot about so.
Yeah, it's something that I really, you know, I think we all go along just thinking well, our information secure and my business partner and my private in my private, private artificial intelligence project is he actually does a cyber security for a living. That's his real job. And every time I say something, he's like, Scott, that's not secure. Scott, that's not secure.
Scott, that's not secure. And I'm like, what is security goes nothing. So so I thought about that today. I was like, we're having a conversation with something we're building and he said, well, how do we protect that?
And I said, well, you just going to put cyber security on and he goes. He finally goes, Scott, even though cyber security means that I kind of went over some of the stuff I'm about to cover with you guys. And he looked at me smiley goes, well, that's part of it. So let me fill in the rest of the gaps.
So. And actually, I think most people who do cyber security don't say cyber security. They say cyber defense. And I thought that's an interesting difference in between being secure and playing defense.
And I'm not a play defense guy. I'm always like a play offense guy. So I thought that interesting that statement. So let me kind of lay this out for everybody.
So I want you to think of cyber security kind of like these digital walls. You know, almost feel like we're going into matrix, right? He's in this cave. He's got all the walls of digital information surrounding him.
And cyber security is these digital walls to protect our data. And in this case, we might talk about patient data. We also might be seeing talking about our own personal data or practice data, provider success data. You know, and historically, we've kind of just said, well, when there's an attack, we'll patch that vulnerability.
You know, every every time you see you have an update, really what those are is vulnerability patches for some defect that somebody found and not trying to fix it. But the challenge is is that same LLM that we were talking about is better than open evidence. Well, now you have, let's play let's say them bad players, right hackers. They're using AI to automate attacks and create these super hyper realistic fishing emails and kind of bypass all these traditional firewalls.
We've already used, but do it in milliseconds, right? And so these static defenses, I go back to like Norton antivirus years ago, you know, we download a program and then every six months you went and bought another one. Those kind of static defenses that those are long gone, right is these dynamic instantaneous machine threats have to be dealt with very, very quickly. And to stop those, let's call them the bad actors.
We have to start thinking about what our cyber defense. Really is like, what are our common measures and how do we stay ahead of those attackers whose job is to feel information. You know, we talk, we talked to a couple of other times, Rayon is that you're the greatest, the single greatest data set that has yet to be attacked. Is patient health data.
You know, I mean financial data, you know, social security numbers bank accounts. None of that even compares the health data and specifically genetic data. So I think is in the healthcare world is something we really have to start paying attention to. So, you know, the challenges when we upgrade to the newest cyber defense, you know, what we're doing is we're shifting from being vulnerable to.
I hesitate to say impenetrable because that's always a danger statement, but it at least gives us some peace of mind. And you know, I think there's lots of hospitals and ophthalmology practice and optometry practice over the last five years, who have been caught with ransomware and have paid a fortune and then still not got their data back. And, you know, all because they said, well, I don't want to pay the 399 monthly defense fee, right? Well, OK, then pay $100,000 later.
And so. The challenge with that is, is we get more and more tools that all integrate. We now run into cyber defense has to be across a lot of different integrated systems, not just a simple like EHR. And so the more players you have plugging in the harder it is to have your data guards that a body guard will call your data guard kind of protecting you.
And yet at the same time, we have to have cyber defense because our patients trust that we're going to protect their data. I always think about it now. I'm like the most unsafe place for data to be these days is an EHR, but it's what we all use. So I think cyber security or cyber defense is what I was thinking about.
So let's say, well, how do we, how do we do these? What are some of those measures? And the first one is that a lot of cyber defense systems are starting to look at behavioral analytics. So it's saying, let's look at network traffic and let's say all of a sudden one account that has been averaging 50 patients a day.
All of a sudden has 300 patients 300 encounters or 300 hits on their website. That is behavior. I say, this is a different behavior than we have seen. And it's going to instantly freeze any accounts or they have this unusual behavior.
So that's called behavioral analytics embedded with AI. And there's things like AI power email filters. And I think about in our office every day, and we get hit with hundreds and hundreds of emails from patients and industry and spam and all that kind of stuff. And like all it takes is one person opening the wrong email and we have a big problem.
And so what type of, you know, quarantine systems or phishing defense systems are out there. And then last, you know, is that I had never really heard this term before and it's called zero trust network access, meaning that candidly, I don't trust anybody unless they have we get to verify every single user that they're the right person. And then that's not username and password anymore. It's going to, you know, dual factor identification.
So you guys will see that DFA and a lot of things, dual factor identification. And that's where you have to somehow use a longer user name and log it and then they send you a code. But our systems are getting more and more sophisticated in that cyber defense now is your iris is match to your fingertips match does your voice match does your face match. So it's no longer going to be a two system dual factor.
Lots of people are talking about triple and quad factor identification where you'll now have not just your username and password, but some other form of biometrics to help make sure that you are actually using it. And you say it are you say why and at the end of the day, though, you know, I think the fourth pillar is psychology, meaning that your team, your culture has to support this and if they don't follow the internal cyber defense rules, we're going to have problems. And you know, so you have to treat every, let's say, every link that looks suspicious and go, is it we sure this is really it. Are we really using multi factor authentication or are we bypassing it because you can bypass if you want that idea.
You know, so though AI is definitely giving us all kinds of cutting edge technologies and opportunities. I think our. Let's call it collective vigilance is still going to be a really important part of what we do over the next few years of creating a system where we have cyber defense to protect everybody's information. So hopefully everybody learned a little bit and made you think a little bit about the next time you go, I won't use my fingerprint or my face.
I'm just going to put my little for digit code in. So think about that in terms of it only takes one person. One bad player to really corrupt what we do. Absolutely.
And now it's getting easier and easier. You probably heard that the claw and bent those and fable models are banned. We can't get access to them primarily because of supposedly they may make it too easy for bad actors to use it as a cyber offensive tech tool. So I think you we talked about I've talked about aculomics a lot.
And so set the stage today for. Oh yeah, I know what you saw and you know, how we're sort of feeling about it. Where we was occupant, what did you see that made us want to talk about today. Well, you know, an ad came across this morning on LinkedIn.
I'm from a gentleman who I don't really know, but I follow him. I kind of every once I see some of this stuff and I think it's interesting. And his quote was his headline, which definitely had my full attention because it said the largest cardiovascular screening site in America. Five years from now.
We'll be lens crafters. And I was like, holy cow, like this is what you and I've been talking about in terms of aculomics is. And I'm not sure it's going to be lens crafters per se, but someplace that's got open access, whether it be Bank of America or whether it be Albertsons grocery stores or someplace that society visits on a daily weekly monthly basis, right. Grocery stores are easy banks less than less people are going to banks these days.
So maybe that's not it. Maybe less and less people are going to lens crafters. Maybe more people are going to go to lens crafters or any other the, you know, type of. I'm going to extend to pass that maybe not a lens crafters, but maybe a diagnostic testing center.
Let's focus around eye care. We've talked about what the future remote telemed might look like in terms of separating history as being something you do pre visit online. Diagnostic testing being a diagnostic center and non surgical or non procedural stuff being done remote in terms of doctor consultation. So you have a sudden you're going to have this place where you have diagnostic technology.
And the OCT that you're using to look at something else is now also looking for cardiovascular risk and we've talked about acutomics and this is like, I mean, this guy, how many comments, let me see how many comments that he had on this. He has had in less than 24 hours. He's had over 10,000 hits to this site to this LinkedIn article. That's crazy.
So obviously, somebody somewhere is going, huh, this is pretty interesting. And so I just think that. You know, that brings us to where we're at is, do we see a future, whether it be, I mean, what's that like, what's our challenges that lens crafters or diagnosed as. I'm telling a challenge.
What's our challenge? As we talk about, I'm a big fan of ocular mix from a scientific perspective when I as a clinician, the ability to look at the red nut. You know, we could see died better not to be we could see how intensive or not be. But the superpower of ocular mix is finding seeing things that we can't see like neurodegenerative disorders, Parkinson's Alzheimer's and predicting things we don't know about.
So I think that the science is very compelling. But let's just talk about some of the issues. And in the back group of us being very pro the idea of ocular mix, but there are challenges. One, there's some huge challenges.
One is the money challenge, which is always that big issue when you follow the dollar who's going to pay for it, right? What do you think about a new technology who's willing? Where is the willing pair? And there's basically three insurance, Medicare, you know, right now there's no general CPT code.
And in the short term for for a clinical screening type of photograph. Some companies do have some breakthrough designations on their technology. But nothing is out yet, right? And then the second option is the patient just paying with cash.
And we see that that some, you know, some companies are doing that. And that's probably a realistic near term model, you know, anywhere from 10 to 30 to 40 to $50. And that's what I mean. I agree with you, but there's a limitation.
I mean, how many people are going to go and spend $50 for screening and they may do it once. And I feel like I'm good. Are they ever going to go do it again? Yeah, that's the question, especially if they don't maybe see the value.
And I think this goes into the another point where, you know, what is the sense to be in specificity of these. And these models, right? A lot of them are compelling, but when you are screening and you are screening not just hundreds and thousands, but potentially millions. And you have a breath, a higher, you know, what is an acceptable false positive rate when you are then plugging, cludging up the system with otherwise normal.
Part of the example, you may send them to the cardiologist, an internal medicine or whoever. And do a whole workout and it's negative. Who's who's the risk bearing entity there? And that's the third.
We have enough. Yeah, like in that situation, we have enough cardiologists to see what this may create. Yeah, and you're and may cause delays in other parts of it, but I think where this sort of makes sense is sort of the capitated model. Where's where you have large plans that are just ensuring lives.
And if you were to find a cardiovascular disease that could give the patient an M.I. 5 years, but you find it beforehand, that is definitely some economic value there. But again, the sensitive and specific has of the test has to rule. It's kind of like what we talked about the news of the day, you have to sort of study it and test your hypothesis, right?
So at a false positive rate of say, you know, 40% like it probably is not going to be a viable economic proposition. So super excited about it, but there are some major. Yeah, I'm going to go back to the I think the two big. Yeah, I think the two big challenges are number one patient accepted.
It's like, I mean, I always think about you go to the grocery store and you know, there's a chair sitting there with a blood pressure cuff. I have a grocery store in the same parking lot as where my office is. And so unfortunately, I go everywhere every day for lunch. And so I have never in seven years, seen somebody sitting at that blood pressure cuff machine.
Never once in seven years and I am there every single day pretty much during the week. And so like, and that cost nothing, right? So now the sun you're going to say I'm going to charge $50, right? So I think there's a problem with potential challenge of patient trust.
So if I have a question, it's acceptance. And patient willing this to pay. That's the first pillar of big hurdle. I see the second big hurdle.
I see is kind of what you were talking about. But okay, so they get a, they give a screening reading, right? What's the workflow of the next step, right? If you get a high blood pressure reading, you may call your GP.
You're like, oh, I got a high blood pressure reading. I'll check it again in three months. Well, three months go by. What happens if you get a, you know, a false, let's say you get a true positive, right?
But you're like, well, so what do I do next? Do I go to my GP? Do I go to my PCP and they go, oh, yeah, I don't really believe in those machines. And you know, there's a lot of people who do they can go, no, it's that.
And we're not, I don't really trust that. You know, and they just get dismissed. And then the patient goes, oh, maybe I shouldn't trust it. And they tell their friend, I don't know.
My doctor said, don't trust it. Or there's the workflow problem when they go and the PCPs are like, no, you need to go right to a cardiologist and kind of like I mentioned earlier. Do we overwhelm the health system? Because we don't know what this because we don't have the specificity sensitivity dialed in.
To mass society kind of like you're talking about. I think the two biggest challenges aren't the technology. It's the patient acceptance and workflow. I think those are huge challenges.
Yeah, I think we both agree the promise is real and very exciting. But as we mentioned before with our previous podcast on the unbundled exam and workflow issues, this retires in directly to that. Well, I think this has been a fascinating episode sort of spanning a bunch of different topics. But the through line is always I think workflow patient acceptance.
Enter the real world problems that we deal with. And I think it's a great way to, you know, even though we're sort of talking about this new technology that seems to almost other world they call AI. Great, great chatting tonight. Look forward to seeing you guys next week on an episode another episode of AI and I can you find all of our stuff on AI and I care.
Come you reach Scott at SCOT at AI and I care. And myself at our EHA and Reagan at AI and I care. Thanks everyone for listening. Have a good week.