Welcome to Real Talk on AI and I care with your host, Dr. Scott Morris. And I'm co host, Dr. Rehan Ahmed.
In each episode, we discuss current news, add in a little AI education, and debate innovative AI tools and topics that will change the industry. Hey, welcome back to Real Talk. This is Scott Rehan. And I ran.
I wanted to let you know, there's a really new and kind of cool thing as read down four of us this morning in my, my flight home today from San Francisco. It was talking about, it's called inside the race to improve the patient experience with AI agents. And I thought it was really interesting that in forms of all places, we're finally starting to get to the point where we're talking about how do we improve the patient experience. I think so much of AI, we're talking about how we improve clinical care and that's great and how we're going to make the doctors break experience better.
And how we're going to affect the income or the income of a practice and the financial thing. And finally in a major, you know, financial journal, it was how do we use AI agents to change the experience. And it offered you know, like there's a bunch of people. I mean, there's, you know, lots of different series A rounds by lots of different companies trying to figure out like who's going to be the first person to develop truly an agent that helps the patient have a better experience.
I think that's interesting because you and I've talked about this over the over the last year about how does that integrate because it's going to have to integrate with an EHR. It's going to have to integrate with workflow. This is not an easy task. I mean, it is going to be a challenge.
If you're trying to link it together with healthcare software. But then I kind of wonder, is that really what they want to do? Are they going to create agents that are independent of healthcare systems? And we have a whole new model of how patient experience works that's based on patient platform, not necessarily healthcare platform.
I think I think it's going to be interesting that that article goes over a lot of that different stuff. I think it's going to be super interesting to see where that goes. Yeah, I totally agree. I think from a consumer standpoint, I think at this point, many people are very comfortable with LLMs.
And now they're starting to dip their water to aid the agent world. And so it's going to be, I can imagine a future where we all have our own agents or army of agents that are doing tasks for us. And one can imagine our agents as consumers talk with our doctors agents. 100%.
I really do see that coming. I agree with you, and I think that's the future. Yeah, like today, like for the past three months, I've been wanting to schedule a dentist appointment for my two kids, two of the three kids I want to get to see a dentist. It's been months and it's just I cannot calling them schedule appointment getting the insurance cards already.
Man, if I could offload that to my agent, sign me up. I'm ready to get that stuff done. I think that's coming. I think we're going to look back.
I really think we're going to look back this podcast and that particular journal and go, here's kind of where this is the infancy. This is where it started, you know, just like we were talking, I envision Expo, you know, the ubiquitousness. I guess that's a word, the ubiquity of mobile phones and smartphones and all the kind of technology not just accepted as, you know, that's self evident truth. And we look at AI agents go, yeah, but that's somebody down the future.
I kind of wonder five years we're not going to go. Can you believe we actually operated without our agent? Yeah, yeah, I could see that definitely happening. It's going to be a whole new world.
So today in our knowledge bite, I'm going to be talking about the dirty H word and the AI, the dirty H word is hallucination. And so, you know, we were I care providers. So, you know, I would say once every few months, maybe I have a patient and I'm seeing things and they're like, not sure if the floater or the vigorous debris that they're talking about is like a hallucination. Is this really there or is it something physical?
In this case, so who's in our context of AI, hallucination means seeing or hearing something that isn't really there and that's the normal definition in AI, it's similar. It's when and we've all experienced this if you use chat GPT or Gemini enough, it's like it produces an answer that sounds so good. But actually it's wrong or entirely invented. And it does it with so much confidence.
It's kind of like, you know, the person in the media, I know some people like that. Exactly. You beat me to the fun slide. Exactly.
There are people like that. And they're there are hard to spot if you don't know what you to look for. And so. So here's the here's the thing hallucination is is a feature in a way it's part of the system.
It's how they work. We have to remember these models, they don't quote unquote no facts the way humans do. As we've said in in our and this is why these knowledge rights are so important. We're sort of building this and now we're going to some of the problems.
It's a prediction device. It's predicting statistically what's the next likely word sentence paragraphs based off their training data. And the model has not been exposed to the fact that you're looking for. Like a kid in third grade, it fills in the blanks.
That's why you get a beautifully phrased answer, but it can be just wrong. And it's worth pointing out like this is not limited to gen A.I. or LLMS. I mean, this is where we talk about it.
We see it in other forms of AI to like take about computer vision or HI or HI. Yeah, you would have told it's the part two. No, I know Scott, you're always in good for depending the robots. I have to really have to explore this.
But yeah, I like to computer vision rental images. If it hasn't been trained on enough images, it's it could misclassify benign lesions as something else. I'm all in Oma. It could perceive something that isn't really there or speech recognition.
It just gets words wrong or incorrectly. So that's that's incorrect. There could be a sort of hallucination term applied there to. So the truth is, you know, all AI systems can't produce outputs as you rightly pointed out as well as human and human intelligence systems produce outputs that are wrong.
And in AI, it's just so good. It's former fabricated text fabricated citations invented concepts. I had, you know, I'm consulting for a company and I see a report. I'm like, wow, this was done really quickly and done really well.
And then there's some citations and they're like, huh? I'm like, is this right? I can't believe like it's so perfect and could not find that citation. And so this is like this.
Actually, we will talk about soon. It's basically this concept of AI slop. Right. These.
Yeah. So I just learned that term yesterday. AI small. Yeah.
And it's going to be we're going to see it a lot more in the workplace. People just handing in stuff that's just slop. Right. And it actually makes thing makes work harder because it takes more time to figure out what's true and what's not.
So, you know, in I care. There, there, there are serious consequences. Right. Can't imagine telling an L having an L that does a diagnostic that says there's a glaucoma when it doesn't exist or misreading OCTs or misreading other things.
For example, in the same way humans do too. And I, you know, researchers are trying to like fix this for language models. One of the great things and we talked about this before this concept called rags or retrieval augmented generation. That's what that stands for.
And that's what that's basically forcing the AI to pull answers from verified sources like PubMed. So you could have a layer on top of your LLM just to double check to make sure on the important stuff that doesn't get it wrong or you at least know where it's getting from. Or having like some probabilistic or uncertainty quantification. Those are things that we can do.
But at the end of the day where we are right now, sort of October 1, 2025, it's how hallucinations aren't a rare glitch. It's fundamental on how AI learn and how they predict. And it's part of the system. And so you just have to be careful when you read this beautifully phrased fluid confident output.
Look, double check. I mean, at the bottom of chat, you be as chat to you, you can make mistakes check important info and you have to take that seriously. So. And this is where I think it's much more of an augmented intelligence.
So for I care providers and for healthcare providers in general, you know, there's an augmentation and ultimately it's up to us to decide what's real and what's what's noise. So we've got to use our great matter. For the time being anyway. Yeah.
Well, that brings up to you know, even I in our pre meeting, we're talking about like what's kind of we try every every week and are in our. We call it our fireside chat, but we really kind of explore a subject. And it's been all over the news this last week is the AI bubble, you know, and their compare to is this the internet bubble in the world of AI. And what is that look in it?
I was speaking yesterday, we're talking about the gardener hype circuit, hype curve, you know, and I think that I've been part of lots of different technological innovations over the last couple decades and every single one of them has went through the height curve. Every single one where there's this peak of exuberance and all the stuff is going to happen, they, you know, they call this like the innovation trigger and then the peak of inflated expectations. Feel like that's almost like a fairytale concept where you see this big huge peak and everybody's you know it's all of the media. Everybody's talking about these great success stories and you know, then comes the, then comes kind of the hallucination piece of it kind of creates these unrealistic expectations.
And then things start to not go so well. And that begins the trough what's called the trough of disillusionment, otherwise known as the crash. And you know, I'll kind of explain how the rest of the curve works if you've never seen this, if you have never seen it, please look it up, just write the gardener, g-a-r-the-n-e-r hype cycle. And then it shows the little graph of what that's like, but after the trough of disillusionment, I eat a crash, then you get this little slope of enlightenment where people go, OK, I'm start understand this and we're getting to the second and third and fourth generation products.
And then you kind of go, OK, now it's really working. And I think about that we were talking about that vision expo during the meeting is where are we out on the hype curve. And we're at the beginning of the trough of disillusionment. I think we're at the crash.
And there's been a lot of press about that the last couple of weeks is a lot of these companies that went in and they fired all their staff or fire those opponents spend tens or hundreds of million dollars saying, AI is going to be our solution. They're learning that there was a little bit of over promise and under deliver. Now, I'm starting to hire back their people and they're going, this was maybe a mistake and blah, blah, blah, blah. Now, I mean, if you look at enough, if you've been around for enough of these innovation curves, you know this is just part of the norm.
It's just, it's what's going to happen. I mean, it's just, it's part of it. And it's almost like this little sanity check of, OK, you know, people who thought it was going to solve all the world's problems. And then, you know, I'm going to use the reality check.
So I don't know, you know, I'm, I try to be as pro AI. Obviously, Ray, and that's why you and I are the editors. We're pretty kind of, yeah, we're pretty biased on pro technology pro innovation. But I also think experience matters and that this is part of normal.
And for all of our listeners, you know, you guys are going to hear a lot of nestly bad press over the last probably next three to six months. Innovation curves used to take years to come forth. And now things are speeding up. And I think it'll take months and months of disillusionment when we're like, oh, this wasn't, you know, maybe what we thought it was.
That doesn't mean it's not going to be the future. It just means that maybe we needed a little reality check. And kind of like after you talked about hallucination, we still need to use our gray matter and kind of be logical about it. I don't know what you, what do you think?
Where are you sit on it? It's hard to know. It's kind of like when you're on a, on a, you know, on a windy road. It's hard to know exactly your altitude or your where you're at.
And I think the same thing with the, with the hype cycle like this is this curve, right. And the tricky part is, I'm not sure where we are. I mean, definitely. It, it sounds very hypey.
The fact Nvidia. And I thought I read this wrong. Like I thought I got my bees mixed for him. And it's a hundred billion dollar.
It's crazy. And so like just the sums of money that are being thrown around is just gargantuan. So one can't help but wonder are we. And in sort of like the peak of the, of like all these inflated expectations.
And now we're sort of as these investments, maybe not pan out. And so I don't know. I think it's like, I think you're going to look back and retrospect right and go, we don't know where we're at. So later, but I kind of have that feeling like about the sea some bad news.
I feel like directly it's obviously no serious person as far as I know this doesn't think AI is going to profoundly impact workflow. Right. I mean, I think everyone sort of sees that happening. Maybe the magnitude are people are, but directly that's what we're going.
I think the issue is the timeline. It's very unclear. And I love this code. I think it's bug it's like we don't know.
Often overestimate what's going to happen in two years. We're impatient creatures. That's just the nature of who we are. But then we underestimate what's going to happen in 10 in 10 years.
So like we overestimate to we underestimate 10. I mean, there's a very, there's a lot of that in 10 years. We have a robotic AI system like walking around in the first time we see it. We're going to say, oh my goodness.
And then it just gets normalized for us and our kids like that's in the realm of, I mean in the realm of possibility. Every other innovation that's ever happened that's been successful, right? It's you just look back and go, oh, well, can you imagine time we didn't have a smartphone or any HR or an electronic vehicle or a internet or Netflix or you know, I mean, we at list goes on and on and on. We just came back from vision and expo and you know, acuolog mix or systemic disease management, putting the I care provider at the center of a patient's journey.
I mean, this really wasn't part of the conversation five years ago. I mean, certainly people will knew about it. They thought about it. We as clinicians, we know how important.
The eye health is and what things we can detect. But this is just taking off. It's on its own. And I just sometimes I get concerned like we are, are we, you know, is this too much like are we over, are we over selling what we have.
But, you know, for the, for the sake of the field, I'm looking forward to seeing all the amazing things are going to be happening. I agree. Everybody will thank you for joining us for this version of real talk, where we talk about some pretty interesting stuff. I mean, we are always open to suggestions.
You can reach me at Scott, a C O T at a I in I care dot AI, or Rayhan and R E H a N at I in I care dot AI. We are happy to have discussions about what you guys want us to talk about in terms of where we are in all different context of AI, how it affects the patient care delivery cycle and how it affects us as an industry. Hope you guys learn something and we look forward to having you on next week. Thank you for listening.
And if you're listening to real talk on AI and I care, you're weekly podcast to keep you informed about AI technologies revolutionizing I care.