Well, welcome everyone to another episode of the AI innovators webcast, sponsored by Jobson and brought to you by Dr. Scott Morris and myself, Dr. Rehan Ahmed. We are so lucky today to have Dr.
Rumasa Chana, who is an ophthalmologist and retina specialist at the University of Wisconsin at Madison. She completed her medical school at the very well regarded autohone university in Pakistan and then did her ophthalmology residency and her retina fellowship at Johns Hopkins at the Boulmore Eye Institute and now is a faculty member at University of Wisconsin at Madison. She actually is a co-director and the reason we're talking of the AI unit at the Department of Ophthalmology there. Rumasa, the first time I met you actually was a couple years ago at the American Academy of Ophthalmology meeting where you're giving a stirring talk on diabetic retinopathy and autonomous grading of that and I know I know we're going to get into that topic.
But I just want to give you an opportunity to tell us about yourself. How you know, how did someone get interested in AI and ophthalmology and did you see this path before you? Is this some part of some grand plan that you came up with and how you became co-director of this really well regarded AI unit in the country? Yeah, absolutely.
Well, first of all, thanks Rehan for inviting me to this podcast. It's super exciting to reconnect and talk about this subject again. So, you know, I really never saw myself as somebody who would be using computer science and autonomous technology to help take care of patients. And it was really fortuitous that when I was at Wilmer and I was a resident, you know, people kind of knew that I was passionate about diabetic retinopathy and improving access to screening.
And one of my seniors at the time was like, well, Google is doing this project in which, you know, originally they used their algorithm to identify cats and dogs accurately. But now they're interested in looking at retinal images. And they want someone to like annotate those images so that they can then train a computer algorithm to detect it in an autonomous fashion. And I was really intrigued by that idea.
So, I got involved and, you know, resident life is busy. But in between things, I would, you know, annotate these lesions. And I really thought there was potential in having computers help us because it was really hard seeing these patients would come in with really advanced disease and knowing that if they had access to care and it was identified early, perhaps their lives would be completely different. And I think this is becoming increasingly important as we all know with diabetes increasing.
So anyway, that was my first foray into it. And then as I got more and more involved in it, I connected with Michael Abramoff who actually invited during my chief here to Wilmer. And that was the year when his algorithm got FDA approval. And so we started this thing about using it in kids.
So that was my very first project trying autonomous AI to detect diabetic retinopathy in children because it wasn't approved for kids. And it's still not because that population is small. From then on, I think it was just more and more projects trying to figure out. And that is what my focus has been now is implementing it into the real world.
Like we have this technology, you know, I was developed by Google many, many years ago, the first paper, I think was like 2016 or something like that. And but we still when we look at the real world numbers, the adoption has been very, very slow despite there being a CPT code where you can bill for it, despite all those things. And so my focus has been, you know, in implementation. And then the other piece has been the follow up piece that I talked about at AO when we've discussed as well.
So we can go more into that. But coming to the AI unit, which I co-direct now, we started it as a way to help our graders at the reading center. So Wisconsin has probably the oldest reading center in the country. And we provide standardized grading for various diseases.
Dibetic retinopathy is one of the most areas that we work on. And so we started this as a way to help our graders. However, as we've grown, and that's where my role comes in as the co-director, we're trying to bring those algorithms into clinical care beyond diabetic retinopathy as well. So, you know, how can we assist our clinicians, taking care of patients and automate some of those things that are tedious.
So that's where my role as the co-director of the AI unit has been. Well, give the audience, let's step back for a second. Maybe a little bit of a historical, I don't know, I walk down the historical lane. So, a plug for ophthalmology, we were the first specialty, of course, as you know, to have an AI based autonomous approval.
Tell us a little bit about that approval. There, and now there's a CPT code that you mentioned, 9-2-2-9. And then you sort of said that, you know, the uptake has not been as maybe as fast as we had thought. So, where, you know, give us a little bit of background here.
Yeah. Where do we stand? And, you know, maybe where has our expectations not hit reality? Yeah.
So, I would say the approval, the FDA approval was in 2017, 2018 time. 2021 was when the CPT code was approved for autonomous AI interpretation. And so, I think ophthalmology absolutely has been at the forefront. I mean, I go to these medical AI meetings and other specialties that are still trying to figure out, you know, how do we build for this?
Is this equitable? How, you know, I think in ophthalmology, we started the discussions for all of those real world implementation issues way back. However, when we look at the uptake, and this is based on claims data that are published, we can see that, you know, for example, in the NEJM AI paper that was published, the uptake was like 15,000 claims. And, you know, when you think about the 34 million plus patients with diabetes, that's a drop in the ocean.
There was another paper after that from the trinetics data set, which showed that 0.09% of patients with diabetes in their data set had used autonomous AI. And I think there's multiple issues. You know, one is that it took a while for peers, different peers to come on board with this technology. I think there's the awareness piece.
There's also true implementation challenges like in the primary care world where we want this to be used, there's so many other aspects to take care of for patients with diabetes that try to carve out time for something like this is a challenge. And a lot of the clinics where we're trying to roll this out are like FQHCs or federally qualified health centers, which tend to have limited resources anyway. So I think that's where that uptake challenge comes in. For example, when we rolled it out at the Hopkins Children's Center, this is, you know, very well resourced pediatric endocrine clinic with lots of personnel, our screening rate was close to like it was 100%.
And in fact, when we published it, the reviewers were like, that is not real. And we're like, you know, it wasn't that hard to do it there. But when we go to the screening rate, we prior to implementing AI, Oh, no, after implementing after implementing AI, we had a screening rate about 60 to 65% in that group. So we still had like over a third of patients not getting screening.
So I think AI screening made its impact. But it was easier to implement. When we go to these underserved clinics, the resource limitation has been a big issue that's limited implementation, I think. Yeah.
So is a larger message. And the way I sort of think about is that the best resourced places actually had are the ones with the least amount of need, maybe, and it seems like it still helps, right? To go from 60 to 70% to near 100%. But your least resources one are the one that's actually where you want to go.
Yes. And they have the hardest time actually doing this, the last mile of, well, we'll actually get to the last mile, but actually getting this screening in place. So tell us a little bit about your research, the AI bridge program, and how that starts, it's into solving this sort of problem that you sort of surfaced. Yeah.
So I think there's obviously no easy solution. And there's lots of like different ways to handle this. We're trying to work with the local institutions to co-design it. So it's not like one size fits all.
And yes, everybody's going to use AI for DR screening, but we're using implementation science methods. So implementation science has developed as a full field now to help with issues like that. So co-designing the intervention with local stakeholders, and then rolling it out. And then once we have the quantitative data going back and interviewing all these different stakeholders, which includes not just the leadership, but also the providers, the front end people actually interacting with the technology, the patients.
And we're hoping, and we haven't completed the project yet, because that's another story where it got terminated and reinstated. But our goal is to sort of come up with some implementable strategies that will increase the uptake of this technology. Because we know, I mean, Wisconsin was involved in the approval process. So we know that it's a rigorously evaluated technology.
It works really well against established gold standards. But how to get it to be used is the main crux of the issue right now. Yeah, I think you're mentioning of all getting all the stakeholders on board and really thinking about it in terms of an implementation science. I mean, it's not just a technical solution, which is there, but getting the tech to comfortable with the camera is probably the key thing.
Even in primary well resource primary care clinics, they're doing a million other things, checking blood pressure, glucose labs, distat and the other. And yet now we're asking them to do one more screening, especially something that they're not comfortable or familiar with. They may not know what a normal retina or normal picture should look like. Exactly, exactly.
So, you know, I mean, this a little bit of it is like the way we train our medical students, right? Like, they're the ones who go on to become primary care physicians, but many of them, throughout their training, have never seen a retina. You know, it's not a core part of the curriculum. And so they feel very uncomfortable having this technology and making this recommendation, even though you and I know it's rigorously validated.
And then, you know, they know, but there's lots of other AI that's not rigorously evaluated and is being used. So that level of trust to be like, okay, I don't really know this pathology, but I'm going to trust the AI and make this recommendation for my patients. So that is another barrier that comes up. Yeah, I don't mean to put you on the hot seat, but one question I often get I something I ask myself, it's like, you know, I see all these algorithms that are getting approved now.
I mean, they're several, definitely. And, you know, I get to the inundated with all that sensitivity and specificity, like the numbers are like swirling in my head. And I have a hard time truly differentiating between one or the other. Yeah.
And so part of this, like, is it just table stakes now that we expect these algorithms to just do really well? I mean, is the technology, is that part more or less solved? Since you started your career in terms of the AI, like labeling it, I wouldn't say like, and the differentiation really is on the implementation side. It's like the ease of use of the camera, how well it takes pictures is that one photo, is it two photos, is it like a degree, you know, all these like sort of small things?
Or would you disagree and say, no, no, actually that's like the underlying technology, actually, you know, the the robustness of the algorithm actually makes a difference. So where do you stand on that in terms of comparing different devices and the algorithms? Yeah, I think I think that is a it's a very loaded question. There's so many ways to look at this.
So I think if we break it down and we think about diabetic retinopathy, I think the algorithms are really good, you know, they're 85% sensitivity and specificity is the minimum and they do go beyond that. However, there's all these caveats though, you know, in the clinical trial, the pivotal trials that were done, 24% of patients were actually dilated and try working with a primary care clinic and getting them to dilate patients. So when real world studies are done, there's a lot of non-diagnosable, not because the algorithm is bad, but because we just didn't get good enough quality photos, especially when you go to these underserved clinics where patients have cataracts, they already have media capacities. So that's what causes the algorithms to not perform as well, in the case of DR.
However, I think when it comes to macular degeneration and glaucoma, especially with glaucoma, the issue we're struggling with right now is what is the reference standard? Like how do you even evaluate it? And you know, I make it a point to mention this in my talks multiple times, but basically whatever your reference standard is, is going to change your sensitivity and specificity drastically. And so, you know, if your reference standard is another greater, a human greater versus you used, you know, a host of human graders versus you used multiple modalities to establish your gold standard, all of that is going to change that sensitivity and specificity.
And so I think as we look at more and more algorithms coming in, I think it's really important to look at what was the reference standard that was tested against because there are algorithms out there for glaucoma and AMD, but the reference standard for glaucoma is not something we've agreed on as a community. So that makes it really hard to see if those algorithms are really picking out things that we want them to pick out. For AMD, at the University of Wisconsin, we've actually started study, it's an international study, it's called the Abbott Study. And we are the goal there is to have a reference data set at the reading center so that each company doesn't have to do what the DR companies have to do.
So basically for DR, every company had to enroll a parallel data set, right? So they had the real world thing, then they had the gold standard thing and they had to compare the two. So what we're trying to do is to have that gold standard set up for AMD so that the algorithms can be tested against a reference data set so we can compare apples to apples. So I think with different diseases, we're at different stages.
And I think that as those algorithms become more robust, perhaps uptake will improve because at this point, people do worry that you're missing glaucoma or AMD if you're using just the app for DR screening. So in some places, we actually have a human AI hybrid approach where the positives are still being reviewed for other diseases. Yeah, I've always wondered that about DR screening is that there's a host of other pathologies that we as ophthalmologists and optometrists will look in someone's eyes that could be hypertensive right now. And by having an algorithm, are we missing that?
So having either multiple algorithms in place and having some type of threshold for all disease could make sense, almost like a simple green light, yellow light red light type of situation. Or as you mentioned, having a human in the loop for these borderline cases. And sort of like the regular thinking through that in the next few years and maybe even longer, how do you see the regulatory? I mean, it seems like that is going to be a big issue.
Having this one sort of reference gold standard for AMD seems like it makes a ton of sense. But I wonder, do you think we're going towards which, if you had to predict the future, which pathway are we going toward? One where we're talking about multiple diseases, you have your AMD algorithm glaucoma, glaucoma algorithm, etc, etc. Or having some sort of like pan algorithm, you know, all knowing algorithm that knows everything, right?
Yeah, yeah, I think it would have to be some sort of thing where these multiple algorithms are, you know, sort of integrated into one system. So the user is just seeing one system, but in the back end, you know, it's giving a more detailed output. And now with LLMs, we can see that, right? You can actually get a more detailed output which covers things like, Oh, the arteries look narrow, more of a human way of looking at it rather than the, you know, these older algorithms, which just give a binary output, and which is great.
I think we still need that because we're starting with such low screening rates and, you know, such need. But I think as things move forward, I see that it would probably be more of a human-like interpretation. And I think that would improve the trust level and uptake as well. Yeah, yeah, some type of combination where you have a natural language output.
It seems like it can make a ton of sense. Yeah. So we've talked a little bit about the issues and maybe the challenges and benefits of these models. But what, okay, what happens after screening?
So we talked about problem screening, but then there's a whole problem of level. It's great once you identify the, you know, it's great to identify. But then what next, right? And I know you have talked a lot about that.
So yeah, I'm glad you brought it up. So basically, you know, as we were doing these AI projects, one of the things that I kept thinking about is like, you know, you and I and others who are eye care providers, what we really care about is like, what level of vision are we preventing? How much vision loss are we preventing on a population level? And that's hard to predict with diabetes because the disease progresses very slowly over a matter years.
So you need like this really large prospective studies, which are not feasible. So what we ended up doing was we ended up doing a computer simulation model to see, well, what happens? What happens in a setting where there's no screening? What happens in a setting where we try to get everybody do our best to get them to the eye care provider?
We know that many people don't go. What happens if you have 100% AI screening? And you know, now that I think about it in retrospect, it's like, well, duh, that was the result. But when we were actually doing the analysis, we're like, there must be something wrong because we have 100% screening, and there's not that much vision loss prevented.
And you know, so that's where we started thinking about, you know, there's so many other aspects to care. You know, once people screen positive, like how many are actually following up on the recommendation? Also, how many are actually taking care of the metabolic control, which is, you know, we, we, off the mologists, I think we often think of like, okay, we fix the problem, but the there's diabetes, there's so much more to think about. And I think the metabolic control, you know, we know is directly related to decreasing progression of disease and then vision loss.
So that's where my focus, I mean, I still want to increase screening uptake, but not just that. So the AI bridge program is designed to also help with that follow up piece. So really working with the patients, providing them education, having a champion who can assist them with scheduling these visits, because a lot of them may have primary care, but then they have no way to access eye care. So there's multiple ways that we are working on with them so that they can actually go and get their eye care done as well as just not just finding out that they have disease.
Yeah. I think, you know, there's a lot of commercial opportunity there because once you, it's almost surfacing another problem and challenge and opportunity that once you, once you find all these people, yes, yes, well, you have to, you have to do something for, I mean, and have an education, you say you have vision directly DR, but sorry, we have no place to or get, you know, or we don't have it accessible and available for, they can't even afford treatment. Yeah, really. I think this is important.
Yeah. And you make a good point here, because, you know, one other related thing that comes up is people are like, well, AI take over our jobs and, you know, really, AI is helping us identify more issues and more disease earlier. And, you know, yes, there's the false positives too, but really trying to get more people into care. So I think ultimately, it might even just make us even busier as eye care providers.
So as part of this podcast, we are, we connect the three O's of eye care, and that's the optical industry, optometry, and of course, ophthalmology. Yeah. For our, for our colleagues in that first two, most Americans actually get their eye care in, by from independent pumps or in actually in retail eye care settings. Yeah.
How do you imagine sort of these types of advanced screening tools? Yeah. Impacting where most Americans get their eye care. I mean, you mentioned FQHCs, which is a great spot.
But what about sort of other other things? You know, I'm glad you brought that up, because I think that primary care is actually super overwhelmed with non-eye care stuff. But I think in ophthalmology, and in the field of eye care in general, we're so lucky that we have opticians, we have optometrists who can really provide that level of primary eye care. And I think AI can, as I see it expanding and us having more diseases that can be picked up by AI, maybe using these in these optical shops where people are actually going on a routine basis and having, and that might help us sort of bypass this problem of adoption.
Because I think we're so used to thinking of using these technologies in primary care, because that's where patients with diabetes go for care. But the problem is that those primary care settings have so many competing priorities. Whereas, you know, optical shops and optometrists, if you can get these patients to go there, perhaps they can be a way to provide ice screening on a more mass scale. Yeah, yeah.
And as you know, oftentimes we get patients who think they just need an updated prescription, but in fact have that, you know, evitreous hemorrhage or diabetic macular edema and had no idea. Yeah. Yeah. And already like, you know, a lot of optometrists have optos cameras and they can get undilated photos.
So we get a lot of referrals, at least in our region here, from optometrists who've seen something concerning in the retina on their optos photos and will send those patients in for care. And I think with more and more AI algorithms coming in, perhaps that whole path we have care can be further streamlined. So Dr. John, you sort of sit at the nexus of a lot of interesting future developments.
I mean, I feel those and sort of academic, you're sort of really doing the work that we're going to be seeing a couple of years later. And so what advice slash predictions slash take out takeaway points that you could give to our audience about, you know, what should they be paying attention to? If you're an optometrist or an optometrist, you know, you're just busy in your private practice, you're busy in your academic job. What should we really be paying attention to given, you know, how much AI news and information there's out there?
Yeah, I think it's really important to separate the noise from the actual good stuff that can be done from AI. And, you know, as we talked about at the beginning of this podcast, of themology has been at the forefront. And I think really paying attention to the new things that are coming out, but looking at them critically in terms of like, what value added value do they bring to our patients? And how can we incorporate those in our day to day care of patients?
I think that's where we will move the needle. Because, you know, right now the number of people aging who are going to have all these diseases is increasing. You know, everybody talks about the I care provider shortage. And so I think we need to really think about how to incorporate these technologies to make our lives more efficient and being able to provide more timely care for patients.
And we'll have more and more opportunity to do that as more and more of these technologies come in. So I think if I had to say one thing, I would be like, just look critically at all the stuff that's coming out and see what can be incorporated into I care. Well, thank you so much, Dr. Chana.
Again, Dr. Chana is the co-director of the AI unit at the University of Wisconsin at Madison. She's an ophthalmologist and a retina specialist. And we hope that you can come to AI and iCare.com to help separate some of that signal from the noise, to look at these new things that Dr.
Chana mentioned with a critical eye and we want to be able to assist in that. Again, Dr. Chana, thank you so much for your time. I really appreciate you taking an opportunity out of your busy schedule to educate us and look forward to continuing the conversation.
Thank you so much. Thanks again for the invite. And it was a real pleasure discussing this with you. So, all right.