AI Isn’t as Smart as You Think

HOT TOPIC

ChatGPT Helpful, But Not a Stand-in for Human Medical Advice

A recent study found that AI chatbot (ChatGPT) may be a valuable adjunct to patient education on ophthalmic diseases, but it is not sufficient without concomitant human medical supervision. In fact, the researchers concluded that ChatGPT on its own provides incomplete, incorrect, and potentially harmful information about common ophthalmic conditions, defined as the recommendation of invasive procedures or other interventions with potential for adverse sequelae, which are not supported by the American Academy of Ophthalmology for the disease in question.

In the study, researchers from the Wills Eye Hospital in Philadelphia sought to assess the reliability and adequacy of the information related to medical knowledge provided by ChatGPT version 3.5. They submitted a standardized set of questions relevant to various eye diseases, divided into eight subspecialties, and selected the five most common diseases. The researchers then initiated a new chat for questions related to each disease to limit learned patterns between assessments and asked ChatGPT the following three questions, in sequence:

1. What is [disease]?
2. How is [disease] diagnosed?
3. How is [disease] treated?

The responses were scored from -3 (unvalidated and potentially harmful to a patient's health or well-being if followed) to 2 (a correct and complete answer).

Interestingly, 22.5% of all replies from ChatGPT scored ±1. Among those, 7.5% obtained a score of -3. These results were published in Eye, and the researchers found that ChatGPT scored best in the “definition” questions and scored lowest in the “treatment” questions. The authors surmise: “The definition of a common disease is usually standard and well-known, and thus the information the chatbot has received in its training regarding the definition of a disease should be very straightforward. When prompted about diagnosis and treatment, it is more likely that inputs contained conflicting information.”

My Two Cents

The use of ChatGPT is likely here to stay. So, as technology moves forward and AI continues to improve, there should at least be some safeguards regarding the medical information given out by these chatbots. The study authors state, “As the use of chatbots increases, human medical supervision of the reliability and accuracy of the information they provide will be essential to ensure patients’ proper understanding of their disease and prevent any potential harm to the patient's health or well-being.”

OUTSIDE THE LANE

Woman Warns Others After Losing an Eye in Car Crash

In August 2021, Hannah Oliver, of Atlanta, Georgia, was involved in a fender bender. She was in her car traveling 25 mph when another driver pulled out in front of her and slammed on his brakes. Ms. Oliver says she did not react fast enough and hit the other car.

She lost consciousness, and when she awoke, she remembers looking into her rearview mirror and thinking that one of her eyes “looked like it had gone through the garbage disposal.” She states “My eye was gone, it deflated on impact. My optic nerve was severed, and my retina was detached. All of the glass had shredded my eye into pieces.” After six surgeries in an attempt to reconstruct her eye, she now wears a prosthetic.

Ms. Oliver attributes her monocular blindness to the fact that she was wearing her favorite sunglasses, which were not shatterproof. Since the accident, she has launched her own company, Blue Eye, which sells shatter-resistant sunglasses to spare someone else from going through the same nightmare she did. Ms. Oliver states, “I think it’s crazy that people don’t know your sunglasses can do this. Over 75% of them [sunglasses] are made with either plastic, nylon, or glass lenses, which will shatter right on impact.”

My Two Cents

This is something that I don’t think enough of us consider when recommending lenses to our patients. Prescribing high-quality, shatterproof lenses could be the difference between your patients losing their vision in an accident or losing their eye, like this woman. I’m going to share this article with our optical staff so they have a story that will illustrate the importance of high-quality lenses to our patients.

CAN YOU RELATE

Well, get ready: Another major tech company has released a virtual reality (VR) headset—this time with a bit more fanfare. On Friday, the Apple Vision Pro became available. I’m extremely excited for the release and to try the product out for myself (although I’ll probably wait a month or so before going to the undoubtedly crazy Apple store for a demo).

When Apple releases a product, it does it right, and it often changes many of our daily lives. Take the Apple watch. Before its introduction, similar watches were available from other brands (eg, Samsung, Fitbit). Apple took the concept, enhanced it, and brought it to the mainstream. The company did the same thing for tablets, apps, in-ear sound-dampening headphones, and even smartphones.

I’ve tried the VR thing when Facebook—or Meta (whatever it’s called now)—released its Quest 2 all-in-one VR headset. Although it was novel and unique, and included some enjoyable apps, such as Beat Saber, it just didn’t have the longevity. How many of you have a Facebook/Meta Quest 2 VR headset rotting away in a closet somewhere? A quick poll of 10 of my friends with kids showed that five of them bought the Quest 2, and all five don’t know where it is and haven’t seen it “for months.”

All joking aside about past VR headsets, I do think the Apple Vision Pro will show us how sometimes, the second company to market actually has the leg up. Apple has successfully been “late” to the market and then lapped the field many times over. Do we have any such opportunities in eye care? One thought that comes to mind is Vuity (AbbVie). I’ll be honest, I’ve had tremendous success with this drop (when prescribed for the right patient who is given realistic expectations), and those for whom it has worked have become some of my most loyal patients. However, many of our readers have had little or no success with the product, and it never seemed to catch on as I expected it would. Now, Qlosi (Orasis Pharmaceuticals) has also entered the presbyopia treatment space. This pilocarpine-based drop could follow the same path as Apple with its VR headset and rejuvenate the presbyopia category. Most of our patients are still not even aware that a presbyopia drop even exists! What say you? Are there any other opportunities in eye care where an Apple-like attack on the market might succeed?