When Kim Hilliard shows up at the clinic at the New Orleans University Medical Center, she's not there simply for an eye exam. The human touches she gets along the way help her navigate her complicated medical conditions.
In addition to diabetes, the 56-year-old has high blood pressure. She has also had back surgery and has undergone bariatric surgery to help her control her weight.
Hilliard is also at risk of blindness, which can result from a condition called diabetic retinopathy. And on this day in February, her vision will be evaluated by a new practitioner: a piece of software.
Automation like this is starting to infiltrate medical care. Depending on how it's deployed, it could help reduce medical errors and potentially reduce the cost of care.
It could also create a gulf between health caregivers and people of more modest means.
"My fear is we will end up with what I've been calling a 'health care apartheid,' " says Sonoo Thadaney Israni, at the Stanford University medical school. "If we create algorithmic care and 'kiosk' it in some fashion — focusing on efficiency and throughput — the people who will end up having access and using it will be the ones who already lack privileges of various kinds."
We are far from that dystopian world at the moment, but are we moving in that direction? That possibility concerns her.
Hilliard's experience at the clinic underscores the importance of human contact. She's here for an annual eye exam to look for signs of blindness that can arise in people with diabetes.
"I got the full diabetes when I made 40," she says. It's a challenge for her to stay on top of all her medical conditions. "I go to so many doctor's appointments I get tired," she says.
The software to identify early signs of diabetic retinopathy, called IDxDR, can do that job without expert intervention, but skilled medical personnel at this clinic are, for the moment at least, still playing a hands-on role.
After Hilliard finishes the exam, nurse practitioner Chevelle Parker shows her images of her eye.
"If we zoom in here, we can see some little fat deposits here, OK?" Parker says. Hilliard leans in and studies the image of her retina.
"That can be from the foods you're eating," Parker says. "Think of some of the fatty foods you're eating — sausage, bacon."
Hilliard says she stopped eating those foods last fall, after her gastric bypass surgery.
"Well, when you were eating those, the deposits were being placed on the eye," Parker explains. "That's why we talk to you about your diet. And now that you know you can't have that, this is the reason why, OK?"
Parker goes on to reinforce the dietary recommendations for diabetes. Hilliard should eat breakfast within an hour or so of waking up, and she should be sure to have some protein, rather than carbohydrates, at the end of the day.
Hilliard gratefully accepts the advice, along with a referral to an ophthalmologist, who will need to get a closer look at the signs of damage in her eye.
"I do what I can do to keep from going blind," Hilliard says. "So whatever they tell me to do that's what I do. At least I try."
Hilliard's experience is a stark reminder that health care is more than a simple transaction. Six in 10 adults in the United States have a chronic disease, and 4 in 10 have two or more, according to the Centers for Disease Control and Prevention.
This is the real world, in which computer algorithms are starting to take off in medicine.
"I think for too long we've had this assumption that any new technology is good, more is better," says Abraham Verghese, a physician who works in partnership with Thadaney at a Stanford center that focuses on the human aspects of medical care.
"New is not always better," he says as the three of us sit together in their office.
Medical care, like so much of our society, creates haves and have-nots, Thadaney says. "We need to make sure that technology doesn't further exacerbate the issues of equity and inclusion."
"Just to carry that thought forward," Verghese says, "AI algorithms we already know are causing inequities in bail bonding, inequities in real estate," as well as in policing. Unconscious racism and other biases get baked in, without the developers even being aware of it. "That same kind of algorithmic approach can easily infect medicine and probably does," Verghese says.
These technologies are driven by companies interested in turning a profit, and that doesn't necessarily lead to better care. In fact, the cost-savings these technologies promise could be the result of reducing the time an individual spends face-to-face with a doctor or nurse.
"One thing that I think is unchanged since antiquity is that when you're seriously ill, you feel bad," says Verghese. "And amongst all the other things you need, you also want someone to care for you — not just your family member but someone with the scientific knowledge to also express care."
Thadaney says a member of her household recently brought that point home. He had been injured in a bicycle accident. Treatment involved a complicated trek through two hospitals and a rehabilitation facility. Thadaney was able to advocate for him. "I was able to call friends who are physicians," she says. "I was able to, you know, call into the leadership of those organizations and request for something different."
That intervention alone provided an edge to her family member, but she says what really helped him was a visit with Verghese. The doctor "didn't tell him anything different than he already knew," she says, but he provided comfort and reassurance, "and I think it hastened his healing."
Verghese says he was recently reading Walt Whitman's accounts of his time caring for the wounded in Civil War medical tents on the Mall in Washington, D.C.
"He did what those young men most needed," Verghese says. "They were so far from home. They needed someone to read to them, to hold their hands and to write letters for them and take care of their every task. And it was the most elemental kind of care. Nothing's changed. You know we're still the same human beings."
Verghese is hopeful that technology, such as artificial intelligence, can improve medical care, but only if it isn't done at the expense of human contact. AI has the potential to free up clinicians to spend more time with their patients, depending on how it ends up being deployed. In principle, AI could also help the most challenging tasks.
"We don't need another image recognition [system]," he says. "They're all nice great and very tidy."
But where the technology can do the most good is to help sort through the clues gathered during medical treatment. "Medicine is messy," he says. "Help us out."
Some of the nuts-and-bolts improvements that AI can bring have their place, Thadaney says. "Yes, the patient wants you to make sure that you have efficiencies in your system so they don't get 19 bills with the same stupid thing."
But patients also want to get better. To help accomplish that, doctors and nurses can't simply be adjuncts to machines. Her mantra to the young doctors she advises is this: "In the end, be present. That matters a great deal."
In March, Stanford inaugurated a new institute to focus on the human dimensions of artificial intelligence.
Dr. Russ Altman, a professor of bioengineering and genetics at Stanford and an associate director of the new institute, says it is important to have best practices in place as technology and medicine commingle. "It's unfair and unrealistic to expect that technologists to be experts at all this."
He shares the concerns of Verghese and Thadaney that machines could degrade the human relationship at the core of medicine.
"Medicine is a combination of art and science," which will be augmented by AI, Altman says. "But the act of laying your hands on a patient, showing that you really care about what is there, what their problem is [and] assuring them that you're going to be with them through an odyssey — that might take a while," he says. "That is very difficult to imagine being replaced by computers."
You can contact NPR Science Correspondent Richard Harris at firstname.lastname@example.org.
AUDIE CORNISH, HOST:
Today we round out our month-long look at the state of artificial intelligence on All Tech Considered.
(SOUNDBITE OF ULRICH SCHNAUSS' "NOTHING HAPPENS IN JUNE")
CORNISH: We've looked at how artificial intelligence is creeping into more aspects of our daily lives, from how we get jobs, how we get medical care. Today we examine some of the ethical questions being raised. For example, technology can improve efficiency but at what cost to the doctor-patient relationship? NPR's science correspondent Richard Harris explores that as part of his reporting on AI in medicine.
RICHARD HARRIS, BYLINE: It's easy to think of medicine as a transaction. You're sick, so you go to the doctor for treatment. But caring for a person is often much more complicated than that.
KIM HILLIARD: How you doing?
UNIDENTIFIED PERSON: How do you do - that time again.
HARRIS: For 56-year-old New Orleans resident Kim Hilliard, care is a journey as she works to manage a tangled array of medical problems. Today she's at the University Medical Center to be screened for diabetic retinopathy, which poses a serious risk of blindness.
HILLIARD: I got the full diabetes when I made 40. And after I got that, that's when they start doing, like, a yearly thing on my eyes. They said it was time.
HARRIS: Sounds like you're careful to keep to schedule, huh?
HILLIARD: Oh, yeah. I mean, shoot, I go to so many doctors' appointment, I get tired.
HARRIS: She has high blood pressure, though she's down from taking three pills to just one. Last year, she had gastric bypass surgery to help her lose weight - oh, yeah, and back surgery.
HILLIARD: The back surgery was a pretty big one. I had that before this.
HARRIS: A back injury left her disabled, also in pain, which shoots down her left side.
HILLIARD: But I'm in physical therapy for that.
HARRIS: On this day, she will add a new practitioner to her long list - a computer program. An algorithm will take data from her eye exam and determine whether she needs to see an ophthalmologist. Nurse practitioner Debra Brown is learning how to use this new system.
DEBRA BROWN: We're going to take two pictures of each eye with this camera. It's going to be like taking a regular picture, but the light will be bright when we flash.
HARRIS: The actual exam takes just a few minutes. The instrument is designed to make its diagnosis without expert help. But this diabetes clinic, at least for now, adds another service - the human touch. The nurse practitioners consider this a teachable moment, so they take the time to review the results. Chevelle Parker shows Hilliard images of her eye.
CHEVELLE PARKER: When you zoom in here, we see some little fat deposits here, OK? That can be from the foods that you're eating, OK? Are you - think about some fatty foods you're eating - Sausage, bacon...
HILLIARD: No, I can't have that.
PARKER: OK, well, then when you were eating those, then the deposits were being placed here on the eye, OK?
PARKER: So that's why we talked to you about your diet. And now that you know you can't have that, this is the reason why, OK?
HARRIS: Hilliard admits that she has long had a taste for fatty foods.
HILLIARD: I was still eating bacon and sausage. And mm-hm, yes, I was. But I'm not eating it now. I haven't had it in - since October.
HARRIS: Her doctors and other health care providers have been nudging her to change her habits so she can improve her health, and this eye exam is another step along that path. It turns out her retina is showing signs of diabetic retinopathy, so she is referred to an ophthalmologist for follow-up.
HILLIARD: I do what I can to keep from going blind. So whatever they tell me to do, that's what I do. Well, at least I try.
HARRIS: So the journey continues for you, I guess.
HILLIARD: Oh, yes, but I'm going to work on it. I am working on it.
HARRIS: This is the real world of medicine into which computer algorithms are starting to take off. Will these new products seen as a boost to efficiency hurt or help the healing process? That question is very much on the minds of Sonoo Thadaney Israni and Dr. Abraham Verghese, who run a center at Stanford University that aims to strengthen the human dimension of medical care.
ABRAHAM VERGHESE: I think for too long we've had this assumption that any new technology is good, more is better. And you know, there's sort of a movement in medicine - slow medicine - a recognition that, you know, we should take our time; new is not always better.
SONOO THADANEY ISRANI: I agree with Abraham. And what I would add is we need to make sure that technology doesn't further exacerbate the issues of equity and inclusion that we already have in our country with lots of things, including health care. And health care is care; it's relational work. And if what we do is digitize it into that which can be measured and only focus on the bottom line, my fear is we will end up with what I've been calling a health care apartheid.
HARRIS: Medicine mirrors so much of society, she says, with the privileged getting more and the neediest getting less. Will algorithms stand in for the poor patients' doctors?
THADANEY: And if we create algorithmic care and kiosk it in some fashion focusing on efficiency and throughput, the people who will end up having access and using it will be the ones who already lack privileges of various kinds.
VERGHESE: Now, just to carry that thought forward, AI algorithms, we already know, are causing inequities in bail bonding, inequities in real estate. Policing in communities is affected by AI algorithms. That same kind of algorithmic approach can easily infect medicine and probably does.
HARRIS: Algorithms can easily pick up unintentional biases, Verghese says, and subtly encode racial stereotypes and the like. Also worrying - if machines take over on the front lines, that could interfere with the healing power of a human connection.
VERGHESE: Now, one thing that I think is unchanged since antiquity is that when you're seriously ill, you feel bad. And amongst all the other things you need, you also want someone to care for you - not just your family member but someone with the scientific knowledge to also express care.
HARRIS: That could be at risk if computer algorithms start taking over the tasks performed today by people like the nurse practitioners in New Orleans. Thadaney and Verghese are part of a conversation that's trying to get ahead of those worries. In fact, last month, Stanford set up a new center to study these issues. Verghese says on the plus side, a future technology could actually help doctors grapple with a messy reality of medical care.
VERGHESE: That's where I think the real frontier for AI is. We don't need another image-recognition something. I mean, they're all nice, great and very tidy and no messiness. But come engage with our mess. Medicine is messy. Help us out.
HARRIS: What would that look like? How would you do that?
VERGHESE: I actually don't quite know. But you know, there are all kinds of complicated social cues that are happening around the patient. The information is never complete, and it's coming in drips and drabs. And we have to make decisions. Help us with that sort of stuff. Help us with the chaos and the uncertainty. To the degree that technology can help, that would be wonderful.
THADANEY: Yeah. And if you look at the patient - in the end, what does the patient want? Yes, they want you to bring the best that science can bring. Yes, the patient wants you to make sure that you have efficiencies in your system so they don't get 19 bills with the same stupid thing.
HARRIS: But Thadaney says the patients also want to get better. To help accomplish that, doctors and nurses can't simply be adjuncts to machines.
THADANEY: In the end, be present. That matters a great deal.
HARRIS: Richard Harris, NPR News.
CORNISH: And in May, All Tech Considered looks at other ethical dilemmas raised by the spread of artificial intelligence. That starts next week.
(SOUNDBITE OF KRAFTWERK'S "IT'S MORE FUN TO COMPUTE (2009 REMASTERED VERSION)") Transcript provided by NPR, Copyright NPR.