Help A Blind Person Identify Everyday Things, Via Smartphone App | KERA News

Help A Blind Person Identify Everyday Things, Via Smartphone App

Dec 15, 2015
Originally published on June 7, 2016 11:14 am

The camera on your smartphone is powerful. You use it to record your baby's first steps. Take a panorama shot or selfies at the Taj Mahal. Every day, we're finding new uses.

And recently, a startup in Silicon Valley realized: That camera on the phone could be used by people who are blind, to get help seeing remotely. The company Be My Eyes has created a novel kind of volunteer opportunity on the Internet.

I'm in the home of Lisa Maria Martinez. She's got a little bit of vision, but grew up reading braille. "Print is not my forte," she says.

That makes it very hard to read labels — especially those fancy fonts on food packaging. Martinez wants to make spaghetti before her husband and kids get home. "I'm cheating and I'm going to use a pre-made sauce," she says.

It's a weekday. Time for shortcuts.

Martinez already has the key ingredients on the table. I ask her what she's laid out, to which she responds, chuckling, "I don't know."

And that is the task at hand: to sort out what's inside the jars she's pulled from her pantry. She's going to use an app on her iPhone called Be My Eyes.

She taps the icon to launch. You could also launch it using a voice command.

Be My Eyes is a pretty simple concept: The person who's blind connects to people with sight, points the smartphone camera at things and gets help seeing what they are. The app rings — a monotonous, droning ring — until both sides are connected.

On our first call, we get connected to volunteer Kayley Bennett. They say hi and Martinez points her phone at a can. Bennett reads the label. "They are chili beans."

Next jar: "That's some pesto."

And finally: "That is parmesan, asiago, romano sauce."

It's a three-cheese sauce, which Martinez decides she's going to use.

That said, people from all corners of the world respond. Bennett, for example, is on vacation in Exmouth, in Western Australia.

Bennett is from that continent and she happens to have an Internet connection strong enough to stream video chat. This is an unconventional volunteer setup.

I ask her why she's volunteering now. "Well, they just call you whenever," she says.

"It just comes up as a notification, so ..."

Micro-Volunteering

Be My Eyes isn't just a way to help the blind. It's an experiment in online engagement. To date, the main way you can do good on the Internet is by signing a petition or donating money. Here, you can hop on a quick call to action and clock in some micro-volunteering.

"You have a chance just to step in or step out of your everyday life just for a brief moment and then do something good within a couple of minutes," says CEO Christian Erfurt. "I think that's very appealing for people."

The numbers suggest he's right. The startup (which, Erfurt says, has spent zero money on marketing) has more than 24,000 blind users registered and 300,000 volunteers.

The company has seen a couple dozen cases of abuse, "mostly prank calls from teenage boys pretending to be blind," Erfurt says.

But mostly, people are just trying to help. And to be a truly on-demand service, they need more volunteers. Though one should not volunteer at any expense.

Cautionary Tale

Back in Lisa Maria Martinez's home, we get connected to Bua Bondkotmanee, a student who's never done community service. She says this is her very first time. The app popped up in her Facebook stream. She decided to download it and try it out instead of playing Candy Crush.

She helps Martinez sort through the colors of some yarn, for knitting.

"This one has many colors," Bondkotmanee says. "Pink, yellow, blue and white."

When they're done going through a few other ones, I ask Bondkotmanee where she is currently, by which I mean: what country. (She has a foreign accent.) But Martinez and I get a much more specific answer.

"Um, I'm in the car," she says.

That's right. She's in Thailand, and in a car. Not only that, she doesn't mean in the backseat. She's in the driver's seat.

It's a very awkward situation. Asked why she's on the app at the same time, Bondkotmanee says, "I want to help you."

While this is very sweet and bizarrely global — the last thing the world needs is a headline about an altruistic young woman in Thailand who crashed her car while helping a blind lady in California see.

Wherever you are, if you're going to try Be My Eyes, please do it safely.

Copyright 2018 NPR. To see more, visit http://www.npr.org/.

ARI SHAPIRO, HOST:

The camera on your smartphone is powerful. It can record your baby's first steps or take a panorama of the Taj Mahal - also selfies. Recently, a startup in Silicon Valley realized a camera on the phone could be used by people who are blind to help them see remotely. NPR's Aarti Shahani reports.

AARTI SHAHANI, BYLINE: I'm in the home of Lisa Maria Martinez. She's got a little bit of vision.

LISA MARIA MARTINEZ: But I grew up reading Braille, so print is not my forte.

SHAHANI: Which makes it very hard to read labels, especially those fancy fonts on food packaging. Martinez wants to make spaghetti before her husband and kids get home.

MARTINEZ: And I'm cheating and I'm going to use a premade sauce.

SHAHANI: Lisa Maria, I'm disappointed in you. You're - premade.

MARTINEZ: (Laughter) I know.

SHAHANI: But it's a weekday - time for shortcuts. Martinez already has the key ingredients on the table.

Tell me what we've got.

MARTINEZ: I don't know (laughter).

SHAHANI: And that is the task at hand, to sort out what's inside the jars she's pulled from her pantry. She's going to use this app on her iPhone called Be My Eyes. She taps the icon to launch.

COMPUTER-GENERATED VOICE: Creating request - connecting to servers.

SHAHANI: You could also launch it using voice command.

COMPUTER-GENERATED VOICE: Waiting for other part.

SHAHANI: Be My Eyes is a pretty simple concept. The person who's blind connects to people with sight, points the smartphone camera to things and gets help seeing what they are. The app rings - this monotonous, droning ring - until both sides are connected.

COMPUTER-GENERATED VOICE: Connected to other part.

MARTINEZ: Yay.

SHAHANI: The other part, so to speak, is volunteer Kayley Bennett. They say hi and Martinez points her phone.

MARTINEZ: So I'm holding up a can.

SHAHANI: Bennett reads the label.

KAYLEY BENNETT: They are chili beans.

SHAHANI: Next jar.

BENNETT: Basil pesto.

SHAHANI: Pesto, and finally...

BENNETT: This is parmesan, asiago...

SHAHANI: It's a three-cheese sauce, which Martinez decides she's going to use. Now, there's a cool little detail about the location of her volunteer.

BENNETT: I'm on a holiday in Exmouth (laughter) at the top of Australia.

SHAHANI: Bennett is from Australia on a family vacation on the northern tip of that continent. And she happens to have an Internet connection strong enough to stream video chat. This is an unconventional volunteer setup. I ask her...

You decided to get on this app while you're on holiday?

BENNETT: Yeah, well, they just call you whenever. It just comes up as a notification.

SHAHANI: Be My Eyes isn't just a way to help the blind. It's an experiment in online engagement. To date, the main way you can do good on the Internet is by signing a petition or donating money. Here, you can hop on a quick call to action, clock in some micro-volunteering.

CHRISTIAN ERFURT: You have a chance just to step in or step out of your everyday life just for a brief moment and then do something good within a couple of minutes.

SHAHANI: Be My Eyes CEO Christian Erfurt.

ERFURT: I think that's very appealing for people.

SHAHANI: The numbers suggest he's right. The startup has more than 24,000 blind users registered and 300,000 volunteers. They have seen a couple dozen cases of abuse.

ERFURT: Mostly prank calls from teenage boys pretending to be blind.

SHAHANI: But mostly people are just trying to help. And to be a truly on-demand service, they need more volunteers. Though one should not volunteer at any expense. For example...

COMPUTER-GENERATED VOICE: Creating request - connecting to servers.

SHAHANI: Back in Lisa Maria Martinez's home, we get connected to Bua Bondkotmanee, a student who's never done community service. She says this is her very first time. The app popped up in her Facebook stream. She helps Martinez sort through the colors of some yarn for knitting.

BUA BONDKOTMANEE: This one has many color - pink, yellow, blue and white.

MARTINEZ: OK, perfect.

SHAHANI: And when they're done, I ask Bondkotmanee where she currently is, by which I mean what country. But Martinez and I get a much more specific answer.

BONDKOTMANEE: I'm in the car.

MARTINEZ: Oh, you're in the car, oh, my gosh.

SHAHANI: That's right. She's in Thailand in a car. And by that she doesn't mean in the backseat.

SHAHANI: You're in the driver's seat?

BONDKOTMANEE: (Laughter) Yeah.

SHAHANI: So why are you on the app also?

BONDKOTMANEE: I want to help you.

SHAHANI: While this is very sweet and bizarrely global, the last thing the world needs is a headline about an altruistic young woman in Thailand who crashed her car while helping a blind lady in California see. Wherever you are, if you're going to try Be My Eyes, please do it safely. Aarti Shahani, NPR News, Silicon Valley. Transcript provided by NPR, Copyright NPR.