U of M engineering researchers use the latest in technology to develop a device that could change how the blind navigate through daily life.

By Laura Fenton

U of M engineering researchers use the latest in technology to develop a device that could change how the blind navigate through daily life.Once this app debuts, walking canes and seeing-eye dogs will be so last century.

Herff College of Engineering researchers at the University of Memphis have spent more than three years developing a smart phone app, or mobile phone application, that will allow the blind to read printed text anywhere they go. It could also eventually act as a GPS-type device for them.

“For blind people, reading printed text is very important because if it is in the computer, they have a reader, but if it is in a paper, they have nothing to read with,” says Mohammed Yeasin, University of Memphis associate professor of electrical and computer engineering.

He and graduate student Akbar Shaik, who has assisted on the project for a year and a half, are working to finish the first complete prototype of the app.

The process of researching the niche for the app, logistics of creating the program and the dedication to create a zero-cost app meant Yeasin and Shaik spent more hours on the project than they can count.

Preparation included discussing the application’s purpose and testing prototypes with blind people, which helped to lead to the current design.

But the sleepless nights and hours reading manuals are entirely worth it, according to both researchers.

“Of all the handy-dandy things we do, my main goal is always to [assist] people who need help most and can’t afford that help,” Yeasin says.

The app captures an image of text through a photo taken by the camera in the phone. The reader then processes the image and reads the text aloud.

The application keys alternate between corners of the phone’s screen.

“Use your right thumb to start the application,” Shaik says. “[Use your] left thumb to enter the capture mode, using again your right thumb to capture it. Use the [left thumb] to click to read the words.”

Mohammed Yeasin, left, and Akbar Shaik, right, will soon complete a smart phone app that will allow blind persons to “read” printed text through the use of the camera in the phone.

“Since I can see, I focus it again,” Shaik says. “A blind person, he might not know it is out of focus. In that case, the phone vibrates.”

The verbal feedback from the first prototype was not well received.

“The blind persons are very responsive to sound, but they don’t like to be spoken [to] that the application has been started,” Yeasin says. He explains that the subjects did not want to draw more attention to themselves in public by having the app talk them through the process.

But the app is not devoid of kinks, which Yeasin and Shaik must mend before its launch.

The largest issue is the clarity of the images on glossy or bright surfaces, such as on medicine bottles or signs in direct sunlight.

Yeasin describes the camera’s inability to always completely capture the text as a “complex interaction of the light” and the camera.

“All of these things depend on if you can take a good picture,” Yeasin says. “If the lighting is good, things work. If the lighting is not, it does not work.”

In addition to the text-reading element, the app will also read barcodes to allow people to know which book or item they are holding.

By using GPS functions within the smart phone, Yeasin and Shaik also intend to include a navigation element to direct a person when he or she is in an open space or within a building.

Standard GPS programs guide a person to locations based on addresses, but spaces such as parking lots or stores within a shopping mall are not detailed or precise enough to allow the device to properly take a person to the exact location needed. Traditional GPS devices can be three to five meters off from the specified location, Yeasin says.

But, if the GPS and inner shield sensors can determine several surrounding addresses, the app can determine the person’s location.

As for maneuvering within a building, the app would detect additional location information from radio frequency identification (RFID) tags placed in the building as points of reference. Tags are inexpensive, costing about 10 cents each or more depending on the sensor specifications.

Also, maps of the buildings will be uploaded to the app database for additional directional support. Blueprints of most public buildings are public record and would be available for Yeasin to upload. Private residences would not be included in the app.

“Those things are not finished products,” Yeasin says. “We are still working on those things.”

Researchers work on formulas for the application.

“The projects have changed significantly over time,” Yeasin says. “I had no clue how to do it. I only had some idea [that] I wanted to do something that is useful for people.”

All elements will be components of the same app to simplify the process.

“Our goal is, without adding any more devices to this phone, is to configure it to provide a number of different services,” Yeasin says. “So the cell phone should not be only a cell phone. The cell phone could be the heart for this person’s day-to-day activities.”

By teaching people how to understand the feedback and the steps to using the app, a person should be able to guide him- or herself through the program.

“There is no way to teach the person to do it right other than an interactive process between the device and the person,” Yeasin says.

Training the blind users will be an essential final step in the process of the app because although people might be able to figure out the non-verbal signals, understanding the program in its entirety will make it more useful to the person.

The app will be compatible with all smart phones on either the Apple or Android platforms. It will be based on the Android platform because app hosting is available for free, a benefit Yeasin wants to pass along to users by offering the app at no cost.

He and Shaik also chose to use open source programs, such as Google’s, because there are no fees for them online.

“We want to do things at every level that are free,” Yeasin says. “This is not a cash cow thing. This is something I want to do because it’s close to my heart.”

Yet, the project is priceless because it will better the lives of others. 

“Whatever it is I do, it has to help people one way or another,” Yeasin says.

Once the complete first prototype is ready, Yeasin will again test the app. The expected launch date of the app has yet to be determined.