Sie sind auf Seite 1von 7

Artificial Intelligence: The Walking Assistant Robotic

System, Is It Worth It ?
Albert Ilham R. . Demas Saputro . Didi Ruhyadi . Nude Alfathama . Pramana Aji . Risma Y. Fazriani

Abstract well. However, it is estimated by WHO that 285

People with impaired vision feel hard to do million people are the visually impaired
their daily activities because they cannot get worldwide. Among these people, 39 million are
information about their surrounding blind, 246 million have low vision and 82 % of
environment. Currently, there is a technology people living with blindness are aged 50 and
developed from the Artificial Intelligence field above[1]. With their condition, they can’t live
called the walking-assistant robotic system which independently. They need to assist or help from
is a device that can guide people with impaired other people to tell them about their surrounding
vision to walk safely. The system is based on environment. Sometimes they also need physical
computer vision and tactile perception. The main support to guide them through the streets, through
components of this system are safe direction their days. In the next stage, there is a question.
extraction, moving object detection, and a How can impaired vision person live normally
vibrotactile indication. The walking assistant and independently or at least they can walk safely
consists of several body parts which are a rollator through any possible obstacles from their
to provide a strong physical support, a Kinect environment?
device used as eyes of the system, and ultrasonic The simple and realistic answer is people
sensors utilized to detect the road surface. The with impaired vision usually use a cane or a dog
purpose of this paper is to review walking to guide them. By using a walking cane, they can
assistant technology device in general, so finally feel the surface condition in front of them, but it
it can be concluded if the walking assistant is is not the best solution. Perhaps they know the
worth it to be used by impaired vision people. condition of the road but it takes a time to feel
Several experiments done show that the walking sure. Clearly, it helps a lot, but it is not the best.
assistant robotic system is effective and helpful in Then, a guide dog can bring them to the
faciliting impaired vision people to walk destination. It can be trusted by his owner, but
independently and safely there is a chance if the dog turns into a dangerous
Keywords Computer vision · Tactile perception · dog. A fact that can’t be refused, a dog keeps
Vibration modes · Walking assistant robotic being dog, dogs are animals, they have a natural
instinct like animals in common.
1. Introduction But currently, there is a development of the
walking cane. Many scientists add a physical
The eye is one of the important parts of a sensor function, like the ultrasonic sensor, laser
human body. A human receives information sensor, and so on. With the sensor, the distance
about their surrounding environment by their between user and obstacle can be measured
vision. So, it can be concluded that if someone exactly and the message can be received by the
with impaired vision or eye disorder, they cannot user through tactile. As mobile robot technology
receive true information about their environment develops, researchers combine canes with mobile
robot technology to design a new device for
guiding the visually impaired. A Guide Cane [2]
is developed for the visually impaired. With
ultrasonic sensors detecting obstacles and an
embedded computer deciding a safe direction,
users can feel a very noticeable force in the
handle which guides the user easily without any
conscious effort on their part.
Figure 2 : The diagram of the walking assistant robotic
Based on a paper that we have reviewed, system
there is a group of Chinese scientists that have
found innovation about a device that can guide
people with impaired vision using some 2.2 Safe direction feature extraction
algorithm to function the tools in this device. The
device that they have planned using a Kinect The walking assistant will give the user front
camera to get the picture of the real condition in free direction and road surface flat direction by
front of the user, also using an ultrasonic sensor processing image from Kinect device and
to detect the possible obstacles or any holes on ultrasonic sensors. The system will process pre-
the road, and so on. The device gives information filter of depth images first for reduce the noise
to the user by a vibrotactile belt and it is used as and fill the holes in depth images before
a trolley but it walks automatically. extracting front free direction.
In order to give real-time direction to the user
a simple and fast depth images pre-filter method
2. Design and System of Walking Assistant
is employed. This method is based on the fast
2.1 Design in General marching method of color images [3].
If user input their voice “Direction”, the
Walking assistant robotic system is based on proposed system can real-time tell the user the
computer vision and tactile perception. This robot front free directions which are the areas with
consists of three main function which are safe fewer barriers. Depth images from the Kinect
direction extraction, moving object detection and device have a resolution of 640 x 480 pixels. Such
position, and vibrotactile belt indication. a low resolution cannot carry the whole
information of the depth images. In order to solve
In this input interface, a special structural
that an algorithm is used in this system.
rollator is designed which is equipped with one
Kinect device, three ultrasonic sensors, a According to Figure 3. First, Kinect device
microphone and a control cabinet as shown in capture front view of user (e) and process pre-
Figure 2. For the output interface vibro-tactile filter images (a) to reduce noise and fill the hole
belt is designed with vibration motor for give in depth images and get (b), after that the
message indication as vibration. Figure 1 depicts algorithm process this images using compression
the main flowchart of systematic process. processing and get (c) and finally (d) is a sketch
after making a count histogram. The lighter color
show the front free direction.

Figure 1 : The main flowchart of systematic process

moving object [4]. The background difference
method detect difference between background
and foreground so it can be calculated moving
object areas, map its position to estimates the
distance between the user and the moving object,
can be computed Figure 4.

Figure 4 : The flowchart of processing moving object

detection and location

2.4 Vibro-tactile indication

The walking-assistant provides two main

message indications. The mic voice input is
designed to input users voice like “Direction” and
“Objects”. Also the vibrotactile belt is designed
as an output interface consisting of sixteen
Figure 3 : The workflow of extracting free directions from
vibration motors which are numbered as shown
depth images caught by Kinect device
in Fig. 6.
Ultrasonic sensors designed to detect tiny
obstacles and holes on the surface of roads, which
is placed on the bottom of the rollator, there are
three ultrasonic sensors which are 20 cm high
from the flat floor, when sensors detect that their
value is less than the threshold value, it means
that there is regarded to be threat.

2.3 Moving object detection and position

Figure 5 : The prototype of the vibro-tactile belt. Vibration
motors are numbered in (a). b and c show the back side and
According to know if there are moving object front side of subjects wearing the vibro-tactile belt
like cars, bike, and so on. User need to input voice
“Object” to the mic and there will be tactile These motors are separated into two groups,
indications for them. In the system, Kinect device the front group (motor 1–6, motor 9–14) are
provides color images which can be detect placed on the front of waist while the back group
moving object and combined with depth images (motor 7–8, motor 15–16) are placed on the back
for computing approximate position of object. of waist. The motors on the belt are driven by a
But it not simple because in real-time outside square wave whose frequency is 100 Hz, and the
moving object should be high speed, and low normal value can vary as needed by adjusting
complexity. Therefore, the background duty cycle.
difference method algorithm is using to detect
3. Trial total. In this experiment, researchers picked ten
subjects aged 20 to 27 as volunteers with normal
It is such a necessary task to test how good vision but are all blindfolded and of course
the system is working before using it. After wearing vibro-tactile belt. Before the experiment,
explaining the design and the system, we will volunteers took twenty minutes to learn the
move to be testing the efficiency of walking correspondences between vibration modes and
assistant robotic system. The writers of this paper indication meanings. Each subject has repeated
actually don’t test the robotic system in real life, the experiment three times. This test program
but we mean to explain the trial of this system that which can record the direction, distance number,
has been done by researchers and also the result and the choice of subjects is used to measure the
of it to finally decide if the system is worth it. perception rate.
A few experiments done by researchers are After testing every main component, testing
dedicated to try or to test every main component the effectiveness of the whole system while it is
of walking assistant robotic system if it is used by used is a must. In this experiment, researchers
impaired vision people. Those experiments picked the same ten subjects as in tactile
consist of three main part; front free direction perception experiment. The volunteer subjects
experiment, tactile perception experiment, and took part in this experiment in two ways; using a
the whole system testing experiment. Every cane and using walking assistant robotic system
experiment was done in several places and while they were walking in the same chosen road,
conditions and some volunteers that have taken all blindfolded and with no other’s help. A road
part of in the experiment. chosen for this experiment is beside researchers’
To test the efficiency of front free direction, laboratory in Southeast University campus,
it took thirty different scenarios to be processed. Nanjing, China. The condition of the road is
Input data was captured by Kinect sensor to likely normal road; there are trees, several cars
produce final free direction processed by front parked, and people walking or riding bikes on the
free direction algorithm as output data. After road. Sketch of the road used for experiment is
being processed, the final output hopefully acts shown in Figure 6.
like an effective direction guide of impaired
vision people which makes right correlation of
actual direction and system indication, also
makes error as least as possible.
Tactile perception experiment is dedicated to
perceive the recognition rate of each tactile mode
of this walking assistant robotic system. There are
two main kind of messages, which are five
direction judgements and three distance
judgements. Five directions are named Direction
1, Direction 2, Direction 3, Direction 4, and
Direction 5. Three distance ranges are Distance 1
which is 3-4 m, Distance 2 which is 2-3 m, and
Figure 6 : The test sketch map
Distance 3 which is 0.5-2 m. Combining different
directions with different distance ranges, there The volunteer subjects were asked to walk as
are fifteen different vibration modes in total and safe as possible with a cane and with a walking
each mode works six times and ninety times in assistant robotic system in route from A to B and
also from B to A. In the first experiment, the Additionally, the average recognition rate of all
system works on SDI mode for all the period of distances in three times are respectively 92.24%,
experiment. After some times, the subjects can 94.52%, and 96.11%. We can conclude from the
get MODI by inputting voice orders. Final results result that tactile indication method is effective,
that were measured are time cost, number of useful, and easy to use [5].
obstacles the subjects avoided, number of voice
“Objects” or “Directions”, and detail walking
routes taken by all subjects that were recorded by
video camera.

4. Result
Researchers collected the results of all
experiments done and they were presented by
statistics data graphically. After testing the front
free direction component by processing thirty
scenarios, we get detailed result as shown in
Figure 7. As we can see the graph, front free Figure 8 : The recognition rate of each mode in the first
direction extraction can make the system experiment
indication the same with correct direction. The
accuracy rate of front free direction extraction is
more than 90%. Therefore, we can conclude from
the result that the method used in front free
direction is effective for the system proposed.

Figure 7 : The accuracy rate of front free direction

From tactile perception experiment, the result can

be shown by presented graphics as in Figure 8 and
Figure 9. The result of first step of this
experiment which produced data shown in Figure
8 concluded that the recognition rate of each
mode is higher than 88%. Beside, the average Figure 9 : The recognition rate of each direction and each
recognition rate of each direction is more than distance in the three times experiments
90%. The average recognition rates of all
directions in three times experiments are
For the whole system testing experiment, the
respectively 92.25%, 94.08%, and 95.66%.
routes taken by every volunteer were recorded by
video camera and then drawn as shown in Figure comfortable level based on every volunteer
10. Subjects can walk independently and can also subject’s feeling. As we can conclude from the
avoid obstacles normally. result, subjects feel comfortable and safe while
walking blindfolded with walking assistant
robotic system than with a walking cane. Subjects
also feel easy to use the system.

5. Conclusion

The walking assistant robotic system is

helpful for blind people or people has visually
impaired to assist walking independently and
safely. From the Table 1 we can conclude that
the user can walk more quickly and safely using
this walking-assistant than using a cane or a dog.
But there are problems with the sensors and the
algorithms, such as Kinect device need to keep
working out of direct sunlight and the algorithms
take long time to interpret moving object. Overall
Figure 10 : The walking route of each subject the walking-assistant is worth it.
Researchers also took several close-up
pictures when one of subjects avoided a stone 6. References
while walking with walking assistant robotic [1] World Health Organization (2013) Visual
system on the road as shown in Figure 10. After impairment and blindness. Media
the ultrasonic sensors had detected a stone, back center,
motors began to vibrate to prompt the subject,
then the subject adjusted the walking. eets/
Meanwhile, data of the number of obstacles fs282/en/. Accessed 8 June 2015
subjects avoided and number of voice “Objects” [2] Ulrich I, Borenstein J (2001) The
or “Directions” measured can be presented as in GuideCane-applying mobile robot
Table 1. technologies to assist the visually
impaired. IEEE Trans Syst Man Cybern
Part A 31(2):131–136
[3] Telea A (2004) An image inpainting
technique based on the fast
marching method. J Graph Tools
Table 1 : Measurement of the experiments 9(1):23–34
[4] Elgammal A, Duraiswami R, Harwood
D, Davis LS (2002) Background and
After all experiments, researchers finished foreground modeling using
the walking assistant robotic system trial by nonparametric kernel density estimation
investigating all volunteer subjects in a for visual surveillance. Proc IEEE
questionnaire. The questions asked are mainly 90(7):1151– 1163
about safety level, practical level, and
[5] D. Ni, A. Song, L. Tian, X. Xu, and D.
Chen, “A Walking Assistant Robotic
System for the Visually Impaired Based
on Computer Vision and Tactile
Perception,” Int. J. Soc. Robot., vol. 7,
no. 5, pp. 617–628, Nov. 2015.

Albert Ilham Rumaidinillah


Demas Saputro

Didi Ruhyadi

Nude Alfathama

Pramana Aji

Risma Yuanita Fazriani