Grasping Action for Impaired Vision Action Using HC2D Barcode Technology

. Nowadays, There are many people who have impaired vision. It is hard to do for weak-eyed man to sense a range because of their blurry vision. Therefore they often feel complication in grasping objects. It is useful to design and implement a system presenting the distance between a hand and an object for low vision people. In this paper, we propose a method to estimate the distance between a camera and an object surface on which a HC2D barcode is pasted. In this research, we assume the camera is worn on hand and the HC2D barcode with information embedded is pasted on the surface of object. The distance between the camera and the HC2D barcode, which is attached on the object, is estimated comparison the actual size of the HC2D barcode with the size of HC2D barcode that snap or capture from the camera.


Introduction
According to the recent day, the lot of number of people with visual impairment in the world is low vision people. And people with visual impairment are elderly people who are old. It seems that the number of low vision people in the world increases much more. From these facts, developing a system, which supports low vision people's daily life, is necessary. Compared to the healthy people's vision ( Fig. 1), low vision people's visibilities vary according to every individual symptom, such as blurry, crooked, and restricted visions. The visibility, of which an example is as shown in Fig. 2, might bring on tipping things and missing in grasping. And also, these difficulties could give low visions much stress. Indeed a lot of studies have been devoted with respect to support systems in reading characters, but there are few attempts to support in grasping things. In this paper, we propose a grasp action support system for visually impaired persons using HC2D barcodes [1] and a camera. This system can present the distance between the camera and the HC2D barcode, which is attached on the object surface by comparing the real size of HC2D barcode with the apparent size of HC2D barcode in the camera. In addition, the system can guide hands to the object's direction. The proposed system can be easily applied to support system presenting the distance perception by wearing the camera on hand. The effectiveness of proposed system is verified through some experiments.

Overviews
In this section, the way to u tem is to guide user's hand rection information from th camera on the hand. Cord from interfering in daily lif processing equipment. And the system detects a HC2D distance and direction inf Therefore, this information barcode which is pasted on lowing the voice sound, vo barcode at the center. By re

Background and
In this paper, we propose a and a camera. uses standing wave, which is generated by phase interference of transmitted and reflected waves. Using standing wave of audible sound [3], distance estimation is possible at the close distance. However, this method takes long time for calculating the distance. In addition, uses stereo images to estimate distance. Distance is estimated by the gap of corresponding areas. However, these methods do not use marker. It is not able to capture the target object. The purpose of this study is to guide a hand to object for visually impaired person. Therefore, it is hard to say this method is suitable for this study. As another approach, there is a method to use a compact compound-eye camera system: TOMBO [4]. However, TOMBO is special camera. In this study, a standard monocular camera is used from the viewpoint of availability and tractability.

Proposed Method
In this study, the distance between camera and the object is estimated by camera and HC2D barcode embedding the HC2D barcode's shape information. Let the camera focal length f [m], distance between HC2D barcode and camera z[m], size of HC2D barcode in the captured image a[pix], and real size of HC2D barcode b[m]. The relation of these symbols is shown in Fig. 3. As we can see, the distance between the camera and the HC2D barcode is calculated based two homologous triangles including camera focal length, the size of HC2D barcode in the captured image, and the real size of HC2D barcode. Processing flow of this study is shown as indicated below.
1. Get a captured image 2. Detect the region of HC2D barcode 3. Extract information of HC2D barcode 4. Estimate distance using the information of HC2D barcode 5. Show the distance the way to detect the region of HC2D barcode, way to estimate distance by using the information of HC2D barcode, and way to show the direction of object are explained in next section.

Detection of HC2D Barcode
In this section, the mathematical descriptions with respect to the distance estimation are explained. Suppose a camera coordinate is represented as (x; y; z) ( Fig. 6), the coordinate is transformed into the image plane coordinate by perspective transformation as follows.

Distance Estimation
Three of four HC2D barcode's vertex has a distinctive mark called "finder pattern" (Fig. 4). HC2D barcode is distinguished by detecting these marks. To detect finder pattern, the following process are applied to the captured image.
2. Extract the contour information.
3. Detect and save a square using the contour information. 4. Check up the square whether it has Small Square in itself.  n "(1)," is true in case of an ideal model, but the an is in the camera coordinate system does not definitely c med by u-axis and v-axis in image plane. By considering e (x; y; z) is transformed to a point (u; v) in the image pl e "(4)," the following notation can be obtained: Assume the real size of HC2D barcode is b[m], it is represented by the following equation Substituting the "(6)," to the "(5)," the following equation is obtained: According to the "(7)," camera's focal length f [m], distance between HC2D barcode and camera z[m], size of HC2D barcode in the captured image a[pix] and real size of HC2D barcode b[m] has proportional relation. From this point forward, the distance estimation is performed on the basis of the "(7)".

Guide Process
Two separated parts construct the guide method. One is a direction guide process to guide the user's hand to the object, and the other is distance guide process to announce the distance between the hand and the object.

Direction Guide
Captured image is separated 9 parts (Fig. 7). Direction guide is decided by detected position of HC2D barcode. There are 8 kinds of direction to guide the hand, such as "upper", "upper left", "upper right", "left", "right", "lower", "lower left", and "lower right". HC2D barcode's gravity center is calculated by detected HC2D barcode and direction of HC2D barcode is announced like "upper", "under left", by considering where the gravity point is found in separated parts. When the gravity point fit in the center of captured image, the hand is guided to the object after the distance between the hand and the object is announced.

Distance Guide
Distance between the camera and the object is calculated by "(7)," and user is informed the information of distance and direction by voice sound. When the HC2D barcode's gravity point fits in the center of captured image, the distance is read aloud for user. If the center of captured image does not fit the gravity point of HC2D barcode, voice sound announces the direction of the HC2D barcode again to fit the center point in gravity point. The system guides hand to the object by repeating this process. When the distance between hand and the object is short in under 5cm, voice sound saying that "please close your hand".

Measuring the Accuracy of Distance Estimation
Distance estimation is conducted by HC2D barcode and camera using "(7)". In addition, the value of f in "(7)" is calculated by camera calibration. Value of is calculated (Fig. 7). In "(7)" substi and are used 838 and 0.686 detection", the camera does in realistic environment. W flected in the form where H fore, distance estimations a estimated distance. The situ 1. When camera confront w which gradually changin 2. Estimate distance is perf on cylinder, prism and co The start of the estimate dis estimate distance interval is will be less than 0.03[m], H

Distance Estimation
At first, distance estimatio attached on flat plate surfac in graph. Axis of abscissas error is calculated by estim longer than true distance, t estimated distance is short value. The equation to calcu Fig. 9 shows result of dista the front. The error was le nd S. Tanaiadehawoot ituting a at z=0.05 [m]. In this experiment, the values o 6, respectively. As described at section of "HC2D barc s not always capture HC2D barcode from an anterior v Within a captured image, there is a possibility of being HC2D barcode was reflected aslant or was distorted. The are attempted in some situations to evaluate the accuracy uation where distance estimation is performed is as follo with HC2D barcode set as 0 degrees and measure dista ng the angle. formed at 3 patterns, which the HC2D barcode is attac one (Fig. 8).
stance between HC2D barcode and camera is 0.03[m], s 0.005 [m]. If distance between camera and HC2D barc HC2D barcode cannot fit within the captured image.
. 7. Separate the captured image system n When HC2D Barcode and a Camera Turn to the Fro on is conducted by camera and HC2D barcode, which ce. The estimated distance and true distance is summari s is true distance D t and axis of ordinate is error E . T ated distance D e and true distance. If estimated distanc the error is plotted with a positive value. By contraries ter than true distance, the error is plotted with a negat ulate the error is shown as "(8)" ance estimation when HC2D barcode and a camera turn ess than 5% and error was further reduced at close ran of f ode iew reerey of ows.

Distance Estimation
When camera confront wit performed whose angle shi grees, HC2D barcode did n image. According to Figs. than Fig.9. However, when a several millimeters error. next experiment, HC2D ba tin can as cylinder, PET bo 13-15, the error of which H Fig. 9. However, the error o thermore, detection of HC2

Assessment of Supp
Next, proposed system is system. First, examinees (s and put the camera on hand attached on, is set up on th can grasp the object or no indicated below. The numb 1. Examinees take a blindf cluttered desk by only gr 2. Examinees take a blindf system at desk put on the nd S. Tanaiadehawoot distance between the HC2D barcode and the camera when distance between the HC2D barcode and the camera when one n Whose Angle Changes Gradually th HC2D barcode set as 0 degrees Distance estimation ifts by every 15 degrees. When the angle exceeds 60 not recognize as HC2D barcode even if it is in the capt 10-12, in each angle, the accuracy of close range is low n the error of distance is replaced to real distance, it is o . Therefore, this error does not affect action of grasp. T arcode was attached on cylinder, prism and cone. We u ottle as prism, and paper cup as cone. According to F HC2D barcode is attached on cylinder and cone, is simila of which HC2D barcode is attached on prism is huge. F 2D barcode is difficult in this case. port System of Grasp Action evaluated by examinee by grasping the object using sighted peoples) cut off the eyesight by taking a blindf d. After the blindfold, object, which the HC2D barcode he desk (Fig. 16). The experiment whether the examin ot is performed. The experiment environment is shown er of examinees is 10 persons.
fold and do not wear the camera. They grasp the objec roping (Fig. 17)

A. Valuation Metho
Examinees are had a quest naire is shown as indicated -Usability of the direc -Usability of the dista -Usability of this syst

Result of Valuation
The result of valuation 1 is grasp the object at the clutt nee could grasp object, but examinees grasp the object nee's hand. Therefore, ther difference between the exam to actual environment, it is thing only groping. If the t feedback from the examinee -It takes a long time to cause it is necessary th -It is dangerous that if t ction guide ance guide tem ns s shown as Table 1. 7 of 10 blindfolded examinees co tered desk by only groping. In this experiment, one exa t he tipped other object on the desk. An average time is 23 second. This experiment used only feeling of exa re are some cases that they tipped an object because of minee's imaged distance and the real distance. If it repla dangerous for visually impaired persons to look for som tipped thing is hot drink, they get burned. We feed off es.
o recognize that a touched object is target object or not hat a touched object is confirmed carefully. there are some easily tipped things on the desk. distance between the HC2D barcode and the camera when ylinder distance between the HC2D barcode and the camera when ism  Table 2. The value of "Usability of the direction guide", "Usability of the distance guide", and "Usability of this system" are shown at average point. In this experiment, there was only the object on the desk. Therefore, all examinees could grasp the object without the tipping. All items of usability value is over 3.5. There is little variability among the examinees, but it seems to be good result. We feed off the feedback from the examinees. Sometimes, examinees feel difficulties that how long the camera moves. If the HC2D barcode's gravity point fits the center of captured image, it can grasp easily the object. It is necessary to get used to this system At last, the result of valuation 3 is shown as Table 3 and Table 4. The value of "Usability of the direction guide", "Usability of the distance guide", and "Usability of this system" are also shown at mean opinion score (MOS) [5]. As compared to the result of valuations 1, all examinees could grasp the object without the tipping. An average time is 4 second faster than the result of valuations 1. Therefore, the information from the camera could guide examinee to the object. In addition, the evaluation of usability is better than the result of valuation 2. It is because that the examinees get used to this system. These results show they alleviate their a burden on grasping action in daily life if users acquire proficiency in this system. We feed off the feedback from the examinees.
-When comparing the experiment 1, it is easily to grasp the object by using this system's information. -If the HC2D barcode out of the captured image, the system cannot inform the information of distance and direction. The target was grasped, and nothing was tipped. 6 The target was grasped, but other object was tipped.
1 The target was tipped.
3 An average time the examinees grasp the object 23 Second The target was grasped, and nothing was tipped 10 The target was grasped, but other object was tipped 0 The target was tipped 0 An average time the examinees grasp the object 19.3 Second Table 4. Mean option score at valuation 3. higher score is better. A number of examinee is 10.

Conclusion
In this paper, we propose distance estimation and guide system using HC2D barcode and a camera. This system can present distance between camera and HC2D barcode, which is attached on the object surface by comparing the real size of HC2D barcode with the size of HC2D barcode in the camera. In addition, the system can guide hands to the object's direction by capturing the HC2D barcode's position. By using this system, the visually impaired person's daily life ameliorates much more. In the experiment, we performed two prominent types of performs considering the actually environment. One is the measure of accuracy about distance estimation. We performed some experiments considering the actually environment and got a good result for this. Another is the validation of usability of guide system. The target object is set up on the desk with other object, and examinee grasped the target object using this system or does not. As a result, the value of valuation is over 3.5 point. All examinees could grasp the object by using this system, but 40% of examinees could not grasp the object by only using groping. Therefore, these results show this system's effectiveness. However, some examinees felt difficulties that how long they move their hand and where they move the hand if the camera drop off the HC2D barcode.
To resolve this problem, the separation of captured image must become finer, and the direction will be announced more properly.