Usability Evaluation for Continuous Error of Fingerprint Identification

. It is generally quite difficult to apply the current usability evaluation methods to the interface for the biometric identification, because the operation time of the biometric identification is too short to analyze. In the current study, we conducted the interview research on the biometrics. From the results of the interview research, it was clarified that 71% users have an experience of continuous error while in use of the biometric identification. So, we conducted the evaluation experiments for continuous error of the fingerprint identification. In the experiment, the unsatisfaction which is one of the usability is evaluated from the aspect of the mental stress during the continuous error of fingerprint identification. Based on the results of the evaluation experiment, we can show a guideline that when the continuous error occurs X times, the fingerprint identification system should be changed to another identification method to avoid an increase in the user’s unsatisfaction.


Introduction
In the current progress of the information society, the technologies which can identify a user specifically and real-time have been attracted. The biometric identification has been considered as one of the most powerful technologies. In the biometric identification, fingerprint, vein and iris are used as the physical characteristics. On other hand, signature, gait and voice are used as the behavioral characteristics. However, the physical characteristics, especially the fingerprint, is mostly used for the identification and the behavioral characteristics are not commonly used. Being one of reasons of this situation, it is considered that the usability of biometrics has not been sufficiently discussed.
On the other hand, it is quite difficult to apply the current objective methods to evaluate the usability of the interface [1]- [6] for the biometric identification, since the biometric identification step is having too short time to analyze. So, it is needed to innovate another approach to evaluate the interface of biometrics.
In the current study, the interview research on the usage of biometrics was conducted. Based on the results of the interview research, we conducted the evaluation experiments for continuous error of the fingerprint identification. The general usability is evaluated from the three view points; effectiveness, efficiency, and satisfaction. In this study, we supposed that the efficiency is related with the continuous error, the unsatisfaction is evaluated from the aspect of the mental stress during the fingerprint identification. And, as for comparison, the unsatisfaction is evaluated during the continuous error of the password identification. Moreover, as a context of use which is mentioned in ISO 9241-11 [7], the time pressure was added as a condition of the evaluation experiment.
Based on the results of the evaluation experiment, we can show a guideline that when the continuous error occurs X times, the fingerprint identification system should be changed to another identification method before the user's unsatisfaction is increased.
The present study is organized as follows: in Section 2, a literature review is summarized; in Section 3, the interview research on biometrics is described in detail; in Section 4, the results of our evaluation experiment of continuous error are discussed; and in Section 5, our conclusions are noted.

Literature Review
Usability is defined by ISO9241-11 [7] as "The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use". Under this definition, there are three critical terms of usability: effectiveness, efficiency and satisfaction. In applying these terms to biometric identification, the effectiveness is about whether the identification is successful or not. The efficiency is for the number of trial and operation time to successful identification and the satisfaction is a kind of emotion arising from the usage of biometric identification. The accuracy of the most biometric systems is still not high enough. For example, the fingerprint identification can be problematic if the user's finger is in different condition from the enrollment; such as wetness, dryness, or oiliness, moved during the reading of the fingerprint, or just a part of the fingerprint is presented. Boutella [8] reported that fingerprint image quality is of great importance for the fingerprint identification, and affects its performance. So, the continuous error is currently unavoidable. When the continuous error is considered from the viewpoint of the usability, the efficiency is mightily related with the continuous error.
Some studies on the usability of biometrics have been conducted. Theofanos [9] validated the elements of usability of biometrics; such as the method for feedback, the physical issues (angle and height of the devices for identification), and presence or absence of the usage experience on biometric identification. Matsumoto [10] discussed the balance between the security and usability. El-Abed [11] evaluated the biometric identification system from the viewpoint of users' acceptance and satisfac-tion. Though these studies were focused on the usability of biometrics, it was not evaluated with experimental approach or the continuous error was not concerned.

3
Interview Research on Biometrics

Method of the Interview Research
The interview research on the biometrics was conducted to 21 university students who were regularly using the biometric identification system. Following items were the interview questions; (Q1) What type of biometrics have you used? (Q2) Have you ever experienced an error of biometric identification? (Q3) Have you ever experienced continuous error of biometric identification? (Q4) If you have experienced continuous error, how did you respond to it?

Results of the Interview Research
The results of the interview research on the experience in using biometrics were shown in Table 1 and on the experience of error of biometrics identification in Table  2. 90% users have experienced an error of biometric identification while 71% users have experienced the continuous error. Most of the users who have experienced the continuous error feels unsatisfied about the accuracy of the biometrics. Similar research was conducted by Sasaki [12]. In his research, he mentioned that some students using the vein identification as the attendance management system have experienced the continuous error, and felt strong unsatiscfaction to the accuracy of identification system.
The summary of the responses regarding the continuous error in the fingerprint identification are noted below; * Wipe finger, or heat it up by breathing on it. * Ask someone who registers the system to login and then use the system. * Give up the fingerprint identification and use another identification method (for example, password).
From these results, the problems of biometrics usage caused by the continuous error must not be ignored. It is necessary to evaluate the usability of biometrics from the aspect of the continuous error.

Method of the Evaluation Experiment
In this experiment, users' mental stress was evaluated when the continuous error had occurred in the identification with fingerprint and password. The time pressure was also added as a condition of the evaluation experiment. The fingerprint reader of the swipe type (SREX-FSU2, RATOC Systems Inc., Japan) was used in the experiment. The experimental view is shown in Figure 1. On the interface for the identification was made by us ( Figure 2).

Fig. 1. Experimental View
In the identification system, there was no identification performed during that period. As the fingerprint reader is triggered by swiping the finger, the interface shows results randomly whether it's successful or not after one second. If the identification fails, the user clicks the retry button and tries it again, if the identification succeeds, a web browser pops up, and a task is finished. The password identification also has the same flow. Each subject used six characters to set the password.
The subjects of the experiment were 20 university students. The experimental task was conducted under four conditions below.  In each 4 conditions, 5 tasks (the successful identification is at the nth [n=1-5]) were conducted randomly. In the condition "with time pressure", the tense situation was assumed by the subjects in the task. The subjects were given 30 seconds to log in using the fingerprint identification while those who were using the password identification had 40 seconds. Both number of seconds were set to be able to login even if there were continuous error. After every task, user's mental stress about the 3 items was evaluated on 5-grade evaluation based on the evaluation method of Tamaki [13].
(E1) How much mental stress did you feel on the number of times of identification? (E2) How much mental stress did you feel on the action of identification? (E3) How much mental stress did you feel on the time of identification?

Results of experiment and consideration
The user's mental stress on the number of times of identification is shown in Figure 3, on the action of identification is shown in Figure 4 and on the time of identification is shown in Figure 5. In all graphs, the vertical axis shows the average score of the men-tal stress, and the horizontal axis shows that the successful identification is at the nth. Four lined graphs show the each conditions from C1 to C4. In Figure 3, 4 and 5, it was shown that the users felt stronger mental stress from the password identification than the fingerprint identification. Especially on the mental stress on the action of identification in Figure 4, the difference of the mental stress between the fingerprint and password identification was clearly large. There was a statistically-significant difference (p<0.01) for the mental stress between the fingerprint identification and the password identification at each successful identification. These results were supported by the users' comments after the experiment that the fingerprint identification was easy to use and the password identification was bothering to type the password. Therefore, it is clarified that the usability regarding the satisfaction of the fingerprint identification is better than the password identification.   The analysis of variance and multiple comparison (Holm's method) were performed to the mental stress on the number of times of identification and the time of identification. In the result, in spite of the presence or absence of the time pressure, a statistically-significant difference (p<0.05) for the mental stress was shown in between the second and the third successful identification. Moreover, a statisticallysignificant difference was not shown in between neighboring conditions over the third successful identification. So, it was shown that the mental stress over the third successful identification is obviously lager than the second one. From these results, if a continuous error occurs twice, the fingerprint identification system should be changed to another identification method before the user's unsatisfaction is significantly increased.
Finally, in Figure 3, 4 and 5, from the results of the analysis of variance, it was shown that the presence or absence of the time pressure didn't influence to the mental stress. There was no statistically-significant difference between the presence and absence of the time pressure. Though the tense situation was assumed with time pressure, it was consider that the users didn't feel the pressure, because the penalty against the failure of the task wasn't set in the experiment. This term will be discussed continuously as a future issue.

Conclusions
In this study, we conducted the evaluation experiments for continuous error of the fingerprint identification. By the advance interview research on the usage of biometrics, it was clarified that the problems of biometrics usage caused by the continuous error must not be ignored. From the results of the evaluation experiment for continuous error, three conclusions were obtained; 1) The usability regarding the satisfaction of the fingerprint identification is better than the password identification. 2) When the continuous error occurs twice, the fingerprint identification system should be changed to another identification method to avoid an increase in the user's unsatisfaction. 3) The presence or absence of the time pressure does not notably influence to the mental stress of the continuous error of fingerprint and password identification. For future experiment, the change of pressure during the fingerprint identification should be measured and the relationship between the data gathered and the subjective assessment should be validated. The data of the changes in pressure will then be considered and be used for the improvement of the usability of biometrics. On the other hand, other physical or behavioral characteristics will be applied to the current proposed method.