Novest: Position Estimation of Finger on Back of Hand with a Small Ranging Sensor Array

. This article suggests a technique to make back of the hand (BOH) a track pad by attaching a distance ranging sensor on the side of a smart watch. It does not only estimate the position of a finger on BoH but also distinguishes the presence or absence of distance between the operating finger and BoH, which functions as an operating surface. Distance data and signal strength data are obtained by distance ranging sensor array, and the presence of distance is distinguished by machine learning. Our performance evaluation shows the accuracy of finger position estimation is 4.2mm(SD=1.2mm).


Introduction
The input method of smartwatches depends on a touch panel and buttons. The size of their interface gets smaller in proportion to the size of the devise; therefore, an operating finger becomes relatively large and hides information on screen, which leads to a problem of lowered operating efficiency.
For this problem, a technique to use the fringe area of the device as an input interface is being suggested, and there are types that use permanent magnet [1], interchange signal [2], and supersonic wave [3].
This article proposes a technique to use BoH like a trackpad with a small ranging sensor array. The two-dimensional position of a finger and distance between the finger and BoH is distinguished by obtaining distance data and signal strength data with a sensor attached to the side of a smart watch. Using BoH as a trackpad allows operation without hiding any information on display. Also, it solves the problem of having a difference between an operating object and an input area on screen, by estimating the two-dimensional position of the finger while distinguishing the presence or absence of distance between the finger and BoH.

Concept
This study estimates the position of a finger on BoH, and distinguishes the presence or absence of distance between the operating finger and BoH by attaching a small sensor on the side of a smart watch. Fig1 (Left) shows the overview of Novest, and 3 states (out of range, hovering, touching) are differentiated while distinguishing the presence or absence of a distance. Out of range is a state where the sensor does not detect the finger, hovering is a state in which there is a certain level of distance between the finger and BoH, and touching is a state in which there is a small amount of distance between the finger and BoH or they are in contact. To distinguish the distance between a finger and BoH, ranging sensors are arranged vertically on BoH. The gap between the operating object on display and the input area on BoH is solved by treating them like the relation between a mouse and a mouse pointer, setting up the touching state. Touching is equivalent to clicking on a mouse. This article only refers to the case of operating with one finger (index finger on right hand) Fig.1:Left).

Implementation
The implemented ranging sensor array is shown in Fig.1 (Rgiht). The size of the sensor board is 15mm in length, 50mm in width and 8mm in thickness. The width is determined in reference to the side of SONY Smart Watch 3 [4] (10mm height, 51mm width). As ranging sensor, 18 (2 x 9) of Time of Flight system STMicroelectronic VL6180Xs are arranged. This arrangement is made during the preliminary experiment to make sure that a finger 20mm away from the sensor is in the view area of at least one sensor. The minimal vertical intervals implementable are set, and the intervals in the luminescent part are approximately 8mm. This sensor board is connected to NXP Semiconductors prototyping board mbed LPC1768 with a flat cable, controlled by I2C communication.
Additionally, the noise of distance data and signal strength data obtained by sensor array is suppressed using a low-path filter.

Finger Position Estimation
Coordinate system is set as Fig.1 (Left). Strength data of 9 signals of a top column on the ranging sensor is used for Y coordinate derivation. Since this time one finger is assumed to operate, strength data of 9 signals is processed with spline interpolation, determines the position with the strongest signal as Y coordinate.
After that, the sensor distance data closest to the derived Y coordinate is used as X coordinate.

Finger State Classification
Support Vector Machine is used to classify the distance between the operating finger and BoH. 44 features are used, and they consist of the signal strength ratio of lower column to upper column (9) and its average value, total value, standard deviation, maximum value, minimum value, range, signal strength difference with the adjacent sensor (16), signal strength difference between the lower column and upper column (9), its average value, standard deviation, total value, and the ratio of total signal strength in lower column to total signal strength in upper column. These are scaled from -1 to 1 for use.

Evaluation
To evaluate the performance of this technique, experiments under 4 situations were carried out, with a finger and a round stick grasped by a robot arm as a direction method, and a flat surface and BoH as a directed surface. Subjects were 9 university and graduate school students (1 female, 8 males, 23.8 years old (SD=3.7 years)). The purpose is to confirm the finger and BoH effect on this technique by comparing the position estimate accuracy and the distance distinguishing accuracy under 4 experimental situations.
In this evaluation experiment, pointing task which is to point a marker on BoH or a surface with one's index finger on right hand was implemented, and 20 markers were set up for a random indication.
The task was implemented under the environments of 1) finger and BoH, 2) finger and flat surface, 3) round stick and BoH. The above 3 environments were of evaluation in touching, and after they were completed, same tasks were implemented with hovering in the environment of finger and BoH. With hovering, it was instructed to keep 10 ~ 15mm distance between a finger and BoH.
As a result, the position estimation accuracy of touching was 4.2mm(SD=1.2mm), and the estimate accuracy of hovering was 5.5mm(SD=2.8mm). Additionally, the distance distinguishing accuracy was 91.8%(SD=8.3%) with per-user classifier.

Conclusion
We proposed Novest which is a technique to make BoH a trackpad by setting up a small ranging sensor array on the side of a smart watch. Novest did not only distinguish the two-dimensional position of a finger, but also the presence or absence of a distance from BoH. Then, accuracies of the position estimate and distinguishing distance between touching and hovering were confirmed by implementing pointing tasks under 4 kinds of environments for experiments. As for the position estimate accuracy, a highly accurate estimate of 4.2mm(SD=1.2mm) was possible with touching. Finally, this technique can be expected to be applied to small devices other than a smart watch such as Smartglasses, because it distinguishes the two-dimensional position of a finger and the presence or absence of distance without attaching a sensor directly on skin.