[English / Japanese]

Ubi-Finger: Gesture Input Device for Mobile and Ubiquitous Environment


We propose a novel interface in mobile environment called "Ubi-Finger" that realizes sensuous operations for information appliances by gestures of fingers. Since gestures are usual methods of non-verbal communications and enables sensuous operations for users, many researches on them carried out especially in the field of Virtual Reality. However most of those existing gesture-input systems are either very expensive or large, and have not been used in mobile environment. In contrast, Ubi-Finger is a gesture-input device, which is simple, compact, and optimized for mobile use. We develop prototype systems, and evaluate how effective our approach works.

Using our systems, a user can detect a target device by pointing with her/his index finger, and then control it flexibly by performing natural gestures of fingers(Fig. 1).

By pointing a light and making a gesture like "push a switch". .
The light will turn on!
Figure. 1 An example to control Home Appliances (Demo videos are here

Device Architecture and System Architecture

As shown in Fig. 2, Ubi-Finger consists of three sensors (a bending sensor, an acceleration sensor, and a touch sensor) to detect gestures of fingers, an infrared transmitter to select a target device in real world and a microcomputer to control these sensors and communicate with a host computer. Each sensor generates the information of motions as follows: (1) a bending degree of an index finger, (2) tilt angles of a wrist, (3) operations of touch sensors by a thumb. We use (1) and (2) for recognition of gestures, and use (3) for the trigger mechanism to start and stop gesture recognition.

As shown in Fig. 3, the system consists of four main factors: Ubi-Finger device, Ubi-Appliance (an information appliance with an infrared receiver, LEDs, and a network connection), Ubi-Host (a host computer of Ubi-Finger device), and Ubi-Server (a server to control Ubi-Appliance).

Figure 2. Basic Concept of Ubi-Finger Figure 3. System Architecture

We explain the prototype of Ubi-Finger hardware (Fig. 4). The size of the prototype device is as compact as a fingertip. The device architectures of the prototype are as follows. We attached a bending sensor (BendMini by Infusion Systems ) on an index finger and two compact switches on the side of an index finger. In addition, we have set a 2-axis acceleration sensor (ADXL202E by Analog Devices), an infrared transmitter, and two LEDs on the portion of an index finger. We used a microcomputer (Tiny-H8 by Akizuki Denshi Tsusho) to convert the information generated by the sensors to numerical values, and transmit them to a laptop computer (OS: Windows 2000 Professional). Then, the Ubi-Host software on the laptop computer recognizes gestures by analyzing the input value in real time, and communicates with Ubi-Server via wireless network.
Figure 4. Prototype

Demo videos

-Controlling a light and a TV with Ubi-Finger.

ubi-finger.mp4 (MP4,320 x 240, 1min,3 M)

#You can play these videos with QuickTimePlayer6.0 or later.


  • Tsukada, K. and Yasumura, M.: Ubi-Finger: a Simple Gesture Input Device for Mobile and Ubiquitous Environment, journal of Asian Information, Science and Life(AISL),Vol.2, No.2, Nova Science Inc., pp. 111-120 (2004). (Selected Paper) [PDF]
  • Tsukada, K. and Yasumura, M.: Ubi-Finger: Gesture Input Device for Mobile Use,
    Proceedings of APCHI 2002, Vol. 1, pp.388-400 (2002). [PDF]
  • Tsukada, K. and Yasumura, M.: Ubi-Finger: Gesture Input Device for Mobile Use,
    Companion Proceedings of Ubicomp'2001, Technical Report: GIT-GVU-TR-01-7 (2001).