Investigation on the research of gesture recognition based on wireless signal

Absrtact: Since 21st century, human-computer interaction technology has been developing continuously, and gesture recognition is one of the representatives. In 2013, researchers at the University of Washington put forward a method of gesture recognition using wireless signals, which poses a challenge to traditional gesture recognition techniques, but the technology is not very mature and is not commercially available for the time being. In this paper, gesture recognition is introduced in detail, and the research status of gesture recognition based on radio frequency is described.
Keywords: wireless signal, gesture recognition, human-computer interaction
I. BACKGROUND
Since 21st century, with the continuous progress of world science and technology and the rapid development and popularization of computer, man-machine interaction technology (Human-computer Interaction, or HCI) has become more and more important in the sustainable development of the world. Human-Computer interaction refers to the process of exchanging information by using a specified interaction (such as a mouse, gesture, sound, etc.) to accomplish a specified task [1]. The main purpose of studying HCI technology is to realize the communication and communication between human and computer, make the computer understand human's intention, help people improve their quality of life and build a more intelligent society.
In recent years, with the advent of virtual reality, augmented reality, wearable computing and other technologies, HCI technology (computer-centric, traditional, mouse-and keyboard-based, single interactive technology) is increasingly unable to meet people's needs. Therefore, the multi-channel natural user interface came into being, as the name implies in the original HCI, adding gestures, lip language, voice, expression, eyes, ideas and other human natural information for multi-channel interaction. Further, we can not only interact with the computer, but also with the mobile phone, smart TV and other smart devices to interact with, such as the most well-known Microsoft Company production of somatosensory game control system Kinect[2], not only greatly enrich people's lives, but also expand the application of HCI technology field.
Gesture is an intuitive, image, easy to learn, informative and has a strong visual impact of human body language, usually use the action sent by the body to convey relevant information, can be called the People's daily life in the second language [3]. Gesture language is clear, natural and friendly, people, especially the relevant researchers more and more extensive attention. Because we can use the hand to make a variety of movements, and each action represents the meaning is also rich and colorful, so, research gesture recognition for the development of HCI technology has a very important role, at the same time to improve the life of human society has great significance: can be used for smart home home to gesture control, weaken the remote control, Mobile terminal dependence, reduce the additional cost, can be used for sign language recognition, to help the deaf and dumb people to improve their standard of living; can be used in automotive networking, such as the use of different gestures to awaken the different functions of the car system, such as open entertainment system, call up navigation system, etc., do not have to press the switch button to facilitate , reduce risk, improve its efficiency; can be used for remote control of the robot, such as fire scene, flood fighting, mining operations, chemical test site and other dangerous inconvenience of direct control of the special occasions, we through the gesture of remote robot operation, can be used to build home entertainment platform, enrich family life, such as the application of the body sense game , to increase the fun and playability of the game, can be used for teaching or meetings, such as the use of gestures to control the page of PPT, opening and closing of documents, presentation of reports, etc., easy to work [4].
At present, gesture recognition mainly includes sensor-based gesture recognition, vision-based gesture recognition, and radio frequency-based gesture recognition. This paper focuses on the radio frequency-based gesture recognition. The more mature radio frequency-based gesture recognition system is: wivi[5], wisee[6], allsee[7], witrack[8] and so on.
Second, the current situation of research
2.1 Overview
In the era of the rapid development of human-computer interaction, gesture recognition, as its extremely important research field, has aroused great attention from all over the world. Depending on the channel in which gesture actions are collected, we can divide it into three main directions (Figure 1): Gesture recognition based on sensors (sensor-based), gesture recognition based on vision (vision-based), and gesture recognition based on radio frequency (rf-based) [4].

Figure 1 Key techniques for gesture recognition

With the increase in wireless technology (Wireless technology) and the expansion of the coverage area of wireless networks, the IEEE 802.11a/g/n protocol-based wireless routers are slowly gaining popularity in our daily lives. Wireless signals (1-5), such as China's mobile CMCC, exist almost every moment around us. WiFi signal has the advantages of low overhead and easy deployment. On Sigcomm ' 13, Fadel Adib and Dina Katabi utilize MIMO Interface nulling and inverse synthetic aperture radar (inverse syntheticaperture Radar, hereinafter referred to as ISAR) Technology to eliminate the reflection of a stationary object, capturing the reflective signal of a moving object, thus identifying the trajectory of the moving object [5]. Subsequently, the University of Washington, Qifan Pu and other people on the USRP-N210 experimental platform using the radio frequency Doppler effect (the target near or away from the wireless hotspot has caused the wireless signal changes), proposed the Wisee, realized the entire family's nine gesture recognition, such as push, pull, kick, flash, etc. Average recognition rate up to 94%[6]. [9] In 2014, Göttingen University's Stephan Sigg and others, from a simple and convenient way of thinking, extracted the strength of the receiving signal strength indicator (Received Signal strength Indicator,rssi) for simple motion recognition. MIT researchers developed the WITRACK using FM continuous wave (Frequency modulated continuous wave, or FMCW) technology to calculate the time from the transmitter to be reflected back to the receiving end by the target object, TOF) to track the target [8]. Their recent research results Wiz can detect multiple user movements and identify three-dimensional gestures pointing to [10]. Both WiTrack and Wiz can be implanted into consumer electronics to determine when a user enters the door, automatically switch lights, identify gesture points, and automatically turn on the air conditioning system. and Bryce Kellogg and others from the power point of view, the production of a special low-cost hardware allsee, from the TV signal and RFID signals to extract information about the gesture, so as to identify eight gestures [7]. Pedro and others implemented fine-grained gesture recognition on the warp (Wireless open-access) platform running the IEEE 802.11 a/g/n protocol [11].
2.2 WiTrack
Witrac is a three-dimensional motion tracking and detection device. Its working process is to transmit the wireless signal and accept the wireless signal from the body back, through the corresponding algorithm, the body parts of the distance information, complete three-dimensional tracking. It does not require the user to carry any wireless transceiver device, and it can be used in a wall or other WiFi signal interference environment. The power consumption of the WiTrack device is very low, it sends a wireless signal 100 times times smaller than the normal WiFi signal, 1000 times times smaller than the phone signal.
WiTrack technology uses the human body to reflect the radio signal to achieve the human body positioning and motion tracking. The test equipment is a T-type bracket, the middle point is placed a receiver, three vertices placed three transmitters, so that the device can be three-dimensional monitoring of human movement.
The advantage of the witrack is that the centimeter-level tracking can be carried out, it can be located in the center of the human body on an x, Y axis 10-13 cm, Z axis 21 cm in three-dimensional space, can also be measured by the human hand of the direction of the coarse tracking, this function has 96.9% accuracy. WiTrack technology can be incorporated into electronic consumer products for a range of applications:
Apply to Games: Users can no longer sit in front of a TV computer to play games when they are equipped with witrack gaming equipment. Users can move freely around the home, and the device can automatically track the user's movements, such as the user can hide behind the sofa or the wall to control the characters in the game to avoid enemies.
Use of care for the elderly: it is well known that falls are the leading cause of fatal or non-fatal injuries to older people over 65 years of age. Two current fall detection methods, a wearable sensor that requires the elderly to wear, make life difficult for the elderly, a need to use a webcam to detect the fall of the elderly, which violates the privacy of the elderly. WiTrack can detect the fall of the elderly by changing the intensity of the wireless signal reflected back from the human body, the accuracy of which can reach 96%.
Applied to smart home: WiTrack can control the user's finger direction of the furniture switch, such as the switch of the lamp, by detecting the direction of the user's finger. Users only need to turn on the light, with a finger to open the light can be.
2.3 Wisee
The researchers confirmed that by using a modified WiFi router in the living room and several wireless devices, users could manipulate their electronics and home appliances in any room in the house with a simple gesture. Khiyam, assistant professor of computer science and Engineering, University of Washington The Galakota called it a new way to reuse existing wireless signals. Gesture recognition with wireless signals eliminates the need to deploy more sensors.
The Washington University research team named the Technology "Wisee", which intelligently detects the Doppler shifts in the movement of people under the WiFi signal. The average recognition rate for the current 9 gestures is 94%, and the standard deviation is 4.6%. The study was submitted to the 19th session of the International Conference on mobile Computing and networking. "Wisee" is conceptually similar to Microsoft Xbox Kinect, but the technology is simpler and cheaper, and users do not have to control their devices in the same room, because WiFi signals can travel through walls without being constrained by sight or sound barriers.
The research team converted a standard WiFi router into a "smart" receiving device that basically listens to wireless signals from all electronic devices, such as smartphones, laptops, and tablets in the room.
When a person moves, it causes a slight change in the frequency of the wireless signal. Movement of the hand or foot will cause the receiving device to detect a change in Doppler frequency shift. These frequency changes are very small, usually only a few hertz, compared to the bandwidth of the WiFi signal at 20 gigabit and the operating frequency of 5 GHz. Researchers have developed an algorithm to detect these subtle changes.
The technology is now able to identify 9 different gestures, such as push, pull, stab, and cast. Wisee in a two-bedroom test proves that it can also put the system into another room where people are located. In addition to its range of advantages, a multi-antenna and MIMO-enabled WiFi base station can support multi-person identification, allowing up to 5 people to operate gestures simultaneously in the same home without confusion. When a person moves, the frequency of the wireless signal is changed slightly. Moving a hand or a foot causes the receiver to detect a pattern change called a doppler shift. The researchers developed a set of algorithms to detect frequency shifts. It can identify nine different body positions, from push, pull, punch to body twisting. In a total of 900 gestures performed, the wisee has a 94% accuracy of identification.
If users want to use Wisee, they must perform a specific sequence of repeated gestures to gain access to the receiver. This cipher concept keeps the system safe and prevents neighbors or hackers from controlling the devices in the user's home. When the wireless receiver and the user are locked, the user can interact with the electronic devices in the home in a normal gesture. The receiver is programmed to understand the specific gestures that correspond to each electronic device.
However, the operation of this technology is simple, whether the user needs to memorize gestures to operate these are the issues to be considered. If the technology can be mature, it will play a very big role in smart home, and this is still in the WiFi itself has become popular in the case. To really be practical, we need to discuss how to make this technology simple and convenient.
2.3 WiG
Wig is a WiFi-based gesture recognition experiment system by the author of He Wenfeng, a student of Shenzhen University, in his master's degree thesis. It is a gesture recognition system based on the existing WiFi device and commercial wireless network card, using the channel state information of the wireless signal physical layer, device-free (no need to carry additional equipment), including hardware modules and software modules.
In the hardware module, the transmitter is a common commercial router called the AP (Access point). The receiving end is a desktop computer with a commercial wireless network card (such as an Intel 5300 wireless card), called a DP (Detect point). The AP and DP are connected over a wireless card, and the AP is continuously sending wireless signals, while the experimenter gestures between the AP and DP.
In the software module, it contains six parts: CSI Data Acquisition, CSI preprocessing, CSI Data denoising, CSI Data anomaly detection, feature extraction and classifier classification.
The wig system is divided into three processes: CSI data acquisition, CSI preprocessing and de-noising, CSI Data anomaly detection, feature extraction and classification.
WISEE[6], al1see[9], witrack[8, etc. are based on software radio platform USRP or dedicated hardware. Although Pedro and others do not use the software radio platform, but the warp platform, but warp and USRP, the use of expensive oscillators, the cost is high. A set of warp experimental platform will need a hundred thousand of yuan, not conducive to large-scale deployment.
The wig system uses the existing wireless facilities and commercial wireless network cards, so the wig system provides a feasible scheme for more common gesture recognition based on wireless signal, but its actual effect needs further research.
Iii. Summary and Outlook
3.1 Summary
Gesture recognition is a very important part in the research of HCI technology. Wireless radio frequency-based systems, such as Wivi, Wisee, Al1see, etc. are based on the software radio platform USRP or dedicated hardware implementation, high cost, not conducive to popularization. Wig system uses the existing wireless facilities and commercial wireless network card, but its actual effect needs further research.
Radio frequency-based gesture recognition has its own somewhat, such as: low overhead, easy to deploy on a large scale, can be used in the dark, can wear walls and so on, but it also has its own limitations, such as the precise definition of gestures and He Wenfeng's paper mentioned in the "cannot be identified" and so on.
3.2 Outlook
According to He Wenfeng's paper mentioned that radio frequency technology "can not be identified", the author believes that posture recognition can be combined with wifi positioning [12], posture recognition, gait analysis and other technologies: first posture recognition and WiFi positioning, the identified posture and location information to the gait analysis system, For a certain period of time can achieve the purpose of identification, and then continue to use WiFi location for character tracking. This approach is highly accurate for wifi positioning techniques and posture recognition and may be difficult to achieve at this point, just to suggest possible guesses.
With the depth of gesture recognition, the accuracy of the algorithm based on the radio frequency gesture recognition will be further improved, so that the requirement of the related equipment is reduced, even the use of universal WiFi facilities and commercial wireless network card can reach the standard of use, and finally realize the popularization of this technology.
Reference documents
[1] Wang Yan. Research on interactive information design [D]. Hunan: South Central University, 2011.
[2] Microsoft.kinect[eb/ol]. [2014-12-12].http://www.microsoft.com/en-us/kinectfowindows.
[3] Baidu encyclopedia. Gesture Deb/old. X2014-12-25].http://baike.baidu.com/view/179746.htm
[4] He Wenfeng. Research on gesture recognition based on WiFi [D]. Shenzhen: Shenzhen University. 2015
[5] Adib F, Katabi D. Seeing Through Walls Using WiFi [C]. Proceedings of the ACM Sigcomm conference. Hong KONG:ACM, 2013.
[6] Pu Q, Gupta S, Gollakota S, et al whole-home gesture recognition using wireless signals[c]proceedings of the 19th Ann UAL International Conference on Mobile Computing & Networking. ACM, 2013:27-38.
[7] Kellogg B, Talla V, Gollakota S. Bringing gesture recognition to all Devices[c] Proceedings of the 11th USENIX Symposi Um on networked Systems Design & implementation. Seattle, WA:ACM, 2014, 14
[8] Adib F, Kabelac Z, Katabi D, et al. 3d Tracking via body Radio Reflections[c] Proceedings of the 11th USENIX Symposium On networked Systems Design &implementation. Seattle, wa:acm,2014, 14
[9] Sigg S, Blanke U, Troster G. The telepathic phone:frictionless activity recognition from wifi-rssi[c]proceedings of the IEEE International Conference On pervasive Computing and Communications. Budapest:ieee, 2014:148-155.
[Adib] F, Kabelac Z, Katabi D. Multi-person Motion Tracking via RF Body Reflections [J]. 2014.
[One] Melgarejo p, Zhang X, Ramanathan p, et al leveraging directional antenna capabilities for fine-grained gesture RECOG Nition}c}//proceedings of the ACM international Joint Conference on pervasive and Ubiquitous Computing. Seattle, WA:ACM, 2014:541-551.
[12] Liu Chunyan. Research on pedestrian indoor positioning algorithm based on wifi and inertial technology [D]. Beijing: China University of Geosciences. 2015

Investigation on the research of gesture recognition based on wireless signal