Discover Hawking's computer, how to solve communication problems?

Hawking first met with Intel co-founder Moore at a meeting in 1997. Moore noticed that the computer Hawking used to communicate with people was using AMD's processors. Moore asked Hawking whether he would like to have a "real computer" with a built-in Intel microprocessor. Since then, Intel has been providing Hawking with tailor-made PCs and related support services and replacing them every two years.

Hawking lost his ability to speak in 1985. At the time, Hawking got pneumonia on his way to CERN, the European Institute of Particle Physics, in Geneva. In the hospital, the doctor equipped his ventilator, his situation is quite serious. The doctor asked Hawking's wife, Jane, if they could close the life support system. Jane flatly refused. Hawking was then transferred to the Addenbrooke Hospital in Cambridge, where doctors controlled the infection. To help him breathe, the doctor underwent a tracheotomy, a technique for punching a hole in the neck and inserting a catheter into the trachea. The operation led to Hawking's inability to speak.

Hawking then communicates through the spelling version, laboriously selecting the letters, and then spelling them into complete words, through the rotation of the eyeball. Martin Kim (Martin King), a physicist who has found a new communication system for Hawking, contacted a company called Words Plus in California and developed a computer program equalizer that allows users to select words and commands by clicking their fingers. Kim asks Words Plus's CEO, Walter Woltosz, whether the product will help a professor of English physics who suffers from amyotrophic lateral sclerosis (ALS). The purpose of the equalizer is to help his mother-in-law, who suffers from the same disease, and thus loses the ability to speak and write. When he learned that the product would be used on Hawking, he offered to donate.

At first, equalizer ran on the Apple II computer, working with a speech synthesizer developed by a company called Speech Plus. Subsequently, David Messen (David Mason) improved the system. Mason is an engineer and husband of one of the nurses who looks after hawking. The improved system can be installed on the armrest of the wheelchair side. Through the system, Hawking's communication speed reached 15 words per minute.

However, the nerves that allow Hawking to move his thumb are deteriorating. Hawking's hand was too weak to use a tap in 2008. His graduate assistant designed a device called a "facial switch" that was mounted on glasses to identify Hawking's facial expressions through an infrared beam. Since then, Hawking has written e-mails, surfed the internet, wrote books and talked through the only controllable muscles. However, his ability to communicate continued to deteriorate. By the time he was 2011, he could only communicate 1 to 2 words a minute, so he wrote a letter to Moore saying, "I'm talking more and more slowly, is there any way Intel can help me?" ”

Moore Justin Ratner Justin Rattner, the Intel CTO, to follow up on the matter. Ratner, a team of human-computer interaction experts from the English laboratory, attended Hawking's 70 birthday in Cambridge. At the meeting, Ratner said it would use cutting-edge technology to help hawking improve communication. "We want the team to be able to get a breakthrough and find a technology to help him get back to the level of communication a few years ago," he said. ”

Hawking was too weak to attend his birthday party. So his meeting with Intel's expert team was a few weeks later, at his office in the Department of Applied Mathematics and Theoretical Physics at Cambridge University. A team of 5 experts included Host Hauseck, director of the Experience Technology Lab (Horst Haussecker), director of the prospective Computing lab Lama Rackman (Lama Nachman), and interactive designer Peter Dengman (Pete Denman). "Hawking has always inspired me," said Dunman, who also sits in a wheelchair. "My mother gave me a brief history of time after I broke my neck and caused paraplegia." She told me that people in wheelchairs could still make a remarkable achievement. Looking back, I think that book is a prophecy for me. ”

The team members then communicated with Hawking on the issues facing him. In 20 minutes, Hawking barely finished 30 words, which made team members feel that the problem was much more serious than originally expected.

Hawking's computer interface is a program called the EZ Keys, which is upgraded from the original program and still comes from the words plus hand. The program provides a soft keyboard as well as a basic word prediction algorithm. The cursor sweeps each letter by line, simply by moving the neck to stop the cursor and select the corresponding letter. The EZ keys also allow Hawking to control the mouse in Windows and to manipulate other programs. Hawking uses Firefox to browse the Web and write in Notepad, and he also has a webcam to use with Skype.

The Intel team intends to radically improve this old system, which requires new hardware. "Justin believes facial expression recognition, eye tracking and brain-computer interface technology can be a test," Luckman said. Most of these attempts ended in failure. Eye tracking cannot be locked because of the sagging face. Hawking has also tried to read brain waves through an EEG (EEG) and then send commands to the computer. For some reason, the technology does not have enough brain-wave signals. "We let hawking choose by displaying flashing letters on the screen, and then identifying and registering brain activity," Wood said. "The technology is good for me, but he can't work at Hawking and can't get a strong enough signal-to-noise ratio," he said. ”

"As we communicate with Hawking more and more, we also feel the urgency of Hawking's eagerness to improve the efficiency of communication," Luckman said. After months of research at the Dunman Laboratory, he sent Hawking a 10-minute video. The video depicts the prototype of a new interactive system, and the team waits for Hawking's feedback. The system does not change in nature, but changes are not small, including the "Back" button, the word prediction algorithm, and the next word navigation that can be removed from the character or returned to the previous step.

The prototype solves the biggest problem-hawking can't hit the key, causing him to pick the wrong letter often and then having to wait for the next round of the cursor scan. This is a time-consuming step that will make Hawking frustrated. To address this problem, team members used a method similar to smartphone input. When the iphone was first marketed, people were very unaccustomed to its word prediction, but after a while, users felt very handy. The key to the problem is that the system takes a little time to learn, and once the system completes its own work, the user can continue to use it with ease.

In June, Hawking visited the English laboratory and was very satisfied with the improved system. "There has been a significant increase in relative to previous versions," Hawking said. "I really like it." ”

They then integrated the interface into Hawking's computer. In the same September, Hawking's feedback pointed out that he was not comfortable with the new system, too complicated, and that he was puzzled by the added functionality. "Hawking is one of the smartest people in the world, but we can't forget that he's not in touch with much modern technology," Dunman said. "He never used the iphone. ”

Dunman and other members realised that they had to do something different.

At the end of 2012, the Intel team set up a system to record how hawking interacts with the computer. They recorded a dozens of-hour video that included various situations: typing, typing when tired, using the mouse, trying to open the window and adjusting the appropriate size. "I watched the video over and over again," Dunman said. "Sometimes I go into the video 4 times times faster, and then I find something new." ”

The team members updated Hawking's computer in September 2013 with the help of Hawking's graduate assistant, Jonathan Wood. This time, they are full of confidence. In the next one months, however, things soon became clear and Hawking was still unable to adapt. One of Hawking's aides said the system was a torment for hawking.

The Intel team has spent too much time trying to solve Hawking's problems. Hawking is currently using a vocabulary predictor from SwiftKey, a london-based start-up company. The tool is able to give a full word prediction after entering a letter, and the previous system requires Hawking to select the word list at the bottom of the interface. The Intel team worked with SwiftKey to put a lot of Hawking's documentation into the system so that, in context-based predictions, Hawking did not have to type a letter every time. "The phrase ' the black hole ' doesn't need any input," Luckman said. Choose ' the ' to predict ' black ' and choose ' black ' to automatically predict ' hole '. ”

Hawking's new version of the user interface includes context menus and offers a variety of shortcuts to text, search, or email. The new lecture manager allows him to control the speed of his speech. There is also a mute button that Hawking can use to turn off the speech synthesizer. "Because he has a facial operating system that can produce random inputs in the course of eating or traveling," Wood said.

Wood's office is next door to Hawking's, which is more like a workshop, stacked with a variety of circuit boards and experimental prototypes. The desktop has a camera installed, part of Intel's ongoing project to try to identify Hawking's face more complex. "He can move his chin up or down so he can even manipulate the mouse while moving the wheelchair." These are cool ideas, but they take time to improve. "said Wood.

Another pilot project was made by Hawking's wheelchair manufacturer earlier last year. The project allows Hawking to operate the wheelchair alone by placing a joystick near the chin.

Wood showed the only replica of Hawking's speech synthesizer, which it called Calltext 5010, which was given to him in 1988 when Hawking visited its maker speech plus. The product is built into a processor that converts text to speech, which was widely used in automatic answering equipment in the 1980s.

Hawking was obsessed with the sound he liked. When speech plus gave Hawking a new speech synthesizer in 1988, he offered a different voice, Hawking asked them to replace the original sound. His voice was created in the 1980s by MIT engineer Dennis Crate (Dennis klatt). Clutter is a pioneer in translating text into speech algorithms, and the invention of DECtalk is the first device in the world to convert text to speech. He initially used 3 of voices, from himself, his wife and his daughter. The voice of the female is called "Beautiful Betty", the child's voice is "Bridget", and the male voice comes from himself-"perfect Paul". "Perfect Paul" is the sound that Hawking uses.

(Responsible editor: Mengyishan)