What do you do when you want to drink water with a virtual reality helmet?

With the development of technology, human beings are entering a world of virtual and reality.

One form of integration is augmented Reality (AR), which is the superposition of virtual elements in a realistic view. For example, pilots have been using their head-worn display devices to get flight data for a long time, or some apps can now embed virtual characters into the real world view, or technology like Google Glass can overlay useful information such as navigation, personal data, etc.

However, the development of virtual reality technology may also lead to the emergence of another form of integration. It's just that a lot of people don't think about it right now--when you put on a virtual reality helmet and immerse yourself in a virtual world quite different from the reality, what if you want to pick up a glass of water on the table?

There are probably two possible solutions. One is to continue to immerse in the virtual reality side with a helmet to explore the physical reality of the cup, the result may be a cup of water splashed, or take off the helmet temporarily away from the virtual reality return to physical reality, but so your experience is interrupted. But Pulkit Budhiraja of the University of Illinois at Urbana-Champaign and his team have come up with a better solution-the flip stack.

The so-called reverse superposition, the physical reality superimposed on the experience of virtual reality. The goal is to find ways to continue to immerse in virtual reality while allowing users to interact with physical reality objects, equivalent to augmented virtual reality.

The research team uses tools such as Oculus Rift helmets and a pair of cameras that generate a physical world stereo map. The two cameras are placed in front of the Oculus Rift and serve as human eyes. Then they came up with four ways to overlay the physical reality of the image into the virtual world, allowing the user to get a cup of water while holding the immersion experience.

Left: Method 1, right: Method 2

The first is to overlay the user's hands and cups into the virtual world. The second adds the contours of the object around the first, such as tables and keyboards, so that people can move their hands with background references. The third is to open a window in the focus of the view to present the real world, while virtual reality. The last one is the form of painting in the lower right corner of the virtual world to show the physical world.

Left: Method 3, right: Method 4

The team found 10 volunteers to test different scenarios and methods. The first scenario is a movie that doesn't need to interact with each other. The second scenario is a first-person shooter game that requires a lot of interaction. The third is a racing game that requires advanced action.

In each case, the subjects were asked to take a glass of water while remaining immersed in the virtual environment. The team originally thought that subjects might prefer the first approach because it only showed that the hands and cups were the least disruptive to the virtual environment.

But the team was slightly surprised by the results. Most subjects preferred the second, which also showed the background profile of the hand movement (such as the convex horn of a table). The reason for this is that the extra contours provided by the background help to find the cup and put it back, and the presentation is not distracting. Others say it's hard to find cups without a background reference.

2 cameras in front of the helmet.

You might think it's a bit out of the question to consider this now. But the speed of technology development cannot be measured by linear thinking. Looking back at the 50-year development of virtual reality, you will find that the evolution of technology is accelerating, Facebook is developing virtual reality for its applications, Apple is recruiting virtual reality hardware engineers, and even people are already considering what to do with virtual reality. The realization of consumption should be within 10 years, perhaps more quickly than expected.

However, in the consideration of the technical problems of combining virtual with reality, the social and ethical problems caused by it should not be neglected. A few days ago, an adult man in Shanghai, after 19 hours of internet cafes, suddenly vomiting blood and coma and then death is a warning: technology has not fully realized immersive experience yet so, wait until the virtual reality of consumption and then what? Perhaps the human spirit can sustain the long-term immersion in the virtual reality, but the human body can not bear-and the following reality is never virtual.