Interpreting temporal distortions of VR (Time-warp) _VR

Explain Time warp

In the original English link. There are some personal auxiliary information added in the translation, marked with parentheses, "Note:" And in bold, for example (note: The following is a personal translation, limited in level, welcome to correct).
In the context of virtual reality, Time warp is a technique to reduce the delay between motion and image.
In an ideal situation, the rendering engine is sent to the display using a measured real-time head posture (orientation and position). However, in reality, rendering takes time, so the rendering engine uses a gesture that is a few milliseconds before the display device is sent. In these milliseconds, the head moves, But the image is going to be a bit more delayed than the actual situation.
Let's take a numerical example, suppose we need to render 90 frames per second (90FPS), in which case it takes 11 milliseconds to render each frame. We assume that the head tracking data is continuous, but the rendering takes 10 milliseconds. Given the rendering time, the later the rendering engine is rendered, the better. The frame that needs to be sent to the display device is 10 milliseconds ago. Therefore, the rendering engine uses the head tracking data 10 milliseconds ago. If the head rotates at 200 degrees per second, then 10 milliseconds is rotated 2 degrees (200*0.01=2). If the HMD horizontal field angle is 100 degrees, The horizontal view is arranged horizontally with 1000 pixels, and a 2-degree delay means that the image is delayed by 20 pixels from the actual image.
However, experiments have shown that even 2-degree changes are not easily detectable in distant images. Therefore, if there is a way to move 20 pixels on the screen (for example, 2 degrees in the example). Then the final image is quite accurate.
This is precisely the time warp (or "TW"): It is very fast (less than 1 milliseconds) to convert the image, which is based on the rotation of the head between the time and time warp of the rendering engine using the header information (rendering begins to the time between the execution of TW).
The processing time distortion is fairly simple: the rendering engine performs rendering, and then the rendering engine finishes, and the time warp is quickly applied to the final image.
But what if the rendering engine takes longer than the frame and frame time? In this case, a version of Time Warp, called Asynchronous Time Warp ("ATW"), is usually used. ATW uses the last valid frame plus the time warp to handle this situation. If the render engine does not finish in time (image rendering work), ATW uses the previous frame and distorts it with time. If the previous frame is used, the head is probably no longer in the position So you need a bigger shift. You can't expect the rendering engine to finish on time, ATW uses the data from a previous frame to process it, still better than the obvious dithering of the lost data. This is why ATW is called a "secure network" for rendering, which is useful when rendering is not done on time . ATW asynchronous comes from the ATW is a thread or process that is independent of the rendering engine and is running at a higher priority than the render engine to update the frame (insert frame) when the render engine does not finish rendering on time.

Here is a detailed description of the technical points: the time warp example can be compensated for by the head movement of the Left and right sides (yaw direction). In fact, all three rotational directions-yaw,pitch and roll-can be compensated, and in some hypothetical case the head position. For example, Osvr actually performs 6DOF distortion based on the assumption that the object is 2 meters away from the center of the projection. It handles rotations about the direction of implementation and approaches all other translations and rotations. Objects in the scene such as: hand, if the rendering engine does not render a frame in time, although the added time distortion, will still appear jitter. In order to make time distort the work better, the render frame needs to be slightly larger than the display resolution. Otherwise, you may end up seeing an empty pixel (for example, the image moves 20 pixels to the left), and the 20 pixel position on the left of the screen is an empty pixel. Actually, how much does the rendering really need, depending on the frame rate, And predicting the speed of head rotation. Larger frame images mean more pixels to be rendered and occupy more content, so time distortions are not completely "free". If the image in HMD is rendered to a single display (instead of two screens, each with one display), Time warping requires a different degree of distortion for each eye, because one eye refreshes the image earlier than the other (60FPS, one eye 1/60/2 earlier than the other eye, about 8 milliseconds). Some objects, such as the menus in the header space (for example, should be fixed relative to the head), require separate code to render and submit, as they are not applied to the correction of the head motion projection. Predictive tracking (estimating future gestures, based on previously read directions, locations, and angular velocities) can help provide input information to the rendering engine, but the actual measured data is often more accurate than predicting future gestures. Depending on the HMD display configuration, the left eye renders more later than the right eye (for example, if the screen is a vertical, render from top to bottom, the left eye corresponds to the top half of the screen. In this case, it is necessary to use a different time warp for each eye (these 2 eyes refresh time difference 1/fps/2 seconds).