Timeout detection and recovery

Timeout detection and recovery of GPUs through wddmupdated: a6l 27,200 9on this page

	Introduction
	Timeout detection and recovery
	Windows Vista SP1 update
	Error messaging
	Registry Keys
	Next steps
	Resources

Introduction

One of the most common stability problems in graphics is when the system appears completely "Frozen" or "Hung" while processing an end-user command or operation. users generally wait a few seconds and then reboot the system by pressing the power button. usually the graphics processing unit (GPU) is "busy" processing intensive graphical operations, typically during gameplay. this results in nothing being updated on the screen, thus appearing to the user that the system is frozen.

This paper briefly describes the timeout detection and rediscovery (TDR) process in Windows Vista. It also events ents the Registry controls so developers can easily debug problems.

What's new for Windows Vista SP1
Changes for Windows Vista SP1 to improve user experience in cases of frequent and rapidly occurring GPU hangs. New registry keys to support these changes.

Top of page

Timeout detection and recovery

Windows Vista attempts to detect these problematic hang situations and recover a responsive desktop dynamically. in this process, the Windows display driver model (wddm) driver is reinitialized and the GPU is reset. no reboot is necessary, which greatly enhances the user experience. the only visible artifact from the Hang detection to the reible is a screen flicker, which results from resetting some portions of the graphics stack, causing a screen redraw. some older Microsoft DirectX applications may render to a black screen at the end of this recovery. the end user wowould have to restart these applications.

The following is a brief overview of the TDR process:

1.	Timeout Detection:The video Scheduler Component of the Windows Vista graphics stack detects that the GPU is taking more than the permitted quantum time to execute the particle task and tries to preempt this particle task. the preempt operation has a "wait" timeout-the actual "TDR timeout. "This step is thus the" timeout detection "phase of the process. the default timeout period in Windows Vista is 2 seconds. if the GPU cannot complete or preempt the current task within the TDR timeout, then the GPU is diagnosed as hung.
2.	Preparation for recovery:The operating system informs the wddm driver that a timeout has been detected and it must reset the GPU. the driver is told to stop Accessing memory and shocould not access hardware after this time. the operating system and the wddm driver collect hardware and other state information that cocould be useful for post-mortem diagnosis.
3.	Desktop recovery:The operating system resets the appropriate state of the graphics stack. the video Memory Manager Component of the graphics stack purges all allocations from video memory. the wddm driver resets the GPU hardware state. the graphics stack takes the final actions and restores the desktop to the responsive state. as mentioned earlier, some older DirectX applications may now render just black, and the user may be required to restart these applications. well-written DirectX 9ex and DirectX 10 applications that handle "device remove" continue to work correctly. the application must release and then recreate its Microsoft direct3d device and all of its objects. directX application programmers can find more information in the Windows SDK.

Top of page

Windows Vista SP1 update

Minor changes were made in Windows Vista SP1 to improve the user experience in cases of frequent and rapidly occurring GPU hangs. repetitive GPU hangs indicate that the graphics hardware has not recovered successfully. in these instances, the system must be shut down and restarted to fully reset the graphics hardware. if the operating system detects that six or more GPU hangs and subsequent recoveries occur within 1 minute, then the following GPU hang is treated as a system bug check.

Top of page

Error messaging

Throughout the process of GPU hang detection and recovery, the desktop is unresponsive and thus unavailable to the user. in the final stages of recovery, a brief screen flash occurs that is similar to the one when the screen resolution is changed. after the desktop has been successfully recovered, the following informational message appears to the user.

The message is also logged in the Windows Vista event viewer. diagnosis information is collected in the form of a debug report that is returned to Microsoft through the online crash analysis (OCA) mechanic if the user opts in to provide feedback.

Top of page

Registry Keys

The following registry keys are supported entedTesting purposes only.These registry keys shocould not be manipulated by any applications outside targeted testing or debugging.

The TDR-related registry keys are located under HKLM \ System \ CurrentControlSet \ Control \ graphicsdrivers.

•	Tdrlevel: REG_DWORD. The initial level of recovery. The possible values are: • Tdrleveloff (0).-detection disabled. • Tdrlevelbugcheck (1)-bug check on detected timeout, for example, no recovery. • Tdrlevelrecovervga (2)-recover to VGA (not implemented ). • Tdrlevelrecover (3)-recover on timeout. This is the default value.
•	Tdrdelay: REG_DWORD. The number of seconds that the GPU is allowed to delay the preempt request from the schedld. This is too tively the timeout threshold. The default value is 2.
•	Tdrddidelay: REG_DWORD. The number of seconds that the operating system allows threads to leave the driver. after a specified time, the operating system bug checks the system with the code video_tdr_failure (0x116 ). the default value is 5.
•	Tdrtestmode: REG_DWORD: Internal test usage.
•	Tdrdebugmode: REG_DWORD: The debugging-related Behavior of the TDR process. • Tdr_debug_mode_off (0) breaks to kernel debugger before the recovery to allow investigation of the timeout. • Tdr_debug_mode_ignore_timeout (1) ignores any timeout. • Tdr_debug_mode_recover_no_prompt (2) recovers without break into the debugger. This is the default value. • Tdr_debug_mode_recover_unconditional (3) recovers even if some recovery conditions are not met (for example, recovers on consecutive Timeouts ).
•	Tdrlimittime:REG_DWORD (Windows Vista SP1 and later versions only): the default time within which a "tdrlimitcount" Number of TDRS are allowed without crashing the system.
•	Tdrlimitcount:REG_DWORD (Windows Vista SP1 and later versions only): the default number of TDRS (0x117) that are allowed in "tdrlimittime" without crashing the system.

Top of page

Next steps

Graphics hardware vendors:

•	Ensure that graphics operations (that is, DMA buffer completion) take no more than 2 seconds in end-user scenarios such as productivity and gameplay.

Graphics Software Vendors:

•	Ensure that the DirectX graphics application does not run at a low frames per second (FPS) rate. as the FPS decreases, the likelihood of the GPU getting reset increases. if the application is running at 10 FPS or lower and a complex graphics operation is about to start, then a flush can be inserted.
•	For running benchmark tests on low-end GPUs, use the aforementioned registry keys that control the TDR timeout. remember that they shocould not be used in production systems because it wocould affect overall system stability and robustness. use these keys only as a final solution.

System manufacturers:

•	Work with the graphics hardware vendor to diagnose the TDR debug reports.
•	Remember that any system that uses the aforementioned TDR registry keys to change the default values is a Windows logo program violation.

Top of page

Resources

Queries:If you have questions that are not answered in this document, send e-mail to directx@microsoft.com.

Windows Driver Kit
DirectX Developer Center on msdn
Windows logo program Requirements
Windows SDK for Windows Vista

Top of page