Windows APC mechanism (I)

Asynchronous Process calling (APCs) is a basic part of the NT asynchronous processing architecture. It is helpful to understand how nt operates and how to execute several core system operations.

1) APCs allows user programs and system components to execute code in the context of a thread in the address space of a process.
2) the I/O manager uses APCs to perform asynchronous I/O operations initiated by a thread. For example, when a device driver calls iocompleterequest to notify the I/O manager that it has ended processing an asynchronous I/O Request, the I/O manager queues an APC to the request initiating thread. Then, at a lower IRQL level, the thread executes APC. APC to copy the I/O operation results and status information from the system space to a buffer of the thread's virtual memory space.
3) You can use APC to obtain or set the context and suspension of a thread.

 

The above is found on the Internet, and the following is the description on msdn:

AnAsynchronous procedure call(APC) is a function that executes asynchronously in the context of a participant thread. when an APC is queued to a thread, the system issues a software interrupt. the next time the thread is scheduled, it will run the APC function. an APC generated by the system is calledKernel-mode APC. An APC generated by an application is calledUser-mode APC. A thread must be in an alertable state to run a user-mode APC.

Each thread has its own APC queue. An application queues an APC to a thread by callingQueueuserapcFunction. The Calling thread specifies the address of an APC function in the callQueueuserapc. The queuing of an APC is a request for the thread to call the APC function.

 

Let's take a look at the important structures:

Kd> DT kthread
NT! Kthread
+ 0x000 header: _ dispatcher_header
+ 0x010 mutantlisthead: _ list_entry
+ 0x018 initialstack: ptr32 void
+ 0x01c stacklimit: ptr32 void
+ 0x020 kernelstack: ptr32 void
+ 0x024 threadlock: uint4b
+ 0x028 apcstate: _ kapc_state
+ 0x028 apcstatefill: [23] uchar
+ 0x03f apcqueueable: uchar
+ 0x040 nextprocessor: uchar
+ 0x041 deferredprocessor: uchar
+ 0x042 adjustreason: uchar
+ 0x043 adjustincrement: Char
+ 0x044 apcqueuelock: uint4b
+ 0x048 contextswitches: uint4b
+ 0x04c state: uchar
+ 0x04d npxstate: uchar
+ 0x04e waitirql: uchar
+ 0x04f waitmode: Char
+ 0x050 waitstatus: int4b
+ 0x054 waitblocklist: ptr32 _ kwait_block
+ 0x054 jsonobject: ptr32 _ kgate
+ 0x058 alertable: uchar
+ 0x059 waitnext: uchar
+ 0x05a waitreason: uchar
+ 0x05b priority: Char
+ 0x05c enablestackswap: uchar
+ 0x05d swapbusy: uchar
+ 0x05e alerted: [2] uchar
+ 0x060 waitlistentry: _ list_entry
+ 0x060 swaplistentry: _ single_list_entry
+ 0x068 queue: ptr32 _ kqueue
+ 0x06c waittime: uint4b
+ 0x070 kernelapcdisable: int2b
+ 0x072 specialapcdisable: int2b
+ 0x070 combinedapcdisable: uint4b
+ 0x074 Teb: ptr32 void
+ 0x078 Timer: _ ktimer
+ 0x078 timerfill: [40] uchar
+ 0x0a0 autoalignment: POS 0, 1 bit
+ 0x0a0 disableboost: POS 1, 1 bit
+ 0x0a0 reservedflags: POS 2, 30 bits
+ 0x0a0 threadflags: int4b
+ 0x0a8 waitblock: [4] _ kwait_block
+ 0x0a8 waitblockfill0: [23] uchar
+ 0x0bf systemaffinityactive: uchar
+ 0x0a8 waitblockfill1: [47] uchar
+ 0x0d7 previusmode: Char
+ 0x0a8 waitblockfill2: [71] uchar
+ 0x0ef resourceindex: uchar
+ 0x0a8 waitblockfill3: [95] uchar
+ 0x107 largestack: uchar
+ 0x108 queuelistentry: _ list_entry
+ 0x110 trapframe: ptr32 _ ktrap_frame
+ 0x114 callbackstack: ptr32 void
+ 0x118 servicetable: ptr32 void
+ 0x11c apcstateindex: uchar
+ 0x11d idealprocessor: uchar
+ 0x11e preempted: uchar
+ 0x11f processreadyqueue: uchar
+ 0x120 kernelstackresident: uchar
+ 0x121 basepriority: Char
+ 0x122 prioritydecrement: Char
+ 0x123 saturation: Char
+ 0x124 useraffinity: uint4b
+ 0x128 process: ptr32 _ kprocess
+ 0x12c affinity: uint4b
+ 0x130 apcstatepointer: [2] ptr32 _ kapc_state
+ 0x138 savedapcstate: _ kapc_state
+ 0x138 savedapcstatefill: [23] uchar
+ 0x14f freezecount: Char
+ 0x150 suspendcount: Char
+ 0x151 useridealprocessor: uchar
+ 0x152 calloutactive: uchar
+ 0x153 iopl: uchar
+ 0x154 win32thread: ptr32 void
+ 0x158 stackbase: ptr32 void
+ 0x15c suspendapc: _ kapc
+ 0x15c suspendapcfill0: [1] uchar
............

............

The red part above is the fields used by the APC mechanism !!

 

Kd> dt _ kapc_state
NT! _ Kapc_state
+ 0x000 apclisthead: [2] _ list_entry. Each Pointer Points to the _ kapc structure.
+ 0x010 process: ptr32 _ kprocess
+ 0x014 kernelapcinprogress: uchar
+ 0x015 kernelapcpending: uchar
+ 0x016 userapcpending: uchar

Obviously, the apclisthead here is the head of the APC queue. However, this is an array of 2, indicating the actual
Each thread has two APC queues. This is because APC functions are divided into two types: User APC and kernel APC.
Queues. The user APC refers to the corresponding APC function located in the user space and executed in the user space. The Kernel
Then the corresponding APC function is the kernel function.

The savedapcstate is also a _ kapc_state structure. apcstate will be copied to the savedapcstate for temporary storage when "Attach" is used to access the address space of another process!

Of course, there must be status information indicating whether the current thread is in the "original environment" or "affiliated Environment". This is the role of apcstateindex. In the code, the value of apcstateindex defines an enumeration type:

Typedef Enum _ kapc_environment {
Originalapcenvironment,
Attachedapcenvironment,
Currentapcenvironment,
Insertapcenvironment
} Kapc_environment;

Only originalapcenvironment and attachedapcenvironment can be used for apcstateindex.

The current value of apcstatepointer [2] In the kapc_state pointer array is used as the subscript, And the pointer in the array can point to one of the two apc_state data structures as needed.

Kd> dt _ kapc; APC object
NT! _ Kapc
+ 0x000 type: uchar
+ 0x001 sparebyte0: uchar
+ 0x002 size: uchar
+ 0x003 sparebyte1: uchar
+ 0x004 sparelong0: uint4b
+ 0x008 thread: ptr32 _ kthread
+ 0x00c apclistentry: _ list_entry
+ 0x014 kernelroutine: ptr32 void
+ 0x018 rundownroutine: ptr32 void
+ 0x01c normalroutine: ptr32 void
+ 0x020 normalcontext: ptr32 void
+ 0x024 systemargument1: ptr32 void
+ 0x028 systemargument2: ptr32 void
+ 0x02c apcstateindex: Char
+ 0x02d apcmode: Char
+ 0x02e inserted: uchar

Kernelroutine, rundownroutine, and normalroutine. Only normalroutine points to the function provided by the requester of the (executed) APC function, and the other two are auxiliary!

 

Ntkernelapi
Void
Keinitializeapc (
_ Out prkapc APC,
_ In prkthread thread,
_ In kapc_environment environment,
_ In pkkernel_routine kernelroutine,
_ In_opt pkrundown_routine rundownroutine,
_ In_opt pknormal_routine normalroutine,
_ In_opt kprocessor_mode processormode,
_ In_opt pvoid normalcontext
);

This function is mainly used to initialize the structure of APC (_ kapc). If environment = currentapcenvironment, APC-> apcstateindex is determined by apcstateindex in kthread, but environment cannot be greater than apcstateindex in kthread!

Finally, the APC request mode is processormode, but there is an exception: if the pointer normalroutine is 0, the actual mode is changed to kernelmode. This is because in this case, no user space APC function can be executed. The only thing that will be executed is kernelroutine!

Finally, keinitializeapc sets the inserted domain to false. However, the initialization of the APC object does not store it in the corresponding APC queue.

 

Ntkernelapi
Boolean
Keinsertqueueapc (
_ Inout prkapc APC,
_ In_opt pvoid systemargument1,
_ In_opt pvoid systemargument2,
_ In kpriority Increment
);

According to the specific situation of the APC request, it is sometimes inserted in front of the queue, usually hanging at the end of the queue.

 

_ Kiserviceexit:

CLI; Disable interrupts
Dispatch_user_apc EBP, returncurrenteax

;
; Exit from systemservice
;

Exit_all norestoresegs, norestorevolatile; this macro will be discussed later

 

Dispatch_user_apc macro tframe, returncurrenteax
Local a, B, c
C:
. Errnz (eflags_v86_mask and 0ff00ffffh)

Test byte PTR [tframe] + tseflags + 2, eflags_v86_mask/010000 h; is previous mode v86?
Jnz short B; If NZ, yes, go check for APC
Test byte PTR [tframe] + tssegcs, mode_mask; is previous mode user mode?
Jz a; no, previusmode = kernel, jump out
B: mov EBX, PCR [pcprcbdata + pbcurrentthread]; get ADDR of current thread
MoV byte PTR [EBX] + thalerted, 0; clear Kernel Mode alerted
CMP byte PTR [EBX] + thapcstate. asuserapcpending, 0
Je A; If EQ, no user APC pending

MoV EBX, tframe
Ifnb <returncurrenteax>; conditional macro assembly. If the returncurrenteax parameter is not empty, compile it!

; Dispatch_user_apc EBP, returncurrenteax, which is compiled here!
MoV [EBX]. tseax, eax; store return code in trap Frame
MoV dword ptr [EBX] + tssegfs, kgdt_r3_teb or rpl_mask
MoV dword ptr [EBX] + tssegds, kgdt_r3_data or rpl_mask
MoV dword ptr [EBX] + tsseges, kgdt_r3_data or rpl_mask
MoV dword ptr [EBX] + tsseggs, 0
Endif

;
; Save previous IRQL and set new priority level
;
Raiseirql apc_level
Push eax; save oldirql

STI; allow higher priority ints

;
; Call the APC delivery routine.
;
; EBX-trap Frame
; 0-null exception Frame
; 1-previous Mode
;
; Call APC deliver routine
;

Stdcall _ kideliverapc, <1, 0, EBX>; 1 is usermode

Pop ECx; (ECx) = oldirql
Lowerirql ECx

Ifnb <returncurrenteax>; Same as above
MoV eax, [EBX]. tseax; restore eax, just in case
Endif

CLI
Jmp B; pay attention to this loop !!

Align 4
A:
Endm

 

This code mainly checks:

• Whether or not the user space to be returned.
• Whether the user APC request is waiting for execution

If the condition is met, kideliverapc is used to deliver the APC. Note that the Code jmp B appears to be called cyclically until no user APC is returned to the user space. Actually, it is not. kideliverapc clears userapcpending every time a user APC is processed, therefore, user APCs can only be shipped once when returning to the user space! Kideliverapc processes all the kernel mode APCs with while, but the user mode APC only processes one! In fact, user APC delivery is very special. It will be mentioned later, and it can only be shipped once at a time!

As mentioned above, kthread has two kapc_state data structures: one is apcstate and the other is savedapcstate. Both have APC queues, but only the queues in apcstate are to be shipped.

Kideliverapc (
In kprocessor_mode previusmode, // It is not better to write delivermode.
In pkexception_frame exceptionframe, // this parameter is almost 0
In pktrap_frame trapframe
)

This function is still complicated and the code is no longer posted.

The previusmode parameter indicates the APC that requires "Shipping". It can be usermode or kernelmode. However, kernelmode does indicate that only the kernel APC is required to be executed, while the usermode indicates that the user APC is executed outside the kernel APC.

 

 

The Windows operating system uses three kinds of APCs:

• User APCsRun strictly in user mode and only when the current thread is in an alertable wait state. The operating system uses user APCs to implement mechanic ISMs such as overlapped I/O andQueueuserapcWin32 routine. (run IRQL = passive_level)
• Normal kernel APCsRun in kernel mode at IRQL = passive_level. A normal kernel APC preempts all user-mode code, including user APCs. Normal kernel APCs are generally used by file systems and file-system filter drivers.
• Special kernel APCsRun in kernel mode at IRQL = apc_level. A special kernel APC preempts user-mode code and kernel-mode code that executes at IRQL = passive_level, including both user APCs and normal kernel APCs. the operating system uses special kernel APCs to handle operations such as I/O Request completion.

The Code is as follows:

User APCs
_ Kapc. apcmode = usermode, _ kapc. kernelroutine! = NULL, _ kapc. normolroutine! = NULL

 

Normal kernel APCs

_ Kapc. apcmode = kernelmode, _ kapc. kernelroutine! = NULL, _ kapc. normolroutine! = NULL

 

Special kernel APCs

_ Kapc. apcmode = kernelmode, _ kapc. kernelroutine! = NULL, _ kapc. normolroutine = NULL

 

Their _ kapc. kernelroutine is certainly not empty. In addition, if normolroutine is not empty, kernelroutine is called before normolroutine is called !!

 

To be continued ......