Security checks based on Checkstackvars

Under VS2008, if an array exists in the stack space of a function, the Checkstackvars check is automatically added, as the name suggests, to check whether local data accesses the bounds. In contrast, this kind of inspection can only play a role, and not all the Cross-border access can be checked, according to the principle of the following introduction will understand this. Since the check is local, then the static type array defined within the function or global array outside the function does not take this check, and since it is a check array, the check does not exist if there is no local array within the function.

Let's first look at a simple example to verify the existence of this check:

void Testvars (void)
{
int BF = 0XEEEEEEEE;
Char array[10] = {0};
int bk = 0xFFFFFFFF;

strcpy (Array, "Masefee");
}

int main (void)
{
Testvars ();
return 0;
}

In this example, there is an array of arrays, where two other variables are deliberately defined to see the memory distribution of the two variables and array arrays. This will be able to clearly understand the checkstackvars the principle of this inspection. Then look at the memory distribution of 3 local variables within the Testvars function under Debug. The breakpoint is hit on the strcpy sentence, distributed as follows:

The

FF FF FF ff cc cc cc CC   cc cc CC CC  00 a 00 CC cc cc CC CC cc  cc cc CC cc ee ee ee ee  

BK Array Bf

The above relationship has been very clear, we found that in C + + code, the BF, array, bk three in the memory distribution should be continuous, next to each other. But this is not the case, look at the BF and array between like a 10 bytes away. The reason for this is that, in the debug version of VS2008, the local variables are not kept in stack memory, but are aligned in 4-byte ways, and there are protective bytes before and after. The protected byte here is 4 bytes, and the value is 0XCC, which is clearly the code byte of the assembly instruction int 3 interrupt. So there will be 4 bytes of 0xcc before and after the BK and BF variables. The green part above is that there are also 4 bytes of 0xcc at both ends of array arrays. Above the black bold part is that the array array accounted for 10 bytes, to 4-byte alignment, so you need to complement two bytes, so two 0xcc, resulting in a 10 byte between the BF and array. The one next to the above array should be two 0xcc to complement the alignment. It was deliberately marked to the back. The purpose of this identification here is to illustrate the principle of checkstackvars this inspection.

OK, clear the memory distribution, then checkstackvars at what time to perform the check, in C + + code can not be displayed to see, so to flip the testvars function of the disassembly code:

Testvars:
004113B0 Push EBP
004113B1 mov Ebp,esp
004113b3 Sub esp,0f0h
004113B9 push EBX
004113BA push ESI
004113BB Push EDI
004113BC Lea EDI,[EBP-0F0H]
004113C2 mov ecx,3ch
004113C7 mov eax,0cccccccch
004113CC Rep STOs dword ptr Es:[edi]
004113CE mov eax,dword ptr [___security_cookie (417004h)]
004113d3 XOR EAX,EBP
004113D5 mov dword ptr [Ebp-4],eax
004113d8 mov dword ptr [Ebp-0ch],0eeeeeeeeh
004113DF mov byte ptr [ebp-20h],0
004113E3 XOR Eax,eax
004113E5 mov dword ptr [Ebp-1fh],eax
004113E8 mov dword ptr [Ebp-1bh],eax
004113EB mov byte ptr [ebp-17h],al
004113EE mov dword ptr [EBP-2CH],0FFFFFFFFH
004113F5 Push offset string "Masefee" (415804h)
004113FA Lea EAX,[EBP-20H]
004113FD push EAX
004113FE call @ILT +160 (_strcpy) (4110a5h)
00411403 Add esp,8
00411406 push edx
00411407 mov ecx,ebp
00411409 push EAX
0041140A Lea edx,[(411438h)]
00411410 call @ILT +130 (@_rtc_checkstackvars@8) (411087h)
00411415 pop eax
00411416 pop edx
00411417 Pop EDI
00411418 pop ESI
00411419 pop ebx
0041141A mov ecx,dword ptr [ebp-4]
0041141D XOR ECX,EBP
0041141F call @ILT +25 (@__security_check_cookie@4) (41101Eh)
00411424 Add esp,0f0h
0041142A CMP EBP,ESP
0041142C call @ILT +320 (__RTC_CHECKESP) (411145h)
00411431 mov esp,ebp
00411433 Pop EBP
00411434 ret
00411435 Lea ECX,[ECX]
00411438DB 01h
00411439 DB 00h
0041143A DB 00h
0041143B DB 00h
0041143C DB 40h
0041143D DB 14h
0041143E DB 41h
0041143F DB 00h
00411440 DB e0h
00411441 DB FFH
00411442 DB FFH
00411443 DB FFH
00411444 DB 0ah
00411445 DB 00h
00411446 DB 00h
00411447 DB 00h
00411448 DB 4ch
00411449 DB 14h
0041144A DB 41h
0041144B DB 00h
0041144C DB 61h
0041144D DB 72h
0041144E DB 72h
0041144F DB 61h
00411450 DB 79h
00411451 DB 00h

It is clear from Testvars's disassembly code that the black bold part is described in the previous blog post, and in this note, after the strcpy call, the _rtc_checkstackvars function is called, which is a function. Let's take a look at his prototype:

void __fastcall _rtc_checkstackvars (void *_esp, _rtc_framedesc *_fd);

This is a fastcall function, so two parameters are passed through registers. The second argument is a struct type, and then look at the definition of the struct:

typedef struct _RTC_FRAMEDESC

{
int varcount; The number of arrays to check
_rtc_vardesc *variables; Information about the array to check

} _rtc_framedesc;

This structure is defined in the Rtcapi.h header file, and the _rtc_vardesc is also a struct type, looking at the definition:

typedef struct _RTC_VARDESC

{
int addr; The offset of the first address of the array relative to the EBP
int size; Size byte number of array
Char *name; The name of the array
} _rtc_vardesc;

After populating the structure with the example above, the data for the structure is:

_rtc_framedesc.varcount = 1;

_RTC_VARDESC->ADDR = ARRAY-EBP; Here array at the low address, so addr is ultimately negative

_rtc_vardesc->size = 10;

_rtc_vardesc->name = "Array";

Okay, this is clear. The information stored, and then back to the disassembly code, before calling the _rtc_checkstackvars function, pay attention to the red bold one of the instructions, the EBP assignment to the ECX register, and then 411438h This address value assigned to the EDX , because the _rtc_checkstackvars function is fastcall, the two registers are used to pass arguments instead of push operations. ECX is the Testvars function of the stack frame, edx this address is a bit strange, should have passed _RTC_FRAMEDESC structure pointer, is this 411438h address value is _RTC_ The memory address in which the FRAMEDESC struct variable resides. From the disassembly code above, you can see that the following from the 411438h address, a bit more strange data, this should not have such a piece of data under the function, in debug most cases are 0XCC filled. Let's take a closer look at this data, or simply copy the 411438h address value to the memory window:

0x00411438 e0 FF FF FF 0a 4c 61 72 72 61 79 00 in the same.

Look at the data above, is not the _RTC_FRAMEDESC structure should have the data. The answer is yes, the red part is the value of the _rtc_framedesc.variables pointer, pointing to the position is immediately followed, this is the compiler intentionally so deal with. Of course it could be somewhere else. This is the compiler's direct record of this information in the code snippet, and immediately after the recorded function code. So do not mistake this information for a _RTC_FRAMEDESC structure that is written in or populated during the execution of the program.

Knowing here, the whole rule is found to be well-founded, and the design is very good. You can feel the greatness of Ms again. Oh, nonsense.

Now that you've given all two parameters to the _rtc_checkstackvars function, let's look at how the function is detected inside, and look at the disassembly of this function:

_rtc_checkstackvars:
00411500 mov Edi,edi
00411502 Push EBP
00411503 mov Ebp,esp
00411505 push ECX
00411506 push EBX
00411507 push ESI
00411508 Push EDI
00411509 XOR Edi,edi//Clear Zero
0041150B mov esi,edx//Assign _RTC_FRAMEDESC structure pointer to ESI
0041150D cmp DWORD ptr [Esi],edi//Comparison Varcount is 0,if (_fd->varcount!= 0)
0041150F mov ebx,ecx//Testvars stack frame assignment to ebx
00411511 mov dword ptr [I],edi/I should be the loop variable, assigning the number of arrays to I,i = _fd->varcount;
00411514 jle _rtc_checkstackvars+58h (411558h)
00411516 mov eax,dword ptr [esi+4]//+4 is the _rtc_framedesc.variables pointer
00411519 mov ecx,dword ptr [eax+edi]//_RTC_VARDESC->ADDR, which is the offset of the initial address of the array relative to Testvars EBP
0041151C add Eax,edi//position EAX to _RTC_VARDESC structure first address
0041151E cmp DWORD ptr [ECX+EBX-4],0CCCCCCCCH//[ecx+ebx-4] is equivalent to ebp-addr-4, which is the front 4 protection bytes of array
00411526 jne _rtc_checkstackvars+36h (411536h)//If not equal to 0XCCCCCCCC error _rtc_stackfailure
00411528 mov edx,dword ptr [eax+4]//Eax+4 is _rtc_vardesc->size, representing the size of the array
0041152B add EDX,ECX//ECX is currently offset, plus size is the offset of the end of the array array relative to the EBP
0041152D cmp DWORD ptr [EDX+EBX],0CCCCCCCCH//EDX+EBX is the rear 4 protection byte of the array's tail, then compares
00411534 JE _rtc_checkstackvars+4ah (41154Ah)
00411536 mov eax,dword ptr [esi+4]//esi+4 as _rtc_framedesc.variables pointer
00411539 mov ecx,dword ptr [eax+edi+8]//Eax+edi+8 is _rtc_vardesc->name, for error warning
0041153D mov edx,dword ptr [ebp+4]
00411540 push ECX//incoming array name across bounds
00411541 push edx//Incoming ebp+4 address, which is _rtc_checkstackvars's return address, for locating
00411542 call _rtc_stackfailure (4110CDH)//calls this function, pops up an exception messagebox and prompts which array is out of bounds
00411547 Add esp,8
0041154A mov Eax,dword ptr [i]///Multiple arrays are useful to check
0041154D Inc EAX
0041154E Add edi,0ch//Navigate to the next _RTC_VARDESC structure
00411551 cmp eax,dword ptr [esi]
00411553 mov dword ptr [I],eax
00411556 JL _rtc_checkstackvars+16h (411516h)//Cycle
00411558 Pop EDI
00411559 pop ESI
0041155A pop ebx
0041155B mov esp,ebp
0041155D Pop EBP
0041155E ret

The above process is somewhat complicated, the main principle is to read the _RTC_VARDESC structure, each array for the front and back bounds check, if the change, then call the _rtc_stackfailure function, the last pop-up error message box, information such as:

Run-time Check failure #2-stack around the variable ' array ' was corrupted.

It is important to note that if multiple arrays need to be checked, the name of each array is next to each other, followed by a plurality of _RTC_VARDESC structures, and the memory is distributed as follows:

[Number of arrays, _RTC_VARDESC address] [Multiple _RTC_VARDESC structures (arrays)] [Name of each array]

These position distributions are written directly in the code by the compiler.

This allows for a simple boundary check, as mentioned earlier, this check will only check the front and back boundaries, if the program in the Cross-border access, but no modification or write value is the value of the boundary check 0XCCCCCCCC, it will not detect the code has been out of bounds. So the main thing is to be cautious. The compiler can't always do everything for us. The above procedure adds a boundary check value to the stack memory, so it is more practical in the debug version. There is no room for such a waste of release, so crossing the line is even more dangerous.

From the analysis above, you can write pseudo code for the _rtc_checkstackvars function, as follows:

[CPP] view plain copy print?       typedef struct _RTC_VARDESC {int addr;       int size;   char* name;      } _rtc_vardesc; typedef struct _RTC_FRAMEDESC {int varcount;