Make MASM support the _ fastcall call Method

Question: let your MASM support the _ fastcall call Method
Author: thebutterfly
Time:
Chain: http://bbs.pediy.com/showthread.php? Threadid = 20938
Details:

Make MASM support the _ fastcall call Method
Author: thebutterfly (cloud)
Anyone familiar with reverse engineering and cracking knows that calling a function must follow certain call conventions. common call conventions include C call conventions (_ cdecl), standard call conventions (_ stdcall), Pascal call conventions, and quick call conventions (_ fastcall. in these call conventions, MASM has good support for the first three, but does not support the _ fastcall call conventions. This is a pity. because of this, MASM is quite inconvenient to process functions of the _ fastcall type. For example, when writing a kernel-mode driver, there is a very common function iofcompleterequest, which is the same as a common method: first, the function prototype declaration is a big problem. The result of the common declaration method is "error XXXX: cannot resolve external symbol yyyyy". Second, the call syntax. Invoke obviously does not work, it will only press the parameter stack and then call it, but this is not the form of _ fastcall. it is also because it does not support _ fastcall, which makes assembly and other languages (such as C) mixed programming difficult once _ fastcall is involved.

We all want to adopt some strategies to make MASM support the _ fastcall call method. In other words, it enables MASM to define and call functions of the _ fastcall type.

To achieve this goal, we must first understand the characteristics of the (standard) _ fastcall call.

_ Fastcall: 1st parameters are put into the ECX register, 2nd parameters are put into the edX register, and if there are other parameters, the stack is pressed from right to left; the operator is responsible for maintaining the stack balance (removing the pressed parameters). If the function has a return value, the return value is stored in the eax register. function Name modification features: Add @ before function name, followed by @ and 4 * parameter count. for example, iofcompleterequest with two parameters is modified to "@ iofcompleterequest @ 8 ".

Let's take a look at how to process the _ fastcall function in the assembly language. Suppose we want to define and call such a function in the assembly language:
Int _ fastcall addnum (int x, int y, int Z ){
Return X + Y + z;
}
How to define this function? It is easy to see that there are three parameters. According to the characteristics of the _ fastcall call, X should be stored in the ECX register, and Y should be included in the edX. Another parameter Z should be passed through the stack, after the function is executed, you should clear the input parameters (retn 4 is returned). It is not difficult to get our initial idea as follows:

Addnum proc Z; here, X and Y are gone because they all run in registers.
MoV eax, ECx; ECx is the X Parameter
Add eax, EDX; edX is the y parameter. Here, X + Y is calculated.
Add eax, Z; add Z. In this case, the return value is X + Y + Z.
Retn 4; clear the Z parameter by yourself
Addnum endp

How to call it? As mentioned above, invoke is definitely not good, because it only puts all the parameters on the stack, and does not meet the characteristics of _ fastcall. we can only use the original method, that is, write a bunch of code processing parameters and then call XXX. according to the features of _ fastcall, the call code is as follows:

Other code
MoV ECx, real parameter X; ECx stores real parameter x (1st parameter)
MoV edX, real parameter Y; real parameter Y in EDX (2nd parameter)
Push real parameter Z; according to the call characteristics, Z should pass through the stack
Call addnum; call
Other code, such as processing return values. The caller does not have to worry about stack balancing.

After testing, it seems that this is possible. Not only is there no syntax error, but the stack is not broken, and the results of disassembly and calling are also very correct. The problem seems to have been solved.

However, further research breaks down our dream. We did this experiment: We used this function to perform mixed programming with the C language and called it in the C program as follows:

Extrn "C" int _ fastcall addnum (int x, int y, int Z); // declare an external Function
...
...
Int result = addnum (1, 2, 3); // call
...

Compilation is okay, but the connection is wrong. The error XXXX: an unresolved symbol "@ addnum @ 12" is prompted"

Strange ?! We clearly define a function named addnum. Why can't we find @ addnum @ 12?

Let's try again. This time, the Assembly Language calls the addnum function written by C. The Code is as follows:

Addnum proto Z: DWORD; declare the prototype, because X and Y are all in the register, there is only one parameter Z
...
MoV ECx, real parameter X; these few words are not mentioned, the same as above. The key is to see the result.
MoV edX, real parameter Y
Push real parameter Z
Call addnum
...

Result: The error message is as follows: Error XXXX: cannot resolve external symbol _ addnum @ 4.

Still strange ?! When the addnum function is declared, the system prompts that _ addnum @ 4 is not found!

Facts make us realize that what we have done above has not completely achieved our goal, but has not solved the "Internal Affairs" problem.

Let me analyze why the connection fails. Every time we define the addnum function, the connector always prompts that the symbol name in the form of xxxaddnumxxx cannot be found. Why?

Yes, symbol name modification! This is the key to the problem!

(By The Way, most of the so-called "mixed programming" should pay attention to three issues: one is the function call method, and the other is the OBJ file format, the last one is the symbol modifier. the so-called "VB and VC mixed programming" and "VB and Assembly mixed programming" can be implemented mainly because they all use the same link.exe program! The more fundamental reason is that, although different compilers are used, the compiled OBJ files are in coff format. It is precisely because the format is uniform that hybrid programming becomes possible. if you want to mix tasm and VC ++ for programming, it will be much more difficult, because the two OBJ formats are different)

Depending on the call type, symbol names are modified in different ways. We will write a sentence when writing a Win32 Assembler:
. Model flat, stdcall
There is no way to do this, because the function declaration in the header file of masm32 does not specify the language type, and there will be errors without the stdcall Statement (as I am confused, this is a weakness of masm32. this causes a lot of trouble for the mixed programming of MASM and other languages ). this makes every function we write is of the stdcall type. The modification method of the stdcall type is similar to that of the fastcall type, except that it is followed by an underscore (_) instead of an at sign @. for example, messageboxa is modified to _ messageboxa @ 16. so in the second experiment, the function we wrote was modified to _ addnum @ 4, and the real addnum was modified to @ addnum @ 12. Of course, the two are different. similarly, in the first test, the function we wrote was modified to _ addnum @ 4, but the connector went to @ addnum @ 12, but of course it could not be found.

You can try to open the compiled OBJ file in the hexadecimal editor and find the addnum string.

What should I do? It seems that an important reason for mixed programming is that MASM cannot automatically generate the fastcall function modifier, which makes it impossible to call the external _ fastcall function, it also makes it impossible to call the function we have written. it seems that the only way to solve this problem is to generate your own symbol modifier! This should be noted that the function modifier name is related to the language type (call method) of the function. As mentioned above, the function we write is caused by that sentence. the model declaration is automatically changed to the stdcall type and will be automatically modified by the compiler during compilation. If we write @ addnum @ 12 proc Z, during compilation, it will be automatically modified to _ @ addnum @ 12 @ 4 (followed by an underscore and @ 4), which violates our intention. therefore, we need to find a call method that meets the following three conditions and serves as an alternative to fastcall.
1. Do not modify the function name (so that we can modify it ourselves)
2. the parameter is pushed from right to left (otherwise, parameters cannot be directly referenced and can only be referenced using [EBP + 8} and other operators)
3. The function clears the stack parameters.

Are there such call methods? Yes! Is the syscall call method!

So our function can be modified as follows:

@ Addnum @ 12 proc syscall Z; Are there any differences? The function name is modified, and the syscall type is added.
MoV eax, ECx; these are the same
Add eax, EDX
Add eax, Z
Retn 4
@ Addnum @ 12 endp; the beginning of the response, which is also modified here

Test 1 again, succeeded !!!

Similarly, the call statement and proto Statement of test 2 must be modified
@ Addnum @ 12 proto syscall: DWORD
And
Call @ addnum @ 12

In this way, Experiment 2 is successful, and the problem is finally completely solved.

Summary:
1. if you define a function by yourself, the function name must be modified according to the fastcall rule, and the function type should be syscall.
2. If you call functions of other modules, the call and proto function names must also be modified.
It's easy. It's just a bit of trouble.

Finally, if you want to call functions of other modules, you can use externdef to declare functions in addition to using proto.
Externdef syscall @ addnum @ 12: Proc
Which of the following is a matter of personal preferences?
In addition, if it is annoying to write or modify a name, defining a text macro with textequ is also a good note, as shown in the following code:
Addnum textequ <@ addnum @ 12>
In this way, you can directly use addnum to call the program.

**************************************** ****************************************
**************************************** ****************************************

To facilitate the definition and call of the fastcall function, I wrote a set of macros. The following describes how to use them.

Define a fastcall function:
Beginfcproc (function name, number of parameters [, distance]) [others such as uses register list and other parameters}
Example: beginfcproc (addnum, 3) uses ebx edi esi z: DWORD
End definition:
Endfcproc (function name, number of parameters)
Example: endfcproc (addnum, 3)
The preceding two macros must be used in pairs.

Call a fastcall function:
Fastcall function name [, parameter 1] [, parameter 2] [......]
The parameters support the ADDR and offset operators. The syntax of this macro is basically the same as that of the invoke macro.
Example: fastcall exampleproc, ADDR dwnum, offset szstring, null

Define a fastcall function:
Fcproto (function name, number of parameters)
This macro is defined by externdef.
Example: fcproto (addnum, 3)

**************************************** ****************************************
The following is the macro content. paste it and save it as an ASM or inc file. You can use it directly.
If you have any questions, please criticize and correct them!
**************************************** ****************************************

Comment> comment ---------------------------------------------------------------------------------------
Fastcall macros Version 1.0
Written by cloud, nju
Copyright by cloud, 2006
This header file provides support for fast call (fastcall ).
Latency>
Ifndef _ fastcall_m _
_ Fastcall_m _ =-1

; Success ;;------------------------------------------------------------------------------------
; (MA) macro returned by advanced functions
;;
; Usage: return value [, number of stack balances]
;;
; Parameter: return value, return value of the Function
; Balance number, the parameter returned by the function when the stack is automatically cleared (there are several pressure stack parameters to write), only fastcall is required, other situations do not
;;
; Affected registers: eax (mandatory)
;;
; Success ;;------------------------------------------------------------------------------------
Ifndef return
Return macro Arg, argsize
Local num, reax
Reax = 0
IFB <argsize>
Num textequ <>
Else
Num textequ % (& argsize & * 4)
Endif

IFB <Arg>
RET num
Else
If @ sizestr (ARG) Ge 8
Ifidni @ substr (ARG, 1, 7), <OFFSET>
MoV eax, ARG
Reax =-1
Endif
Endif
If @ sizestr (ARG) Ge 10
Ifidni @ substr (ARG, 1, 9), <lroffset>
MoV eax, ARG
Reax =-1
Endif
Endif
If not reax
If (opattr (ARG) and 0001_ B; is a register value
Ifdifi <Arg>, <eax>; Do not move eax onto itself
MoV eax, ARG
RET num
Reax =-1
Else
RET num
Endif
Elseif (opattr (ARG) and 00000100b; is an immediate value
If (opattr (ARG) and then 0001b
MoV eax, ARG
RET num
Else
If Arg EQ 0
XOR eax, eax
RET num
Elseif Arg EQ 1
XOR eax, eax
INC eax
RET num
Elseif Arg eq-1
Or eax, not 0
RET num
Else
MoV eax, ARG
RET num
Endif
Endif
Reax =-1
Else
MoV eax, ARG
RET num
Reax =-1
Endif
Endif
Endif
Endm
Endif

; Success ;;------------------------------------------------------------------------------------
; (MF) Flip parameter list
;;
; Usage: $ argrev (parameter table)
;;
; Parameter: original parameter list (vararg)
;
; Return Value: list of parameters after flip
;
; Affected registers: None
;;
; Success ;;------------------------------------------------------------------------------------
Ifndef $ argrev
$ Argrev macro ARGs: vararg
Local Arg, y
Y textequ <>
For Arg, <& ARGs>
Y catstr <Arg >,<!,>, Y
Endm
Y substr y, 1, @ sizestr (% Y)-1
Exitm @ catstr (<! <>, Y, <!> )
Endm
Endif
; Success ;;------------------------------------------------------------------------------------
; (MA) quick call
;;
; Usage: fastcall function name, parameter 1, parameter 2 ,....
;;
; Parameter: function name + parameter list
;;
; Affected registers: eax, ECx, EDX (must be changed)
;;
; Success ;;------------------------------------------------------------------------------------
Ifndef fastcall
Fastcall macro API: req, P1, P2, PX: vararg
Local Arg, line, recx, reax, redx
Recx = 0
Reax = 0
Redx = 0

Ifnb <PX>
% For Arg, $ argrev (<PX>)
If @ sizestr (<Arg>) Ge 6
Ifidni @ substr (<Arg>, 1, 5), <ADDR>
Lea eax, @ substr (<Arg>, 6)
Push eax
Reax =-1
Else
Push ARG
Endif

Else
Ifidni <Arg>, <eax>; Do not overwrite eax
If reax
Line textequ % @ line
% Echo @ filecur (line): Error! Eax register value overwritten by fastcall macro.
. Err
Else
Push ARG
Endif
Else
Push ARG
Endif
Endif
Endm
Endif

Ifnb <P1>

If @ sizestr (<P1>) Ge 6
Ifidni @ substr (<P1>, 1, 5), <ADDR>
% Lea ECx, @ substr (<P1>, 6)
Recx =-1
Endif
Endif
If @ sizestr (<P1>) Ge 8
Ifidni @ substr (<P1>, 1, 7), <OFFSET>
MoV ECx, p1
Recx =-1
Endif
Endif
If @ sizestr (<P1>) Ge 10
Ifidni @ substr (<P1>, 1, 9), <lroffset>
MoV ECx, p1
Recx =-1
Endif
Endif

If (not recx)
If (opattr (P1) and 00000100b; is an immediate value
If (opattr (P1) and then 0001b
MoV ECx, p1
Else
If P1 EQ 0
XOR ECx, ECx
Elseif P1 EQ 1
XOR ECx, ECx
INC ECx
Elseif P1 eq-1
Or ECX,-1
Else
MoV ECx, p1
Endif
Endif
Recx =-1
Elseif (opattr (P1) and 0001_ B; is a register value
Ifdifi <P1>, <ECx>
Ifidni <P1>, <eax>
If reax
Line textequ % @ line
% Echo @ filecur (line): Error! Eax register value has changed.
. Err
Endif
Endif
MoV ECx, p1
Recx =-1; no more ECx
Endif
Else
MoV ECx, p1
Recx =-1
Endif
Endif
Endif

Ifnb <P2>

If @ sizestr (<P2>) Ge 6
Ifidni @ substr (<P2>, 1, 5), <ADDR>
% Lea edX, @ substr (<P2>, 6)
Redx =-1
Endif
Endif
If @ sizestr (<P2>) Ge 8
Ifidni @ substr (<P2>, 1, 7), <OFFSET>
MoV edX, p2
Redx =-1
Endif
Endif

If @ sizestr (<P2>) Ge 10
Ifidni @ substr (<P2>, 1, 9), <lroffset>
MoV edX, p2
Redx =-1
Endif
Endif

If (not redx)
If (opattr (P2) and 0001_ B; is a register value
Ifdifi <P2>, <edX>; Do not move edX onto itself
Ifidni <P2>, <eax>
If reax
Line textequ % @ line
% Echo @ filecur (line): Error! Eax register value has changed.
. Err
Endif
Endif

Ifidni <P2>, <ECx>
If recx; If ECx was used report error
Line textequ % @ line
% Echo @ filecur (line): Error! ECX register value has changed.
. Err
Endif
Endif

MoV edX, p2
Redx =-1
Endif

Elseif (opattr (P2) and 00000100b; is an immediate value
If (opattr (P2) and then 0001b
MoV ECx, p2
Else
If P2 EQ 0
XOR edX, EDX
Elseif P2 EQ 1
XOR edX, EDX
INC edX
Elseif P2 eq-1
Or edX,-1
Else
MoV edX, p2
Endif
Endif
Redx =-1
Else
MoV edX, p2
Redx =-1
Endif
Endif
Endif

Call API
Endm
Endif

; Success ;;------------------------------------------------------------------------------------
; (MF) Quick macro function call
;;
; Usage: return value = $ fastcall (function name, parameter 1, parameter 2 ...)
;;
; Parameter: Same as fastcall
;;
; Return value: the return value of the function (eax)
;;
; Affected registers: Same as fastcall
;;
; Success ;;------------------------------------------------------------------------------------
Ifndef $ fastcall
$ Fastcall macro API: req, P1, P2, PX: vararg
Fastcall API, P1, P2, PX
Exitm <eax>
Endm
Endif

; Success ;;------------------------------------------------------------------------------------
; (MF) Quickly call the function name to modify the macro
;;
; Usage: $ fcallfuncnamegen (function name, number of parameters)
;;
; Parameter: Function Name, number of function parameters
;;
; Return Value: modifier of the Function
;;
; Affected registers: None
;;
; Success ;;------------------------------------------------------------------------------------
Ifndef $ fcallfuncnamegen
$ Fcallfuncnamegen macro fname: req, argsize: req
Local num
Num textequ % (& argsize & * 4)
Exitm @ catstr (<@ >,< fname >,< @>, % num)
Endm
Endif

; Success ;;------------------------------------------------------------------------------------
; (MF) defines a quick call function Header
;;
; Usage: [$] beginfcproc (function name, number of parameters [, distance]) uses register list contains parameters other than parameter 1 and 2
;;
; Parameter: Needless to say
;;
; Return Value: No worries
;;
; Affected registers: None
;;
; Success ;;------------------------------------------------------------------------------------
Ifndef $ beginfcproc
$ Beginfcproc macro fname: req, argsize: req, distance
Fname textequ $ fcallfuncnamegen (fname, argsize)
IFB <distance>
Exitm <$ fcallfuncnamegen (fname, argsize) proc syscall>
Else
Exitm <$ fcallfuncnamegen (fname, argsize) proc & Distance & syscall>
Endif
Endm
Beginfcproc textequ <$ beginfcproc>
Endif

; Success ;;------------------------------------------------------------------------------------
; (MF) The end of the quick call function, which must be used in pairs with $ beginfcproc
;;
; Usage: $ endfcproc (function name, number of parameters)
;;
; Parameter: Do not speak more
;;
; Return Value: No worries
;;
; Affected registers: None
;;
; Success ;;------------------------------------------------------------------------------------
Ifndef $ endfcproc
$ Endfcproc macro fname: req, argsize: req
Exitm <$ fcallfuncnamegen (fname, argsize) endp>
Endm
Endfcproc textequ <$ endfcproc>
Endif

; Success ;;------------------------------------------------------------------------------------
; (MA) dedicated return macro for quick function call
;;
;;
; Usage: fcret return value, number of parameters
;;
; Parameter: Same as return, the stack clearing parameter is automatically corrected.
;;
;;
; Affected registers: eax (required)
;;
; Success ;;------------------------------------------------------------------------------------
Ifndef fcret
Fcret macro retv, argsize: req
Local num
If argsize GT 2
Num textequ % (& argsize &-2)
Else
Num textequ <0>
Endif
Return retv, num
Endm
Endif

; Success ;;------------------------------------------------------------------------------------
; Is used to declare an external fastcall Function
;;
; Usage: $ fcproto (function name, number of parameters)
;;
; Parameter: Needless to say
;;
;;
; Affected registers: None
;;
; Success ;;------------------------------------------------------------------------------------
Ifndef $ fcproto
$ Fcproto macro fname: req, argsize: req
Fname textequ $ fcallfuncnamegen (fname, argsize)
Exitm @ catstr (<externdef syscall>, <$ fcallfuncnamegen (fname, argsize) >,<: Proc>)
Endm
Fcproto textequ <$ fcproto>
Endif

; End of File
Endif; fastcall. inc