An alternative method for detecting ollydbg

An alternative method for detecting ollydbg

Author: pumqara

Preface

Nowadays, ring-3 debuggers are increasingly used because they have a graphical interface and are more convenient than ring-0 debuggers (such as SoftICE. In this articleArticleI will show you how to detect ollydbg, one of the best ring-3 debuggers. Many people have heard of isdebugerpresent and FS: [20] detection methods. But is there any other new method? Next I will introduce some of my own detection methods. I will give a detailed explanation, so you can use your imagination to perfect them.

Method 1: findwindow

This method is based on the findwindow function. Like all the dialogs, the Main Dialog Box (window?) of the ollydbg ?) It also has its title and class name. Using this API function, we can determine whether the main ollydbg window is opened. Microsoft writes:

Bytes ------------------------------------------------------------------------------------------------

The findwindow function can obtain the top-level window handle of the window class name or the title is a specific string. This function does not search for subwindows.

Hwnd findwindow (

Lpctstr lpclassname, // address of the window class name

Lptstr lpwindowname // address of the window title

);

Parameter description

Lpclassname

A pointer to a string ending with null that represents the window class name, or an atom that identifies the window class name string. If this parameter is an atom, it must be a global atom pre-created by the globaladdatom function. The 16-bit atom must be placed in the low 8 bits of the lpclassname, and the high 8 bits of the lpclassname must be 0.

Lpwindowname

Pointer to a string ending with null that represents the window name (I .e. the title. If this parameter is null, all windows are considered qualified.

Return Value

If the search is successful, a window handle that meets the search criteria is returned.

If the search fails, the return value is null. To obtain detailed error information, call getlasterror.

Bytes ------------------------------------------------------------------------------------------------

MyProgramParts:

Code:--------------------------------------------------------------------------------

. Data

Strollyclsname DB "ollydbg", 0

. Code

Invoke findwindow, ADDR strollyclsname, null

CMP eax, 00000000 H

JNE olly_detected/

--------------------------------------------------------------------------------

Method 2: createconlhelp32snapshot, process32first/next

This is an interesting method. It is based on four API functions (createquallhelp32snapshot, process32first, process32next, getcurrentprocessid) and a structure (processentry32 ). Let's take a look at what msdn says:

Bytes ------------------------------------------------------------------------------------------------

[Createconlhelp32snapshot]

The createconlhelp32snapshot function creates a snapshot [snapshot] for the specified process (including the heap [heap], module [module], and thread [thread] used by the process).

Handle winapi createconlhelp32snapshot (

DWORD dwflags,

DWORD th32processid

);

Parameter description:

Dwflags

[Input] th32cs_inherit-the snapshot handle can be inherited.

Th32cs_snapall-the snapshot contains all processes and threads in the system, it also includes the heap of the process specified in th32processid and the th32cs_snapheaplist of the module. The snapshot contains all the heap of the process specified in th32processid. List the heap of a process and view heap32listfirst.

Th32cs_snapmodule-the snapshot contains all the modules of the process specified in th32processid. List the modules of the process and view module32listfirst.

Th32cs_snapprocess-the snapshot contains all processes in the system. List all processes and view process32first.

Th32cs_snapthread-the snapshot contains all the threads in the system. List all threads and view thread32first.

Th32processid

[Input] the ID of the process to be crawled. If this parameter is set to 0, the current process is crawled. This parameter is valid only when th32cs_snapheaplist, th32cs_snapmoudle, or th32cs_snapall is set. In other cases, this parameter is ignored and all processes are crawled.

Return values

If the call is successful, the snapshot handle is returned.

If the call fails, invaid_handle_value is returned.

Remarks

The snapshot taken by this function is examined by the other tool help functions to provide their results. access to the snapshot is read only. the Snapshot handle acts like an object handle and is subject to the same rules regarding which processes and threads it is valid in.

List the heap and module status of all processes, specify th32cs_snapall, and set th32processid to 0. Then, for each newly added process in the snapshot, call createconlhelp32snapshot again to set the ID of the new process to be captured and th32cs_snapheaplist or th32cs_snapmoule.

To delete a snapshot, use the closehandle function.

Bytes ------------------------------------------------------------------------------------------------

[Process32first]

Process32first obtains the information of the first process in a system snapshot.

Bool winapi process32first (

Handle hsnapshot,

Lpprocessentry32 lppe

);

Parameter description

Hsnapshot

[Input] handle to the System Snapshot returned when the createconlhelp32snapshot function was previously called

Lppe

[Input, output] pointer to a processentry32 Structure

Return Value

If the first entry in the process list has been copied to the buffer, true is returned. Otherwise, false is returned. If the snapshot does not contain process information, getlasterror will return an error_no_more_files error.

Remarks

The program that calls this function must set the member dwsize in processentry32 to the size of the structure (expressed by the number of nodes ). Process32first changes the dwsize to the number of bytes written to this structure. This value will never be larger than the initial value of dwsize, but it may be smaller. If the value is smaller, any member whose offset is greater than the value is unreliable.

To obtain information about other processes in the same snapshot, use the process32next function.

Bytes ------------------------------------------------------------------------------------------------

[Process32next]

Process32next can get information about the next process in a system snapshot.

Bool winapi process32next (

Handle hsnapshot,

Lpprocessentry32 lppe

);

Parameter description

Hsnapshot

[Input] handle to the System Snapshot returned when the createconlhelp32snapshot function was previously called

Lppe

[Output] pointer to a processentry32 Structure

Return Value

If the next entry in the process list has been copied to the buffer, true is returned. Otherwise, false is returned. If the snapshot does not contain process information, getlasterror will return an error_no_more_files error.

Remarks

To obtain information about the first process in the snapshot, use the process32first function.

Bytes ------------------------------------------------------------------------------------------------

[Processentry32]

After a snapshot is created, processentry32 describes one of a series of processes in the system address space.

Typedef struct tagprocessentry32 {

DWORD dwsize;

DWORD cntusage;

DWORD th32processid;

Ulong_ptr th32defaultheapid;

DWORD th32moduleid;

DWORD cntthreads;

DWORD th32parentprocessid;

Long pcpriclassbase;

DWORD dwflags;

Tchar szexefile [max_path];

} Processentry32, * pprocessentry32;

Member variables

Dwsize

The Structure Size expressed by the number of nodes. Set this member to sizeof (processentry32) before calling process32first ). If you do not initialize dwsize, process32first fails to be called.

Cntusage

The number of times the process is referenced. This process exists only when the number of times referenced by a process is not 0. Once the number of times it is referenced is 0, the process is terminated.

Th32processid

The ID of the process. (ID)

Th32defaultheapid

Identifier of the default heap for the process. the contents of this Member has meaning only to the tool help functions. it is not a handle, nor is it usable by functions other than the toolhelp functions.

Th32moduleid

Module identifier of the process. The contents of this Member has meaning only to the tool help functions. It is not a handle, nor is it usable by funther other than the toolhelp functions.

Cntthreads

The number of threads started by the process.

Th32parentprocessid

Create the identity of the parent process of the process.

Pcpriclassbase

Base priority of any threads created by this process.

Dwflags

Unused, retained

Szexefile

Pointer to a null string that specifies the executable file to which the process belongs.

In Windows ME/98/95, the file name contains the path.

Bytes ------------------------------------------------------------------------------------------------

[Getcurrentprocessid]

Getcurrentprocessid returns the identifier of the process that calls the function.

DWORD getcurrentprocessid (void)

Parameter description:

This function has no parameters.

Return Value

The returned value is the identifier of the process that calls the function.

Remarks

This process identifier (ID) identifies a unique process in the system until the process is terminated.

Bytes ------------------------------------------------------------------------------------------------

Our goal is to check whether the parent process of our program is ollydbg.

Plan:

1) Use getcurrentprocessid to get the process ID of our program.

2) use createconlhelp32snapshot and process32first/next to compare whether the th32processid member in each processentry32 structure is our process id until it is found out.

3) Obtain th32parentprocessid in the processentry32 structure. (Parent process ID)

4.) Use createconlhelp32snapshot to start a new comparison. However, this comparison compares the ID of each th32processid member to the parent process we obtained in step 1 until we can find it.

5.) obtain the szexefile member in processentry32 to see if it is "ollydbg.exe"

6.) If yes, we will know that our program is running under the control of ollydbg.

Sample Code:

Code :--------------------------------------------------------------------------------

. 586

. Model flat, stdcall

Option Casemap: None

Include D: \ masm32 \ include \ windows. inc

Include D: \ masm32 \ include \ user32.inc

Include D: \ masm32 \ include \ kernel32.inc

Includelib D: \ masm32 \ Lib \ user32.lib

Includelib D: \ masm32 \ Lib \ kernel32.lib

. Data

Strcaption DB "ollydbg detector! ", 0

Strfound DB "ollydbg found! ", 0

Strnotfound DB "ollydbg not found! ", 0

Strollydbg DB "ollydbg. EXE", 0 h

Valcurrentpid dd 0

Valparentpid dd 0

Hsnapshot dd 0

. Data?

Proces processentry32 <>

. Code

Start:

; Create a snapshot

Invoke createconlhelp32snapshot, th32cs_snapprocess, null

MoV hsnapshot, eax

Obtain the ID of the current process.

Invoke getcurrentprocessid

MoV valcurrentpid, eax

Lea ESI, offset proces

Assume ESI: PTR processentry32

MoV [esi]. dwsize, sizeof processentry32

; Start the first round of Search

; Use valcurrentpid to find the current process

Invoke process32first, hsnapshot, ADDR proces

Lea ESI, offset proces

Assume ESI: PTR processentry32

MoV EBX, valcurrentpid

Cmp ebx, [esi]. th32processid

JNE nope1

Nope1:

Invoke process32next, hsnapshot, ADDR proces

Lea ESI, offset proces

Assume ESI: PTR processentry32

MoV EBX, valcurrentpid

Cmp ebx, [esi]. th32processid

JNE nope1

Push [esi]. th32parentprocessid

Pop valparentpid

Invoke closehandle, hsnapshot

; Create a snapshot again

Invoke createconlhelp32snapshot, th32cs_snapprocess, null

MoV hsnapshot, eax

MoV [esi]. dwsize, sizeof processentry32

; Start the second round of Search

; Use valparentpid to find the parent process of the current process

Invoke process32first, hsnapshot, ADDR proces

Lea ESI, offset proces

Assume ESI: PTR processentry32

MoV EBX, valparentpid

Cmp ebx, [esi]. th32processid

JNE nope2

Nope2:

Invoke process32next, hsnapshot, ADDR proces

Lea ESI, offset proces

Assume ESI: PTR processentry32

MoV EBX, valparentpid

Cmp ebx, [esi]. th32processid

JNE nope2

; Extract the file name from the complete path

Lea eax, [esi]. szexefile

Push eax

Invoke lstrlen, eax

Sub eax, 11

Pop EBX

Add EBX, eax

Converts the file name to uppercase, and compares it with "ollydbg. EXE ".

Invoke charupper, EBX

Invoke lstrcmp, EBX, ADDR strollydbg

. If eax = 0

Invoke MessageBox, 0, ADDR strfound, ADDR strcaption, 0

. Else

Invoke MessageBox, 0, ADDR strnotfound, ADDR strcaption, 0

. Endif

Invoke closehandle, hsnapshot

Invoke exitprocess, 0

End start

--------------------------------------------------------------------------------

Method 3: setunhandledexceptionfilter

The article <seh in ASM> about seh in Hume has a classic discussion. It is omitted here and only lists the code, for more information, see <seh in ASM> and <seh in ASM>)

Code :--------------------------------------------------------------------------------

. 586

. Model flat, stdcall

Option Casemap: None

Include D: \ masm32 \ include \ windows. inc

Include D: \ masm32 \ include \ user32.inc

Include D: \ masm32 \ include \ kernel32.inc

Includelib D: \ masm32 \ Lib \ user32.lib

Includelib D: \ masm32 \ Lib \ kernel32.lib

. Data

Strcaption DB "ollydbg detector", 0

Strnotfound DB "ollydbg not found! ", 0

. Code

Excphandler proc

MoV eax, dword ptr [esp + 4]; eax = prediction_pointers

MoV eax, [eax + 4]; eax = Context

Assume eax: PTR Context

MoV [eax]. regeip, offset safe_address; change regeip

Pushad

Invoke MessageBox, 0, ADDR strnotfound, ADDR strcaption, 0

Popad

XOR eax, eax ;\

;) Set prediction_continue_execution

Dec eax ;/

Retn 4; normalize stack and return

Excphandler endp

Start:

Invoke setunhandledexceptionfilter, offset excphandler

MoV EBX, dword ptr [0 ffffffh]; Exception is here!

Safe_address:

Invoke exitprocess, 0

End start

--------------------------------------------------------------------------------

Method 4: API relocation

This is a method I have discovered. It is based on the processing method when ollydbg calls an API function. When the program being debugged calls the API function, Oleh yuschuk (author of ollydbg) uses the API relocation method in its Debugger for processing. The following is an example:

0401000> push asd.00403033;/filename = "kernel32.dll"

00401005 call; \ loadlibrarya

|

|

'--> 0040105c jmp dword ptr ds: [402004]; jmp dword ptr ds: [<& kernel32.loadlibrarya>]

|

|

'--> 87ff4120 push bff776d0; push kernel32.loadlibrarya

87ff4125 JMP kernel32.bff957ca

It seems that he manually loads the input table and fills the IAT manually. All the API addresses are reset to a allocated buffer. In this buffer, he calls a function in a strange way. But here is a more important question: He also imitates the getprocaddress function, so this function returns the relocated address. For example, the actual address of isdebuggerpresent in the memory is bff946f6, but this function returns the address 87ff4110 after the relocation. How can we use this to implement ollydbg detection? Well, there are many methods. The following is the simplest one:

1) load the kernel32.dll library, then it will return its base address (it is loaded to the memory location)

2) Call the getprocaddress function to obtain the exitprocess address.

3) Compare the above return value (the exitprocess address obtained by getprocaddress) with the base address of kernel32.dll.

4.) If the returned value (obtained in step 1) is greater than the base address (obtained in step 2), we can be sure that this API function is directly called. Otherwise, this API is indirectly called.

Example program:

Code :--------------------------------------------------------------------------------

. 586

. Model flat, stdcall

Option Casemap: None

Include D: \ masm32 \ include \ windows. inc

Include D: \ masm32 \ include \ user32.inc

Include D: \ masm32 \ include \ kernel32.inc

Includelib D: \ masm32 \ Lib \ user32.lib

Includelib D: \ masm32 \ Lib \ kernel32.lib

. Data

Strcaption DB "ollydbg detector", 0

Strfound DB "ollydbg found! ", 0

Strnotfound DB "ollydbg not found! ", 0

Strlibrary DB "kernel32.dll", 0

Strfunction DB "exitprocess", 0

. Code

Start:

Invoke loadlibrary, ADDR strlibrary

Push eax; eax is the base address of kernel32.dll

Invoke getprocaddress, eax, ADDR strfunction

; Eax is the exitprocess address or the address after the relocation

Pop EBX; EBX is the base address of kernel32.dll

CMP eax, EBX; Whether to exitprocess address <kernel32.dll base address

Jl olly_detected

Invoke MessageBox, 0, ADDR strnotfound, ADDR strcaption, null

Invoke exitprocess, 0

Olly_detected:

Invoke MessageBox, 0, ADDR strfound, ADDR strcaption, null

Invoke exitprocess, 0

End start

--------------------------------------------------------------------------------