Core Programming Learning Summary

The road to success was trial and error development, recompilation, and lots of crashes.

Winter vacation over the past one months, plans are often unable to keep up. But on the whole, it's learning and harvesting something. Things in the past cannot be changed, so there is no need to be overly tangled and remorseful. The early days of the holiday also planned to learn English systematically, my personal task is to draw the wrong time, with the wrong force, to see the time of professional English can be used to open the translation and kernel programming related development documents, so more meaningful, in the discovery of problems can not be put down, this is a bad disadvantage. So, people, there is no need to be too stubborn, sometimes to make a choice, in the realization that they are doing something very useless and boring, you should be brave to give up.

Have always wanted to enrich their knowledge of other aspects, but has not been able to implement. Knowledge usually brush, read the introduction of computer, English, philosophy did not begin. Usually do not pay attention to summary and reflection. But this is not a review, because there is no systematic work.

The following should be a good summary of the one months, the kernel programming aspects of learning work. Say is one months, but also not to, early because progress not to catch up, oneself again more stubborn, put this core work to overwhelm the back, after really should change!

Know how to give up, but also understand the importance of things, not too willful!

Keep doing without the pursuit of perfection. Jot down things that distract you, so you don't have to be distracted to think about it. Work to be single-minded, a short period of focus on a breakthrough!

Data collection Work

The collection of information is actually a very important process, which also wrote a "serious investigation is key" such a diary, to remind themselves of the importance of this work. But think about it, the idea was a little too absolute. Although I did not do a good job of research before the beginning of work, but it is not too practical to expect to find all the knowledge of related fields together. After all, in the beginning, they have not been able to understand very deeply, for many problems only met will be relatively deep, so some information is to really start work after the clear, so it will be more targeted.

So data collection is important,Keep Doing as important. No matter research, study, work or life, no one can be foreseen, a person's experience is limited, we just keep Doing, constantly groping, it is possible to come to the truth.

Misunderstanding the relationship between Minifilter and Sfilter

Misunderstanding of the relationship between Minifilter and Sfilter led to some deviations in the direction of follow-up work. I thought this was two different things, always thought that minifilter development needs to re-establish the environment, sfilter that the set is no longer applicable to minifilter. This makes me slow to carry out the related work of minifilter, that is, the environment for sfilter is not applicable in minifilter.

It didn't dawned on me until mid-March. actually minifilter just put Sfilter part of the filesystem development package, the tools used and the development package are all the same.

In addition, always think Sfilter can do things minifilter can not do, in fact, accidentally saw an article, said minifilter like can achieve sfilter function, but will reduce some compatibility, This is the same as embedding the assembly in C. In the subsequent learning process, it is very correct to find such a view. Comprehensive development time, as well as engineering volume, as well as compatibility and many other considerations, the ultimate conviction of the choice minifilter framework to develop transparent filtration system.

detours are unavoidable, but often as long as they persist, everything will be clear.

Development environment and debugging environment

WINDOWS7+WDK+VS2010+VISUALDDK, Windows7+wdk+vs2010+ddkwizard, both of which are configured in the debug environment in VS, it is said that the speed and convenience, but this is not the first approach I adopted, do not know the situation, Not much to comment on. There are also people in the VC6.0 to configure the development environment, but it is said that debugging is not very convenient.

Now the scenario is that the development environment: WINDOWS7+WDK_VS2010, Debug Environment windbg + VM virtual machine dual-machine debugging, using DbgView to view the core output, SRVINSTW used to install Sfilter services, For minifilter, the INF file is used. The use of tools is also organized on the blog.

With VS, even programs that can be compiled through the WDK will show errors, and errors of course cannot compile the. sys file, but if you change the compiled type. exe to a file type, you can remove the. sys file anyway. read some information and suggestions on the Internet, including NS3 It is also recommended to use the WDK Compile tools provided to facilitate the discovery of some VS cannot be found in the error. for ease of writing, you can write code using VS, and then compile with the WDK.

WinDbg use is not very skilled, but really is very strong, there are netizens support the use of WinDbg plus virtual machine for two-machine debugging, but not quite agree with the use of the configuration in the debug environment in VS, or feel that it is not necessary. In short, I feel that this is really a good choice for me to use now.

There are a number of options that can be adopted at some time, but should be determined as soon as possible in order to avoid wasting the matter in doubt.

Sfilter Learning is the foundation

even with the minifilter development, it is necessary to sfilter for systematic learning. after all, minifilter is the interface above sfilter. Furthermore, knowledge of the real underlying processes, messaging mechanisms, memory access control, and so on is useful for understanding minifilter. In addition, sometimes in order to achieve special functions, such as to fill out and send an IRP, or will involve sfilter related concepts.

Learning, mainly around the "Han River single-fishing Windows kernel security programming," a book carried out. The Sfilter instance code provided by Microsoft is carefully studied. The detailed content is in the study notes of "The Han River exclusive fishing". Learn the following basics:

The difference between the kernel programming environment and the application layer environment is that the kernel layer is not controlled by the operating system, and the user layer is protected and constrained by the operating system, so the kernel layer development is very difficult.

Windows Development Model:

NT (KDM), WDM, WDF (the upgraded version of WDM)

WDM: Layer-based, physical device objects (PDO) and Functional Device Objects (FDO). Fdo attached to the PDO, there will be a filter drive before, Attacheddevice point to pdo,stacksize indicates how many layers are still reaching the underlying physical device.

[Figure] Device object stack

Device hierarchy:

Vertical hierarchies: Hierarchies are primarily implemented by attaching a device to another device. The order of the device stacks is to create the underlying PDO first, create the FDO on top, and from the underlying device to the high-level device. A variety of filter drives are mixed between PDO and FDO. The lower layer finds the upper device through the Attacheddevice domain, and the upper layer finds the lower device through the device extension.

Horizontal hierarchy: Describes the relationship between device objects created by the same drive object.

[Figure] Horizontal structure of the device

Important Data Structures:

Drive Object (Driver_object ): Each driver corresponds to a unique drive object.

[Figure] Structure of the drive object

Device Object (Device_object ): A Drive object creates one or more device objects, each with a pointer to the drive object, and a pointer to the next device object to form the device chain. In kernel programming, most "messages" are passed through an IRP, and the device is the only recipient.

device Extension (device_extension ): defined by the programmer, the I/O Manager is created to store the necessary global information to avoid the use of global variables to mitigate the problem of non-reentrant.

Input Output Request package (IRP , I/O request packet ): the upper application layer communicates with the underlying driver through an I/O request, and the operating system translates the I/O request into the corresponding IRP data, which is handled by the corresponding distribution function.

[Figure] IRP package structure

The MdlAddress (pmdl) field points to a Memory descriptor table (MEMORYDESERIPTIONLIST,MDL) that describes a user-mode buffer associated with the request.

The Asociateirp field is a three-pointer union that points to a data buffer that is in non-paged memory in kernel mode.

IoStatus (Io_status_block): is a structure that contains only two domains, and the driver fills in the structure when the request is finalized.

UserBuffer for Irp_mj_device_control requests that read and write in a "no way" format, the domain contains the user-mode virtual address of the output buffer.

IRP the main processing is: According to the IRP direct operation of specific hardware, depending on IRP production of new IRP sent to other drives, the IRP is directly forwards to the lower driver.

re -entry: The re-entry means that in multi-threaded programs, multiple functions run in parallel, and the result of the function will not be different depending on the order in which the function is called.

Symbolic Connection: The device name of the driver can only be visible in the kernel state, while in user mode the device must be found by means of a symbolic link or device interface

The particularity of kernel programming:

Kernel programming Call Source: The calling source is the most initial function that a programmer can see with a piece of code. A generic single thread has only one main call source. However, in kernel programming, there may be multiple invocation sources: DriverEntry, Driverunload, callback functions, distribution functions, completion functions.

function Multithreading Security: When multiple threads call this function, they are guaranteed to be reentrant.

code Interrupt Level: mainly have pasive_level and Dispach_level, and the former interrupt level is lower than the latter interrupt level, the program running in the lower interrupt level can call the program in the advanced interrupt level, and vice versa, and cannot be implemented by forcing a lower interrupt level, you can also create a thread that runs in a low-level interrupt to complete the task. in addition, page breaks may occur when paging memory is used. Fault-pages interrupts are allowed in Pasive_level-level programs, but not in dispach_level and more advanced interrupt levels, or the system crashes, so for dispach_level and more advanced interrupt levels, Non-paged memory must be used.

Filter Concept: filtering is to add a new layer to the Windows system kernel without affecting the upper and lower interfaces, thus adding new functionality without the need to modify the upper-level software or the underlying real drivers.

Interruption: IRQ (Hardware interrupts, what hardware an IRQ comes from, to a large extent, a few defined by the user), NT (soft interrupt), causes the CPU to pause, jumps into the interrupt handler function, and the first address of the interrupt handler is in the IDT (Interrupt descriptor list).

File system:

Control equipment (CDO): CDO The main task is to modify the internal configuration of the entire drive, so generally a driver corresponds to only one CDO.

Volume device: Generally speaking, a volume corresponds to a logical disk (for example: C, D, E, F).

Normal distribution function: The IRP for a particular main function number is processed and specified in DriverEntry. Special processing is generally required to give a separate distribution function, the simplest processing is directly issued.

Fast distribution function: Handles fast I/O because the file system has fast I/O requests in addition to ordinary IRP requests, and if not handled, an error occurs.

Load function (DriverEntry): Typically, some initialization work is done, such as creating and binding devices, creating symbolic links, assigning distribution functions, and so on.

Unload function (driverunload): Complete cleanup work, such as the device's unbind, the user to open up the destruction of memory and other work.

Drivers created by the device generally have three ways to read and write:

Buffer mode: The operating system allocates memory in a section of kernel mode, copying the data from the application-supplied buffers to memory in the allocated kernel mode;

Direct mode: The operating system will lock the system in user mode. The operating system then maps this segment of the buffer to the kernel-mode address again. (MDL)

is not the way: The distribution function directly reads and writes the buffer address provided by the application.

Minifilter Learning-related basics:

Miniflter Some explanations are as follows:

Routines (Routine): I don't know what the routines and functions are different. I think routines are functions. Called routine rather than function may be to avoid other C programmers understanding too easily.

Interface (API): A programming development interface, a function that is provided to you for invocation.

Stream: NTFS file system exclusive. The extra information used to save a file. It seems that it can be opened as a file.

Field: A data member in a data structure. A person who likes a database may be called a field. Like object-oriented is called a data member.

Transparent (transparence): Invisible means that there is no need for a tube. Note, however, that transparent anti-Italian words are definitely not opaque (opacity).

Opaque (opacity): not known. such as a null pointer. Because the null pointer is pointing to something, the null pointer itself is not known. So it's called an opaque pointer.

Callback (Callback) function: A function that is called by the system and in principle you cannot tune yourself.

Pre-operation (pre-operation) callback: If you intend to filter an action, the callback appears before the operation is complete.

Post Action (post-operation) callback: If you intend to filter an action, the callback appears after the operation is complete.

Filtering Manager (Filter manager/fltmgr): A new driver developed by Microsoft's Windows kernel developer. This driver is precisely the traditional file system filter driver, but this driver provides an interface to receive some registered kernel modules. Microsoft has given the Basic IRP processing to fltmgr. It is relatively simple for users to develop according to this specification.

The kernel function definitions used by Miniflter are all in fltKernel.h

minifilter programming Framework for:

Registration and opening of micro file system filtering (Fltregisterfilter, fltstartfiltering)

Micro-filter data structure (flt_registration, flt_operation_registration)

Unload callback function (Fltunregisterfilter)

Pre-operation callback function: represents minifilter before processing I/O operation completes

Flt_preop_callback_status

Npprecreate (

__inout Pflt_callback_data DATA,

__in pcflt_related_objects Fltobjects,

__deref_out_opt PVOID *completioncontext

)

Post Action callback function: Represents minifilter has completed all processing of I/O and returns control to the filter manager.

Flt_postop_callback_status

Nppostcreate (

__inout Pflt_callback_data DATA,

__in pcflt_related_objects Fltobjects,

__in_opt PVOID completioncontext,//User customizable, incoming context from pre-operation

__in Flt_post_operation_flags FLAGS

)

Other callback functions (the following callback function is not required):

Instancesetupcallback (): The main purpose of this callback function is to allow the kernel programmer to decide which volumes need to be bound and which do not.

Instancequeryteardowncallback (): Controls the instance destruction function, which is called only when a request is manually unbound.

Instanceteardownstartcallback (): Unbind the control function for the instance, and when called, indicates that it has decided to unbind.

Instanceteardowncompletecallback (): The completion function that is unbound for the instance.

Minifilter and the Application Communication module (which can be made into DLLs), the relevant user layer API is defined in # include <FltUser.h>.

Installation of minifilter (using INF file).

Sample code--"Han River exclusive Fishing":

Minifilter: Kernel code, compiled with the WDK compilation environment, gets the. sys file.

Minifilter_app: interface with user, compile get exe, "vs Compile Pass"

Minifilter_dll: interface with the kernel program, for Minifilter_app compiled exe dynamic load call, to implement and kernel program communication, compile the. dll file. "vs compilation Passed"

Place the EXE and DLL in the same directory, then use the INF to install the. sys file, and after starting the service, run the EXE to communicate with the kernel layer's. SYS service.

————————————————————————————————

Other:

ULONG gtraceflags = 0;

#define PT_DBG_PRINT (_dbglevel, _string) \

(Flagon (Gtraceflags, (_dbglevel))? \

Dbgprint _string: \

((void) 0))

Pt_dbg_print (Ptdbg_trace_routines,

("npminifilter! Npinstancesetup:entered\n "));

Flagon is not found in the current document, I think it should be found in the Sfilter source code, because Minifilter uses the filter manager is based on sfilter implementation. See Sfilter source for answers later.

————————————————————————————————

The definition of Flagon has been found in the Ntifs.h header file:

//

These macros is used to test, set and clear flags respectivly

//

#ifndef Flagon

#define FLAGON (_F,_SF) ((_f) & (_SF))

#endif

————————————————————————————————

The callback packet structure represents an I/O operation. Both the filter manager and the micro-filter driver Use this structure to initialize and process I/O operations, including many nested structure definitions, which can find more data in the WDK header file fltkernel.h.

————————————————————————————————

Like legacy filter drivers, minifilter drivers attach in a particular order. However, the order of attachment is determined by a unique identifier called an altitude. The attachment of a minifilter driver at a particular altitude on a particular volume are called an instance /c2>of the minifilter driver.

The Filter Manager provides the following support routines for explicit load and unload requests, which can is issued from User mode or kernel mode:

Filterload

FilterUnload

Fltloadfilter

Fltunloadfilter

The following routines is used to register and unregister callback routines for instance setup and teardown:

Fltregisterfilter

Fltstartfiltering

Fltunregisterfilter

It is thought that these APIs can be called through either the kernel or the user layer to protect the current minifilter process (that is, transparent cryptographic processes), and in the process monitoring process, if the discovery process is closed, another process invokes the associated API to restart it. In addition, the server side can control the client's miniflter process through these APIs.

————————————————————————————————

Sysinternals Company's file system monitoring tool source code

Using Filemon

-------------

Start Filemon.exe from its home directory. Complete usage

Instructions is available in the on-line Help file.

See Sysinternals for more monitoring tools, including

A Registry monitor.

Building Filemon

----------------

Fielmon consists of a device driver and a GUI. The NT driver is built with

The Windows NT DDK Build Environment and the GUI were compiled with

Microsoft Visual C + + 6.0. The VxD was built with NuMega ' s

(www.numega.com) VTOOLSD 2.0.

The help is entered with Microsoft Word, and help Workshop

For Word 2000.

To install, copy Filmon.exe, Filemon.vxd and Filemon.sys to the same

Directory.

(The Filemon.vxd isn't necessary that has been confirmed. But the procedure isn't careful enough so, I have not found any abnormal.)

Filmon is actually a file-monitoring program. The operation of each request is listed in detail and can be written according to Filmon.exe's source code to match its own UI interface. Which works with VC open--late

Not very detailed, but the effect of its work and from "see Snow" up and down the FileMonitor function almost

Issues with secure strings

Debugging output Setup Tips

Global variable structure: Globals

In the sample code given by Microsoft, global variables are stored using a custom global structure. This facilitates a unified management of all variables, a look at how many global variables, feel good, you can follow.

WinDbg Commissioning

Today engaged in an afternoon, WinDbg debugging, the old thought is the virtual machine crashes. In fact, it is because WinDbg automatically jumps to the stop state, which means that the virtual machine stops (that is, ctrl+break) effect. Should use go to make the virtual machine run up, the virtual machine can move, has been thought to be a virtual machine crashes, now just shake God ... Continue to refuel tomorrow!!

Some resource sites

Everything Windows Driver DEVELOPMENT,OSR online:http://osronline.com/

See Snow:

http://bbs.pediy.com/

Msdn:

http://blogs.msdn.com/b/alexcarp/

Microsoft website:

https://msdn.microsoft.com/en-us/library/windows/hardware/ff540402 (v=vs.85). aspx

CODE PROJECT:

http://www.codeproject.com/

Drive Network:

Http://bbs3.driverdevelop.com/thread.php?fid-39.html

There are some articles in CSDN, blog Park and so on. Will Baidu, will Google can also find.

Core Programming Learning Summary