Build an asynchronous file IO system from zero using C + + boost

Objective

• Because the undergraduate graduation design to do a distributed file system, one of the modules need to implement file IO. To verify my understanding of asynchronous Io, I decided to build the wheel of an asynchronous file IO. The operating system has given a variety of asynchronous operation of the API, such as overlapping IO, iocp,kqueue,select,poll,epoll and other mechanisms, and C + + also has a lot of cross-platform asynchronous IO libraries, such as Libevent,boost::asio. I refer to the existing implementation to improve the function of this small system.
• Slag technology, Slag code, there are problems please point out.

Overview

• Synchronous vs. Asynchronous:

Synchronization: If I want to handle a file (the same way as a socket), then the general process is:

　　

1 fstream file; 2 File.Open (); 3 File.read (); 4 // Do something 5 file.close ();

Typically, when this thread runs to read (), it is blocked until the file read is complete.

Asynchronous:

Or the above code, I read () through the operating system or library provided by the asynchronous mechanism, tell the operating system I want to reading a file, the data read after the execution of a function, and the current thread after the completion of the operating system to do what work, there are other things (threads do not have to wait for file IO to complete).

• Thread pool

To prevent the IO blocking thread from causing the program to be unresponsive, you can create a thread for each file operation so that you can process multiple files at the same time. But creating threads, switching threads, destroying threads is also a resource overhead, and if you want to reuse existing threads, you can use the thread pool. As a thread pool, provide at least the ability to create threads and submit tasks, a bit more complicated to intelligently control the number of threads constructor. The thread pool is used in this file system. The ThreadPool in Windows API works well, but since it's built on wheels, it doesn't matter if you build wheels for wheels, simply write a simple thread pool.

• Libraries that you use

This module only uses STL and boost two libraries.

STL mainly involves containers and fstream. Boost involves smart pointer shared_ptr, thread synchronization shared_mutex,lock_guard, path in Boost::filesystem, and some file operations, thread operations creating exits, and so on.

• Smart pointers

Since ancient times, memory management is the play of C + +, the function of smart pointers is the allocation of memory by the library management, if a smart pointer to the memory, through other smart pointers can also be accessed (that is, there are multiple references), then the smart pointer is immediately destroyed, pointing to the memory will not be destroyed Only this memory is not referenced and will be released by the library.

• Boost::filesystem

This library provides a number of APIs for cross-platform file operations, such as folder traversal, viewing properties, deleting files, and so on. The path class can record cross-platform paths.

• Because of the limited level, this module basically does not appear with the implementation of the template. (Say later)

Implementation: Talk is cheap,show me the Code.https://github.com/nemoofnemo/cluster/tree/master/filesystem

--not to be continued

Build an asynchronous file IO system from zero using C + + boost