& Lt; Simple setup based on Qt and POSIX Threads & gt; multi-thread download tool, qtposix

<Based on Qt and POSIX Threads> simple setup of multi-thread download, qtposix

For original blog reposted, contact the blogger!

This project has been hosted in my Git remote Library:Https://github.com/yue9944882/Snow

 

 

 

 

 

Project objectives

Major Functionality

 

Development Environment:CentOS7-Qt4

 

Implement a LINUX-based multi-thread download device with the following functions:

 

(HTTP only) multi-thread Remote resource download

(Pause/resume) resumable upload

 

 

 

Main technologies of the project

Major Technique

 

 

POSIX Threads and their collaboration

TCP socket programming

Qt interface implementation

Qt signal slot mechanism SIGNAL/SLOT

// Linux Signal Processing

 

 

 

Project Conception

Major Architecture

 

 

Throughout the use process, the main interface of Qt operates on several global queues to implement system collaboration. In the process, if a synchronization lock is used to allocate threads, collaboration and competition are implemented.

 

Why is this design?

 

The biggest reason for this design is to achieve interaction between POSIX Threads and Qt encapsulation classes. POSIX Threads are implemented based on the C language in LINUX, the entrance to the creation of the call must be a C-style declared function. If these functions are declared as static functions in the Qt control inheritance class, their encapsulation will be damaged (I personally prove that, ). After several complete failures, this method is also one of the few feasible solutions.

 

Why use POSIX Threads?

 

Qt is a portable project environment. If you use the FORK/VFORK series functions that are unique in linux, it will restrict the running environment. threads under POSIX are more common and more extensive. As to why QThread is not used, I can only say that I don't want to use it--|

 

 

The following figure shows the relationship between the overall modules:

 

 

Shows the architecture of the network and dynamic display controls:

 

 

 

The logical task thread does not directly participate in the download. The affiliated thread is encapsulated in the logical task and invisible to other tasks. The dynamic control queue changes with the user's operation length and sequence, while the logic download task queue will only grow continuously, and one-to-one matching with dynamic controls through indexes.

 

The application class architecture of the Qt main interface is defined in the *. ui file corresponding to it. I will not repeat it here. Communication between classes is achieved through the SIGNAL/SLOT SIGNAL SLOT Mechanism of Qt.

 

Shows the POSIX lock architecture:

 

 

--- Declaration ------ global. h: extern declares external global variables

|

| --- Global_t.h: C-style struct definition, mainly used for parameters passing by POSIX Threads

|

| --- Global_f.h: extern declares an external global function

|

| --- Missionbar. h missioncheck. h: C ++ style declaration Dynamic Control

|

| --- Mainwindow. h newdialog. h: C ++ style declaration Static Control

|

|

|

--- Defination ----- global. cpp: global variable definition

|

| --- Global_f.cpp: POSIX thread and log system-C-Style Function Definition

|

| --- Main. cpp: program entry, program environment initialization ..

|

| --- *. Cpp: class definition inherited from the Qt Library

|

|

--- Makefile ------ SnowLINUX. pro: QMake script

|

--- Makefile: automatically compile scripts

 

 

 

 

Code Implementation

Implementation

 

Network download

 

1. TCP socket:

 

Because the multi-thread download device needs to ensure file integrity, we select a socket under the TCP protocol for download, each download first sends a HTTP-HEAD request to get the length and full name of the file and declares that the gzip format is not used for compression, otherwise multi-thread writing will not work! After the complete length of the file is obtained, sections are divided based on the number of threads, and several POSIX Threads use the random long port number and port 80 of the server for TCP connection, then, use the interfaces provided by the Linux kernel for download. Each task maintains an update and write process lock to update the real-time information of the current progress and speed, and compete for the global lock to refresh the global statistical variables.

 

 

2. lockless data structure:

 

The Linux atomic file read/write operation is not a standard file operation to ensure the atomicity and integrity of multi-thread writing. The pwrite/pread function is our final choice. Due to the read/write atomicity, file descriptors do not require lock classes to protect synchronization.

 

 

3. Log System

 

The log system was originally designed to use the Linux-signal library for scheduled task log updates, but this brought too many additional task consumption to the CPU and reduced the download speed to varying degrees, more importantly, this will undermine the encapsulation of the Qt inheritance class, because the signal_handler-style function must be a global function. We use a single log to trigger the logging system when the user needs it. For the relevant functions, see the following in global_f.cpp:

 

Init_log (), write_log (), read_log () series functions. The log format is not described in detail. It is a pure ASCii encoded file.

 

 

 

Qt and global lock

 

1. Global Competitive lock:

 

TimeLock(TimeMutex) is used to regularly call functions on the main interface to obtain the execution time of tasks that continuously update progress asynchronously in the task queue. the time lock is used to ensure data synchronization between the main interface and dynamic controls, the data of the dynamic control is locked by the dynamic control to ensure the synchronization of the Data refreshed by the internal POSIX thread.

 

Table lock (tableMutex) is used to ensure that the global table modification is caused by the deletion/restart task. At the same time, the program will not crash because of the timed refresh curve and the wrong access to the task table.

FinishMutex is a program crash caused by modification of the table content when the dynamic control and the task table access each other by recording the index of the other party.

 

2. Global Collaboration lock:

 

The small QDialog window popped up when a new task is started is an independent window. The SIGNAL of its internal space is bound to the SLOT of the main interface and the SLOT of the dynamic control respectively, however, the slots on both sides are called at the same time by default, but we must require their order, or the program will crash. For example, after selecting the file path and starting the download task, we need to call the slot on the main interface to update the table for preparation before creating dynamic controls and downloading tasks.

 

 

 

 

 

 

Program operations

Demostration

 

The operation method is very concise: