People are very concerned about the speed of download, for users of non-monthly broadband and use of paid CDN, this is a typical take money to buy time behavior, I pay the higher the cost, the faster you want to download, the greater the cumulative bandwidth used. About a variety of speed measurement method is also voluminous, this paper introduces the speed of TCP transmission.
Overview of TCP Rates first, the TCP rate is affected by multiple delays, including:
1. The processing time delay of the machine and the end machine refers to the transmission and receiving host due to operating system scheduling, interruption, network card packet scheduling and other processing introduced delay, basically belongs to the scope of the operating system, if a TCP packet can be sent (window enough to accommodate), However, because the CPU is switched to another process by the operating system caused the delay, or the packet has indeed been sent, but because the network card performance is very low, it is necessary to copy the contents of the packet byte by bit, this also introduces additional delay, this part of the time should be counted in the host processing delay.
2. Intermediate network Equipment Processing delay This is the intermediate device "host processing delay", and the end of the host processing delay similar, the difference is that this part of the delay is the most queuing delay, compared to queue delay, router or switch processing delay can be ignored, the current backbone
Network equipment PPS Line speed are ultra-high, if you want to optimize those devices of the forwarding, routing and other logic, that is not find the right place, now is the line card hardware fast forwarding, Exchange network processing is nanosecond, if you want to optimize the route forwarding, then go to Huawei to do light cat department for a job, There will be a talent for you (no ads!). )。 Again, the processing time delay of intermediate network equipment is mainly referred to as queue delay.
3. Transmission delay of the intermediate network this part is not much to say, we know that the speed of light, but also has a value, that is, the light from one place to another place also takes time, so the data in the carrier through the media from one host to another host or intermediate node also takes time, This part of time is the network transmission time, it is mainly affected by crosstalk, refractive index, optical dispersion and other physical factors.
Knowing the above factors, what should we do if we want to measure the data rate of a TCP connection?
The theory is very simple but very pale! The total amount of data divided by the time. But the question is, at what level do I measure the rate and what precision do I need? If I want to know the processing rate of my host, then time refers to the processing time delay of the machine, the amount of data is in the applied through the amount of data, more generally, if I want to know a file from a Web server to download the rate, I need to use the total file size divided by the download time required to complete, This part of the time includes the sum of the three types of delay mentioned above.
This involves the problem of dot sampling, which is essentially two problems, first, where to start the sampling, where to end the dot sampling, and secondly, how to sample.
Application Layer Download Speed 1. Average moving index of instantaneous velocity open the browser, download a file, various download tools or the browser comes with the download will show a download speed and time remaining, how these values are obtained? Very simple, the download tool or the browser itself is an application, it can be in-house data sampling, such as fixed interval, and then statistics of the amount of data received at this interval, the two divide to get the rate, as for the remaining time, generally through the protocol, such as the response of the HTTP protocol generally have this information, Then dividing the remaining size by the current rate is the time remaining. It's so simple!
The problem is that the current rate is really simply dividing the amount of data at the fixed sampling interval by the time interval? Not too! In fact, most of the download tools and browsers are using the moving exponential averaging algorithm, as far as I know, the earlier IE did not use the algorithm, but the use of instantaneous sampling values directly divided by the time this, you will see with the early IE download files, the remaining time of the jitter is particularly large, a few minutes, a few hours a moment.
I will not write the formula, it is troublesome, a reference to the mathematical formula, understand people do not answer you, do not understand the people said you installed, also very big head.
2. The accurate measurement of the average velocity above is the method of measuring instantaneous velocity, if you want to count the average speed of a download? In fact, this statistic is made after the download is complete, so at this point I have enough information to calculate the average speed, the total size of the downloaded file I have, the total time I can be counted by the HTTP protocol, the two divides is the final answer.
In general, like the HTTP protocol, the response head will have the size of the file bytes, the application receives so many bytes to end the sampling, it can be counted to download so much data used time. This is entirely the credit of the application layer protocol like the HTTP protocol, which tells me enough information.
It's hard to count these in the kernel stack! Because the kernel sees only one TCP stream, unless you receive fin, you don't know when it's going to end, and you don't know how much data will be transferred in the first ...
protocol stack transmission speed before the time encountered a demand, said to be able to at the network card level to reach a certain port traffic, I said can not ... I'm not lying, even if I can do it, I don't want to do it.
1. Instantaneous speed moving index average the instantaneous speed of the download tool is visible, and people are more concerned about the amount of time left to be calculated through this instantaneous speed, so that people have an expectation that there is time to buy a few cans of beer downstairs at a convenience store before a movie is downloaded. What's the use of TCP for this speed?
The instantaneous speed at the TCP stack level is used to guide the TCP sending strategy itself, and we know that TCP is a feedback system that is involved in the whole world (which sees it as a world-wide chaotic system affected by the butterfly effect) that requires feedback from the network to guide its future behavior. I used to bother to say how the ACK clock affects the TCP sense of congestion, and so on, admittedly those are not wrong, but in this article, we go up a level, look at the ACK of the amount of data confirmed and the common role of RTT. This function responds to the TCP transfer rate. We need a rate to represent a trend, and then use this trend to guide the TCP Congestion window adjustment, which requires smoothing out the rate of noise, so again we encounter the moving index average!
Let's take a look at how the instantaneous velocity is measured.
You can use the same algorithm as the user-state method to count the amount of data received within a fixed interval, but in the protocol stack, we have a better algorithm, which is because we can help with the ACK. We can describe the algorithm by:
Assuming the ACK is smooth at a uniform speed, we need to prove that:
This proves to be very easy:
After proving the equation above, we also have a method for calculating the instantaneous rate:
1. Arbitrarily take a point set to T1, get the amount of data that was sent out but not yet confirmed, and the associated serial number
2. When an ACK is received, it is recorded as T2, calculating the amount of data for this ACK acknowledgment, minus the amount of data confirmed by the first step
3. Iterate over the above procedure each time an ACK is received, doing the moving index average
Well, we've simply done the calculation of the instantaneous rate value by the ACK clock flow. Using this can guide a lot of things, but not in the scope of this article.
2. Accurate measurement of the average rate in the previous section we took care of the instantaneous velocity, but we lost some of the information because we made the moving index average smooth noise, so what if we need a precise average speed?
In other words, instantaneous speed is a guide to the behavior of the connection, the trend of the instantaneous value can be learned in the connection, and reaction to the subsequent rate adjustment, it reflects the current network state and the TCP connection to the network state of the response, but the average rate of response is a long-term behavior, For example, we can understand the general condition of the link by analyzing the average rate. This average is more statistically significant for long-term machine learning behavior.
As with the average speed measurement of the application layer, it is easier to measure than the instantaneous value, but we need to change the idea, because no personnel first tells you how much data to download. In TCP, we can later count that the serial number at the time of Fin minus the initial serial number of the connection is the number of bytes sent. Generally speaking, the test average speed is calculated two, one is the total amount of data, one is the total amount of time, but the two divide:
Note that the denominator is a small segment of the time plus, rather than just taking a time period, which is why? Because for TCP, it has no data to send is the application decision, many applications just keep a TCP connection, and then transfer a portion of the data at the appropriate time, the middle of the idle interval, should not be counted into the transmission time. The question now is how to figure out the amount of data for each segment. We're going to be one by one.
First, the calculation of the addition of molecules is simple, is nothing more than fin when the serial number minus syn-ack when the initial sequence number, the difficulty lies in the denominator of time slices plus. In fact, it is not difficult, if only theoretical analysis, I think it can be finished, but we must achieve it, the difficulty lies in your understanding of the protocol stack code depth, if you are familiar with the code, it is rinsed, but do not go biased, do not let others influence your judgment, Very simple thing many people have no solution to the discussion. I'll give you the answer first:
1). When the application layer data enters the TCP send queue, Judge
If the queue length is 0 at this point, start a new round of timing, or do nothing.
2). Ack arrives to clear the TCP send queue when the judgment
If the ACK clears the send queue, ends the last round of timings, counts the time difference, and adds up the time difference.
Then we look at how the code is implemented, very simple code no more than 20 lines, except debug information, less! The code is as follows:
When the TCP client Scocket is created, initialize the field I added several fields in the Tcp_sock structure, and I did not paste the code, give the function directly.
struct sock *tcp_v4_syn_recv_sock (struct sock *sk, struct sk_buff *skb, struct request_sock *req, struct dst_ Entry *dst) { ... INET_CSK (NEWSK)->icsk_ext_hdr_len = 0; #if 1 //Initialize time slice difference field newtp->vstamp = ktime_set (0, 0); Initialization time summation and field Newtp->pvstamp = ktime_set (0, 0); Record the first serial number NEWTP->FIRST_SEQ = newtp->snd_nxt; #ifdef real_time_debug //In order to compare the difference between pure transmission time and connection time, Introduce a field record connection time start Newtp->first = 1; The following two fields in order to confirm the "queue/empty" conservation newtp->pri1 = 0; Newtp->pri2 = 0; #endif #endif ... }
When the data enters the sending queue, the TCP data is unified via skb_entail into the send queue
static void Skb_entail (struct sock *sk, struct Sk_buff *skb) { ... Tcb->flags = tcpcb_flag_ack; tcb->sacked = 0; #if 1 //If the Send queue is empty, start a new round of timing if (tcp_write_queue_head (SK) = = NULL) { Tp->vstamp = Ktime_get_real (); #ifdef real_time_debug tp->pri1 + +; If the data was first sent, the time if (Tp->first = = 1) { Tp->bstamp = tp->vstamp; Tp->first = 0; #endif } } #endif ...}
When an ACK is received, detects whether the send queue has emptied the ACK of the TCP data uniformly through Tcp_clean_rtx_queue cleanup
static int tcp_clean_rtx_queue (struct sock *sk, int prior_fackets, u32 prior_snd_una) { ... while ((SKB = Tcp_write_queue_head (SK)) && SKB! = Tcp_send_head (SK)) { } #if 1 //If the queue has been emptied, Counts the time difference between queuing from the first packet to emptying the queue and accumulating it on the counter. if (tcp_write_queue_head (SK) = = NULL && tp->vstamp.tv64! = 0) { ktime_t delta = Net_timedelta (tp-> Vstamp); Tp->pvstamp = Ktime_add (Tp->pvstamp, delta); Tp->vstamp = ktime_set (0, 0), #ifdef real_time_debug tp->pri2++; #endif } #endif ...}
4. When TCP shuts down the connection, the output I think this thing is no use at present, so just simple printk out to see can, so do not wear leather shoes:
void Tcp_v4_destroy_sock (struct sock *sk) {struct Tcp_sock *tp = Tcp_sk (SK); #if 1 struct Inet_sock *inet = Inet_sk (s k); U32 delta_seq = tp->snd_nxt-tp->first_seq-1; TSval is a pure data transfer when the struct Timespec tsval = Ktime_to_timespec (Tp->pvstamp); Ttot is the total time from the beginning of the data transfer to the end of the connection in a TCP connection struct timespec Ttot = Ktime_to_timespec (Ktime_sub (Ktime_get_real (), tp->bstamp)); #ifdef real_time_debug printk ("Saddr:%pi4 sport:%u dport:%u tot_byte:% #x time:%lu%09lu [%lu%09lu],%u%u\n", &inet->saddr, Ntohs (Inet->sport), Ntohs (Inet->dport), Delta_seq, (unsigned long) tsval.tv_sec, (unsigned long) Tsval.tv_ns EC, (unsigned long) ttot.tv_sec, (unsigned long) ttot.tv_nsec, tp-& Gt;pri1, tp->pri2); #endif #endif}
OK, let's take a look at the test results.
Test
If you use Curl,ab,wget to download a file on your own Web server:
Client:
[email protected] ~]# Curl HTTP://1.1.1.2/BIG10 >/dev/null
% total% Received% xferd Average speed Time Time current
Dload Upload Total spent
9765k 9765k 0 0 1210k 0 0:00:08 0:00:08--:--:--1210k
DMESG information on the Web server:
saddr:1.1.1.2 sport:80 dport:60463 tot_byte:0x989783 time:8 061474566 [8 062002622], 2 2
It can be seen that the client statistics information, the service side of the connection length, the transmission length of the three are the same, but the browser download this BIG10 test, I also give the Web server DMESG output:
saddr:192.168.44.100 sport:80 dport:61023 tot_byte:0x9897bc time:8 110491001 [156020871], 4 4
Why is the connection time and data transfer time so much worse? I think this is part of the time that the so-called browser action consumes. To make sure, I modified the AB.C and added a sleep before the Write_request function (5)
The results are the same as the results of a browser visit. In order to confirm in a more elegant way, I also specially consulted the former company's colleague (Web expert, high performance server expert, PHP master, interested can contact me), he told me a very useful test method:
[[email protected] ~]# Curl HTTP://1.1.1.2/{BIG10,SLEEP.PHP,BIG10} >/dev/null
% total% Received% xferd Average speed Time Time current
Dload Upload Total spent
9765k 9765k 0 0 1214k 0 0:00:08 0:00:08--:--:--1213k
0 0 0 0 0 0 0 0--:--:--0:00:05--:--:--0
9765k 9765k 0 0 1215k 0 0:00:08 0:00:08--:--:--1213k
The sleep.php in the command are as follows:
<?phpsleep (5);? >
The server-side DMESG output is as follows:
saddr:1.1.1.2 sport:80 dport:53362 tot_byte:0x1312fb2 time:16 067200380 [069762118], 5 5
It can be seen that the connection time is 5s more than the transmission time, which is a controllable test. I wanted him to write a bigger php for me afterwards, and he ignored me. I think it's too simple, it's really a profession, interlacing as Foster.
Written in the last enough, and now has basically completed the article, who see the later to use it. There are a lot of things you need to think about, not everyone, but the premise is that you already have a 0.1 version, and if not, the discussion is nonsense!
Although I prefer math, physics and other theoretical things, I prefer show me the code, not document. You do not understand, I do not understand, he also does not understand, everybody together a chit chat when, everybody all understand, this is the story. To be clear, the above-mentioned things are discussed when we were not dropped, but afterwards calmly think, not ah, test shows all OK ah, in the end who is wrong, I do not want to let the idea, wrote this article ... I am not against brainstorming, but more agree with expert brainstorming, otherwise, everyone is full of excitement, everything wants to challenge others everywhere, jianfengchazhen, I think this is not malicious, but the atmosphere caused.
The person who did not want to participate is just a little comment, the result is all started mad spray Ah spray, meaningless, no results, reason is fading, victory is the key, as long as someone proposed a plan, there will be people deny, nothing is wrong, nothing wrong, but the fact is not so bad. The leaders of various countries in the performance of the press conference I think everyone has seen, the best way is to mouth shut, or will be angry people throw shoes. In a sense, "challenge" is a derogatory word, it will disturb your thinking, confuse your audio and video, let you in order to deal with the challenges of the disorder, and then after the meeting, 80% of the people in the collation of their ideas will find themselves not wrong.
Finally, I would like to say a few questions about avoiding challenges, if someone does not understand the details of you, then someone will ask, if you face him without gas field, then you will undoubtedly be introduced to his ideas, and then you are completely disorderly, everyone has had this experience, I try to cite a few examples:
1. During the interview
If you are not an expert in any way, don't talk nonsense, or you will be very embarrassed to be asked. The interviewer is right in front of you, maybe he doesn't understand, but the gas field is stronger than you, what do you do?
2. When dealing with the project Xianzhanhouzou
My personal experience, a few years ago, I think the way of leadership is not right, I work according to their own ideas, fortunately, I succeeded. But the secret to success is to quickly take out your version 1.0, rather than talk to him about the program. The emperor asked the general to meet to discuss how to arrest, the general to avoid the meeting, and leave without saying goodbye heard, the best way is to bring the enemy will come back, all say.
3. When the wife checks the phone
is also my personal experience. When the phone was checked, even if nothing happened, I still feel very flustered, this expression was seen by the wife, the constant challenge began, in fact, I did not do anything, but the answer is still slightly incoherent, this is why? Am I wrong? No! The key is in the gas field! Gas field is not enough, yes also become wrong, the other side will say, no problem why do you nervous!
4. When borrowing money from others
This topic does not say much, borrow once money you will understand.
Finally, it is 2016/06/25 07:53, I began to write this from 3 o'clock in the morning, and now, in the middle of a number of calculation process, but also pulled a doesn't excrement. Why do I get up so early, in fact, I would like to get up 5:30, helpless last night, the daughter of my three feet, a foot in the belly, a foot in the temple, a foot in the ribs ... I can't stand it, I just get up in the middle of the night and write something ... Explosion!
A measurement method of TCP transfer rate
