The entire process from the input URL to the page load

Previous words

This article will detail the entire process from input URL to page load

Overview

The skeleton flow from the input URL to the page load is as follows:

1. Browser Building HTTP REQUEST requests

2. Network Transmission

3. Server Building HTTP Response response

4. Network Transmission

5. Browser Rendering page

Build Request

1, the application layer for DNS resolution

Resolves the domain name to an IP address through DNS. During the parsing process, the cache is read in the order of,,,,, and, 浏览器缓存 系统缓存 路由器缓存 ISP(运营商)DNS缓存 根域名服务器 顶级域名服务器 主域名服务器 until the IP address is received.

DNS pre-parsing is used here, can be based on browser-defined rules, pre-resolution may be used after the domain name, so that the parsing results cached 系统缓存 in, shorten the DNS resolution time, to improve the site's access speed

2. Application layer generates HTTP request message

The application layer then generates an HTTP request message for the target Web server, which includes the start line, header, and body part

If the google.com is accessed, the starting line may be as follows

GET https://www.google.com/HTTP/1.1

The header includes the domain host, keep-alive, User-agent, accept-encoding, accept-language, cookies and other information, possibly as follows

 host:www.google.comconnection:keep - Aliveupgrade -insecure-requests: 1  user -agent:mozilla/5.0  (Windows NT 10.0 ; Win64; x64) Applewebkit/537.36  (khtml, like Gecko) chrome/ 65.0 . 3325.181  safari/accept:text /html,application/xhtml+xml,application/xml;q=0.9 , Image/webp,image/apng,*/*  ;q= 0.8x-client-data:ckm1yqeihbbjaqijtskbcmg2yqeiqz3kaqioo8obaccept-encoding:gzip, deflate, BRACCEPT-LANGUAGE:ZH-CN, zh;q=0.9,en;q=0.8  

There is a carriage return (CRLF) between the header and the main content, and the subject content is the content to be transferred. If it is a GET request, the principal content is empty

3, the transport layer to establish a TCP connection

Transport Layer Transport protocol is divided into two types of UDP and TCP

UDP is a non-connected protocol, and TCP is a reliable and connected protocol, mainly in the following: The receiver will confirm the received data, the sender will retransmit the receiver unacknowledged data, the receiver will receive the data in the correct order reordering, and delete duplicate data, providing a mechanism to control congestion

Because the HTTP protocol uses the TCP protocol, in order to facilitate communication, the HTTP request message is divided into multiple message segments (segment), and each packet is encapsulated. Using a local random TCP source port greater than 1024 (this is assumed to be 1030) established to the destination server TCP80 port (HTTPS protocol corresponding port number is 443) connection, TCP source port and destination port is added to the message section, called the Protocol Data Unit ( Protocol Data Unit, PDU). Because TCP is a reliable Transmission Control protocol, the transport layer will also include the serial number, confirmation number, window size, checksum and other parameters, a total of 20 bytes of header information added

The TCP protocol is connection-oriented, so it needs to establish a connection before it can begin transmitting data. To establish or initialize a connection, both sides of the host must synchronize the initial sequence numbers. Synchronization is done by establishing data fragmentation and initial sequence numbers through the interchange connection, including a SYN (synchronous) control bit in the connection creation data segment. Synchronization requires both parties to send their initial sequence numbers and to send an ACK acknowledgment. This process is a three-time handshake.

First handshake: Host A to Host B, host A's initial sequence number is x, set the SYN bit, no ACK bit set

Second handshake: Host B to host A, Host B's initial sequence number is Y, confirmation number (ACK) is the x+1,x+1 confirmation number implies that the host a sent to Host B synchronization sequence number. Set the SYN bit and ACK bit

Third handshake: Host A to Host B, the serial number of host A is x+1, the confirmation number is the y+1,y+1 confirmation number implies that Host B has received a synchronization sequence number to host A. Set ACK bit, no SYN bit set

The three-time handshake solves not only the serial number problem, but also solves other problems including window size, MTU (Maximum transmission Unit, Maximum transmission unit), and expected network delay.

Building TCP Requests can add a lot of network latency, and the usual optimizations are as follows

(1) Resource packaging, merging requests

(2) More use of cache, reduce network transmission

(3) using keep-alive to establish a persistent connection

(4) Use multiple domain names to increase the number of concurrent loads of resources in the browser, or use HTTP2 's pipelined connection multiplexing technology

4, the Network layer uses IP protocol to select the route

Processing the data segment from the Transport Layer segment, segment the data segment into the packet packet, filling the header, the main is to add the source and destination IP address, and then send the data. In the process of data transmission, the IP protocol is responsible for selecting the route to be routed, called the routing function

5. The data link layer realizes the reliable data communication between adjacent nodes in the network.

In order to guarantee the reliable transmission of data, the packet packet is encapsulated into frame, and the frames are transmitted sequentially. Because the physical circuit is unreliable, the data frame emitted may be error or lost on the line, the CRC (cyclic redundancy test) is computed for each chunk of data, and the CRC is added to the frame, so that the receiver can determine the correctness of the data reception by recalculating the CRC. Re-transmit once it's wrong

Encapsulates a packet packet into a frame, including the frame header and the end of the frame. The end of the frame is the part of the cyclic redundancy check that adds the called CRC. The frame header is the address that adds the data link layer, namely the source address and destination address of the data link layer, namely the source MAC address and the destination MAC address between neighboring nodes of the network.

6, Physical layer transfer data

The frame of the data link layer is converted into a binary form of bit (bit) stream, sent from the network card, and then converted to electronic, optical or microwave signal transmission in the network

Summary

The above 6 steps can be summarized as follows: DNS resolves URL address, generates HTTP request message, constructs TCP connection, chooses transmission route using IP protocol, Data link layer ensures reliable transmission of data, physical layer converts data into electronic, optical or microwave signal for transmission

Network transmission

From client to server requires many network devices, typically including hubs, switches, routers, etc.

"Hub"

A hub is a physical layer device, and after the bitstream reaches the hub, the hub simply amplifies the bitstream and forwards it from all ports except the receive port

Switch

The switch is a data link layer device, the bitstream arrives at the switch, the switch in addition to the bit stream amplification, but also according to the source MAC address to learn, according to the destination MAC address for forwarding. The switch polls the MAC address table based on the destination MAC address in the data frame, sending the bitstream from the corresponding port

Router

The router is the network layer device, the router receives the bitstream, converts the frame to the data link layer, the router compares the destination MAC address of the data frame, and if there is the same MAC address as the router receive port, the router's data link layer encapsulates the data frame and then uploads it to the router's network layer. The router locates the destination IP address of the packet and queries the routing table to forward the data from the inbound port to the out port. Then the network layer is repackaged into packet packet, sinking to the data link layer to re-encapsulate frames, sinking to the physical layer, converted into binary bitstream, sent out

Server processing and reverse transfer

The server receives this bitstream, converts the bit stream into frame format, uploads to the data link layer, the server discovers the destination MAC address in the data frame is the same as the MAC address of this network card, and the server removes the package of the data link layer and uploads the packet to the network layer. The network layer of the server compares the destination IP address in the packet and discovers that the IP address of the machine is the same, and the server removes the encapsulation of the network layer and uploads the data to the transport layer. The transport layer confirms, sorts, and reorganizes data segments to ensure the reliability of the transmission. Data is finally uploaded to the application tier of the server

An HTTP server, such as Nginx through a reverse proxy, locates it to the actual port location of the server, such as 8080. For example, the 8080 port corresponds to a Nodejs service, generates a response message, the message body content is the HTML page of the Google homepage

Then, through the Transport layer, network layer, the layer of Data Link layer encapsulation, the response message is eventually encapsulated into binary bitstream, and converted to other signals, such as electrical signals to the network transmission

The process of reverse transmission is similar to the process of forward transmission, and it is not

Browser rendering

After the client receives the binary bitstream, it converts the bit stream into frame format, uploads the data link layer, and the client discovers that the destination MAC address in the data frame is the same as the MAC address of the network card, and after removing the package of the data link layer, the packet is uploaded to the network layer. The network layer compares the destination IP address in the packet, discovers the same IP address as the native, and then uploads the data to the transport layer after removing the encapsulation of the network layer. The transport layer confirms, sorts, and reorganizes data segments to ensure the reliability of the transmission. Data is finally uploaded to the application layer

1. If the HTTP response message is 301 or 302 redirect, the browser will send the request again in the corresponding header

2. The browser handles the main contents of the HTTP response message, and first uses the loader module to load the corresponding resources.

The loader module has two resource load paths: The main resource load path and the derived resource load path. The master resource is the index.html file for the Google home page, and the derived resource is the resource used in the index.html file.

When the master resource arrives, the browser's parser module parses the contents of the master resource, generates the DOM structure corresponding to the derived resource, and then triggers the load flow of the derived resource as required. For example, during parsing, if an IMG start tag is encountered, the corresponding image element htmlimageelement is created, and the Htmlimageelement property is set based on the contents of the IMG tag. When the SRC attribute is set, the picture resource load is triggered, and the load resource request is initiated

The common optimization point here is to use the cache for derived resources

3. Parsing HTML, CSS, JavaScript resources using the parse module

"Parsing HTML"

HTML parsing can be divided into decoding, segmentation, parsing, build four steps

(1) Decoding: Decoding the encoded byte stream received on the network into Unicode characters

(2) Participle: The Unicode character stream is cut into words (Tokens) in accordance with the rules of the cut words.

(3) parsing: Create corresponding nodes (node) according to the semantics of the words

(4) Achievements: Associating nodes together to create a DOM tree

"Parsing css"

All of the CSS in the page is composed of a collection of style sheet cssstylesheet, and Cssstylesheet is a collection of Cssrule, Each cssrule is composed of the selector Cssstyleselector part and the declaration cssstyledeclaration part, and Cssstyledeclaration is the Key-value collection of CSS properties and values

After parsing the CSS, the cssrule matching process is found, which is to find the HTML element that satisfies the selector part of each CSS rule, and then apply its declaration declaration part to the element. The actual rule matching process takes into account factors such as default and inherited CSS properties, matching efficiency, and rule precedence.

"Parsing JS"

JavaScript is typically parsed by a separate scripting engine, and its role is usually to dynamically change the DOM tree (such as adding an event response handler for DOM nodes), which maps a DOM tree to another DOM tree based on time (timer) or events (event).

Simply put, after the parser module is processed, the browser transforms the page text into a node with a CSS Style, a styled DOM tree that responds to custom events

4. Build Dom tree, render tree and Renderlayer tree

The parsing process of the browser is the process of constructing the content of the Web page in the form of byte stream into DOM tree, render tree and Renderlayer tree.

The process of parsing HTML using parse has completed the construction of the DOM tree, and the next step is to build the render tree

"Render Tree"

The render tree is used to represent the visual information of a document, documenting how each visual element in the document is laid out and rendered

RenderObject is the base class for all nodes of the render tree and acts like the node class of the DOM tree. This class stores the style and layout information needed to draw the visual elements of the page, and the RenderObject object and its subclasses know how to draw themselves. In fact, the process of drawing a render tree is to renderobject the process of drawing itself in a certain order.

The nodes on the DOM tree are not one by one corresponding to the nodes on the render tree. Only the root node and visual node of the DOM tree will create the corresponding RenderObject node

"Render Layer Tree"

The render layer tree organizes the visual information for a document as a layer, and each layer on the page corresponds to a render layer object. The Renderlayer tree can be seen as a sparse representation of the render tree, and each node of the Renderlayer tree corresponds to a subtree of the render tree, which subtrees all the render nodes in the tree are displayed on the same layer of the Web page.

The Renderlayer tree is based on the RenderObject tree, which satisfies certain conditions renderobject will establish the corresponding Renderlayer node

Here is the creation condition for the Renderlayer node:

(1) Root node of Web page

(2) There is an explicit CSS position property (relative,absolute,fixed)

(3) Element set transform

(4) The element is transparent, i.e. opacity is not equal to 1

(5) The node has an overflow (overflow), alpha Mask, or reflection (reflection) effect.

(6) The element has CSS filter (filter) Properties

(7) 2D canvas or WebGL

(8) Video element

5. Layout and rendering

Layout is the process of arranging and calculating geometric information such as the size and position of each element in a page. HTML uses a streaming layout model, the basic principle is that the page elements in the sequential traversal process by the left-to-right, top-to-bottom arrangement to determine their respective location area

In simple cases, layouts can traverse the render tree once in a while, but there are cases where iterations are required. When the size position of an ancestor element depends on a descendant element or interdependence, a traversal cannot complete the layout, such as when the width of the table element is not explicitly specified and one of its sub-element tr specifies that its height is 30% of the parent table height

The Paint module is responsible for mapping the render tree to a visual graph, which traverses the render tree calls each render node's drawing method to display its contents on a single canvas or bitmap, and ultimately renders the actual page that the user sees in the Browser application window

The main drawing order is as follows:

(1) Background color

(2) Background image

(3) border

(4) Child rendering tree node

(5) Contour

6. Hardware acceleration

Turning on hardware rendering, or compositing acceleration, creates a graphicslayer for each layer that needs to be drawn separately

Hardware rendering refers to the composition of each layer of the Web page is done through the GPU, it uses the strategy of block rendering, block rendering refers to: the content of the Web page is covered by a set of tiles, each tile corresponding to a separate back-end storage, when the content of the Web page updates, only update the content of the tile changes. Sub-block strategy can be updated locally, rendering more efficient

A render layer object that, if it needs back-end storage, creates a render layer backing object that is responsible for the various kinds of storage required for Renderlayer objects. If a render layer object can create back-end storage, then the Renderlayer is called the composition layer (compositing layers)

If a render layer object has one of the following characteristics, then it is the composition layer:

(1) The Renderlayer has CSS 3D properties or CSS perspective effects.

(2) Renderlayer contains a RenderObject node that represents a HTML5 "video" element that uses hardware-accelerated decoding technology.

(3) Renderlayer contains a RenderObject node that represents a CANVAS2D element or WEBGL technology that uses hardware acceleration.

(4) Renderlayer uses the CSS transparency effect animation or CSS transform animation.

(5) Renderlayer uses hardware-accelerated cssfilters technology.

(6) Renderlayer uses a trim (clip) or Reflection (reflection) property, and its descendants include a composition layer.

(7) Renderlayer has a sibling node with a z-coordinate that is smaller than its own, and the node is a synthetic layer

The final rendering process is as follows:

"Redraw and Reflow"

Redrawing and reflow are two concepts that are important in the page rendering process. After the page is generated, script actions, stylesheet changes, and user actions can trigger redraw and reflow

Reflow Reflow is a term in Firefox that is called rearrangement in Chrome Relayout

Reflow refers to the layout process that is triggered when the window size is modified, a scrolling operation occurs, or when the element location-related property is updated, and the location information of all elements is calculated during layout. Because HTML uses a streaming layout, if the size of an element in the page changes, then the subsequent element position changes, that is, the process of re-flow layout, so called reflux

The 3 trees generated by the rendering engine are described earlier: the DOM tree, the render tree, and the render layer tree. Reflux occurs on the render tree. Often said to be out of the flow of the document, that is, from the render tree

Redrawing refers to the drawing process that is triggered when the value of a visual-related style property is updated, and the visual information of the element is recalculated during the drawing so that the element presents a new appearance

Because the redraw of the element repaint only occurs on the render layer render layers. So, if you want to change the visual properties of an element, it's best to make the element a separate render layer.

Here are some ways to reduce the number of reflux

(1) Instead of modifying the DOM style one by one, modify the classname or modify the Style.csstext

(2) Operate the node multiple times in memory and add it to the document after completion

(3) For a complex operation of an element, you can hide it before the operation is completed and displayed

(4) Cache into variables when you need to get the property values that cause the browser to reflow frequently

(5) Do not use the table layout, because a small change may cause the entire table to be re-layout. and table rendering usually takes 3 times times the same element time

In addition, the elements that require multiple reflow are independent of the render layer, such as setting absolute, which allows for less redrawing of the range, and hardware rendering for some animated elements to avoid redrawing and reflow

The entire process from the input URL to the page load