Web Programming for Python

Python's web-related modules, with two different types,urlparse and urllib, handle URLs with different functions and compatibility, including Urlparse (), Urlunparse (), respectively. and Urljoin (), which can use Urlopen () or urlretrive () to create a Web client. The URLLIB2 module is a more advanced module than the Urllib function.

Web development, there is no doubt to use the Uniform Resource Locator URL, the URL format is as follows:
Prot_sch://net_loc/path:params?query#flag

Prot_sch: Network protocol or download planning.
Net_loc: Server location, and perhaps user information.
Path: A slash/limit file or a common Gateway interface for CGI application paths.
Params: Optional parameter.
Query: Connector & connection key value pairs.
Flag: Splits the special anchor in the document.

Among them,Net_loc can be further split into several parts, as follows:
User:[email protected]:p ORT

User: Login name.
passwd: User's password.
Host:web the machine name or address that the server is running on is a required field.
Port: Port number, default 80.

urlparse Module--
Urlparse (urlstr,defprotsch=none,allowfrag=none): Splits the URL string into a 6-tuple, If there is no given protocol in URLSTR or the plan will use Defprotsch, the parameter Allowfrag determines whether a URL component is allowed.
Urlunparse (Urltup): The Opposite of the Urlparse () function.
Urljoin (Baseurl,newurl,allowfrag=none): The base part of the URL BaseURL and the new part newurl a complete URL.

urllib Module--
Urlopen (Urlstr,postquerydata=none): Opens the URL that the Urlstr points to, and Urlopen () opens a local file if a given protocol or download plan is not available, or if the file plan is already passed in. When opened successfully, returns a file type object, supported by Read ([bytes]), ReadLine (), ReadLines (), Close (), Fileno (), info (), Geturl (), and so on.
Urlretrieve (Urlstr,localfile=none,downloadstatushook=none): In addition to reading content from a URL like Urlopen (), You can also easily download the entire HTML file Urlstr to the local hard disk, you can save the downloaded data into a local file or a temporary file, if the file has been copied to a local or is already a local file, the subsequent download action will not occur, if possible, Downloadstatushook This function will be called after each piece of data is downloaded or transmitted.
QUOTE (urldata,safe= '/'): Get the URL data and encode it to apply URL strings, especially some special strings that cannot be printed or are not received by the Web server as valid URLs must be converted, Quote_plus () function Similarly, The difference is that you can also encode spaces into plus + numbers.
Unquote (), Unquote_plus (): Opposite to the quote* () function.
UrlEncode (dict): Receive dictionary key-value pairs and compile them as part of a CGI request URL string, with different key-value pairs signed & divided.

URLLIB2 Module--
URLLIB2 can handle open problems with more complex URLs, such as login authentication.

web crawler--
A common application of Web programming is web crawlers, which search and download pages on the Internet for different purposes, such as indexing search engines, offline browsing, downloading and saving history or frames, Web page caching, and so on.

cgi--
CGI, the Common Gateway Interface, is a universal Gateway interface that helps the Web server process client data, which represents the interaction between a Web server and an application that can process user forms, generate and return dynamic HTML pages. However, due to the limitations of CGI, many typical Web application products are now no longer using CGI.
The CGI module helps us build CGI applications by first installing a Web server, configuring it as a pattern for processing Python's CGI requests, and then getting the Web server to access CGI scripts. If it is a simple server, you can use a Python-brought Web server with the following command:
$python-M Cgihttpserver
The above command will create a Web server with a port number of 8000 in the current directory of the current machine, and then you can create a cgi-bin in the directory where the service is launched, place python CGI scripts in this directory, or some HTML files directly in the directory where the service is launched. You can then enter these address classes in the Address bar to access the Web site, as follows:
Http://localhost:8000/mypython.htm
http://localhost:8000/cgi-bin/pythons.py

To establish an HTTP-based Web service, a basic server and a processor are required, the server completes the necessary HTTP interaction on the client and server side, the processor is used to handle client requests, and returns the appropriate files, such as static text or dynamic files generated by CGI, The complexity of the processor determines how complex the Web server is. The Python standard library provides three different processors, each of which can be found in the corresponding module, as follows:
Basehttpserver module: Provides basic Web services and processor classes, respectively, Httpserver and Basehttprequesthandler processors.
Simplehttpserver module: Contains the Simplehttprequesthandler class that performs the get and head requests.
Cgihttpserver module: Contains the Cgihttprequesthandler class that handles post requests and executes CGI.

Web Programming for Python