[Python] Several domain name resolution methods from URLs, pythonurl

[Python] Several domain name resolution methods from URLs, pythonurl
Several Methods for parsing domain names from URLs in Python

Find the domain name from the url, first think of using regular expressions, and then find the corresponding class library. There are a lot of imperfections in regular expression parsing, including domain names in URLs and increasing domain name suffixes. Several methods can be found through google. One is to use the Python built-in module and the regular expression to resolve the domain name, and the other is to enable the third party to directly resolve the domain name with the prepared parsing module.

Url to be parsed

urls = ["http://meiwen.me/src/index.html",          "http://1000chi.com/game/index.html",          "http://see.xidian.edu.cn/cpp/html/1429.html",          "https://docs.python.org/2/howto/regex.html",          """https://www.google.com.hk/search?client=aff-cs-360chromium&hs=TSj&q=url%E8%A7%A3%E6%9E%90%E5%9F%9F%E5%90%8Dre&oq=url%E8%A7%A3%E6%9E%90%E5%9F%9F%E5%90%8Dre&gs_l=serp.3...74418.86867.0.87673.28.25.2.0.0.0.541.2454.2-6j0j1j1.8.0....0...1c.1j4.53.serp..26.2.547.IuHTj4uoyHg""",          "file:///D:/code/echarts-2.0.3/doc/example/tooltip.html",          "http://api.mongodb.org/python/current/faq.html#is-pymongo-thread-safe",          "https://pypi.python.org/pypi/publicsuffix/",          "http://127.0.0.1:8000"          ]

Use urlparse + Regular Expression

import refrom urlparse import urlparsetopHostPostfix = (    '.com','.la','.io','.co','.info','.net','.org','.me','.mobi',    '.us','.biz','.xxx','.ca','.co.jp','.com.cn','.net.cn',    '.org.cn','.mx','.tv','.ws','.ag','.com.ag','.net.ag',    '.org.ag','.am','.asia','.at','.be','.com.br','.net.br',    '.bz','.com.bz','.net.bz','.cc','.com.co','.net.co',    '.nom.co','.de','.es','.com.es','.nom.es','.org.es',    '.eu','.fm','.fr','.gs','.in','.co.in','.firm.in','.gen.in',    '.ind.in','.net.in','.org.in','.it','.jobs','.jp','.ms',    '.com.mx','.nl','.nu','.co.nz','.net.nz','.org.nz',    '.se','.tc','.tk','.tw','.com.tw','.idv.tw','.org.tw',    '.hk','.co.uk','.me.uk','.org.uk','.vg', ".com.hk")regx = r'[^\.]+('+'|'.join([h.replace('.',r'\.') for h in topHostPostfix])+')$'pattern = re.compile(regx,re.IGNORECASE)print "--"*40for url in urls:    parts = urlparse(url)    host = parts.netloc    m = pattern.search(host)    res =  m.group() if m else host    print "unkonw" if not res else res

The running result is as follows:

meiwen.me1000chi.comsee.xidian.edu.cnpython.orggoogle.com.hkunkonwmongodb.orgpython.org127.0.0.1:8000

Acceptable

Urllib to resolve domain names

import urllibprint "--"*40for url in urls:    proto, rest = urllib.splittype(url)    res, rest = urllib.splithost(rest)    print "unkonw" if not res else res

The running result is as follows:

meiwen.me1000chi.comsee.xidian.edu.cndocs.python.orgwww.google.com.hkunkonwapi.mongodb.orgpypi.python.org127.0.0.1:8000

Www. will also be taken, and further analysis is required.

Use third-party module tld

from tld import get_tldprint "--"*40for url in urls:    try:        print  get_tld(url)    except Exception as e:        print "unkonw"

Running result:

meiwen.me1000chi.comxidian.edu.cnpython.orggoogle.com.hkunkonwmongodb.orgpython.orgunkonw

Acceptable results

Other available parsing modules:

• Tld
• Tldextract
• Publicsuffix

This article is from the "orangleliu notebook" blog, please be sure to keep this http://blog.csdn.net/orangleliu/article/details/39545821

How to resolve the main domain name of a url

The first segment of the URL with a suffix (such as COM, CN, ORG, and NET). If the prefix is changed to WWW or WAP, it is the primary network address, such

Domain Name forwarding hiding and URL resolution problems?

Go to the domain name self-service resolution Platform
Open DNS records
Directly forward the URL in the domain name settings, and then hide the original address!

URL (URL forwarding)
Host Name TTL URL
Mode: 0 indicates hidden forwarding, and 1 indicates not hidden forwarding