Python script generates Sitemaps

The project needs to generate a sitemap with a script that learns the format of the Sitemap and how to use the lxml library. Record the results and make it easy for you to use them directly later.
Generate Sitemaps from Python script

Installing lxml

First, you need to pip install lxml install the lxml library.

Suppose you encounter the following error on Ubuntu:

#include "Libxml/xmlversion.h"Compilation terminated.Error:command' X86_64-LINUX-GNU-GCC 'Failed withExit status1----------------------------------------Cleaning up...Removing temporary Dir/tmp/pip_build_root...Command/usr/bin/python- C "Import setuptools, tokenize;__file__= '/tmp/pip_build_root/lxml/setup.py '; exec (compile (getattr, ' Open ', open) (__file__). Read (). replace (' \ r \ n ', ' \ n '), __file__, ' exec ') "Install--Record/tmp/pip-o4cin6-record/install-record.Txt--Single-version-externally-managed --Compile failed withError code1 inch/tmp/pip_build_root/lxmlexception Information:traceback (most recent): File"/usr/lib/python2.7/dist-packages/pip/basecommand.py", line122,inchMain status=  Self.Run (options, args) File"/usr/lib/python2.7/dist-packages/pip/commands/install.py", line283,inchRun Requirement_set.Install (install_options, global_options, Root=Options.Root_path) File"/usr/lib/python2.7/dist-packages/pip/req.py", line1435,inchInstall requirement.Install (install_options, global_options,*Args**Kwargs) File"/usr/lib/python2.7/dist-packages/pip/req.py", line706,inchInstall CWD= Self.Source_dir, Filter_stdout= Self._filter_install, Show_stdout=False) File"/usr/lib/python2.7/dist-packages/pip/util.py", line697,inchCall_subprocess%(Command_desc, Proc.ReturnCode, CWD)) Installationerror:command/usr/bin/python- C "Import setuptools, tokenize;__file__= '/tmp/pip_build_root/lxml/setup.py '; exec (compile (getattr, ' Open ', open) (__file__). Read (). replace (' \ r \ n ', ' \ n '), __file__, ' exec ') "Install--Record/tmp/pip-o4cin6-record/install-record.Txt--Single-version-externally-managed --Compile failed withError code1 inch/tmp/pip_build_root/lxml

Please install the following dependencies:

sudo apt-get install libxml2-dev libxslt1-dev

Python code

Here is the code that generates the sitemap and the Sitemapindex index. Be able to pass the required number of parameters according to demand. Or add a field:

#!/usr/bin/env python#-*-coding:utf-8-*-import ioimport refrom lxml import etreedef generate_xml (filename, url_list): "" "Generate a new XML file use Url_list" "" root = etree.        Element (' Urlset ', xmlns= "http://www.sitemaps.org/schemas/sitemap/0.9") for each in url_list: url = etree. Element (' url ') loc = etree. Element (' loc ') Loc.text = each url.append (loc) root.append (URL) header = U ' <?
XML version= "1.0" encoding= "UTF-8"?
>\n ' s = etree.tostring (root, encoding= ' utf-8 ', pretty_print=true) with io.open (filename, ' W ', encoding= ' utf-8 ') As F:f.write (Unicode (header+s)) def update_xml (filename, url_list): "" "Add new Url_list to Origin XML file." " f = open (filename, ' r ') lines = [I.strip () for I in F.readlines ()] F.close () old_url_list = [] for each_line In lines:d = Re.findall (' <loc> (http:\/\/.+) <\/loc> ', each_line) old_url_list + = d url_list + = Old_url_list generate_xml (filename, url_list) def generatr_xml_index (filename, sitemap_list, lastmod_list): "" "Gen Erate Sitemap Index XML file. "" root = etree. Element (' Sitemapindex ', xmlns= "http://www.sitemaps.org/schemas/sitemap/0.9") for Each_sitemap, Each_lastmod in Zip (Sitemap_list, lastmod_list): Sitemap = etree. Element (' sitemap ') loc = etree. Element (' loc ') Loc.text = Each_sitemap Lastmod = etree. Element (' lastmod ') Lastmod.text = Each_lastmod sitemap.append (Loc) sitemap.append (lastmod) root.append (sitemap) Header = U ' <? 
XML version= "1.0" encoding= "UTF-8"?
>\n‘ s = etree.tostring(root, encoding=‘utf-8‘, pretty_print=True) with io.open(filename, ‘w‘, encoding=‘utf-8‘) as f: f.write(unicode(header+s))if __name__ == ‘__main__‘: urls = [‘http://www.baidu.com‘] * 10 mods = [‘2004-10-01T18:23:17+00:00‘] * 10 generatr_xml_index(‘index.xml‘, urls, mods)

Effect

The resulting effect should be in such a format:
Sitemap format:

<?xml version="1.0" encoding="UTF-8"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">  <url>    <loc>http://www.example.com/foo.html</loc>  </url></urlset>

Sitemapindex format:

<?
xml version="1.0" encoding="UTF-8"?
> <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <sitemap> <loc>Http://www.example.com/sitemap1.xml.gz</loc> <lastmod>2004-10-01t18:23:17+00:00</lastmod> </sitemap> <sitemap> <loc>Http://www.example.com/sitemap2.xml.gz</loc> <lastmod>2005-01-01</lastmod> </sitemap> </sitemapindex>

Lastmod problems with time format

The format is the ISO 8601 standard, assuming that the Linux/unix system can use the following function to obtain

def get_lastmod_time(filename):    time_stamp = os.path.getmtime(filename)    t = time.localtime(time_stamp)    return time.strftime(‘%Y-%m-%dT%H:%M:%S+08:00‘, t)

REF:
Create Site Map
Sitemap
lxml
Description of the XML document format for the Google site map
Google sitemap file format

Python script generates Sitemaps