Build local Debian package cache with apt-p2p

Build local Debian package cache with apt-p2p

I have several computers running Debian GNU/Linux testing in my home, which are in a small LAN. Because Debian testing is a rolling update, I will update all of them every few days. However, as a result, the network speed of Chinese households is still not very fast. It takes a lot of time for each machine to download hundreds of megabytes of software packages from the upstream software source servers; second, such operations also impose a great burden on the software source server.

Later I remembered that Debian had developed a software called apt-p2p designed to cache downloaded software packages for other peers that run the same apt-p2p, as a supplement for downloading from software sources. Can I use this software to build a Debian software package cache in the lan?

The apt-p2p is written in Python and listens to port 9977 of TCP and UDP by default after installation./etc/apt-p2p/apt-p2p.confChange its behavior. TCP port is also used for apt download software packages on the local machine, while UDP port is used to build a common DHT network in p2p file sharing tools. You only need/etc/apt/sources.listThe url of the software source in the file, suchhttp://mirror.server/debian/In front of the server domain name, insert localhost: 9977 to becomehttp://localhost:9977/mirror.server/debian/, The specific work to obtain software packages from the Internet to the apt-p2p is responsible:

 
 

  
  
When receiving a File Download request from the local apt, The apt-p2p first checks whether the file already exists in the cached directory tree 
   
 
    
If it already exists, check whether it is up to date (by sending a HEAD request to the software source server) 
     
 
      
If yes, the cached file is directly returned.
 
      
Otherwise, the downloaded files from the Internet are provided to apt, and the files are stored in the cache and Their hash values are recorded.
 
     
 
   
  
  
The downloaded file will first be downloaded from the peer, and will not be downloaded from the software source server.
  
  
Cached files and hash values can be used by other peers. This process can be observed in depth through the apt-p2p's log file/var/log/apt-p2p.log.
 
 

Below is the host where I installed the apt-p2pmycastleSources. list:

1. deb http://localhost:9977/mirrors.ustc.edu.cn/debian/ testing main
2. deb-src http://localhost:9977/mirrors.ustc.edu.cn/debian/ testing main

3. deb http://localhost:9977/mirrors.ustc.edu.cn/debian-security/ testing/updates main
4. deb-src http://localhost:9977/mirrors.ustc.edu.cn/debian-security testing/updates main

5. deb http://localhost:9977/mirrors.ustc.edu.cn/debian/ testing-proposed-updates main
6. deb-src http://localhost:9977/mirrors.ustc.edu.cn/debian/ testing-proposed-updates main

7. deb http://localhost:9977/mirrors.ustc.edu.cn/debian/ unstable main
8. deb-src http://localhost:9977/mirrors.ustc.edu.cn/debian/ unstable main

However, the apt-p2p cache can only be used on the local machine, and other hosts will only get error 9977 if they try to access port 404 directly. Therefore, you also need to set up an http reverse proxy on the host where the apt-p2p is installed so that in the apt-p2p's view, all requests come from the local machine.

The reverse proxy I use is Pound, which is disabled by default after installation. Configure the server in/etc/pound. cfg, enable it in/etc/default/pound, and then start the server using the service.

The configuration I used is as follows:

1. ...(keep default)
2. ListenHTTP
3. Address0.0.0.0
4. Port9978
5. ## allow PUT and DELETE also (by default only GET, POST and HEAD)?:
6. xHTTP 0
7. Service
8. BackEnd
9. Address127.0.0.1
10. Port9977
11. End
12. End
13. End

Listening on port 9978, and the background server on port 9977 of the local machine.

As a result, other hosts can also use the apt-p2p cache through port 9978. My approach is to use the cache on other hosts./etc/apt/sources.listMove/etc/apt/sources.list.d/50_main.listAnd then create/etc/apt/sources.list.d/10_apt-p2p-home.list, Just putlocalhost:9977ChangeYou can, of course, keep the software source server the same. In this way, apt will give priority to using the apt-p2p's cache through reverse proxy, when the cache is unavailable, you can still directly connect to the software source server to get the software package:

10_apt-p2p-home.list:

1. deb http://mycastle.local:9978/mirrors.ustc.edu.cn/debian/ testing main
2. deb-src http://mycastle.local:9978/mirrors.ustc.edu.cn/debian/ testing main
3. deb http://mycastle.local:9978/mirrors.ustc.edu.cn/debian-security/ testing/updates main
4. deb-src http://mycastle.local:9978/mirrors.ustc.edu.cn/debian-security testing/updates main
5. deb http://mycastle.local:9978/mirrors.ustc.edu.cn/debian/ testing-proposed-updates main
6. deb-src http://mycastle.local:9978/mirrors.ustc.edu.cn/debian/ testing-proposed-updates main
7. deb http://mycastle.local:9978/mirrors.ustc.edu.cn/debian/ unstable main
8. deb-src http://mycastle.local:9978/mirrors.ustc.edu.cn/debian/ unstable main

50_main.list:

1. deb http://mirrors.ustc.edu.cn/debian/ testing main
2. deb-src http://mirrors.ustc.edu.cn/debian/ testing main
3. deb http://mirrors.ustc.edu.cn/debian-security/ testing/updates main
4. deb-src http://mirrors.ustc.edu.cn/debian-security testing/updates main
5. deb http://mirrors.ustc.edu.cn/debian/ testing-proposed-updates main
6. deb-src http://mirrors.ustc.edu.cn/debian/ testing-proposed-updates main
7. deb http://mirrors.ustc.edu.cn/debian/ unstable main
8. deb-src http://mirrors.ustc.edu.cn/debian/ unstable main

If apt-transport-https is installed, you can replace http in 50_main.list with https.

Add ". the domain name format of local is a method provided by the mDNS/DNS-SD protocol to locate hosts and publish services in the broadcast domain. It mainly implements Bonjour of Avahi and Mac OS X.

In this way, in the LAN, any host is given priority to get software packages through the apt-p2p, an update can benefit the whole family; even if a short period of time to update multiple hosts, only need to obtain the software source once; in addition, large-scale data flow occurs in the LAN with high bandwidth, which shortens the Update Time and improves the efficiency.

In this way, two software sources are configured on a non-Cache host, when the two software sources repeat different steps (for example, the laptop updates the software package list of the directly connected software source server outside the LAN ),apt-get sourceIs not usable, because apt checks the source code package more strictly.

This article permanently updates the link address: