Write your own PHP Curl library to implement _php instance of whole station cloning function

Sometimes the use of some online manuals, such as domestic or foreign, some are slow to visit, some is the author directly to the website shut down, and some of the server is always down, so or the overall cloning to their own server relatively cool.

Library Features:

Given an initial connection, all files are copied locally at the level below the initial link.
Multiple clones can be configured to overwrite.
You can configure whether to download pictures.
All links are replaced by relative links, so you can easily rewrite.
There is absolutely no problem with file overwrite.
The most NB feature is that there is no more NB than this library.

svn:http://svn.phpdr.net/repos/ares/php/library/trunk/lib/curlmulti/mycurl/clone.php

Cloud Habitat Community Download: Http://xiazai.jb51.net/201502/other/CurlMulti.rar

Clone results display (this cloning operation is completed in seconds):

Clone source Web site: http://www.laruence.com/manual/
Clone Result: http://manual.phpdr.net/yaf/

Demo Code:

Copy Code code as follows:

<?php
Class Controller_spider extends myyaf_controller_base{
function init () {
Parent::init ();
if (! $this->getrequest ()->iscli ()) {
ares_http::error403 ();
}
Include ' curlmulti/curlmulti.php ';
Include ' curlmulti/mycurl.php ';
Include ' phpquery.php ';
}
}

Copy Code code as follows:

<?php
Class Yafdoccontroller extends Controller_spider {
function init () {
Parent::init ();
Include ' curlmulti/mycurl/clone.php ';
}
function Indexaction () {
$url = ' http://www.laruence.com/manual ';
$dir = Yaf_application::app ()->getappdirectory (). '/data/manual ';
$cacheDir = $this->getbasedir (). '/cache/curl ';
if (! Is_dir ($cacheDir)) {
mkdir ($cacheDir);
}
$curl = new Curlmulti ();
$curl->maxthread = 10;
$curl->cache [' enable '] = true;
$curl->cache [' enabledownload '] = true;
$curl->cache [' dir '] = $cacheDir;
$curl->cache [' compress '] = true;
$clone = new Mycurl_clone ($curl, $url, $dir);
$clone->overwrite = true;
$clone->start ();
return false;
}
}