Recommend a Curl library for the entire site cloning function

Sometimes the use of some online manuals, such as domestic or foreign, some are slow access, some are the author of the direct bar site is closed, some servers are always down, so or the overall cloning to their own server relatively cool,

Here to recommend to everyone touched a curl library, can achieve the whole station cloning function, for copying some manual site is very practical.

Library Features:

given an initial connection, all files are copied locally at the level below the initial link.

multiple clones can be configured to overwrite.

You can configure whether to download pictures.

all links are replaced by relative links, so you can rewrite them casually.

There is absolutely no file overwrite issues.



Cloning results display(This cloning operation was completed in a few seconds):

Clone source Web site:http://www.laruence.com/manual/

Clone Result: http://manual.phpdr.net/yaf/

Technology Exchange: http://www.lai18.com/content/368643.html

Demo Code:

<?php//author http://www.lai18.comclass Controller_spider extends myyaf_controller_base{function init () {Parent::  Init ();  if (! $this->getrequest ()->iscli ()) {ares_http::error403 ();  } include ' curlmulti/curlmulti.php ';  Include ' curlmulti/mycurl.php '; Include ' phpquery.php '; }}

<?php//author http://www.lai18.comclass yafdoccontroller extends controller_spider  { function init ()  {  parent::init  ();  include  ' curlmulti/ Mycurl/clone.php ';  } function indexaction ()  {   $url  =  '/http Www.laruence.com/manual ';   $dir  = Yaf_Application::app  ()->getappdirectory  ()  .  '/data/manual ';   $cacheDir  =  $this->getbasedir  ()  .  '/cache /curl ';  if  (! is_dir  (  $cacheDir  ))  {   mkdir  (   $cacheDir  );  }   $curl  = new CurlMulti  ();   $ curl->maxthread = 10;   $curl->cache [' Enable '] = true;  $ curl->cache [' enabledownload '] = true;   $curl->cache [' dir '] = $ cachedir;   $curl->cache [' compress '] = true;   $clone  = new MyCurl_Clone  (  $curl ,  $url,  $dir  );   $clone->overwrite = true;   $clone->start   ();   return false; }}

Recommend a Curl library for the entire site cloning function