運用php充當shell指令碼（轉載）

使用php充當shell指令碼（轉載）
任務：過濾出2010-08-18的apache訪問日誌，並放到本機資料庫。
解決方案：寫兩個php檔案解決這個問題

假定linux系統
假定全utf-8
假定php已經放在$PATH裡
假如有這麼一個日誌/site/data/log/access_log_20100818，內容樣本如下：

[120.42.16.230] [-] [-] [2010-08-17 08:36:41] [GET] [www.site.com] [/membercenter/ordinary/score] [] [HTTP/1.1] [200] [2585] [-] [Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; QQDownload 618; GTB6.5; 360SE)][121.229.144.193] [-] [-] [2010-08-17 08:36:41] [GET] [www.site.com] [/bbs/jiehunzhenhao/wosikainv_49602.html] [] [HTTP/1.1] [200] [12631] [http://www.site.com/bbs/forum/jiehunzhenhao/filter/0/orderby/2/ascdesc/desc/page/4] [Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8][121.229.144.193] [-] [-] [2010-08-17 08:36:41] [POST] [www.site.com] [/bbsmanage/moderatorsetajax] [] [HTTP/1.1] [200] [21] [http://www.site.com/bbsmanage/moderatorset?id=4650] [Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; CIBA; 360SE)][60.190.125.3] [-] [-] [2010-08-17 08:36:41] [GET] [www.site.com] [/bbs/fangchanzatan/jiangjiatong_49458.html] [] [HTTP/1.1] [200] [10435] [http://www.site.com/membercenter/ordinary/bbssend?page=6] [Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; SE 2.X)][118.120.207.138] [-] [-] [2010-08-17 08:36:41] [GET] [www.site.com] [/bbs/jingcaitietu/tianshangrenjian_51533.html] [] [HTTP/1.1] [200] [13418] [http://www.site.com/bbs/forum/jingcaitietu/] [Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; QQDownload 627; GTB6.5; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 2.0.50727)][121.229.144.193] [-] [-] [2010-08-18 08:36:41] [GET] [www.site.com] [/bbsmanage/setmoderator] [] [HTTP/1.1] [200] [451] [http://www.site.com/mange/magframe] [Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; CIBA; 360SE)][121.229.144.193] [-] [-] [2010-08-18 08:36:42] [POST] [www.site.com] [/bbsmanage/moderatorxml] [] [HTTP/1.1] [200] [3699] [http://www.site.com/bbsmanage/setmoderator] [Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; CIBA; 360SE)][60.211.96.212] [-] [-] [2010-08-18 08:36:42] [GET] [www.site.com] [/member/index/id/7651] [] [HTTP/1.1] [200] [5308] [http://www.site.com/membercenter/ordinary/friend] [Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; SE 2.X; .NET CLR 2.0.50727; .NET CLR 4.0.20506)][113.205.59.70] [-] [-] [2010-08-18 08:36:43] [POST] [www.site.com] [/register/checkcaptcha] [] [HTTP/1.1] [200] [21] [http://www.site.com/register/ordinary/member_id/8326] [Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648)][123.4.197.242] [-] [-] [2010-08-18 08:36:43] [GET] [www.site.com] [/bbsoperate/tuijian] [?act=tuijian&id=33936] [HTTP/1.1] [200] [4448] [-] [Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; CNCDialer)]。。。。。。。。。。

當然很大，幾百M。


shell_filter.php檔案內容如下：

#!/usr/local/php/bin/php 1 )   $date = $argv[1];else   $date = '2010-01-01';   //迭代$j =0;while (!feof($handle)) {    $buffer = fgets($handle);    process($buffer);}//關閉輸入資料流，並結束fclose($handle);//篩選處理function process($str){    global $j;    global $date;      $str = strval($str);   $str = trim($str);   $str = preg_replace('#\n|\r\n#',"", $str);   //首先要確保符合日誌格式   if (preg_match('#\[.*?\] \[.*?\] \[.*?\] \[.*?\] \[.*?\] \[.*?\] \[.*?\] \[.*?\] \[.*?\] \[.*?\] \[.*?\] \[.*?\] \[.*?\]#', $str)) {      if (!preg_match('#::1#', $str)) { //這是無用的記錄          if (preg_match('#'. $date .'#', $str)) { //關鍵點，匹配              $j++;              echo $str . "\n";                    //這裡通過管道輸出到下一個檔案          }      }          }}?>

檔案save_echo.php內容如下：

#!/usr/local/php/bin/php $arr[0],          'access_time' => $arr[1],          'get_post'=> $arr[2],          'httphost' => $arr[3],          'url'=> $arr[4],          'http_type' => $arr[5],          'code'=> $arr[6],          'length' => $arr[7],          'source'=> $arr[8],          'agent' => substr( $arr[9],0, 250),          'engine_name' => $engine_name,       );       $db->insert('table1', $result);        //這裡只是輸出到控制台給人看       echo $i .': ' .$arr[1].'  '. $arr[0]  . "\n";}?>

最後
進入兩個php檔案所在目錄，
cat /site/data/log/access_log_20100812 | php shell_filter.php 2010-08-18|php save_echo.php

解釋：
cat輸出記錄檔內容，有緩衝，機器自動處理
管道至 shell_filter.php檔案的輸入
shell_filter.php檔案截取出2010-08-18的記錄並輸出，如果願意，可以改參數為任意日期，就截取那個日期的記錄
管道至 save_echo.php檔案的輸入
save_echo.php檔案儲存記錄到資料庫，並有控制台輸出提示。

