從github下載最新Android源碼

今年5月底開始，Google徹底被牆，所有Google的網站都不能訪問了，這次包括了android.org，googlesource.com，code.google.com。Android官方的資源不能訪問，想下載Android代碼當然是困難重重了。

本文就為大家解決這一問題——介紹如何使從github.com/andorid下載Andorid最新源碼。俗話說“授之以魚不如授之以漁”，本文不只是給出一個已經打包好的源碼的連結，而是重現一下我是如何下載的，希望對大家遇到的類似的問題能有所協助。

github.com/android簡介

開啟github.com/android可以看到：

中間黑體列出的的是各個項目的連結。github.com/android上託管的Android源碼是用git單獨管理每個項目的，而沒有像googlesource.com那樣使用repo（Google開發的基於git的命令列工具）管理所有項目。所以，如果你只想下載一個項目的源碼，可以單獨下載（比如開發App想跟一下SDK的一些源碼可以下載platform_frameworks_base項目）。

怎麼下載一個項目的代碼。

用過git的同學當然知道，可以使用下面的命令擷取遠程倉庫的代碼：

git clone [倉庫URL] 

github項目頁面的右側邊欄給出了倉庫URL，如下圖（clone URL下方的文字框）：

當然，如果只是要配置“App源碼調試”環境，可以點“Download ZIP”按鈕，下載zip格式的源碼包，然後在Eclipse調試的時候選擇下載好的源碼包。
如何批量下載github上的Android源碼

這裡在是本文的重點——批量下載github上的android源碼。

根據單獨下載一個項目的過程，下載所有項目的源碼可以有兩種方法： 得到所有項目的git倉庫URL 得到所有項目的ZIP包下載連結

接下來的問題是如何得到這樣的URL?

當然是從網頁上（也就是頁面的HTML檔案上）。

現在問題就是如何得到這些網頁。

我瞬間想到的辦法就是——用CURL

ps: CURL是一個命令列上傳下載工具，支援多種協議包括HTTP, HTTPS, FTP等。

沒有curl的同學可以用如下命令下載：

sudo apt-get install curl

有了CURL用來抓頁面，就不愁得不到倉庫URL或ZIP包URL了：

首先要抓取github.com/android的頁面，再“順藤摸瓜”地抓取各個項目的頁面，再從各項目的頁面上抓取倉庫URL或ZIP包URL。

抓取頁面

在github.com/android頁面的下方可以看github.com/andoid一共分5頁，點擊後面幾個頁面，可以發現這幾個頁面的URL形式一致為:

https://github.com/android?page=N

其中N表示“第幾頁”，比如第二頁，就是https://github.com/android?page=2

由此，可以用bash指令碼輕易寫出抓取這5個頁面的代碼（可以直接在終端裡執行下面的命令）：

for ((i=1; i<=5; i++)); docurl https://github.com/android?page=$i > android.github.com.$i.html;done

有捷徑

再看項目的首頁URL和倉庫URL可以發現：

倉庫URL = 首頁URL + .git

比如，platform_frameworks_base的首頁URL是：

https://github.com/android/platform_frameworks_base

倉庫URL是：

https://github.com/android/platform_frameworks_base.git

當然，如果你有github.com的帳號，並且當前環境的pubkey已加入到了profile下，也可以使用ssh協議下載（而不是https），ssh下載的URL是：

git://github.com/android/platform_frameworks_base.git

由此可知，擷取倉庫URL可以不用進一步抓起項目首頁，更簡單（下面只介紹擷取所有倉庫URL）。

解析頁面

現在的問題就是——如何從這些頁面上得到倉庫的URL了。

可以通過瀏覽器的“審查元素”菜單快速找到頁面上一個連結對應的HTML代碼塊，如下圖：

可以看到platform_frameworks_base項目的超連結（<a href="xxx">）被一個<h3 class="repo-list-name">標籤包圍，粗略瀏覽一下剛才下載的幾個子頁面，可以看出每個項目首頁的連結都是如此（在<h3 class="repo-list-name">的下一行）。

根據剛才找到的兩個規律： 項目首頁URL總是出現在<h3 class="repo-list-name">的下一行

倉庫URL = 首頁URL + .git

可以寫出解析各個項目首頁（倉庫URL）的python代碼（getGitRepos.py）：

#!/usr/bin/pythonimport sysflag = Falselink_prefix = 'https://github.com' # 如果使用ssh協議下載，可將https改為gitallLines = sys.stdin.readlines()for curLine in allLines:if curLine.find('repo-list-name') >= 0:flag = Trueif flag:pos = curLine.find('href="')if pos >= 0:pos += len('href="')last = curLine[pos:]end = last.find('"')link = last[:end]# name to path.name = link[link.rfind('/')+1:]prefix = name.find('platform_')if prefix >= 0:name = name[len('platform_'):] # ignore platform_path = name.replace('_', '/')link = link_prefix + link# print 'curl', link, '>', name# https://github.com/android/platform_external_qemu.gitprint 'git clone', link + '.git', path # 輸出 git clone 命令flag = False

之所以用python解析，只是因為python寫起來簡潔，也可以用其他語言。

這段python程式從標準輸入讀入文本並解析，實際使用時可用重新導向操作符實現，實現用之前抓取的github.com/android的幾個子頁面替代標準輸入。

程式輸出的git clone命令的path參數會忽略倉庫名稱前面的platform_首碼，這使得最終下載下來的源碼結構和repo sync下來的類似。

可以先用一個頁面測試一下，看看解析出來的URL對不對（上面的python代碼就是這樣改正確的），終端輸入下面命令：

./getGitRepos.py < android.github.com.1.html

該命令的輸出為：

git clone https://github.com/android/platform_dalvik.git dalvikgit clone https://github.com/android/platform_bionic.git bionicgit clone https://github.com/android/platform_system_core.git system/coregit clone https://github.com/android/kernel_common.git kernel/commongit clone https://github.com/android/platform_external_qemu.git external/qemugit clone https://github.com/android/platform_build.git buildgit clone https://github.com/android/platform_development.git developmentgit clone https://github.com/android/platform_frameworks_base.git frameworks/basegit clone https://github.com/android/platform_manifest.git manifestgit clone https://github.com/android/platform_frameworks_support.git frameworks/supportgit clone https://github.com/android/platform_packages_apps_settings.git packages/apps/settingsgit clone https://github.com/android/platform_external_dhcpcd.git external/dhcpcdgit clone https://github.com/android/platform_external_webkit.git external/webkitgit clone https://github.com/android/platform_external_protobuf.git external/protobufgit clone https://github.com/android/platform_packages_providers_mediaprovider.git packages/providers/mediaprovidergit clone https://github.com/android/platform_external_elfutils.git external/elfutilsgit clone https://github.com/android/platform_external_strace.git external/stracegit clone https://github.com/android/platform_hardware_libhardware.git hardware/libhardwaregit clone https://github.com/android/platform_external_tinyxml.git external/tinyxmlgit clone https://github.com/android/platform_external_oprofile.git external/oprofile

測試基本上出現問題，因為getGitRepos.py輸出的是git clone命令，執行之後在能執行實際的下載動作；所以，需要將這些輸出儲存下來；

使用如下命令可將每個子頁面解析出的git clone命令儲存到一個文字檔內：

for page in `ls android.github.com.*`; doecho parse $page./getGitRepos.py < $page >> gitRepos.txt;done

有了寫滿git clone命令的gitRepos.txt後，可以使用如下命令執行之：

sh gitRepos.txt

一鍵下載

為了方便廣大網友使用，我已將上面需要用的命令寫道了下面的cloneRepos.sh：

#!/bin/bash# clone AOSP each repo from https://github.com/android# this shell depends on git, curl and python 2.# INSTALL depends:#sudo apt-get install git curl python## by xu(xusiwei1236@163.com).# download github AOSP sub pagesfor ((i=1; i<=5; i++)); doecho get AOSP subpage https://github.com/android?page=$icurl https://github.com/android?page=$i > android.github.com.$i.html;done# parse git repo URL from each sub pagesmkdir reposcat /dev/null > gitRepos.txtfor page in `ls android.github.com.*`; doecho parse $page./getGitRepos.py < $page >> gitRepos.txt;done# clone repossh gitRepos.txt

另外，我已將上面的cloneRepos.sh和getGitRepos.py打包在了一起，可以到 http://download.csdn.net/detail/xusiwei1236/7960925下載使用。

當然，如果覺得自己運行麻煩，可以直接copy產生的gitRepos.txt，直接粘貼到終端裡執行之，即可下載全部Android源碼：

git clone https://github.com/android/platform_dalvik.git dalvikgit clone https://github.com/android/platform_bionic.git bionicgit clone https://github.com/android/platform_system_core.git system/coregit clone https://github.com/android/kernel_common.git kernel/commongit clone https://github.com/android/platform_external_qemu.git external/qemugit clone https://github.com/android/platform_build.git buildgit clone https://github.com/android/platform_development.git developmentgit clone https://github.com/android/platform_frameworks_base.git frameworks/basegit clone https://github.com/android/platform_manifest.git manifestgit clone https://github.com/android/platform_frameworks_support.git frameworks/supportgit clone https://github.com/android/platform_packages_apps_settings.git packages/apps/settingsgit clone https://github.com/android/platform_external_dhcpcd.git external/dhcpcdgit clone https://github.com/android/platform_external_webkit.git external/webkitgit clone https://github.com/android/platform_external_protobuf.git external/protobufgit clone https://github.com/android/platform_packages_providers_mediaprovider.git packages/providers/mediaprovidergit clone https://github.com/android/platform_external_elfutils.git external/elfutilsgit clone https://github.com/android/platform_external_strace.git external/stracegit clone https://github.com/android/platform_hardware_libhardware.git hardware/libhardwaregit clone https://github.com/android/platform_external_tinyxml.git external/tinyxmlgit clone https://github.com/android/platform_external_oprofile.git external/oprofilegit clone https://github.com/android/platform_external_neven.git external/nevengit clone https://github.com/android/platform_external_netperf.git external/netperfgit clone https://github.com/android/platform_external_netcat.git external/netcatgit clone https://github.com/android/platform_external_libxml2.git external/libxml2git clone https://github.com/android/platform_external_libpng.git external/libpnggit clone https://github.com/android/platform_external_libpcap.git external/libpcapgit clone https://github.com/android/platform_external_libffi.git external/libffigit clone https://github.com/android/platform_external_jpeg.git external/jpeggit clone https://github.com/android/platform_external_jhead.git external/jheadgit clone https://github.com/android/platform_external_jdiff.git external/jdiffgit clone https://github.com/android/platform_external_iptables.git external/iptablesgit clone https://github.com/android/platform_external_icu4c.git external/icu4cgit clone https://github.com/android/platform_external_giflib.git external/giflibgit clone https://github.com/android/platform_external_freetype.git external/freetypegit clone https://github.com/android/platform_external_fdlibm.git external/fdlibmgit clone https://github.com/android/platform_external_expat.git external/expatgit clone https://github.com/android/platform_external_esd.git external/esdgit clone https://github.com/android/platform_external_emma.git external/emmagit clone https://github.com/android/platform_external_dropbear.git external/dropbeargit clone https://github.com/android/platform_external_apache-http.git external/apache-httpgit clone https://github.com/android/platform_external_openssl.git external/opensslgit clone https://github.com/android/platform_external_sonivox.git external/sonivoxgit clone https://github.com/android/platform_packages_providers_telephonyprovider.git packages/providers/telephonyprovidergit clone https://github.com/android/platform_packages_providers_downloadprovider.git packages/providers/downloadprovidergit clone https://github.com/android/platform_packages_providers_contactsprovider.git packages/providers/contactsprovidergit clone https://github.com/android/platform_packages_providers_calendarprovider.git packages/providers/calendarprovidergit clone https://github.com/android/platform_packages_apps_voicedialer.git packages/apps/voicedialergit clone https://github.com/android/platform_packages_apps_stk.git packages/apps/stkgit clone https://github.com/android/platform_packages_apps_soundrecorder.git packages/apps/soundrecordergit clone https://github.com/android/platform_packages_apps_phone.git packages/apps/phonegit clone https://github.com/android/platform_packages_apps_packageinstaller.git packages/apps/packageinstallergit clone https://github.com/android/platform_packages_apps_music.git packages/apps/musicgit clone https://github.com/android/platform_packages_apps_mms.git packages/apps/mmsgit clone https://github.com/android/platform_packages_apps_htmlviewer.git packages/apps/htmlviewergit clone https://github.com/android/platform_packages_apps_email.git packages/apps/emailgit clone https://github.com/android/platform_packages_apps_contacts.git packages/apps/contactsgit clone https://github.com/android/platform_packages_apps_camera.git packages/apps/cameragit clone https://github.com/android/platform_packages_apps_calendar.git packages/apps/calendargit clone https://github.com/android/platform_packages_apps_calculator.git packages/apps/calculatorgit clone https://github.com/android/platform_packages_apps_browser.git packages/apps/browsergit clone https://github.com/android/platform_hardware_ril.git hardware/rilgit clone https://github.com/android/platform_external_zlib.git external/zlibgit clone https://github.com/android/platform_external_yaffs2.git external/yaffs2git clone https://github.com/android/platform_external_tcpdump.git external/tcpdumpgit clone https://github.com/android/platform_external_tagsoup.git external/tagsoupgit clone https://github.com/android/platform_external_srec.git external/srecgit clone https://github.com/android/platform_external_sqlite.git external/sqlitegit clone https://github.com/android/platform_external_skia.git external/skiagit clone https://github.com/android/platform_external_safe-iop.git external/safe-iopgit clone https://github.com/android/platform_external_ppp.git external/pppgit clone https://github.com/android/kernel_msm.git kernel/msmgit clone https://github.com/android/platform_system_bluetooth.git system/bluetoothgit clone https://github.com/android/platform_prebuilt.git prebuiltgit clone https://github.com/android/platform_packages_providers_drmprovider.git packages/providers/drmprovidergit clone https://github.com/android/platform_external_ping.git external/pinggit clone https://github.com/android/platform_external_dbus.git external/dbusgit clone https://github.com/android/tools_repo.git tools/repogit clone https://github.com/android/android.github.io.git android.github.iogit clone https://github.com/android/platform_external_wpa_supplicant.git external/wpa/supplicantgit clone https://github.com/android/platform_packages_apps_googlesearch.git packages/apps/googlesearchgit clone https://github.com/android/platform_system_wlan_ti.git system/wlan/tigit clone https://github.com/android/platform_packages_providers_googlecontactsprovider.git packages/providers/googlecontactsprovidergit clone https://github.com/android/platform_packages_apps_im.git packages/apps/imgit clone https://github.com/android/platform_packages_providers_improvider.git packages/providers/improvidergit clone https://github.com/android/platform_external_tremor.git external/tremorgit clone https://github.com/android/platform_packages_apps_sync.git packages/apps/syncgit clone https://github.com/android/platform_packages_apps_alarmclock.git packages/apps/alarmclockgit clone https://github.com/android/platform_external_aes.git external/aesgit clone https://github.com/android/platform_external_clearsilver.git external/clearsilvergit clone https://github.com/android/platform_external_googleclient.git external/googleclientgit clone https://github.com/android/platform_packages_apps_launcher.git packages/apps/launchergit clone https://github.com/android/platform_external_bluez.git external/bluezgit clone https://github.com/android/platform_external_gdata.git external/gdatagit clone https://github.com/android/platform_frameworks_opt_com.google.android.git frameworks/opt/com.google.androidgit clone https://github.com/android/platform_external_elfcopy.git external/elfcopygit clone https://github.com/android/platform_frameworks_policies_base.git frameworks/policies/basegit clone https://github.com/android/platform_packages_apps_updater.git packages/apps/updatergit clone https://github.com/android/platform_external_opencore.git external/opencore