Download the latest Android source from GitHub

At the end of May this year, Google was completely blocked by the wall, and all Google sites were inaccessible, including android.org,googlesource.com,code.google.com. Android's official resources cannot be accessed, and it is of course difficult to download the Android code.

This article is for everyone to solve this problem--how to make download from Github.com/andorid andorid the latest source code. As the saying goes, "to give the fish as the award of fishing", this article is not just to provide a package of the source of the link, but to reproduce how I downloaded, I hope that the similar problems encountered by everyone can be helpful.

Github.com/android Introduction

Open github.com/android to see:

The middle black list is a link to each item. The Android source code hosted on github.com/android is used by git to manage each project separately, instead of managing all projects with repo (Google-developed GIT-based command-line tools) like googlesource.com. So, if you only want to download the source of a project, you can download it separately (for example, the development app wants to follow some of the SDK source code can download Platform_frameworks_base project).

How do I download the code for a project?

The classmate who used git knows:

Yes, the right column of the GitHub project page gives the warehouse URL, such as (text box below the clone URL):

Of course, if you just configure the "app source debugging" Environment, you can click the "Download zip" button, download the zip-format source package, and then choose the download good source package when the Eclipse debugging.

How to bulk download the Android source on GitHub

Here is the focus of this article--Download the Android source code on GitHub in bulk.

Depending on the process of downloading one item separately, there are two ways to download the source code for all projects:

1. Get git repository URLs for all projects
2. Get a Zip package download link for all items

The next question is how do I get a URL like this?

Of course from the Web page (that is, the HTML file on the page)!

Now the question is how to get these pages?

The way I think of it in a moment is--with curl

Ps:curl is a command line upload download tool that supports a variety of protocols including HTTP, HTTPS, FTP, and more.

Students without curl can download it using the following command:

sudo apt-get Install Curl

With Curl used to capture the page, there is no need to get the warehouse URL or zip package URL:

The first is to crawl github.com/android pages, and then "go through" to crawl the pages of each item, and then crawl the warehouse URL or zip package URL from each item's page.

Crawl page

In the Github.com/android page below you can see github.com/andoid total of 5 pages, click on the following pages, you can find the URL of these pages in the form of the same:

Https://github.com/android?page=N

where n means "page number", like the second page, is https://github.com/android?page=2

As a result, you can easily write code that crawls these 5 pages with a bash script (you can execute the following command directly in the terminal):

for ((I=1; i<=5; i++)); Docurl https://github.com/android?page= $i > Android.github.com. $i. Html;done

There are shortcuts.

Then look at the project's homepage URL and the warehouse URL can be found:

Warehouse url = home URL +. Git

For example, Platform_frameworks_base's homepage URL is:

Https://github.com/android/platform_frameworks_base

The Warehouse URL is:

Https://github.com/android/platform_frameworks_base.git

Of course, if you have github.com account, and the current environment PubKey has been added to the profile, you can also use the SSH protocol download (instead of https), ssh download URL is:

Git://github.com/android/platform_frameworks_base.git

It is also easier to get a warehouse URL without having to grab the project home page (just get all the warehouse URLs below).

Parse page

Now the question is--how to get the URL of the repository from these pages.

The HTML code block for the previous link in the page can be quickly found through the browser's "Review elements" menu, such as:

You can see the hyperlink to the Platform_frameworks_base item (<a href= "xxx" >) is surrounded by a

According to the two rules just found:

1. The project home URL always appears in the next line of

2. Warehouse url = home URL +. Git

You can write the Python code (getgitrepos.py) that parses the home page (warehouse URL) for each project:

#!/usr/bin/pythonimport Sysflag = Falselink_prefix = ' https://github.com ' # if you download using SSH protocol, you can change HTTPS to Gitalllines = Sys.stdin.readlines () for curline in Alllines:if curline.find (' repo-list-name ') >= 0:flag = Trueif Flag:pos = curline.f IND (' href= ') if pos >= 0:pos + = Len (' href= "') last = Curline[pos:]end = Last.find ('" ') link = last[:end]# name to Path.na me = Link[link.rfind ('/') +1:]prefix = Name.find (' platform_ ') if prefix >= 0:name = Name[len (' platform_ '):] # Ignore Plat Form_path = Name.replace (' _ ', '/') link = link_prefix + link# print ' curl ', link, ' > ', name# Https://github.com/android/ Platform_external_qemu.gitprint ' git clone ', link + '. Git ', path # output git clone command flag = False

Python parsing is only because Python is simple to write and can be used in other languages.

This Python program reads the text from the standard input and parses it, using the redirect operator when it is actually used, to replace the standard input with several sub-pages of the previously fetched github.com/android.

The program output git clone command's path parameter ignores the PLATFORM_ prefix in front of the warehouse name, which makes the final downloaded source structure similar to the repo sync down.

You can test it with a page first, and see if the parsed URL is correct (as the Python code above does), and the terminal enters the following command:

./getgitrepos.py < android.github.com.1.html

The output of this command is:

git clone https://github.com/android/platform_dalvik.git dalvikgit clone https://github.com/android/platform_ Bionic.git bionicgit clone https://github.com/android/platform_system_core.git system/coregit clone https:// Github.com/android/kernel_common.git Kernel/commongit Clone Https://github.com/android/platform_external_qemu.git External/qemugit clone https://github.com/android/platform_build.git Buildgit clone Https://github.com/android/ Platform_development.git Developmentgit Clone Https://github.com/android/platform_frameworks_base.git frameworks/ Basegit clone https://github.com/android/platform_manifest.git Manifestgit clone Https://github.com/android/ Platform_frameworks_support.git Frameworks/supportgit Clone Https://github.com/android/platform_packages_apps_ Settings.git Packages/apps/settingsgit Clone Https://github.com/android/platform_external_dhcpcd.git external/ Dhcpcdgit clone https://github.com/android/platform_external_webkit.git External/webkitgit clone https://github.com /ANdroid/platform_external_protobuf.git External/protobufgit Clone Https://github.com/android/platform_packages_ Providers_mediaprovider.git Packages/providers/mediaprovidergit Clone Https://github.com/android/platform_ External_elfutils.git External/elfutilsgit Clone Https://github.com/android/platform_external_strace.git external/ Stracegit clone https://github.com/android/platform_hardware_libhardware.git Hardware/libhardwaregit clone https:// Github.com/android/platform_external_tinyxml.git External/tinyxmlgit Clone Https://github.com/android/platform_ External_oprofile.git External/oprofile

The test basically has a problem, because the getgitrepos.py output is the git clone command, after execution can perform the actual download action, therefore, need to save these output;

Use the following command to save the git clone command parsed by each child page into a text file:

For page in ' LS android.github.com.* '; Doecho parse $page./getgitrepos.py < $page >> Gitrepos.txt;done

With gitRepos.txt full of git clone commands, you can do this with the following command:

SH gitRepos.txt

One click Download

In order to facilitate the use of the vast number of users, I have to use the above command wrote the following clonerepos.sh:

#!/bin/bash# clone AOSP Each repo from https://github.com/android# this shell depends on git, curl and Python 2.# INSTALL Depends: #sudo apt-get install git curl python## by Xu ([email protected]). # download GitHub AOSP Sub Pagesfor ((I=1; i<= 5; i++)); Doecho get AOSP subpage https://github.com/android?page= $icurl https://github.com/android?page= $i > Android.github.com. $i. html;done# parse git repo URL from each sub pagesmkdir reposcat/dev/null > gitrepos.txtfor page In ' LS android.github.com.* '; Doecho parse $page./getgitrepos.py < $page >> gitrepos.txt;done# clone repossh gitRepos.txt

In addition, I have packaged the above clonerepos.sh and getgitrepos.py together, can be used to download http://download.csdn.net/detail/xusiwei1236/7960925.

Of course, if you feel you are running trouble, you can directly copy the generated gitRepos.txt, directly pasted into the terminal to execute, you can download all the Android source code:

git clone https://github.com/android/platform_dalvik.git dalvikgit clone https://github.com/android/platform_ Bionic.git bionicgit clone https://github.com/android/platform_system_core.git system/coregit clone https:// Github.com/android/kernel_common.git Kernel/commongit Clone Https://github.com/android/platform_external_qemu.git External/qemugit clone https://github.com/android/platform_build.git Buildgit clone Https://github.com/android/ Platform_development.git Developmentgit Clone Https://github.com/android/platform_frameworks_base.git frameworks/ Basegit clone https://github.com/android/platform_manifest.git Manifestgit clone Https://github.com/android/ Platform_frameworks_support.git Frameworks/supportgit Clone Https://github.com/android/platform_packages_apps_ Settings.git Packages/apps/settingsgit Clone Https://github.com/android/platform_external_dhcpcd.git external/ Dhcpcdgit clone https://github.com/android/platform_external_webkit.git External/webkitgit clone https://github.com /ANdroid/platform_external_protobuf.git External/protobufgit Clone Https://github.com/android/platform_packages_ Providers_mediaprovider.git Packages/providers/mediaprovidergit Clone Https://github.com/android/platform_ External_elfutils.git External/elfutilsgit Clone Https://github.com/android/platform_external_strace.git external/ Stracegit clone https://github.com/android/platform_hardware_libhardware.git Hardware/libhardwaregit clone https:// Github.com/android/platform_external_tinyxml.git External/tinyxmlgit Clone Https://github.com/android/platform_ External_oprofile.git External/oprofilegit Clone Https://github.com/android/platform_external_neven.git external/ Nevengit clone https://github.com/android/platform_external_netperf.git External/netperfgit clone https:// Github.com/android/platform_external_netcat.git External/netcatgit Clone Https://github.com/android/platform_ External_libxml2.git External/libxml2git Clone Https://github.com/android/platform_external_libpng.git external/ LibpNggit clone https://github.com/android/platform_external_libpcap.git External/libpcapgit clone https://github.com/ Android/platform_external_libffi.git External/libffigit Clone Https://github.com/android/platform_external_ Jpeg.git external/jpeggit clone https://github.com/android/platform_external_jhead.git external/jheadgit clone HTTPS ://github.com/android/platform_external_jdiff.git external/jdiffgit Clone https://github.com/android/platform_ External_iptables.git External/iptablesgit Clone Https://github.com/android/platform_external_icu4c.git external/ Icu4cgit clone https://github.com/android/platform_external_giflib.git External/giflibgit clone https://github.com/ Android/platform_external_freetype.git External/freetypegit Clone Https://github.com/android/platform_external_ Fdlibm.git external/fdlibmgit clone Https://github.com/android/platform_external_expat.git external/expatgit clone Https://github.com/android/platform_external_esd.git External/esdgit Clone Https://github.com/androId/platform_external_emma.git External/emmagit Clone Https://github.com/android/platform_external_dropbear.git External/dropbeargit Clone Https://github.com/android/platform_external_apache-http.git External/apache-httpgit Clone Https://github.com/android/platform_external_openssl.git External/opensslgit Clone https://github.com/ Android/platform_external_sonivox.git External/sonivoxgit Clone Https://github.com/android/platform_packages_ Providers_telephonyprovider.git Packages/providers/telephonyprovidergit Clone Https://github.com/android/platform _packages_providers_downloadprovider.git Packages/providers/downloadprovidergit Clone Https://github.com/android/ Platform_packages_providers_contactsprovider.git Packages/providers/contactsprovidergit Clone https://github.com/ Android/platform_packages_providers_calendarprovider.git Packages/providers/calendarprovidergit Clone https:// Github.com/android/platform_packages_apps_voicedialer.git Packages/apps/voicedialergit Clone https://github.com/android/platform_packages_apps_stk.git Packages/apps/stkgit Clone Https://github.com/android/platform_packages_ Apps_soundrecorder.git Packages/apps/soundrecordergit Clone Https://github.com/android/platform_packages_apps_ Phone.git Packages/apps/phonegit Clone Https://github.com/android/platform_packages_apps_packageinstaller.git Packages/apps/packageinstallergit Clone Https://github.com/android/platform_packages_apps_music.git Packages/apps /musicgit clone https://github.com/android/platform_packages_apps_mms.git Packages/apps/mmsgit clone https:// Github.com/android/platform_packages_apps_htmlviewer.git Packages/apps/htmlviewergit Clone https://github.com/ Android/platform_packages_apps_email.git Packages/apps/emailgit Clone Https://github.com/android/platform_ Packages_apps_contacts.git Packages/apps/contactsgit Clone Https://github.com/android/platform_packages_apps_ Camera.git Packages/apps/cameragit Clone Https://github.com/android/platform_packages_apps_calendar.git packages/ Apps/calEndargit Clone Https://github.com/android/platform_packages_apps_calculator.git Packages/apps/calculatorgit clone Https://github.com/android/platform_packages_apps_browser.git Packages/apps/browsergit Clone https://github.com/ Android/platform_hardware_ril.git Hardware/rilgit Clone Https://github.com/android/platform_external_zlib.git External/zlibgit clone https://github.com/android/platform_external_yaffs2.git External/yaffs2git clone https:// Github.com/android/platform_external_tcpdump.git External/tcpdumpgit Clone Https://github.com/android/platform_ External_tagsoup.git External/tagsoupgit Clone Https://github.com/android/platform_external_srec.git external/ Srecgit clone https://github.com/android/platform_external_sqlite.git External/sqlitegit clone https://github.com/ Android/platform_external_skia.git External/skiagit Clone Https://github.com/android/platform_external_ Safe-iop.git external/safe-iopgit clone Https://github.com/android/platform_external_ppp.git external/pppgit clone Https://github.com/android/kernel_msm.git Kernel/msmgit Clone Https://github.com/android/platform_system_ Bluetooth.git system/bluetoothgit clone https://github.com/android/platform_prebuilt.git prebuiltgit clone https:// Github.com/android/platform_packages_providers_drmprovider.git Packages/providers/drmprovidergit Clone https:// Github.com/android/platform_external_ping.git External/pinggit Clone https://github.com/android/platform_external _dbus.git external/dbusgit clone https://github.com/android/tools_repo.git tools/repogit clone https://github.com/ Android/android.github.io.git Android.github.iogit Clone Https://github.com/android/platform_external_wpa_ Supplicant.git External/wpa/supplicantgit Clone Https://github.com/android/platform_packages_apps_ Googlesearch.git Packages/apps/googlesearchgit Clone Https://github.com/android/platform_system_wlan_ti.git System /wlan/tigit Clone Https://github.com/android/platform_packages_providers_googlecontactsprovider.git packages/ Providers/googLecontactsprovidergit Clone Https://github.com/android/platform_packages_apps_im.git Packages/apps/imgit clone Https://github.com/android/platform_packages_providers_improvider.git Packages/providers/improvidergit Clone Https://github.com/android/platform_external_tremor.git External/tremorgit Clone Https://github.com/android/ Platform_packages_apps_sync.git Packages/apps/syncgit Clone Https://github.com/android/platform_packages_apps_ Alarmclock.git Packages/apps/alarmclockgit Clone Https://github.com/android/platform_external_aes.git external/ Aesgit clone https://github.com/android/platform_external_clearsilver.git External/clearsilvergit clone https:// Github.com/android/platform_external_googleclient.git External/googleclientgit Clone Https://github.com/android/ Platform_packages_apps_launcher.git Packages/apps/launchergit Clone Https://github.com/android/platform_external_ Bluez.git external/bluezgit clone https://github.com/android/platform_external_gdata.git external/gdatagit clone hTtps://github.com/android/platform_frameworks_opt_com.google.android.git Frameworks/opt/com.google.androidgit Clone Https://github.com/android/platform_external_elfcopy.git External/elfcopygit Clone https://github.com/ Android/platform_frameworks_policies_base.git Frameworks/policies/basegit Clone Https://github.com/android/ Platform_packages_apps_updater.git Packages/apps/updatergit Clone Https://github.com/android/platform_external_ Opencore.git External/opencore

Download the latest Android source from GitHub