[Draft] automatically downloads complete. NET source code

Updated: programmers have similar thinking modes. Kerem Kusmezer and John Robbin released a similar tool. NET Mass Downloader. Since some of them will not reinvent the wheel with me, the following project will stop maintenance, and friends who need it will use the tool directly. For detailed instructions, refer

Download All the. NET Reference Source Code at Once with Net Mass Downloader

Update: you have applied for a project on sourceforge. If you are interested, you can download the code or submit bugs and suggestions for improvement. However, it is estimated that I will not be able to continue to deal with the problem in a few years. I will try to dig a hole for the time being.

Since last year, MS began to shout about opening. NET Source code. Now, in a vague authorization protocol, the Source Server is used. Although it is open-source, it currently only provides several core modules, and it also limits online access to VS2008.
In fact, since the decision to open up, why do we have to make so many restrictions, so that even when it is not easy to use, you have to do it yourself, write a program to directly simulate VS2008 to get the source code, all drag back for backup at a time.
Upload a System. Xml source code package and a prototype of the program that can be run theoretically. If you are interested, try it. Now we have time to sort out the relevant code and then release it. Let's take a look at the ideas below.

First, although VS2008 requires a patch to be used for source code debugging, this is not a necessary condition for downloading the source code. The Source Server mechanism on which MS releases Source code has been introduced through the WinDbg release package for a long time. If you are interested, you can use the WinDbg tool to set up the Source Server. for detailed steps, see the documentation and scripts in the % Debugging Tools for Windows % \ sdk \ srcsrv \ directory.
In other words, the Source Server is from. in the pdb debugging symbol file, obtain the list of bound indexed files, splice a URL Based on the server specified by the system _ NT_SOURCE_PATH environment variable, and download the file over http. For details about the configuration, refer
Tracing ing Visual Studio to Debug. NET Framework Source Code
What's annoying is that MS has made some effort in dbghelp. dll, and it is not allowed to directly download debugging symbols and source code through URLs. Just as scz has previously studied the behavior of the symbol server, it is basically determined that the ms server is determined based on the user agent. That is to say, when a request is initiated, you can change the current http user agent to the ms approval method to simulate the work completed by dbghelp. dll through srcsrv. dll. The specific code is similar

Public const string DefaultUserAgent = "Microsoft-Symbol-Server/9.9.9 (C; V ;;)";

Public void Restart (Uri uri)
{
_ Request = (HttpWebRequest) WebRequest. Create (uri );
_ Request. UserAgent = DefaultUserAgent;

IAsyncResult result = (IAsyncResult) _ request. BeginGetResponse (
New AsyncCallback (ResponseCallback), this );
ThreadPool. RegisterWaitForSingleObject (result. AsyncWaitHandle,
New WaitOrTimerCallback (TimeoutCallback), this, _ timeout, true );
}

The other thing is that after each link is created, the MS Server Returns a loose license statement. The srcsrv. dll dialog box will pop up to allow you to continue downloading.
For example, you can use the command line to download an object. Wget -- user-agent = Microsoft-Symbol-Server/9.9.9.9
Http: // ReferenceSource.microsoft.com/source/.net/8.0/DEVDIV/
Depot/DevDiv/releases/whidbey/REDBITS/ndp/fx/src/Misc/
InvariantComparer. cs/1/InvariantComparer. cs

Will directly return a pair of junk information, similar SYMSRVCOMMAND:
YESNODIALOG:
IDYES: Accept = 0cf80f849aa7449e9d064ade971bd887
IDNO: Decline = 0cf80f849aa7449e9d064ade971bd887
TITLE: End User License Agreement
BUTTONDESCRIPTION: Please read carefully and understand the license agreement above.
If you want to accept the license agreement, please click the "Accept" button.
TEXT: This license applies to the. NET Framework components that Microsoft makes available to you
In source code form. To print these terms, select the contents of this area and copy then paste
Into an application that can print text.

MICROSOFT. NET FRAMEWORK REFERENCE LICENSE
. NET FRAMEWORK REFERENCE SOURCE CODE


This license governs use of the accompanying software. If you use the software, you accept this license.
If you do not accept the license, do not use the software.
1. Definitions
The terms "reproduce," "reproduction" and "distribution" have the same meaning here as under U. S. copyright law.
 

The solution is very simple. You can use a regular expression to find a randomly generated id, and then the program automatically Accept. The code is similar

Private static Regex _ accept = new Regex (@ "IDYES: Accept = (\ w *)");

If (task. _ auth)
{
}
Else
{
String license = Encoding. Unicode. GetString (task. _ buf, 0, read );

Match m = _ accept. Match (license );

If (m. Success)
{

Task. _ auth = true;

String id = m. Groups [1]. Value;

_ Logger. DebugFormat ("Accept license id {0} for {1}", id, task. _ request. Address );

Uri uri = new Uri (string. Format ("{0 }? Accept = {1} ", task. _ uri. ToString (), id ));

Task. Restart (uri );
}
}

There is nothing else to note, that is, what is a basic http request call?

To retrieve the file name list from. pdb, you need to call a group of APIs in dbghelp. dll.
SymInitializeW/SymCleanup is used to initialize and analyze the symbol engine
SymLoadModuleExW/SymUnloadModule64 is used to load the module for convenient source code
SymEnumSourceFilesW is used to traverse a list with the. pdb symbol and containing the indexed source code.
Two APIs, SymGetSourceFileW and SymGetSourceFileTokenW/SymGetSourceFileFromToken, can be used to obtain the download URL of the specific source code. Then, you can use the preceding two methods to process the address.

Note: The current implementation is just a prototype and needs to be manually downloaded and configured. pdb files and directly initiate a large number of asynchronous http requests. For friends with slow network speeds, you may need to use the-I parameter to increase the timeout time (indeed 15 minutes)