F # Tour 5-little Practice download Web page (crawler base Library)

Reference Article: https://swlaschin.gitbooks.io/fsharpforfunandprofit/content/posts/fvsc-download.html

The reference article teaches us if we use it in F #. NET Library to download a Web page, here, let me scatter it and make it into a base library that can be used to help write Crawlers.

first, I've made several changes to the downloaded code:

1, removed the callback, directly changed to save the text to the file, note that if the download picture can not be written like This.

2, use the flow to step-by-step Call. net, and Adds exception handling.

3. Add an async async method, which is the same as C # async, Await.

next, with async, It's natural for this download to be able to be concurrent, and the underlying function for auxiliary concurrency is this:

1, found on the StackOverflow a throttle auxiliary function, can be used to control the number of concurrent, Great.

2, with the flow and async.parallel implementation of Concurrency.

F # Tour 5-little Practice download Web page (crawler base Library)