Recently began to learn Python language, learned "with children to learn programming", read the official tutorial. Now is looking at the Chinese version of the "Learn Python", I feel this book is really too much, more like a reference book, learning interest is also reduced, I would like to get some web crawl, crawl data, and so on, but feel that the foundation is not enough, you have experienced predecessors to request advice (I am not a computer professional, But in this area is still very interested in, about 2 hours a day to learn about Python.
Reply content:
Interest will be consumed in the time of reading and not working.
If you want to learn reptiles and worry about the foundation is not enough, you can first try to use Scrapy.
Take 1 days to go through the document in 2 hours of progress per day.
Scrapy 0.24 Documentation
Since the master is a non-computer major, you may also need to know the following:
- The basics of HTML tags, you can read the meaning of the label is good
- HTTP basics, GET, post,json what the heck are these messy nouns?
Above are available in W3school online tutorials
This site gets a basic understanding.
Assuming that the above basic problem has been, you can find the watercress or some other small station, reference to the document began to write a reptile
If you encounter some need to verify or a cookie to give up first, the introduction of the first focus on allowing the crawler to run, which is to maintain learning interest in the method
When you reach the point where you can write a crawler crawl data, you can consider optimization, what anti-crawling, user authentication, cookies, impersonation login, multi-threading to improve crawl efficiency ...
I was first to the watercress under the hand, on the 2nd day of writing a small reptile climbed more than 1 million of the book data stored in the database, with the data but do not know what to do, feel no meaning. (It is worth mentioning that the watercress has an anti-crawling verification, the crawl speed will require you to lose the verification code to determine that you are not a robot. Remember to set the delay, or your crawler will be very annoying)
So plan to climb know, I heard that the anti-climb more serious (before trying to be known to the IP segment), so intends to get a proxy pool through high anonymous agent to crawl know, in the Internet to find an agent to buy half a year of service, the results found that the agent is not even, feel too troublesome, simply to climb agent, During the feel scrapy to climb agent this little thing a little overkill feeling, so oneself with Pycurl + Beatifulsoup wrote a simple crawler to climb, now have some short time more stable free agent can use, feel can start to crawl know. Currently only crawls a person's dynamic concurrent mail to their own mailbox, well, as this guy did.
What cool, interesting, useful things can you do with crawler technology? -Woosen's answer
The above experience from the beginning of the document to the present, it took about 10 days, during the most time to check the information. Also consider the first to write their own or use the framework and other unnecessary consumption, so do not think so much, decide a direction straight walk, you will find that you feel like difficult things, in a few days after the feeling is not so difficult I learned python in the winter vacation and learned to write reptiles for only 10 days ... While doing a reading to read other people's blog, find source analysis.
In the process of writing, you can find out what you lack, what you need to learn, how much you want to learn, and the most important things to learn about and what to do.
After writing some simple reptiles and small scripts, the system began to read, because this time know how to learn the system.
Purposeful learning progresses much faster than aimless learning. After understanding the basic grammar, began to try to write something, write a small game, and so on, although it will find it difficult, but more than simple to learn more efficient multi-fish C Forum 0 Basic introductory Learning Python video, there are teaching you how to make a crawler crawling sister pictures, very interesting. Forum also has a lot of users to write small projects can refer to. Url / http bbs.fishc.com
Recommend a book "Basic Python Tutorial", the end of the book has several small projects, the implementation is very interesting.
In short, learning programming language is to solve the practical problems in life, do not learn to learn, do not forget the original intention, to have a steady flow of learning power. After mastering the basic grammar, you will be interested in an in-depth study, such as I am interested in pygame, I wrote a plane game. 1, a little can read English books
2. Type the electronic library in the browser. Download Books free. Finding Boooks
3. Enter "with Python" and press Enter
4. Find the Book of interest and click Download
5, PA PA Since you are interested in crawling Web pages, also learned a book and the official tutorial, I feel completely can learn Web crawl. Don't think of it so complicated, I think you have to look at the "Learn Python" courage, certainly not have any people or things forcing you to crawl the web, since it is playing, why wary it? But the book is so thick, and so you watch the lily is cold, if not to learn the language itself, it is recommended not to chew it. Also, web crawling is not just about learning this python, you may need to know some other knowledge.
I am also a python lover, learn it purely out of interest, and occasionally can use it in learning. I learn python only a few months, but also simple to try to crawl the web, feel like to get started is not so difficult, as long as you are good English or search the dictionary, to crawl the library of the official documents of the Web to learn slowly. It is recommended that you first look at these related libraries: Requests, BeautifulSoup, lxml. Then you can look at Scrapy, and then I don't know.
Progress together! Go get some damage.
Have read so many things, you don't want to write something?
Where do you start?
1 reptiles. You can get a lot of interesting stories when you search the crawler, and maybe get someone else's code.
2 robots. This refers specifically to chat bot and social network bot. For example, based on the WeChat framework and everyone API write a confession wall, such as based on XMPP write a turn ICQ into a browser (by the wall? ) Robots and so on. Implement a chat room on a service that does not support group chat or something, open a query for the weather public number, use a chat room to open an DND (Dungeons and Dragons), these code architectures are not too big
3 Make a website. The most repeated invention of the wheel is the blog. Do not spit groove. But doing this may require a bit of JS Foundation. Well, "point" and "base" may be removed, perhaps not.
4 fork. On which website did you see a nice little game? Fork one out and try it! You can also get a very interesting basic version of the game when the artwork is not enough. For example, make a 2048 out of the chant.
5 kit. Gadgets are also a good choice, the same is 2048, write a heuristic search of the 2048 helper program lets you determine the current state which direction is more advantageous is not also good.
Well? You don't know how to do this, do you? Never learned anything?
However, you have mastered the majority of Python, and the rest is not a python problem.
What you need to learn is to collect information, such as a reptile tutor, which is simple in a world of Google, but in a world without Google, it may take a little more traditional approach or a way to access Google, which is simpler and harder to say.
What you need to learn is to analyze the requirements, break them down, and then analyze them, so that they are arranged only within the logic.
What you need to learn is to organize the code, and this may help you with Python, and it helps to have a look at PEP8.
From now on, the book will no longer be your teacher, but your tool, you will learn these things from a variety of places, you may not learn them everywhere, they may be in other people's code, other people's needs in the document, even in your dreams.
Research, design, coding, refactoring, programming is in it, and maybe the world is in it.
To continue writing, Kung Fu is already out of the poem. If you're getting started with Python, the first tweet, "A Byte of Python," takes a short time to read, and you can write something small after you've read it. Back in practice when you can more online to find some examples of reference, see more Python Manuals, familiar with the good. As said on the second floor, interest will always read and do not work the time is consumed, want to learn a technology, is the need to constantly hone, and not just to see, only from the hard work to find experience, if really do not know how to carry out the project actual combat, You can see the use of the paginator above the Wheat Academy.
This project is practical, can imitate slowly, then become own knowledge. Start by looking at head first Python and then write a simple web, and then follow your own research direction to go deeper. Have good code to remember to share, look at the increase in download volume will be very fulfilling:)
