Java web crawler Technology

Recently encountered in the project to apply to the web crawler technology problems, write a summary of the record, at the same time give a small example, to provide a friend to download research and study.

The project adopts web crawling image data, streaming layout, responsive layout, lazy loading, dynamic switching load to realize the crawl of the webpage.

Here are some of the image effects that are implemented:

1. Implement Web Capture image Data

2. Implement dynamic switch loading

3. Implement lazy Loading

4. Implementing responsive Layouts

The above is a few key points of implementation, the specific code please download: alioth_spider; Development tools: MyEclipse 2014

Only in this record learning to share, and constantly accumulate progress.

Java web crawler Technology