Development Experiment of general network Collection System

Collection, as its name implies, is to collect things from others' websites to their own databases and publish them to their own websites.

Although this is a bit immoral, we can study it technically. This is understandable as long as you don't have to harm others and do not have a negative impact on society.

In other words, you can have no car, but you need a driver's license, don't get someone else to give you a free car, but you can't drive it, it's boring...

Okay, no nonsense.

 

During the National Day, I studied collection out of interest.

The goal is to collect the main content of a website and download the images to the local device. You can only achieve this purpose.

First, let's look at the two figures:

 

Click the open link to view the specific collection results

 

The following is an explanation:

 

The entire process takes only two steps,

Step 1: Specify collection rules for collected websites

Step 2: Perform collection

 

The most important step is the first step, because this is the source of this collection. The collection of specific URLs extracts valid addresses based on regular expressions.

For example, you set the Domain Name of the website to QQ. Com, you must first set this domain name, and then set a specific valid address

,

Each website contains two headers: the title and the specific content. Therefore, you only need to set the two headers to extract the content. As for the content

Images are also extracted using regular expressions and downloaded to the local device using data streams.

 

Now you can create thumbnails or add watermarks locally.

 

 

 

 

Development Experiment of general network Collection System