[Crawler] Use heritrix 3

Https://webarchive.jira.com/wiki/display/Heritrix/A+Quick+Guide+to+Running+Your+First+Crawl+Job

1. Download heritrix3 and decompress it.

2. Run the command line in the bin directory> heritrix. CMD-A admin: Admin to start

You can use heritrix -- help to view help information.

3. Open the browser address 127.0.0.1: 8443. the user name and password are admin and admin (the previous version seems to be 127.0.0.1: 8080)

I cannot access it in the browser. I checked the exception and found that HTTP is safe or something. I used the address https: // 127.0.0.1: 8443.

4. On the homepage, create a job

Then edit the configuration in the job.

In

1), metadata. operatorcontacturl = http://www.archive.org here is not seed URL

2) populate<Prop>Element ofLongeroverridesBean with the seed values for the crawl.

This person places seed URLs.

Then save

5. Build, launch, and unpause in sequence.