GitHub scrapy-redis library has been upgraded to make it compatible with the latest version of scrapy

1. issues before code upgrade:

With the popularity of the scrapy library, scrapy-redis, as a tool that supports distributed crawling using redis, is constantly being discovered.

However, the problem you will find during use is that scrapy-redis is based on the older version of scrapy. Therefore, when you are using the new version of scrapy, a warning is inevitable, or even errors.

With regard to this question, I have mentioned it in the scrapy-redis database and opened an issue and discussed it with the author. After determining the idea, I started to upgrade the code.

Solution:

It took some time to modify and submit it to the original code library.Pull requestThe author already has merge. Address: https://github.com/darkrho/scrapy-redis

Others:

If you find any problems during use, if it is related to the compatibility upgrade, please alsoSend an emailOr open an issue directly on GitHub.

2. About Travis Ci

I started to understand it only when I was involved in this project. The build icon often found in readme indicates the status after being built using Travis CI.

Travis CI, a cloud continuous integration service provided for the open-source community. Its role is to build on the cloud and feedback the build status. You only need to install the Travis hook in the corresponding code library configuration and configure the. Travis. yml file in the root directory of the library. For details, see Doc.

GitHub scrapy-redis library has been upgraded to make it compatible with the latest version of scrapy