Scrapy wrong record

Before using Scrapy to write a crawler on the server, it has been used well. Results the day before yesterday a classmate installed on the top of the NLTK can no longer be used (whether using shell or crawl), the error is as follows:

Traceback (most recent): File "/usr/local/bin/scrapy", line 9, in <module> load_entry_point (' scrapy= =0.24.4 ', ' console_scripts ', ' scrapy ') () File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 143, In Execute _run_print_help (parser, _run_command, cmd, args, opts) File "/usr/local/lib/python2.7/dist-packages/scrap y/cmdline.py ", line-I, in _run_print_help func (*a, **kw) File"/usr/local/lib/python2.7/dist-packages/scrapy/cmdlin e.py ", line, in _run_command cmd.run (args, opts) File"/usr/local/lib/python2.7/dist-packages/scrapy/commands/sh ell.py ", line, in Run self.crawler_process.start_crawling () File"/usr/local/lib/python2.7/dist-packages/scrapy/c rawler.py ", line 124, in Start_crawling return Self._start_crawler () are not None File"/usr/local/lib/python2.7/dist -packages/scrapy/crawler.py ", line 139, in _start_crawler crawler.configure () File"/usr/local/lib/python2.7/dist-pa Ckages/scrapy/crawler.py ", lINE, in Configure self.extensions = Extensionmanager.from_crawler (self) File/usr/local/lib/python2.7/dist-packa ges/scrapy/middleware.py ", line, in From_crawler return cls.from_settings (crawler.settings, crawler) File"/usr/l ocal/lib/python2.7/dist-packages/scrapy/middleware.py ", line, in from_settings mwcls = Load_object (Clspath) File "/usr/local/lib/python2.7/dist-packages/scrapy/utils/misc.py", line, in Load_object raise Importerror ("Error Loadi  Ng object '%s ':%s '% (path, e)) Importerror:error loading object ' scrapy.telnet.TelnetConsole ': No module named Conch

Hint Conch This module is not found, this may be Sys.path was changed. Luckily, there was a Python interactive window on the Tmux that could look up the old Sys.path. In contrast to the present Sys.path, two more are found:

'/usr/local/lib/python2.7/dist-packages/jieba-0.36.1-py2.7.egg ',
'/usr/local/lib/python2.7/dist-packages/ Setuptools-15.0-py2.7.egg '

It should be said that there is nothing to be found Sys.path missing something is, this temporary see what.

Then along the python error message, try to reproduce the errors simply.
The error is thrown in the/usr/local/lib/python2.7/dist-packages/scrapy/utils/misc.py, and the code is as follows:

    Try:
        mod = import_module (module)
    except Importerror as e:
        raise Importerror ("Error loading object '%s ':%s '% (path, E))

So it can reproduce this error:

>>> from importlib import import_module
>>> import_module (' scrapy.telnet ')

The same is obtained No module named Conch Error. The first line of code in the telnet.py of the Scrapy project is this:

From twisted.conch import Manhole, Telnet

This line of code was not successful because the Conch module could not be found. Attempting to import the Twisted.conch directly is also a failure.

Python's third party package is in the Dist-packages directory,/usr/local/lib/python2.7/dist-packages I found the twisted directory, there are conch.
Then I use the locate command to see where the twisted directory is in the system, because it is possible to replace what was originally available.

$ locate Twisted

Finally found that there is a twisted directory under the/usr/lib/python2.7/dist-packages, and it does not conch this subdirectory. To view _version.py, there is a line:

     Version = versions. Version (' twisted ', 11, 1, 0)

And look at the original/usr/local/lib/python2.7/dist-packages/twisted in the _version.py, this line is:

     Version = versions. Version (' twisted ', 14, 0, 2)

This means that the current search along the Sys.path is the old version of the twisted (who may be installed before), Sys.path was changed, and pointed to the old twisted. Careful comparison of Sys.path, there are two lines of change in order.
This is the normal Sys.path:

"'/usr/local/lib/python2.7/dist-packages/requests-2.0.0-py2.7.egg '/usr/local/lib/python2.7/dist-packages/ Kafka_python-0.8.1_1-py2.7.egg '/usr/local/lib/python2.7/dist-packages/tox-1.6.1-py2.7.egg '/usr/local/lib/ Python2.7/dist-packages/py-1.4.19-py2.7.egg '/usr/local/lib/python2.7/dist-packages/ Virtualenv-1.10.1-py2.7.egg '/usr/local/lib/python2.7/dist-packages/pymongo-2.6.3-py2.7-linux-x86_64.egg '/usr /lib/python2.7 '/usr/lib/python2.7/plat-linux2 '/usr/lib/python2.7/lib-tk '/usr/lib/python2.7/lib-old '/usr/              Lib/python2.7/lib-dynload '/usr/local/lib/python2.7/dist-packages ' ### attention to this line of '/usr/lib/python2.7/dist-packages ' ### also has this line of '/usr/lib/python2.7/dist-packages/pil '/usr/lib/python2.7/dist-packages/gst-0.10 '/usr/lib/python2. 7/dist-packages/gtk-2.0 '/usr/lib/pymodules/python2.7 '/usr/lib/python2.7/dist-packages/ubuntu-sso-client '/ Usr/lib/python2.7/dist-packages/ubuntuone-client '/usr/lib/python2.7/dist-packages/ubuntuone-control-panel '/ Usr/lib/python2.7/dist-packages/ubuntuone-couch '/usr/lib/python2.7/dist-packages/ubuntuone-installer '/usr/lib/ Python2.7/dist-packages/ubuntuone-storage-protocol '

This is being loaded after something, that is now the Sys.path:

"'/usr/local/lib/python2.7/dist-packages/requests-2.0.0-py2.7.egg '/usr/local/lib/python2.7/dist-packages/ Kafka_python-0.8.1_1-py2.7.egg '/usr/local/lib/python2.7/dist-packages/tox-1.6.1-py2.7.egg '/usr/local/lib/ Python2.7/dist-packages/py-1.4.19-py2.7.egg '/usr/local/lib/python2.7/dist-packages/ Virtualenv-1.10.1-py2.7.egg '/usr/local/lib/python2.7/dist-packages/pymongo-2.6.3-py2.7-linux-x86_64.egg '/usr /local/lib/python2.7/dist-packages/setuptools-15.0-py2.7.egg '/usr/lib/python2.7/dist-packages ' the line was moved to the front '/US R/local/lib/python2.7/dist-packages/jieba-0.36.1-py2.7.egg '/usr/lib/python2.7 '/usr/lib/python2.7/ Plat-linux2 '/usr/lib/python2.7/lib-tk '/usr/lib/python2.7/lib-old '/usr/lib/python2.7/lib-dynload '/usr/local /lib/python2.7/dist-packages ' # This line is in the back of '/usr/lib/python2.7/dist-packages/pil '/usr/lib/python2.7/ dist-packages/gst-0.10 '/usr/lib/python2.7/dist-packages/gtk-2.0 '/usr/lib/pymodules/python2.7 '/usr/lib/ python2.7/dist-packages/ubuntu-sso-Client '/usr/lib/python2.7/dist-packages/ubuntuone-client '/usr/lib/python2.7/dist-packages/ Ubuntuone-control-panel '/usr/lib/python2.7/dist-packages/ubuntuone-couch '/usr/lib/python2.7/dist-packages/ Ubuntuone-installer '/usr/lib/python2.7/dist-packages/ubuntuone-storage-protocol '

Note that the two lines,/usr/lib/python2.7/dist-packages in the new Sys.path was mentioned in front, so first found that the old version of the twisted. Finally know why a mistake, long sigh ~
Next, the old version of the Twisted directory deleted (or renamed) on the line, the same deal with the corresponding number of egg-info files. The file name is as follows:

Twisted  twisted_core-11.1.0.egg-info  twisted_names-11.1.0.egg-info  twisted_web-11.1.0.egg-info

Of course also can change the default Sys.path, put/usr/local/lib/python2.7/dist-packages in front. But given the likelihood that it will affect others, or simply throw away the old version, it's no use anyway.

The final solution, although simple, but still spent a lot of time to find this problem, the server is wrong in itself is a test of patience.
Writing this article may have no direct reference value, because everyone's environment is different, the cause of the error is not the same. Just wrong way of thinking, may give a little help to helpless friends, because at the beginning I encountered this problem, is also very annoyed, the Internet can not find anything to help the information. Eventually, you have to calm down and deepen your understanding of Python. In short, there is the belief that problems can always be solved.