Runtimeerror: Maximum recursion depth exceeded Python recursive Depth Error

During this time, I wrote a crawling script using python, and used the breadth-first traversal to crawl pages. However, when the number of traversal attempts reaches 900, the following error occurs:

Runtimeerror: Maximum recursion depth exceeded

I checked on the Internet and found that the default recursive depth of python is very limited, probably more than 900. When the recursive depth exceeds this value, this exception will be thrown.

The solution is to manually set the recursive call depth

 

1. Import sys
   
2. SYS. setrecursionlimit (1000000) # For example, set this parameter to 1 million