Algorithms:
Design and Analysis, part 1
This assignment is the most difficult of an algorithm course. In addition to algorithms, I think implementation is also involved. Because many programming languages cannot process a large number of recursive calls.
Question description
Download the text file here. Zipped version here.
(Right click and save link)
The file contains the edges of a directed graph. vertices are labeled as positive integers from 1 to 875714. every row indicates an edge, the vertex label in first column is the tail and the vertex label in second column is the head (recall the graph is
Directed, and the edges are directed from the first column vertex to the second column vertex). So for example, 11th Row
Looks liks: "2 47646". This just means that the vertex with label 2 has an outgoing edge to the vertex with label 47646
Your task is to code up the algorithm from the video lectures for computing stronugly connected components (SCCS), and to run this algorithm on the given graph.
Output Format: You shoshould output the sizes of the 5 largest SCCs in the given graph, in decreasing order of sizes, separated by commas (avoid any spaces ). so if your algorithm computes the sizes of the five largest SCCs to be 500,400,300,200 and 100,
Then your answer shoshould be "500,400,300,200,100 ". if your algorithm finds less than 5 SCCs, then write 0 for the remaining terms. thus, if your algorithm computes only 3 SCCs whose sizes are 400,300, and 100, then your answer shocould be "400,300,100 ".
Warning: This is the most challenging programming assignment of the course. because of the size of the graph you may have to manage memory carefully. the best way to do this depends on your programming language and environment, and we stronugly suggest that
You exchange tips for doing this on the discussion forums.
Algorithm Implementation
The implementation of the algorithm is relatively simple, divided into three steps: the first step is to find the transpose graph, the second step is to traverse the transpose graph and get the topological sequence, step 3: Use this topological sequence to perform DFS on the graph. The point obtained by each DFS is a set of SCC/strong connections.
The Code implemented using python is as follows:
def firstdfs(vertexind): global fs,isexplored,visitordered,mapDictT if len(mapDictT[vertexind])>0: for ind in mapDictT[vertexind]: if not isexplored[ind-1]: isexplored[ind-1]=True firstdfs(ind) visitordered[fs-1]=vertexind #print(str(vertexind)+' fs: '+str(fs)) fs=fs-1def seconddfs(vertexind): global s,secisexplored,header,mapDict if len(mapDict[vertexind])==0:return for ind in mapDict[vertexind]: if not secisexplored[ind-1]: secisexplored[ind-1]=True seconddfs(ind) header[s-1]+=1maplength=875714#maplength=8f=open('SCC.txt','r')mapDict={x:[] for x in range(1,maplength+1)}mapDictT={x:[] for x in range(1,maplength+1)}for line in f.readlines(): tmp=[int(x) for x in line.split()] mapDict[tmp[0]].append(tmp[1]) mapDictT[tmp[1]].append(tmp[0])f.closefs=maplengthisexplored=[False for x in range(1,maplength+1)]secisexplored=[False for x in range(1,maplength+1)]visitordered=[0 for x in range(1,maplength+1)]header=[0 for x in range(1,maplength+1)]for ind in range(1,maplength+1): if not isexplored[ind-1]: #print('Begin from: '+str(ind)) isexplored[ind-1]=True firstdfs(ind)print('Second DFS')for ind in visitordered: if not secisexplored[ind-1]: s=ind secisexplored[ind-1]=True seconddfs(ind)header.sort(reverse=True)print(header[0:20])
The graph used for testing is stored in a text file. The content of the file used for testing is as follows:
1 22 62 32 43 13 44 55 46 56 77 67 88 58 7
Note: Change maplength to 8 during testing. The first five outputs should be
3,3,2,0,0
Python iterations
Python has a default Function Iteration limit, which generally does not exceed 1000 by default. If this limit is exceeded, stack overflow errors may occur. Use the following code to change the default iteration limit and display
import syssys.setrecursionlimit(80000000)print(sys.getrecursionlimit())
Use the following code to test the number of iterations that can be achieved
def f(d):if d%500==0:print(d)f(d+1)f*(1)
After the above Code is tested, the maximum number of 8g iterations of python3.3 memory in Win8 is about 4000, and the maximum number of 16g iterations of python3.2 memory In Debian 6 is about 26000. The memory is not exhausted. Although the frequency limit is relaxed, it is still restricted for some reasons. In this case, no stack overflow error is reported, but the program will also crash.
Where is the problem? I went to the Forum and looked at other people's discussions to find out whether the stack size is insufficient.
Improved version of the program
Set the stack size to 64 MB and then OK.
The complete code is as follows:
import sys,threadingsys.setrecursionlimit(3000000)threading.stack_size(67108864)def firstdfs(vertexind): global fs,isexplored,visitordered,mapDictT if len(mapDictT[vertexind])>0: for ind in mapDictT[vertexind]: if not isexplored[ind-1]: isexplored[ind-1]=True firstdfs(ind) visitordered[fs-1]=vertexind #print(str(vertexind)+' fs: '+str(fs)) fs=fs-1def seconddfs(vertexind): global s,secisexplored,header,mapDict if len(mapDict[vertexind])==0:return for ind in mapDict[vertexind]: if not secisexplored[ind-1]: secisexplored[ind-1]=True seconddfs(ind) header[s-1]+=1def sccmain(): global mapDict,mapDictT,fs,isexplored,visitordered,s,secisexplored,header maplength=875714 #maplength=11 f=open('SCC.txt','r') mapDict={x:[] for x in range(1,maplength+1)} mapDictT={x:[] for x in range(1,maplength+1)} for line in f.readlines(): tmp=[int(x) for x in line.split()] mapDict[tmp[0]].append(tmp[1]) mapDictT[tmp[1]].append(tmp[0]) f.close fs=maplength isexplored=[False for x in range(1,maplength+1)] secisexplored=[False for x in range(1,maplength+1)] visitordered=[0 for x in range(1,maplength+1)] header=[0 for x in range(1,maplength+1)] for ind in range(1,maplength+1): if not isexplored[ind-1]: #print('Begin from: '+str(ind)) isexplored[ind-1]=True firstdfs(ind) print('Second DFS') for ind in visitordered: if not secisexplored[ind-1]: s=ind secisexplored[ind-1]=True seconddfs(ind) header.sort(reverse=True) print(header[0:20])if __name__ =='__main__': thread=threading.Thread(target=sccmain) thread.start()