Complete example of hierarchical clustering of documents using R (TM package)

# R attempts to perform hierarchical clustering on documents
# Connect to the database and read the files in the database # load the package Library (rmysql) # establish a connection conn <-dbconnect (dbdriver ("MySQL"), dbname = "eswp ", user = "root", password = "root") # read Table 2008 yearnewtext = dbreadtable (Conn, "2008 yearnew ")[, 2: 2] # Read Only the column of the mesh word and modify the number of rows read using the first subscript # load the TM package Library (TM) # Build a corpus = corpus (vectorsource (text) # create a document-word matrix from the corpus and use TF-IDF for representation. stopwords = stopwords ("mesh ") indicates that the mesh disabled Word Table is used. The disabled Word Table is placed in the stopwords folder of the TM package. DTM = documenttermmatrix (corpus, control = List (stopwords = stopwords ("mesh"), weighting = weighttfidf )) # Calculate the similarity between documents using the angle cosine dist_dtm <-dissimilarity (DTM, method = 'cosine') # perform hierarchical clustering based on the similarity, and use average (class average method distance, other available distances include single, complete, median, mcquitty, average, centroid, Ward, and other HC <-hclust (dist_dtm, method = 'ave ') # cluster graph plot (HC) # if the number of classes is large, the number of classes will not be clear. Use the following method to draw a large pixel graph PNG (width = 4000, Height = 3000) # change the output device to PNG. The pixel size is as large as possible. This is a clear setting for 200 documents. # cex indicates the label size. At the same time, you can use cex. axis attribute to change the number size in the coordinate system, using cex. lab changes the size of the matrix name below # Use cex. main to change the title size, use cex. sub changes the size of the cluster method name below. LWD is the width of the midline graph. In this case, the graph will see plot (HC, cex = 2, cex. axis = 3, cex. lab = 3, cex. main = 3, cex. sub = 3, LWD = 1.5) Dev. off () # Use the cutree function to view the clustering result. k = 2 indicates dividing the result into two types. This method is not effective and is often used by many types, A special less Ct = cutree (HC, K = 2) # Use the cutree function to view the cluster result by height CT1 = cutree (HC, H = 0.9) # result statistics table (CT) # view the documents in the first class. If you want to view other classes, and so on, CT1 <-CT [Ct = 1]; CT1