PHP redemption auto Get generate keyword function

PHP implementation automatically generate keyword function
previously written programs have been avoiding this problem, tag what is required to use the program's self-input, for some lazy people and for the experience of the program, it is hoped to have automatic generation of article keywords, automatic access to the article tag similar function, this time in order to meet the new project, so tinkering a night, studied this feature.
to achieve the ability to automatically get keywords, you can probably divide into three steps
1, the title and content are segmented separately by the word segmentation algorithm, and the keywords and frequency are extracted. At present, the main two algorithms are Ictclas and hidden Markov models of CAs. But these two are too high-end, have a certain threshold, and are only support C++/java. There are currently two PHP-based PSCWS and HTTPCWS that are worth recommending. SCWS released the 1.0.0 official version in 2008-03-08, and the latest version has now reached 1.0.4. PSCWS is the PHP version of it. And Httpcws is a feast developed, before called PHPCWS. PHPCWS First Use the "Ictclas 3.0 share Chinese word segmentation algorithm" API for the first word processing, and then use the self-written "inverse maximum matching algorithm" for word segmentation and Word merging, and increase the punctuation filtering function, to obtain the results of the word segmentation. Unfortunately, only Linux systems are currently supported, but not yet ported to the win platform.
2, the extraction results and the existing thesaurus to compare, processing, removing useless words to get the most consistent with the rules of the keyword. The main thing here is to look at the thesaurus, we can define our own thesaurus, we can also use the existing mature thesaurus. For example, Sina and NetEase blog all have this function. They should have a good thesaurus, because they are large sites, and I, just a small programmer, it is impossible to get any authoritative thesaurus, so only from the existing open source program to start, look at their thesaurus.
3, in the process of extracting the results of the selection of appropriate as the final keyword, to get the most consistent with the current content of the key words, at this stage is the specific situation of the specific analysis, in any case can not reach the kind of intelligent people. Most of it. The current PHP class CMS has its own extraction keyword system.
At present, the most widely circulated in the network is the dedecms of the word source, I did a test, found that quite stay, the effect is very bad. It first set a keyword length, determine the number of keywords to get, and then take the word, it thought the title is good words is the required keyword, in addition to read from the text of the keyword only to reach the length of the set, is the ultimate keyword. In addition, such as "we" and other meaningless words have not been removed and be listed as the frequency of the keyword is too high, and even sometimes put the space of the HTML as a key word, urgent need to improve. But if it's an auxiliary feature, it's good enough. And Discuz is slightly better, but Discuz does not provide the source code, just provides an online API.
and Dede's participle also has several versions, the best should be the latest version of it, the frequency of everything has, the following is the dede5.7 of the word and Discuz API results comparison
Test Examples:

1. $title="thinkphp official is about to stop support for version 2.0";
2. $body="To better develop, maintain and support the thinkphp framework, the official announces the maintenance and support for the 2.0 and previous versions from May 1, 2012 onwards, to save energy and low carbon, and also to cancel the corresponding version of the official website and document download.
3. This is the memory of those years, the development of the thinkphp version of it!
4. About thinkphp version 2.0
5. Thinkphp was born in 2006, dedicated to the rapid development of Web applications, its 2.0 release on October 1, 2009, in the previous 1.* version of the new refactoring and Leap, was an epoch-making version, the foundation for the new edition, Also accumulated a lot of user groups and sites, with the rapid updating of the framework, and the release of the new version 2.1, 2.2 and 3.0, heralding the arrival of the thinkphp of the 3.0 era, 2.0 of the life cycle of the end. But basically 2.0 of many features are extended or perfected to version 2.1, and upgrading from 2.0 to 2.1 and 2.2 is also relatively easy. Version 2.2 is the final version of the 2.* version, no longer features updated, only bug fixes. ";

first, Dede participle
sort the results as follows

2. title Array
4. (
6. [thinkphp] = 1
8. [Official] = 1
10. [Upcoming] = 1
12. [Stop] = 1
14. [to] = 1
16. [2.0] = 1
18. [Version] = 1
20. [of] = 1
22. [Support] = 1
24. )
26. content Array
28. (
30. [Version] = A
32. [the] = +
34. [and] = 8
36. [thinkphp] = 5
38. [2.0] = 5
40. [also] = 3
42. [2.2] = 3
44. [2.1] = 3
46. [Development] = 3
48. [3.0] = 2
50. [Yes] = 2
52. [Quick] = 2
54. [to] = 2
56. [Release] = 2
58. [Maintenance] = 2
60. [Previous] = 2
62. [up] = 2
64. [New] = 2
66. [Support] = 2
68. [frame] = 2
70. [at the same time] = 2
72. [from] = 2
74. *******

How do you take out the key words for the final need? The initial idea is to remove the words "," "some", and then according to the sort order of the content, in turn, to see if the title appears in the heading is required, so that you can take out a quantitative word the most final keywords. As a result, we can get

1. version thinkphp 2.0 support stop

five keywords. It seems that the results are acceptable.
second, in view of Discuz, using the API to get an XML document, the key words to be parsed is

1. , rapid, version upgrade, development, user

Five words, the first one is "the" ...
Comparing these two ways to find the first dede+ follow-up processing of the more close to the content of the document, should be slightly better, and discuz deviate from the subject of the article, but its access to the word has a certain popularity.

