IK Chinese Word breaker Introduction _ participle

Participle is an important part of using SOLR and luence, and this article will introduce one of the Chinese word breakers ik

IK Analyzer is an open-source, lightweight Chinese word segmentation toolkit based on Java language development. Starting with the 1.0 release in December 2006, Ikanalyzer has launched 3 major editions. Initially, it is an open source project Luence as the main application, combining dictionary segmentation and Grammar Analysis algorithm of Chinese phrases. The new version of IK Analyzer 3.0 is developed as a Java-oriented common word breaker, independent of the Lucene project, while providing a default optimization implementation for Lucene.

1.1 IK Analyzer 3.0 Structural Design


1.2 IK Analyzer 3.0 Features
The unique "forward iterative most fine-grained segmentation algorithm" is adopted, high-speed processing with 800,000 characters/sec using the multi-processor analysis mode, support: English letter (IP address, Email, URL), number (date, commonly used Chinese words, Roman numerals, scientific notation), Chinese vocabulary (name, Place name processing) the word processing. Optimized dictionary storage, smaller memory footprint. Support User dictionary Extension definition Query Analyzer ikqueryparser for Lucene full-text search optimization; Using the ambiguity analysis algorithm to optimize query keyword search permutation, can greatly improve the hit rate of lucene retrieval.

1.3 Example of Word segmentation effect

Text Original 1:
Ik-analyzer is an open-source, lightweight Chinese word segmentation toolkit based on Java language development. Starting with the 1.0 release in December 2006, Ikanalyzer has launched 3 major editions.
Word Segmentation Result:
Ik-analyzer | is | One | One | A | Open Source | of | Based on | java | language |  Development | of | Lightweight | Measure Level | of | Chinese | participle | Tool Kits |  Tools | from | 2006 | Years | 12 | Month | Launch | 1.0 | Version | Start | Ikanalyzer |  Already | Launch |  Out of the |  3 |  A big | A | Version

Text Original 2:
Yonghe Garments Accessories Co., Ltd.
Word Segmentation Result:
Yonghe | Kimono | Clothing | Decorations | Decoration | Jewelry |  Limited | Company