Big Data Spirit Changtong Collection platform release

December 20, the company in order to let all staff familiar with the company's new product-"Ling Changtong with the acquisition platform," the technical principle, the main features and performance advantages, convenient for everyone in the technology call, user operations and customer development, the acquisition platform has a deeper understanding and grasp, invited to the acquisition platform of the original developer-Gao, For all staff to do the theme of the second generation of acquisition platform-"Ling Changtong with the collection platform," the introduction of the training course.

Gao said that data acquisition is the most important foundation of big data Mining, and "Changtong acquisition platform" is a system platform which can be customized to the depth of the website, can also use the simplest configuration and fast acquisition, it uses intelligent matching and advanced HTML5 module editing tools to satisfy the configuration of dynamic static field. , equipped with comprehensive and intuitive runtime monitoring system, rich development interface and detailed SDK documentation, support distributed collection deployment, scheduling, data processing, can easily deal with big data in the acquisition of various problems.

First of all, Gao introduced the main content of this training course is: Platform technology innovation point, data acquisition system, platform monitoring system, performance and stability, development plan and so on several aspects, then the collection system platform is introduced in detail. Gao said that the data collection first to make the acquisition request, the acquisition system will be based on the requirements, according to acquisition instructions for the collection task distribution, and then to the distributed stream data analysis platform for data comparison, data source settings, data capture, entity extraction, data classification, and finally to the distributed data storage platform for storage.

in the training Gao the focus is to demonstrate the intelligent dynamic increase and decrease collector set up and use method. Intelligent dynamic increase and decrease collector is through the data ID, data address, acquisition function add, collect the number of functions such as the setting to carry out data collection, and the way to collect two modes: one is a common mode, both using common function settings to collect data, In general, the data collected by this module is much more but the effect is relatively poor, the other is the special setting mode, which can set the function of the collector according to the requirement, and the result of this collection is better and the accuracy rate is high.

"Changtong acquisition platform" is a multi-functional platform for data collection in structured and unstructured text documents, images and videos in the Internet, which is composed of data collection, entity extraction, deep learning, text categorization, text summarization, data storage and pick-up, data search, data statistics, Collection and monitoring of more than 10 sets of components, which in the work and maintenance of the need to collaborate in order to play the best collection results. With the continuous improvement of "Changtong acquisition platform", the effect will be better in future data collection work.

Data acquisition is one of the important services provided by the company to customers, the company in the original first generation acquisition platform -the "Golden Eye" data acquisition platform on the basis of experience, absorb insufficient, a new design and development of the second generation of data acquisition platform-"Ling Changtong with the acquisition platform", Better compatibility, higher acquisition efficiency, more accurate collection quality and more personalized collection settings than the previous generation. On the basis of this, the third generation of acquisition platform-"Ling Jiu collection Cloud Platform" is also in the development of key technical demonstration stage.

Big Data Spirit Changtong Collection platform release