Big data "difficult problem" highlights deep research is imperative

For large data, in the industry after a few years, has been a gradual landing from the air, the user's perspective from the understanding of the concept of how to change from the business, application point of view, so that large data truly realize value.

In fact, the large data is not a new concept, it in physics, biology, environmental ecology, automatic control and other fields of science and military, communications, financial and other industries exist for some time. In essence, the big data mainly solves the massive data storage, the computation, the excavation, the demonstration question, based on this may birth a series of applications or the business model.

At this stage, the problem of large data is mainly divided into 3 categories:

Expand the traditional business intelligence (BI) domain. Previously for large data Statistics, association analysis, trend prediction from sampling to full volume analysis, the data back to various reports.

Business process Improvement. The aggregation analysis of various data is used as the basis of business process improvement and evaluation.

Data products and commercial applications. Data products or data services are formed by servicing or packaging existing data or processing capabilities.

Concrete to the implementation level large data is divided into four stages:

1. Learning Stage

2. Exploration Stage

3, involved in the stage (select a few business points, advance research, try to fall)

4, the implementation phase (business scene into production practice)

Since large data is now a novelty for most users, and there are many business model innovations and changes, many users have reflected many difficulties and challenges:

First, because large data is largely business-driven, and the implementation cycle spans are long, IT staff need to be able to articulate business scenarios and business values to gain the support of the business executives.

Second, users entering the stage face the technical challenges, which can be subdivided into two categories. The first is technical skills, including traditional data warehouse skills, new technology skills (e.g., new technologies based on Hadoop, IBM's stream computing, and so on). The second is the challenge of analysis ability, the traditional report can not meet the needs of analysis, the future users need some real-time analysis, operation analysis and forecast analysis data. Third, the landing of the core business challenges, the moment should focus on the quality of data and authenticity.

Third, when the business is small, open source technology can be used to build a set of data processing system. As your business grows, you need to modify or develop systems that meet the needs of specific areas, depending on your business characteristics.

Finally, in some companies, the rapid growth and expansion of business, business difficulties and technical difficulties are also prominent, such as the continuous production of data, data quality, security, cost how to protect it? What are the consequences of metadata changes in front-end business libraries to the backend data processing? These problems are easily short, making the data a huge pile of rubbish and a costly business burden.

It is with this series of challenges and difficulties that we see in order to help the current China's large data market users comb the current large data providers in China's numerous and diverse, find out the value of each program, China Sadie adopted a new evaluation method, from the user's core concerns and solutions provider's supply capacity and value, Based on key industries (including intelligent cities, telecommunications, finance), a survey on the challenges of large data has been launched, with a view to improving the acceleration of large data applications by Governments, enterprises and individual users.