Data-driven enterprise two-time takeoff

Data Center Overall solution

1      Preface

With the development of the global economy, the rapid transformation of the internet era to the DT era, the development of enterprises to a certain stage, integration, industrialization, interconnection is bound to become a new opportunity to take off two times. But within the enterprise, long-term business dispersed development, resulting in information silos, business chain breakpoints and so on, these problems are essentially data problems, data does not realize deep-seated value. Enterprises to achieve rapid development, solve the problem of data, mining the value of data is an imperative road. Among them, the three main ideas of data value discovery are:

1. Solve the enterprise Information Island, realize business integration through; This idea is the main solution of the enterprise to solve the information island.

2. Data integration, to provide deep-rooted value mining for enterprise development; The essence of this idea is to find the intrinsic essence of the enterprise and solve the enterprise problem with the data as the phenomenon.

3. Data operation, the external services and operations of their own data, control the entrance, the idea is currently in the Internet industry and the industry has mastered the data of more than the mining data through the value of the data to achieve value-added.

Combined with the current data value-added ideas, as well as the current problems encountered by large group enterprises, Beijing Airlines Computing Communication Institute after several years of research and practice, successfully released the Enterprise data resource management and analysis platform V2.0, the goal is to solve data problems for enterprises, to achieve data appreciation.

2      The overall solution of the Aviation Enterprise Data Center

Aviation Data Center Total Solution 1.0 version starting from 2011 The project landed, starting from the user's basic data management needs, after several years of development and practice, innovative proposed business subject data, data full-cycle collection, management, application, analysis of integrated management concepts. In the Aerospace Science and engineering group and subordinate units have been successfully applied.

In 2015, after nearly 5 years of accumulated experience, the company successfully published the Data resource management and analysis platform for the Enterprise data center, and delivered the best business practices to different industries.

2.1    A metadata-centric approach to modeling data resources

Figure one: Metadata as the core concept

Based on the experience of the informatization construction of Beijing Airlines, a set of flight data resource modeling methods in line with group enterprises can be adapted to the different definition of metadata model description method and the adjustment of the metadata model description method in different application stages of the system. At the same time, the modeling method provides a custom extension interface for the user.

Data modeling is a description of data objects, including data identification, data composition elements, metadata model is a data specification, covering data naming, interpretation, data structure and other information, can be used to create database structure, understand the data model, to perform cross-system and data exchange and sharing. Create a metadata model under the resource directory for master data Model Management. The reference metadata model definition, the main data element carries on the data description, forms the complete data information, the data model information, the data reference relation information and so on, supports the data management application. At the same time, the meta-data model is versioned to adapt to the change of data management, and the data entity model and data content have influence.

Based on the data resource modeling method, the data modeling of data resource object is realized through the platform, and the storage structure, data format and relationship with other models are established through the data model. Adapting to the data model that needs to be done with the change of time and management method, the system configuration can realize the change and upgrade of the model, and guarantee the security of data resources and the smooth change of data model.

2.2    Data resource management and application based on data model

Based on the data model, the data maintenance, management, query and retrieval, support the process audit of data, and realize multi-role data co-management. Through the configuration to achieve the data resource management interface, data item customization, different interface data content applied to different management scenarios.

At the same time, Beijing Airlines combined many years of experience, put forward a set of enterprise management as the core of the data model set, provides a standard data model. At the same time, the platform provides a flexible model relationship definition to achieve a unified view of enterprise data.

2.3    a flexible coding engine

For enterprises, especially group-based enterprises, the need for coding management of the basic data, personnel need unique code, contract needs coding, material needs code, product needs coding and so on, coding data is the key node of enterprise business through, like the human joint, is the key lifeline of all business in series.

The flight data resource management and analysis platform combines the requirements of enterprise coding data, and combines the problems of coding change, old and new coding and multi-coding rules, which can be used to provide a flexible and configurable coding engine. The engine supports various forms of code segment rule management, such as fixed code segment, feature code segment, date code segment, pipelining, and so on. Among them, the characteristic code segment manages the characteristic value and the characteristic code comparison table of the feature attribute. Through the coding engine, to meet the enterprise coding data requirements.

2.4    data quality improvement based on rule engine

In order to ensure the effectiveness and accuracy of the data center, the platform provides a data quality management module based on the rules engine, that is, by defining the quality rules and associating the rules with the data model, and by evaluating the scheduling to generate the evaluation reports and analysis results.

The quality rule definition creates the quality rules that are required for the data evaluation, and configures the elements in the data model rule set that need to be validated in each type of data model in the system and the validation rules. Set the frequency and checksum content of the generated evaluation results in the quality evaluation schedule. Data quality assessment provides manual generation of evaluation reports on elements of the data model.

2.5    based onMPPThe Data mart

Using the self-developed MPP Data Mart scheme, this scheme uses a similar MapReduce computing framework, which is based on the concept of incremental flow calculation, uses the new technologies such as memory computing, in-Library computing, distributed communication, and Columnstore Distributed file system to realize a distributed parallel real-time processing, complex query optimization, Large data processing database system supported by complex analysis.

2.6    Fast analysis and continuous iteration based on agile

Data Warehouse +olap The era of business intelligence systems that require user pre-proposed analysis and statistical requirements. Based on this, expand the data modeling effort, import the data, and then create the cube. After these efforts are completed, business intelligence applications can be developed, which is a typical data-driven model. Business-driven business intelligence system, the direct import of detailed data, no longer require users to advance specific analysis and statistical requirements, there is no process of creating a cube, which greatly simplifies the work of the data layer, shorten the data layer response period, the entire business intelligence system from data driven to business-driven.

in the +olap era of data warehousing, a new analytical requirement may take one months to achieve, now only a week or even a day. Building a business intelligence system in the past may take a year, and in less than a week we can develop the first data analytics application.

2.7    User-service business intelligence based on exploratory

The exploratory BI system believes that data applications such as reporting and dashboard are portals, portals, and not endpoints of business intelligence systems. In such a system, analysis techniques based on filtering (filter), drilling (drill), brush (brush), correlation (Associate), transform (Transform), dynamic calculation, etc. The user can further interact with the data (Interactive).

The data layer is thin, and the business layer has the conditions to thicken up. Industry-leading enterprises in the good grading planning and classification management, from the general manager to the frontline staff, all levels of departments can put forward and develop their own data analysis applications, and ultimately create a self-service business intelligence system on demand. Since most data analysis applications are developed by users or people close to the user, the associated personnel of the development application are reduced, the whole organization management is more flat, and the response time is greatly shortened, and the management ability and the strategic decision level of the enterprise are improved. Compared to traditional business intelligence systems, self-service business intelligence systems are optimized and more efficient.

This data modeling technique, which directly imports detail data, transforms the relationship between data and applications from tight coupling to loose coupling, so that most analytical applications do not cause any changes in the data layer, whereas the MPP-based business intelligence system is able to perform high-performance analysis of detail data directly. This allows users to quickly develop data applications and then perform real-time analysis.

3      Concluding remarks

DT the arrival of the times, the value of data will be more and more people find, especially for large groups of enterprises, data from the production, transmission, use, replication, reporting, graphics and other life cycle process, each stage will play a different role, only the deeper discovery of the role of data, to give the data more role, The data will have more effect on the enterprise.

This is where the flight data center is born!

This article is from the "Insect Cloud Computing Journey" blog, be sure to keep this source http://4017782.blog.51cto.com/4007782/1685346

Data-driven enterprise two-time takeoff