Data Warehouse--digging in "beer and diapers"

Data Warehouse--digging in "beer and diapers" 01-5-21 04:19:25

Interlocutor: Host: Cheng  Hung--"Computer World" reporter  Home: Mengxiao--Renmin University of China School of Information Professor  Qi-"Data Warehouse Road" website host it Manufacturers: Yang Shunsheng--NCR Greater China Market and partner general manager of the  households: Chen Daobin --the Director of Information Management Department of ICBC (Ph. D) Dialogue topic: How is the application prospect of data warehouse in China? How high is the threshold of the  data Warehouse? Moderator: Since the story of "beer and diapers" has been widely circulated in China, the data warehouse in China has been lively for a while, many entrepreneurs have a great hope for it, but why soon after the Data Warehouse application into the "hibernation period"? What are the constraints on the application of data warehousing technology in China? Mengxiao: Data Warehouse is developed on the basis of database, it usually has three parts: Data Warehouse (information warehousing), online analytical Processing (OLAP) and data Mining (datamining), they have very strong complementary relationship. Data Warehouse is to meet the needs of data analysis on the basis of high data accumulation, but because of the shortage of the basic data accumulation in China, the application of Data Warehouse technology has not been popularized. The emergence of a technology is generally due to the innovator to put forward new concepts, the researchers to solve some problems; Although the data warehouse has crossed this stage, but at present in the Data Warehouse application promotion process, has encountered one threshold, that is this technology how is the majority accepts. I think the first problem that needs to be solved is how data mining can be combined with existing business technologies to make the data warehouse acceptable to most people. There are many common data mining systems in the market that are applicable to all business models, but in practice these systems are extremely difficult, and only those who are very familiar with data mining technology can understand and use them, and it is hard for ordinary users to use these technologies to solve their own business problems. Yang Shunsheng: We are a real business of data Warehouse products, from our experience in contact with domestic enterprises, data warehousing technology in China has not been well developed, mainly in the following reasons: First, China's information infrastructure is relatively imperfect, such as the current telecommunications industry billing data is very dispersed, Billing tools have 40~50, data collection are each of the various, for the future analysis brings many technical difficulties. Second, the enterprise's sense of competition and service is not strong enough, the demand for decision analysis is not so urgent, because enterprises do not have the opportunity to implement the Data warehouse, it also causes the lack of technical personnel in the phenomenon. Third, the Data warehouse is a system of data sharing, different levels of people from the information will be very large, it is a good tool for enterprise decision-making, but at present Chinese enterprises have not set up a management mechanism to promote the sharing of data, whether it is the ability of people, the organization of the enterprise or the quality of data is not a continuousManagement mechanism, it is very difficult to build useful data analysis on such basis. Qi: In fact, in the foreign market competition is very fierce environment, each store for its own survival, has tried to do, many can be discovered by the law has long been discovered, in this case, the use of data mining technology to solve the problem is a very normal idea. But data mining tools don't directly tell policymakers that they should sell beer and diapers together. Domestic enterprises have a lot of implementation of the Data warehouse, but most of the results are not satisfactory, the key reason is that the construction of data warehouse must first identify why to invest in the data Warehouse? What problems do you want to use the Data Warehouse for? What is the purpose of the data warehouse to come to an end? Otherwise do not know how to use the Data Warehouse, how to evaluate its success or not. Second, the Data warehouse is not the kind of software products that can be used to buy, in fact, the data warehouse is more like a process, a user gradually understand themselves, improve their own process. Third, the data warehouse should not only reflect the status quo of the enterprise, but also rely on the user to make the final decision. In short, the domestic data warehouse is not very successful reasons in addition to the user's application level, the level of business management needs to be improved, data warehouse high prices of products also affect more or less the smooth promotion of its domestic. Chen Daobin: I am personally engaged in information analysis work, for why to use the Data Warehouse is also doing some research. As a user I feel that the banking industry most needs data warehousing technology, but also the most should vigorously develop data warehouse technology. Several major banks in recent years have made some attempts in this area, but so far there have been many losers and fewer winners. The main reason is that many banks in the establishment of data Warehouse system, the system to achieve the function of the location is not clear. A data warehouse system should be distinguished from the business processing system, because the requirements of the business system are often quick response and simple interface. The Data warehouse and the business system is not a parallel relationship, it should be based on all business systems, the business information collection, analysis, collation and release, should be a stable, with time parameters of the data collection. The Data warehouse technology itself does not have the new content, it is the management science, the computer Science, the network science and the analysis method big Fusion. is the data warehouse technology easy to use? Moderator: is the Data Warehouse application is not ideal for technical reasons? Is there a contradiction between the user's level and the Data Warehouse front-end tools? Mengxiao: In the three concepts of data warehouse, Data Warehouse is the foundation of enterprise data analysis, its main work is to generalize the raw data in the database and assemble it into a data collection which can be used for high level. On the basis of data warehouse, there are two kinds of analytical tools, one is OLAP for analytical work, the other is data mining of predictive work. The idea of data mining is to find a correlation rule like "beer and diapers". But the current technology system, whether in China or in other countries of the world, shouldare subject to certain constraints, the main reason is that this technology has not yet reached the maturity of the database technology and ease of use. But for now, the availability of all products is questionable, because if you're not a database expert, a statistical expert, or an AI expert, you'll have a hard time using such analytical tools. At present, the data Warehouse products are based on a common technology platform design, although this product can solve the different user's analysis needs, but it does not have the special domain of business logic and Data Warehouse technology integration, so the analysis effect can not reach the peak. Another technical bottleneck is the current many algorithms, has not experienced a tidal wave of sediment, and the retrieval technology in the database after many years of exploration, has formed several fixed, mature technology model, this is the Data Warehouse product failed to achieve the database product practicability of another reason, Therefore, the development of data Warehouse technology is still in the accumulation stage. Chen Daobin: ICBC in the establishment of a data warehouse system in the unified understanding is that the market can not buy directly to use the Data Warehouse products, must be tailored to their own business, must be their own data sources and business needs of the clear, and then the middle of the bypass work well, This bridging work requires the support of data Warehouse products on the market, and the coordination of business and technology must be noted from the outset. Qi: Since the Data Warehouse is produced from the western countries, it has a strong Western cultural color, the most typical is the Data Warehouse report presentation. Foreign products are focused on the content of the report, but China requires content and format as important, and sometimes even more important format than content, at this point, foreign reporting tools are difficult to meet the needs of Chinese users. Data Warehouse as a tool, users at every level of the enterprise can use it to generate benefits, but the real implementation, there is a user level and demand issues, we can not require each user to be able to get data from the Data warehouse, and security measures are not allowed to do so. At this point, we need a series of different data warehouse front-end tools, which is currently all the Data Warehouse products are missing. Most of the current products offer only one tool to try to meet everyone's needs, and everyone is dissatisfied. What is the scope of the Data Warehouse application? Moderator: Which industries have a greater demand for data warehouses? What are the current Data warehouse technology in domestic applications better? Why? Yang Shunsheng: I used to analyze the maturity of some industries and enterprises with data warehouse based on some hypothetical conditions. In the 2000 Global Fortune magazine's 500 list, nearly 50% of companies have implemented enterprise-class data warehouses or departmental data marts, and we understand that telecommunications, banking, retail, aviation, railways, postal, food, consumer manufacturing, automotive, medical, insurance and other industries are the most powerful data warehouse technology needs. In all of these data warehousing industries, the proportion is: Retail industry 17, aviation industry 16, landline 15, mobile communicationsEnterprise 14, banking 13. In addition, we also statistics the number of enterprises that have implemented data warehouse in different industries in the world at present: Based on the experience of foreign countries, we find that the four factors that lead to the requirement of the leadership, the information technology infrastructure, the analytical application and the competition are the key factors that affect the enterprise to implement the Data Warehouse progress. The larger the size of the enterprise, the more historical data, the higher the urgency of implementing the Data Warehouse, the more the retail and manufacturing enterprises pay more attention to cost control, the first is to use the analytical application of operation and production; Due to the difficulty of historical data collection, the speed of the implementation of the data warehouse is relatively slow in the government supervision Department. The Data Warehouse will be used as the information technology means to deal with and analyze the large concentration, and the leadership of Business Administration education and the scientific decision means are more important to support the construction of Data warehouse. Chen Daobin: At present in China's financial system only ICBC in the application of data warehousing has a breakthrough, the reason is: first, the large scale of ICBC, customer base is large, it is necessary for customers to carry out in-depth research to achieve customer-centric service model; second, as early as September 1, 1999, ICBC has proposed that all the business is concentrated in Beijing and Shanghai two centers to deal with, in fact, solve the data warehouse necessary to set up the problem; third, ICBC has one of the biggest advantages, is that it has implemented a unified development of integrated business systems, providing customer information integration conditions. From the perspective of leadership support, the current president is specialized in High-tech environment of business development, he has a unique view on the use of information technology to develop banking business, so ICBC is currently in the Data Warehouse first project (Customer relationship Management) has made substantial progress. How to cross the threshold of data Warehouse? Moderator: Is China's data Warehouse market mature? What solutions can we suggest for the constraints mentioned above? What measures and means should be taken to promote the application of Chinese enterprise Data Warehouse? Qi: Good question! Gartner Group once had a data warehouse market share report, from the report can be seen that by 2003, the United States data Warehouse sales will occupy 58% of the world, Asia accounted for 7.5%, it is not difficult to see our gap. But at present the technology development speed of the two difference is not much, so the Data Warehouse application market in China still exists. I think the only solution is to let enterprises directly to the competitive market, change management ideas, so that will soon produce demand, reading and learning boxing will never understand the rich experience of the champ. Mengxiao: Now more and more companies are building web-based electronic stores that can collect a lot of raw data, so E-commerce has become a promising application area for data warehousing technology. And how to provide a special set of Data Warehouse solutions for e-business applications, should be more acceptable to the market than a general-purpose solution, thus spanning the Data Warehouse applicationThe threshold of the process. There are many customization requirements in the field of Data Warehouse application, and users need a tool that can provide both data analysis and customer personality analysis. Yang Shunsheng: From the experience of advanced countries, we find that online transaction processing system (OLTP) and enterprise network two information infrastructures are necessary to implement the Data Warehouse. The more competitive environment needs data Warehouse system, enterprises need to understand customer needs, need to find business risk also need to conduct business analysis and management, all these analyses belong to the analysis of large amount of data, the use of traditional information technology will have many limitations, You must rely on a TB-level data warehouse system to solve the above problems. Data Warehouse is an analytical application, and it is the most suitable information technology to solve complex business problems. But are these ideas suitable for the ecological environment of Chinese enterprises? Is there a domestic case to support this view? Recently, Shanghai Securities Central registration and clearing company and China Civil Aviation Information Network company have successfully implemented TB class Data Warehouse system, these two cases are the most powerful proof of the necessity of implementing data warehouse in Chinese enterprises and institutions. Chen Daobin: From the implementation of data Warehouse in ICBC, Data Warehouse technology has great development prospect in China. The Data warehouse itself has a lot of technical and methodological system, but in the establishment of data Warehouse application, but also should be problem-oriented and can not be a method-oriented, based on the problem to find products and tools. There are too many examples of failures in this area, mainly because many enterprises buy data Warehouse products First, feel that this product is very good, must use it, and then start to build their own system, this practice has been proved not feasible. The development of data warehouse applications can not be anxious, hope that one time to be able to solve all the problems is impossible. The construction process of data warehouse should pay attention to methodology, in a large demand framework, in the business and technical staff to communicate well, one time to solve a problem. "Reporter comments" "Shooter" and "gun" in the dialogue process, the reporter's biggest feeling is, because the data warehouse can not directly tell decision-makers should put beer and diapers together, so the enterprise can not be all the "treasure" pressure on the Data warehouse. If the Data warehouse is a good "gun", then the decision maker should be "the person who plays the gun". The data warehouse can only reflect the status quo of the enterprise, the final decision or need to do. There are two kinds of applications in data Warehouse, namely online analysis and data mining, online analysis focuses on the presentation of all transactions, while data mining focuses on the discovery of unknown laws contained in the transaction. From the business perspective, both can be used to find and summarize the law, one is to verify some conjecture to find the law, the other is through the data to find hidden unknown law. The success of data mining depends on the reasonable processing of data and algorithms, it is not a universal tool to find any rules, so the more familiar users of their business, the more able to provide comprehensive data mining help and guidance, blindly use data to digDigging, only left to the data mining technology regret.