One week of big data visual analysis-agile Bi helps iResearch Consulting Group achieve big data analysis on the Internet

Compared with traditional analysis methods, Alibaba Consulting Group's business efficiency has been improved several times through the complementarity of agile Bi and hadoop: The offline report delivery cycle has been shortened from 3 to 4 weeks to less than 1 week, software Delivery is shortened from half a year to one month.

At present, when talking about big data, people will think about hadoop, which seems to be the "spokesperson" of big data ". It is undeniable that hadoop has huge advantages in cluster scalability and cost. However, hadoop is not suitable for real-time analysis systems.

Therefore, many enterprises use hadoop for data storage and other tools for high-speed big data capture and real-time analysis. Here, we will use a real case from iResearch Consulting Group to explain how agile Bi complements hadoop and help it implement Internet big data analysis.

Inefficient customization Projects

IResearch is a new economic field that focuses on Network Media, e-commerce, online games, and wireless value-added services, specialized market research institutions that provide market research and strategic consulting services for customers in the network industry and traditional industries.

Currently, iResearch Consulting Group can provide enterprises with two types of customized consulting report services: offline reporting and software. However, the customization requirements of enterprise customers are changing. It takes three to four weeks for iResearch Consulting Group to generate an offline report, and half a year for the delivery of software. In addition, due to the increase in labor costs and the extension of iteration cycles required by the project, iResearch Consulting often does not dare to undertake too many customized projects.

Through research, I found the real needs of iResearch Consulting Group: based on the time dimension and website summary, the user's source region, origin domain name, page access times, stay time, valid access times, bounce rate, visitor, new visitor, return visit times, and return visit are separated number of days and other related data for statistical analysis, after dynamically adding conditions, you can analyze the data obtained by monitoring user behavior to obtain more detailed and clear user behavior habits.

Therefore, iResearch urgently needs a more agile and efficient big data analysis tool to improve the efficiency of customized services.

Big Data front: Agile Bi PK traditional bi

When solving the difficulties faced by iResearch Consulting Group, the traditional bi approach is to build a model based on the needs analysis in advance, create a secondary table or create a cube, and summarize the data in advance, the business personnel can view the analysis result report at the front end. Although this practice is very mature, it cannot solve the problem of iResearch Consulting Group.

First, the reports viewed by business personnel are relatively static. The calculation methods of dimensions and measurements for analysis are preset During modeling and cannot be changed. For example, if you have fixed the sum or average, and want to change it to variance, you must modify the model.

Second, when the analysis requirement changes, the business personnel cannot directly adjust the report. It personnel need to re-model or modify the existing analysis model, which takes a long time and responds slowly.

Finally, some enterprises have a small amount of data and need to carry out a large cost-effective data analysis according to this process and architecture.

The root cause of these problems is that the previous technical architecture lacks the computing capability for massive data. enterprise users need to perform data calculation and summary in advance through modeling, secondary tables, and cubes.

IResearch Consulting Group hopes to submit such an analysis report for enterprise customers, which can be viewed and analyzed dynamically. For iResearch, data presentation should be the starting point rather than the end point. When you see the data, you must be able to perform interactive analysis, explore the data in depth, discover problems and find answers, and take actions. The process of interacting with data must be fast enough. If you wait for three or five minutes for the result to be displayed each time you click, you cannot perform interactive analysis.

In addition, the analysis report should allow non-IT Department colleagues to make it directly on the analysis platform. You cannot submit all the analysis report requirements to the IT department, which seriously increases the burden on the IT department. At the same time, the implementation and operation of agile Bi should be simplified so that business personnel can use it directly.

At the same time, the requirements for analysis reports often involve changes to the data layer. It needs the IT department to improve the data layer and business layer. It takes a month or two for traditional bi platforms to complete model sorting. Without prior modeling, agile Bi can flexibly adjust the analysis dimensions and report presentation during the analysis process. demand changes can respond within one day, improving the enterprise's insight and decision-making capabilities.

Unlike traditional bi's weight modeling and unified view, agile Bi uses lightweight modeling and N views without creating two tables and cubes. data can be directly analyzed after being imported, in addition, business personnel can adjust the calculation methods of dimensions and measurements of analysis in real time, which greatly increases flexibility and truly communicates with data.

Since there is such a convenient method, why does traditional bi not adopt this architecture? This is because the traditional technical architecture does not introduce big data technology. In the face of massive data volumes, the analysis results required by enterprise customers cannot be displayed within a few seconds after a user clicks, therefore, data must be summarized in advance through modeling to ensure the speed at which analysis reports are presented.

Therefore, the premise for implementing agile Bi is to use a new architecture to process data. The technologies involved include distributed computing, memory computing, column storage, and in-database computing. Agile Bi allows enterprises to quickly gain insight into the meaning and value of data through lower costs and shorter launch cycles.

650) This. width = 650; "src =" http://www.ciw.com.cn/Portals/52/7 (259).jpg "/>

Improved business efficiency several times

After in-depth research on the data to be analyzed by iResearch Consulting Group, the author found that iResearch Consulting Group has tens of millions of data records per day, and different enterprise customers have different analysis requirements. Therefore, the complex and Variable Multi-dimensional analysis requirements pose higher challenges to the analysis performance of analysis tools. The traditional database and hadoop architecture cannot meet the needs of high performance and real-time analysis.

For this reason, iResearch Consulting Group has visited some well-known foreign products, but they can only give up after learning about the product price and subsequent service fees. Most of the analysis tools in China are mostly previous-generation Bi, which requires prior modeling and analysis. It is difficult to cope with flexible multi-dimensional analysis and change requirements, and the processing capability for large data volumes cannot meet the requirements.

In the end, iResearch Consulting Group chose yonghong agile Bi technology. When iResearch Consulting Group imported three months of detailed data (about 5 billion records) into the agile Bi system, it can directly present custom analysis reports. Compared with the original analysis method based on Excel and SQL programming, the business efficiency of iResearch Consulting Group has been improved several times: the offline report delivery cycle has been shortened from 3 to 4 weeks to less than 1 week, software Delivery is shortened from half a year to one month.

At the same time, iResearch Consulting Group had been unable to deliver many projects due to fear of changes in demand. After using agile Bi tools, iResearch Consulting Group can quickly build a prototype to show the customer in a few days. Any demand changes can be adjusted within one week. This fast prototype trial and error method enables iResearch Consulting Group to undertake many such projects.

As the business efficiency is greatly improved and more projects can be undertaken, iResearch Consulting Group's revenue space has also increased several times. At the same time, iResearch Consulting Group's customer satisfaction has also steadily increased.

Furthermore, in order to provide more intuitive and interactive analysis reports and improve enterprise user experience, iResearch Consulting Group has built a new SaaS Platform Based on Agile Bi tools. IResearch Consulting Group imports the data stored by enterprise customers in the hadoop architecture to the dataset city through the interfaces provided by agile Bi, and then quickly presents the results through agile bi.

In fact, both hadoop and agile Bi have their own different business scenarios, and the two are complementary relationships. At present, many enterprises use hadoop to store data, and then import hadoop data into agile Bi high-performance datasets based on distributed memory computing, and then perform visual data analysis. Given that hadoop is widely used in enterprises, yonghong agile Bi also supports hadoop data source connection.

IResearch Consulting Group has made reasonable use of its previously spent manpower and money to build a hadoop architecture, so that their previous investment is not wasted. However, yonghong agile Bi can also be well integrated for enterprises that have no hadoop architecture before.

Through the sales SaaS account, we also added a long-term stable revenue for iResearch Consulting Group, which was originally based on a single business model of independent projects. The subsidiary of iResearch Consulting Group quickly followed up on changes in the use of agile Bi and the new model. At the same time, the construction of the SAAs platform also enables iResearch Consulting Group to shift its value from a media/consulting service company to an Internet application provider that provides big data services, this greatly increased the value of the company in the capital market.

This article is from the agile Bi blog, please be sure to keep this source http://9220305.blog.51cto.com/9210305/1542036