10 major inspirations brought by the big data pioneer (1)

Fast query ensures efficiency and timeliness

Undoubtedly, the big data era has come. So how can we deal with this situation? Next, let's hear what experts with experience in this field say.

First, we need to know how to make full use of big data in hundreds of terabytes of information. This depends entirely on individual needs and preferences. Interclick Advertising Service found a solution that provides near real-time data analysis while providing more efficient solutions. Harvard Medical School also learned that data will also grow significantly when studying the number of patients and data that remains unchanged for years. ComSCore, an Internet traffic monitoring agency, has 12 years of experience using column-store databases to compress data. In fact, it uses Sorting Technology to optimize compression and reduce processing requirements.

Currently, Hadoop is an idealized low-cost processing platform for unstructured data, including Yahoo, Facebook, Twitter, NetFlix, and eHarmony. It not only meets the needs of Internet giants, but also meets the needs of JP Morgan and other mainstream traditional enterprises. InfoChimps, a data vendor, also finds that Hadoop is a fast and mature solution that provides deployment support with more and more additional and auxiliary applications.

Applications in the big data age

Of course, not all big data deployments are measured based on the overall scale. For example, Linkshare only retains data for several months but needs to load and quickly analyze up to dozens of GB of data every day. Therefore, it is a large-scale deployment for the daily data scale. In addition, we also need to pay attention to the six dimensions of data warehouse scalability. Only in this way can we develop a more accurate solution to meet the most demanding test requirements and obtain technical investment to meet future needs.

1. Fast query ensures efficiency and timeliness

Large-scale parallel processing platform, column-store database, in-database processing technology, and memory computing technology can greatly reduce the Data Query time, from days, hours to minutes or even seconds. But this is not enough. New York advertising company Interclick found that the main benefit of rapid analysis is efficiency. Quick response can win more time for more and more in-depth queries. The second advantage is that the results of near real-time analysis can be obtained, which helps improve the decision Response Level and accuracy.

Big Data analysis technology used by Interclick

With quick response, Interclick can segment the behavior of online surfers in hours or even minutes. It can send network name behavior information such as visiting travel sites and booking hotel sites to airlines, hotel chains, and car rental companies. Interclick is deployed on the ParAccel column storage database. The memory cluster can accommodate TB of data.