Big data start-ups can look for inspiration on Facebook

To predict the trend of big data and to be clear about your concerns, http://www.aliyun.com/zixun/aggregation/1560.html ">facebook is the choice, because it collects huge amounts of data (100PB, 102400TB). In order to process this data, Cassandra nosql data storage +hive Query Language +hadoop distributed database is the best partner. This article talks about how big data start-ups should learn from Facebook about their breakthroughs.

One of the opportunities: the popularity of Hadoop

Infrastructure-level innovation through Hadoop and NoSQL is an opportunity.

Facebook uses almost every aspect of Hadoop, ranging from friend referrals to directional ads to data center analytics, where large data is fragmented into byte size fragments. However, the need to serve all this means ensuring that users in all departments can interact with Hadoop in a meaningful way.

Customized tools, interfaces, and virtual layers help solve this problem. Facebook's non-tech users can also use Hadoop to generate reports and view analytics after the technology hurdles are lowered. Several former Facebook employees who helped create Hive also launched a cloud version of Hive-qubole, which provides request-access to Hadoop via the hive signed SQL interface. Facebook wants to create tools that will help reduce the difficulty of using hadoop and increase the efficiency of the use of large data.

Second chance: Beyond Hadoop

But sometimes jumping out of existing frameworks, such as Hadoop and NoSQL storage, may also make a new world. It all depends on demand. Everyone uses Hadoop because it is free and open source. However, to achieve your needs, you often need to do a lot of work on Hadoop. There are many big data problems that have nothing to do with Hadoop, so a new one may be a solution. Facebook's Atlas database uses MySQL, and it's developed timeline and Newsfeed's background, and everything should be chosen as needed.

But for startups, it's a trade-off when it comes to choosing an application development platform. Accel's Ping Li's advice is that good enough is a great enemy. To achieve greatness, you may have to break through Hadoop.

Three opportunities: Bigger, as big as the data center

Facebook launched a new in-depth storage strategy for the data center in August this year, with the intention of designing a data center from scratch, with a view to dealing with less-than-constant data storage, rather than a more stable web transaction stream.

This change is by no means a progressive change, much different from the data centers of the past. This energy-intensive data center tries to allocate every degree of computing savings to a much smaller amount of power demand, but these processes still need to deliver data to users and analysis engines. This is a huge challenge because more and more companies are aware of the importance of historical data.

Facebook is going to open its design through the Open Compute Project, where some of the management work is done in the Apache Hadoop project, which is good news for startups, and they just have to do what's left.

(Responsible editor: The good of the Legacy)