What is a A/b test that brings Google $10 billion a year?

650) this.width=650; "src=" Http://s2.51cto.com/wyfs02/M01/76/CC/wKiom1ZcbYiwv-8AAAFJoNnxcVI760.jpg "title=" Google.jpg "alt=" Wkiom1zcbyiwv-8aaafjonnxcvi760.jpg "/>

Recently, a former Facebook engineer, Shup, has been greatly concerned about the history of Facebook's revamped website from 2010 to 2015. According to his record, in 2012, then the vice president of the Facebook product led a team of top experts in the company, which lasted half a year to develop a new homepage, a major revision and update. Then, Facebook conducted A/b test on the home page: When testing to the 5% user base, the data metrics were slipping, and when tested to the 12% user base, the downward trend was still noticeable. As a result, Facebook had to give up the results of nearly a year of efforts by the team of more than 30 people.

what is a A/b test?

A/B testing is already a proven test and development methodology that is commonly used by Google's development teams. Industry insiders told the Titanium Media reporter, every new Google advertising product changes to go through a rigorous online testing to verify the effect, in ensuring the search experience while improving the conversion rate of advertising, this data-based Internet product development and operations, can greatly accelerate innovation, improve the user experience, Play a multiplier effect. Wang Ye, a former Google ad quality control engineer, introduced A/B testing optimization that could generate $10 billion in revenue per year for Google.

In a nutshell, A/B testing is a test of a and B versions of a page to count which version has a higher click-through rate. The most common A/b test form is called grayscale testing, that is, a company's own Internet user base for 1%, 2%, 5%, 10% and other scale of user testing, and then choose the best Test results page to promote the entire user base.

Virtually all successful internet companies have used A/B testing to varying degrees to optimize Web pages and improve click-through rates. Wang Ye said that Google's various products have hundreds of different test versions run simultaneously, the test data determines that only a few changes can be finally online, these better changes can achieve a monthly revenue of about 2% per cent, and eventually reach 20% annual growth.

And Facebook's experience, more fully explained in the Internet world without victorious general, experience and intuition are not the internet world rules of the game. Only through constant testing and repeated data analysis can we prove that an opinion or judgment is correct. From the example of Facebook, it can be seen that the only thing that is fast and constantly trying and wrong is that the "data speak" mindset, A/B testing is a powerful tool to let the data speak.

technology Support and the cross-boundary of data thinking

A/B testing sounds simple, but it is a cross-border area where technical support and data thinking are combined.

In Silicon Valley, reporters interviewed the product manager of Optimizely, A/B testing startup company, Byron Jones, who introduced the five-step approach for A/B testing: Analysis, assumptions, build tests, run tests, performance evaluations. Among them, the analysis, hypothesis and build test belong to the data analysis stage, and the operation test and effect evaluation belong to the technical support category, which are complementary with each other.

In data analysis, the A/b test mainly carries on the mathematical hypothesis to the sampling user group: According to the statistic principle and the algorithm, discovers the most statistical significance sample user group characteristic, the scale, the experiment group number and the number and so on. In terms of technical support, A/B testing is mainly based on large-scale Internet traffic segmentation technology, different Web pages for different user groups to divide, and then through the server and network control to complete traffic segmentation, to different users to send different pages.

There is also a very interesting Simpson paradox in the area of data analysis. In the image, sales increased when a promotion was introduced to 50% of female users, and sales increased when the same promotion was introduced to 50% of male users, but sales fell when the two groups were merged and the same promotion was promoted. Therefore, the Simpson paradox needs to be eliminated by the relevant mathematical processing algorithms.

A/B testing cloud service, has become a vertical business direction

Most of the Internet company's A/b tests are owned by an internal team, and the scale-up and specialization of A/B testing is becoming more sophisticated. With the rapid development of the Internet, more and more enterprises need to carry out Internet services, all the use of self-built A/B testing team is obviously very expensive, so there is a SaaS model based third-party specialization A/B testing company.

Two teams with a background of Google engineers are doing similar service startups: one of them, founded Optimizely in the United States, has completed the C-round financing, and the company was founded in five years to complete a total of $145 million in financing. Optimizely's investors include the famous Silicon Valley VC a16z, Bain Capital Ventures, Salesforce Ventures, and more than 400 employees, and the company has a very rapid business development.

Another team returned to China last year to create a yell technology, which is Wang Ye's team. Wang Ye has been involved in a number of technology innovation projects at Yale, Microsoft Research and NEC North America Labs, including China's Next Generation Internet (CNGI) and web App traffic optimization (ALTO), at Tsinghua and Yale. From 2012 to 2014, Wang Ye worked in the Advertising quality control department at Google headquarters in the United States. In 2014, he saw the domestic Internet + craze after deciding to return to business, the mature A/B testing technology in the public cloud way to provide domestic enterprises and developers.

Yell Technology products called Appadhoc Optimizer, which is based on A/B testing optimization platform, in April 2015 fully on-line, currently has more than 100 trial customers. Yell Technology Support test flow dynamic control, multi-variable combination test, a large number of parallel tests, targeted test for a specific population, the SDK not only supports web (HTML5) end A/B testing, but also supports the native mobile app (IOS, Android) side. Yell Technology current investment institutions include peak Rui Capital, geek state and so on.

The 2016 will be a key year for Internet + action, whether traditional corporate Internet services or a multitude of e-commerce startups, can explore online user behavior, improve operational efficiency, and avoid the pitfalls of empiricism and intuitive thinking through A/B testing of professional tools.

A/B testing tells us that the Internet + era "leader said" does not count, "data speak" only counts. (Wen/Ningchuang, the first titanium media in this article)

"More exciting content in the era of cloud technology " No.: Cloudtechtime "

This article is from the "Cloud Technology Age" blog, please be sure to keep this source http://cloudtechtime.blog.51cto.com/10784015/1718360

What is a A/b test that brings Google $10 billion a year?