Web page Test Common error, do not make such a mistake

Web page Production WEBJX article introduction: 5 ways to test Web pages that have led you astray.

Web-page testing can have a lot of benefits, but there are 5 of a/b tests and multivariate tests that make you go astray, so don't make mistakes like that.

Single variable, multivariable, "disruptive design" test

A univariate test is to test multiple values of only one variable, as we normally say, A/b test. Note that A/b test does not only have A/b two version, a/b/c/d/e/f/g is a/B testing!!。

Multivariate testing, that is, there will be multiple variables, such as the size of the logo, color, layout, logo size has 3 values, color has 3 values, layout also has 2 values, need to test the version of the 3*3*2=18! This is called the whole-factor experimental design. But the general manufacturing does not like this, the cost is too high, so there is the orthogonal experimental design. This can be a short time to find a significantly better scheme, but does not apply to the internet!

Subversive design is the new (radical redesigns), used in new products, totally different, test Look-and-feel.

There are a lot of benefits to testing a Web page, when to use A/b, and when to use a variable quantity?

Applicable conditions of A/b test

1. Novice!
2. Only one variable to test (nonsense)
3. "New Design" (1.1 is too slow!) ）
4. Traffic is too low, multivariate testing results in statistically insignificant

Applicable conditions of multivariable testing

1. Enough traffic, with multiple variables
2. Catching up with the industry boss
3. Unable to test "new design"

But there are 5 of a/b tests and multivariate tests that make you go astray, so don't make a mistake like that.

1. Web analytics programs are not optimal.

Before starting the test, make sure that the analysis program is OK, so that you can not only ensure that the analysis before the test is correct, the test process and the test can be run correctly. (To add, it is to ensure that before the analysis, the method of collecting data is unified.) Even if there are errors there are errors, compared to each other will not take you to the ditch. ）

2. No user segmentation

An average, comprehensive indicator would mask the real problem. For example, you are testing the site's average bounce rate of 59%, if you go to breakdown by user type, the new user may be 74%, return users may be 38%. So the goal should be to reduce the bounce rate for new users.

This same situation exists in domestic and international users, mail subscribers ... Sadly, not all programs can subdivide users.

(Supplement: The correct user segmentation and business combination is very close, generally need to be modeled, through a reasonable variable to subdivide the user.) In order to achieve more effective segmentation)

3, using the wrong measure

4, the test time is too short. Learn statistical knowledge, mean test, card statistics, etc., do not prematurely pronounce a version of the victory.

The above content is mostly taken from http://www.getelastic.com/5-common-ab-multivariate-testing-mistakes/, onlookers please.