No matter what field of research or statistical investigation you are in, the significance test is widely used as a method to determine whether there is a difference between two or more datasets. The author, as a researcher in the field of scientific research, has also suffered a lot in the aspect of significance testing. Later, fascinated by the statistical theory half-load to touch the significance of the examination of the fur, but also for the significance of the test theory of the exquisite, variety, the logic of the rigorous. Here, a close-up of this blog post, for those who are still struggling in the significant test mire of non-statistical professional colleagues in the field of reference. Because I am not a statistical professional graduate, the view is humble, laughable is also looking at your industry predecessors, the field leader of the Liberal enlighten. I thank you for watching officer. This blog post is dedicated to solving a few questions, listed here: 1. What is the significance test? 2. Why do I have to do a significant test? 3. What is the significance of the test? Below please follow the author's step step into the significance of the test of "past and present life." One:
significance test prequel: What is the significance of the test? How does it relate to statistical hypothesis testing? Why do we have to do a significant test? "Significance test" is actually a Chinese translation of English significance test. In statistics, the significance test is a "statistical hypothesis test" (statistical hypothesis testing), a significant test is used to detect whether there is a difference between the experimental group and the control group in the scientific experiment and whether the difference is significant. In fact, the "Portal Background" (statistical hypothesis test) to understand the significance test is more helpful for a novice researcher to understand the significance of the test. The term "statistical hypothesis test" actually points out that the precondition of "significance test" is "statistical hypothesis", in other words "no hypothesis, no test". Anyone who has to be aware of their own research before using the significance test
AssumptionsWhat is the significance of the test is "water in the Moon, Mirror Flower", elusive. In more popular terms, it is necessary to make a hypothesis about the scientific data and then use the test to check if the hypothesis is correct. In general, the hypothesis to be tested is referred to as the original hypothesis, which is recorded as H0, and the hypothesis that corresponds to the H0 (contrary) is called the alternative hypothesis, which is recorded as H1. If the original hypothesis is true, the conclusion of the test is to persuade you to abandon the original hypothesis. At this point, we call this error the first type of error. The probability of the first type of error is usually recorded as α if the original hypothesis is not true, and the conclusion of the test advises you not to abandon the original hypothesis. At this point, we call this error the second type of error. Generally, the probability of a second type of error is recorded as β, which usually only qualifies the maximum probability α of the first type of error, regardless of the probability of the second type of error β. We call this hypothesis test a significant test, and the probability α is called the significance level. The significance level is the mathematics field to be established, generally has the α=0.05,0.025.0.01 these three kinds of situations. The conclusion that represents a significant test is that the error rate must be less than 5% or 2.5% or 1% (statistically, the events that occur in the real world with a probability of less than 5% are called "impossible events").
(this passage actually teaches the relationship between the significance test and the statistical hypothesis test)To facilitate the next lecture, here is an example. Mr. Zhao opened a daily-use company, which opened branches in Zhengzhou and Hangzhou respectively. The following data are now available as sales for two subsidiaries, each of which represents the company's sales for a period of one months in a year. Zhengzhou Branch z = {23,25,26,27,23,24,22,23,25,29,30} Hangzhou Branch h = {24,25,23,26,27,25,25,28,30,31,29} now, Mr. Zhao wants to know whether there is a significant difference in sales between the two companies (whether there is sales in Zhengzhou branch > sales in Hangzhou branch or vice versa) in order to plan the strategic business adjustment of the company. Subordinates know Zhao boss's difficulties, have suggested that "just ask for the average to know which branch of the sales is greater." But as a highly educated Mr. Zhao understands such a philosophy as "we live in a world of probabilities". That means that the average is not a problem, even if the average sales value of the Hangzhou branch is greater than the sales average of the Zhengzhou branch, the sales of the Hangzhou branch must be greater than the sales of the Zhengzhou branch, because " Such a seemingly existing greater than the relationship is in fact accidental and not necessarily ". Mr. Zhao finally decided to check the two data using the variance test. (Please ignore why the variance test, the selection of the test method will be detailed below) Finally, Mr. Zhao found that the P-value of the variance test = 0.2027, which means that Although the average annual sales of Hangzhou branch is 26.63 more than the sales of Zhengzhou branch 25.18, but in essence, the sales of two subsidiaries are not significantly different. (believe that at this time you have thousands of grass mud horse ran: How the variance test is done?) What's the P-value? Why does p=0.2027 mean there is no significant difference in sales? How much information is good and swollen? ）
don't worry, don't panic, let's start over, and sort out what happened to Mr. Zhao here. It is necessary to understand the "slow motion" rooted in Mr Zhao's mind. 1th: As mentioned above, "no hypothesis, no test", what kind of assumptions did Mr. Zhao make (hypothesis)? Since Mr. Zhao wants to know whether there is a significant difference in sales between the two companies, his hypothesis is that there is no significant difference between the sample set Z (Zhengzhou Branch) and the sample set H (Hangzhou branch), in other words, there is no difference between the two sets (there is no difference between the sales)! This is Mr Zhao's hypothesis. So the question is, why does Mr. Zhao want to assume that there is no difference between the two sample sets, rather than assuming that the two sample sets are different. Because this hypothesis (hypothesis) is the original hypothesis of the variance test (null hypothesis). Then the question comes again, what is the original hypothesis. The so-called original hypothesis is the "original hypothesis" that the mathematics community defaults to facilitate discussion. There is no such thing as a conventional word. 2nd: P-Value What's going on? This does not have to do with how the P-value is obtained, directly to the conclusion. In the case of significant level α=0.05, the P-value <0.05 rejects the original hypothesis. Our original hypothesis is that there is no significant difference between the sample set Z and the sample set H, but because of the p=0.2027>0.05, we accept the original hypothesis that there is no significant difference between the sample set Z and the sample set H. Of course there is the refusal of acceptance, if the P value is less than 0.05, then the original hypothesis is rejected, that is, the set Z and set H have significant differences between. 3rd: How to do the variance test and why do the variance test after the fine talk, here for the moment not the table. At the end of this chapter,
give the answers to two questions in this chapter, I believe you can now understand: 1 What is a statistical hypothesis test? The so-called statistical hypothesis test is a priori to the total (random variable)
ParametersOr
Overall distributionform to make a hypothesis, and then use the sample information to determine whether the hypothesis is reasonable. The statistical hypothesis test, which only limits the first kind of error probability, is called the significance test. In the above example, our hypothesis is a kind of significance test. Because the variance test does not apply to estimating parameters and estimating the overall distribution, it is used to test whether there are differences between the two groups of tests. And the variance test is exactly what we are concerned about is whether there is a difference between the mean values of the two sets (two distributions). 2. Why do I have to do a significant test? Because we want to determine whether the difference between a sample and our assumptions about the whole is purely an opportunity variant, or is caused by inconsistencies between the assumptions we make and the overall realities. In our case, the difference is that the mean of H is higher than the mean of Z, but the final conclusion is that the variance is purely opportunistic (h mean >z mean is accidental, when the number of samples in H and Z tends to infinity, The mean of H will approach equal to the mean of Z, rather than the assumption that it is inconsistent with the real situation. If the P-value is <0.05, then it means that our hypothesis (the H-set and the z-set are not different) is inconsistent with the real situation, which makes the hypothesis not true, that is, the H set and the Z-set are different.
Second: How to do the significance of the test? (based on MATLAB)The significance test can be divided into parameter test and non-parameter test. The parameter test requires that the sample be derived from the normal population (subject to normal distribution), and that these normal populations have the same variance, and that under such basic assumptions (normal assumptions and variance-homogeneity assumptions) the values of each population mean are equal and belong to the parameter test. When the data does not satisfy the normality and variance homogeneity assumptions, the parameter test may give the wrong answer, and the rank-based nonparametric test should be used. The method of parameter test and the explanation of corresponding knowledge point (this gives only the common
Variance Analysis): Variance analysis is mainly divided into ' ① single factor Anova '; ' ② two-factor univariate anova '; ' ③ multivariate univariate anova '; ' ④ univariate variance analysis '. The following section introduces the implementation methods for various variance analysis. But before I do, I'm going to go through two important points of "spoiler", and understanding these points helps to differentiate between different types of variance analysis.
what is called a factor, what is a dollar? First explain what is called "Yuan". I assume that the person who is reading this blog must have the level of literacy, then you must be on the "one-yuan two-time equation" "Two-yuan one-time equation" "Multivariate one-time equation" This concept is not unfamiliar. The so-called "yuan" refers to the number of unknown variables. In the statistical hypothesis test, the unknown variable to be tested is called "Yuan" and the Behavior (event) which affects the unknown variable is called "factor". The students who have had the basis of machine learning can interpret the "meta" and "factor" as the "number of features" and "number of tags" in machine learning respectively. Having multiple features is "multiple" and having multiple labels is a "multi-factor".
① method and case of single-Factor univariate variance Analysis:Related MATLAB functions: function One: ANOVA1 (x, Group, displayopt) parameter interpretation: In the first usage, X is an array of n rows and 1 columns, and Group is an array of n rows and 1 columns. X is the set of samples to be tested, which includes all the data from several control and experimental groups. So how does the machine know which data belongs to which group? Very simply, the column vector one by one by group corresponds to the specified. This example comes from the help document of Matlab, which is used here to illustrate: Suppose there are now three sets of data Group One (ST): 82 86 79 83 84 85 86 87 Group Two (AL1): 74 82 78 75 76 77 Group Three (AL2): 79 79 77 78 82 79 Now we need to do the variance test of these three sets of data, the method of using ANOVA1 function is as follows 1. First put all the data in the same array strength:>> strength = [82 86 79 83 84 85 86 87 74 82 78 75 76 77 7 9 79 77 78 82 79];2. Set the label corresponding to the strength corresponding position alloy:>> alloy = {' st ', ' St ', ' St ', ' St ', ' St ', ' St ', ' St ', ' st ', ' al1 ', ' al1 ', ' Al1 ', ' al1 ', ' al1 ', ' al1 ', ' al2 ', ' al2 ', ' al2 ', ' al2 ', ' al2 ', ' Al2 '};3. Call anova1 function >> p = anova1 (Strength,alloy) The result will be a numeric value and two graphs, and a value of P value. P worth the view has been introduced above, here is no longer a thin introduction. In this example, the value of P is as follows P = 1.5264e-004 Obviously, from the P-value, there is a significant difference between the three groups of values. One thing to mention: the significant difference in P here does not mean that 22 of the three groups have significant differences, but only that significant differences exist between the three groups. The first picture is a table, which is called the Anova table. I believe that many non-statistical students see the ANOVA watch is a moment of collapse, a bunch of problems rushing out: source is what ghost? SS is what ghost, DF is what ghost, MS is what ghost, F is what ghost, Prob>f is what ghost, etc. here to solve the "what" problem, the table gives a detailed explanation: source represents the source of variance (whose variance), The variance sources here include groups (inter-group), Error (intra-group), TotAl (total); SS (sum of squares) means that the sum of squares and DF (degree of freedom) represents the degrees of freedom MS (Mean squares) means that the mean variance f is the F value (f statistic), and the F value equals the ratio of the mean square to the intra-group. It reflects the size of the random error action. Prob>f indicates that P-values need to draw two small questions here: The first small question is how the F-value is used, and the second small question is what is the relationship between the P-value and F-value? The first to popularize the relationship between P-value and F-value: F Actual value >f Check table value, then p<=0.05f the actual value <f table value, then the F value is not difficult to see in this example is equal to 15.4, it is the ratio between the group variance 92.4 and the group Variance 6. Check F distribution table (), according to N=19 (Total DF), m=2 (groups DF) can get F0.05(m, n-m-1) = F0.05(2, 16) = 3.634. f The actual value of the 15.4>f table value of 3.634, so you can determine the significance of the difference exists, and the P-value is less than 0.05 describes how to use only f-value to determine the significance of the difference of the method and the F-value of the relationship between the P-value. The following table is a box chart, its view as shown in the table note: here it is necessary to mention the role of the parameters in the ANOVA1 function displayopt. In a large-scale ANOVA1 call (for example, put ANOVA1 in the for loop repeatedly called), you need to set displayopt to ' off ', otherwise anova1 each call will draw two pictures, which will quickly consume the computer's memory, easy to cause the program to crash. In addition to the first method of calling Anova1, described earlier, there is also a way to use a balanced variance analysis. Equilibrium is the requirement that the number of statistical data in different groups must be the same. The statistics for each group appearing in the previous example are {8,6,6} and are non-balanced. In a balanced state, the data for each group constitutes a single column in X, so that the parameter group can be omitted, and the invocation can be simplified to ANOVA1 (X) in the above, as we mentioned. The variance analysis must satisfy two assumptions, namely the normality hypothesis and the variance homogeneity hypothesis. Therefore, in a complete statistical project, it is necessary to first detect the normality hypothesis of data and the assumption of variance homogeneity, which involves two other functions lillietest normal test function (this is exactly the distribution hypothesis test we mentioned above, not the parameter test, It examines the target of what kind of distribution the data set obeys and the Vartestn Variance homogeneity test (which is exactly what we referred to as the parameter test rather than the distribution hypothesis test, which detects the target of the data set to follow what parameters, here is the variance) function two: Lillietest (X) >> [H,p] = Lillietest (Strength (1:8)) H = 0p = 0.5000 Explanation: H = 0 The data is considered to be normally distributed, and h=1 is considered to be non-compliant. The distribution P >0.05 can be assumed to accept the original hypothesis h = 0, then the data obeys the normal distribution >> [h,p] = Lillietest (Strength (9:14)) H = 0p = 0 .5000>> [H,p] = Lillietest (Strength (15:20)) H = 0p =&NBsp 0.5000 It can be concluded that all three groups of strength obey normal distribution function three: Vartestn (X, Group) >> p = vartestn (Strength,alloy, ' off ') p= 0.5142 Note: X and group must be a column vector, or you will get an error. The different group in X is homogeneous, that is, the variance is homogeneous.
② method and case for two-factor unary Anova: As mentioned above, since it is a two-factor, there are multiple labels. Therefore, two-factor univariate ANOVA can be interpreted as "single-feature double-label machine learning technology". Since the two-factor univariate variance analysis requires that the data be balanced, its label can be omitted, as is the second use of the ANOVA1 described above. Here's an example of a ANOVA2 help document in MATLAB that explains how anova2 is used. Here's a batch of popcorn data, and now we know that the popcorn quality score is related to two factors, one is the brand of popcorn (there are three brands: Gourmet,national,generic) and the other is the process of making popcorn (frying, air pressure). These data are described below: brand Gourmet national genericmethods frying & nbsp 5.5000 4.5000   ; 3.5000 frying 5.5000 4.5000 4.0000 & nbsp; frying 6.0000 &NBS P &nBsp 4.0000 3.0000 pressure 6.5000 5.0000 4.00 00 air pressure 7.00 5.5000 5.0000 pressure &nbs P 7.0000 5.0000   ; 4.5000 now needs to know three goals, first: whether there are significant differences between columns and columns (significant differences between brands), the original hypothesis is that significant differences do not exist; second: Is there a significant difference between rows and rows, The original hypothesis is that the significant difference does not exist; Thirdly: whether the interaction between the brand and the method is obvious, the original hypothesis is that the interaction is not obvious in order to complete the above three problems, so the ANOVA2 function is introduced specifically, the parameters of the ANOVA2 function are as follows: P = anova2 (X, Reps, displayopt X is the array you want to test. where each column of x represents a factor, each of the lines of x represents another factor, and several of these uses reps indicate. Displayopt, like ANOVA1, is no longer detailed here. The return of the ANOVA2 is a one-value picture. Here is the specific Matlab method:>> popcorn =[ 5.5000 4.5000&nbsP 3.5000 5.5000 4.5000 4.0000 6.0000 4.0000 3.0000 6.5000 5.0000 4.0000 7.0000 5.5000 5.0000 7.0000 5.0000 4.5000]; >> [P,table,stats] = ANOVA2 (popcorn,3) p = 0.0000 0.0001 0.7462 Explanation: P (1) = 0.0000, overturning the original hypothesis, So significant differences between columns and columns exist (significant differences between brands); P (2) = 0.0001, overturning the original hypothesis, so significant differences between rows and rows exist (significant differences between methods exist); P (3) = 0.7462, which preserves the original hypothesis, the interaction between the brand and the method is not obvious. The columns in the chart represents columns, rows represents lines, interaction represent interactions, others are exactly the same as what we described in Anova2, and are no longer analyzed in detail.
③ Method and case study of multivariate anova of multiple factors:p = Anovan (x, Group, Opt); where x represents the data to be tested; The group represents the factor of x, because it is a multi-factor, so the group is composed of multiple columns. Opt can be selected as ' model ', which can be filled with ' full ' and ' interaction ' after model. For example, the factor has three x, Y, Z, then if the model is interaction, the results will include the significance of X. Y, the significance of Z, the significance of the Xy,xz,yz interaction, if the model is full, the results will include the significance of X, the significance of Y, The significance of Z, the interaction of Xy,xz,yz and the significance of the interaction of XYZ. The example here still comes from the help document of Matlab, Y is the data to be tested, G1,G2,G3 is the 3 factors corresponding to data one by one in Y (data label) y = [52.7 57.5 45.9 44.5 53.0 57.0 45.9 44.0] '; g1 = [1 2 1 2 1 2 1 2];g2 = {' Hi '; ' Hi '; ' Lo '; ' Lo '; ' Hi '; ' Hi '; ' Lo '; ' Lo '};g3 = {' May '; ' May '; ' May '; ' May '; ' June '; ' June '; ' June '; ' June '}; >> p = Anovan (Y,{g1 G2 G3}, ' model ', ' interaction ') p = 0.0347 0.0048 0.2578 0.0158 0.1444 0.5000 This There's a little trick to use, if you want to do a non-equilibrium two-factor univariate Anova then you can also use the multivariate one-element variance analysis function.
④ methods and examples of multivariate Anova for single-Factor analysis: [d, p] = MANOVA1 (x, Group);p, X and group are the same as before. The original hypothesis of this variance analysis is that "the group mean of each group is the same multivariate vector" here is the interpretation of D: d=0, accept the original hypothesis d=1, reject the original hypothesis, that the group mean values of each group are not identical, but can not reject the assumption that they collinear. d=2, rejecting the original hypothesis, the group mean vectors of each group may be coplanar, but not collinear. Four products (X1,X2,X3,X4) are sold according to two different sales methods, data are as follows: Ref. X1 x2 x3 x4 Sales Method 1 338 ( ) 119 &NBS P 233 , , 260 203 429 15 403 205 Sp 480 260 + ( ) 468 295 + 416 265 &NBSp $ 377 260 to + 299 &NBSP 211 390 ; 212 103 310 416 213 &nbs P 507 2 >> X = 125 &nbs P 60 338 210 119 80 233 330 63 51 260 203 65 51 429 150 &N bsp;130 65 403 205 65 33 480 260& nbsp 100 34 468 295 65 63 416 265 &NBSP;110&NBsp 69 377 260 88 78 299 360 73 63 390 320 103 54 416 310 & nbsp 64 51 507 320 >> groups = 1 1 1 1 1 1 2 &N Bsp 2 2 2 2 2 2>> [D, p] = MANOVA1 (X, Groups); d = 0p = 0.0695 Therefore, reject the original hypothesis, Group mean values for each group are not the same multivariate vectors.
Non-parametric inspection:To this kind, the parameter examination part even is finished saying. We can recall that the four functions of the parameter test are divided into ANOVA1,ANOVA2,ANOVAN,MANOVA1. They are based on a common two hypothesis: the normality hypothesis and the variance homogeneity hypothesis, respectively, corresponding to the function lillietest and VARTESTN. However, in the actual work, we can not always meet these two assumptions of statistical data, at this time, if the forced use of parameter testing will cause errors. At this point, a non-parametric test based on the rank sum can be used. Here we introduce two kinds of nonparametric tests: Kruskal-wallis Test, Friedman Test. Through the part of the parameter test, the reader has already been introduced to the significance of the test, some details are no longer detailed here, reserved for interested readers self-query. Here, we only need to introduce the test of sub-parameters.
①kruskal-wallis InspectionKruskal-wallis test is also called single-factor nonparametric variance analysis, non-parametric version of ANOVA1. The original hypothesis of this test is that K independent samples come from the same normal population. Its MATLAB functions are as follows: P = Kruskalwallis (X,group) x,group,p is identical to the parameter test. No more detailed description.
②friedman InspectionThe Friedman test is also called a two-factor rank-variance analysis, and is a non-parametric version of ANOVA2. As with ANOVA2, the data to be tested must also be balanced. However, it is important to note that the Friedman test and the ANOVA2 test are not identical, ANOVA2 also pay attention to the impact of two factors on the test data, but the Friedman test only focus on one of the 2 factors of the impact of the test data, The other factor is used to differentiate between groups. As shown in the Matrix X,friedman test, where there is no significant difference between the individual columns of x (Factor a), he is completely uninterested in the lines (factor B, also known as the District Group). Therefore, the original hypothesis of the Friedman test is that K independent samples (columns of x) come from the same normal population. As to why the Friedman test is not interested in factor B, here is an example to illustrate. This example is derived from "MATLAB statistical analysis and application of 40 case analysis" 4 food judges 1234 on the four regions of the ABCD chef's famous dish boiled fish evaluation score, the data is as follows: Area A B C D Food jury   ; 1 85 82&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;82&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;79&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;2 87 + 82 3 90 Bayi 76 4 75 Now I We would like to know whether the quality of boiled fish in these four places is the same. Data Analysis: Our goal is to determine whether the quality of boiled fish is the same in four places. Then the same jury scored on four regional chefs, and there was little reference between the judges in different regions about the same chef (the jury's own subjective consciousness was too strong). Therefore, we think that four districts are factor A, and the judges are factor B (district factor), the data between different district groups is not comparable. >> X = 85 82 82 79 87 75&N Bsp 86 82 90 81 80 76 80 75& nbsp 81 75>> p = Friedman (x,1) p = 0.0434 Therefore, it can be considered that the level of the production of boiled fish in four regions is significantly different. There is also a one by one comparison between the two that have significant differences.
Conclusion:In this case, the common significance of the test method is even finished. Hope that through this blog post can make the significance of the test is no longer the heart of the crossing great misfortune, do not have to talk about "check" color change. If we can do this, I hope it will suffice.
About the significance test, all you want is here!!
