R Language and significance test study notes

First, what is the significance of the test

The thought of the significance test is very simple, that is, the small probability event cannot occur. Although we have always stressed that the small probability event must occur in probability theory, the significance test still believes that the small probability event did not occur in the test I did.

The significance test is used to determine whether there is a difference between the experimental treatment group and the control group or the effects of the two different treatments, and whether the difference is significant.

A hypothesis that is to be tested is often referred to as H0, which is called the original hypothesis (or 0 hypothesis), and H0 's contradictory hypothesis is H1, called the alternative hypothesis.

⑴ when the original hypothesis is true, decides to abandon the original hypothesis, called the first kind of error, the probability of its occurrence is usually recorded as α;

⑵ in the original hypothesis, the decision to accept the original hypothesis, called the second type of error, the probability of its occurrence is usually recorded as beta.

usually only the maximum probability α of the first type of error is defined, and the probability of the second type of error is not considered beta. This hypothesis test is also called the significance test, and the probability α is called the significance level.

We commonly used the significance of the test is T-Test, chi-square test, correlation test, etc., in doing these tests, what need to pay attention to?

Two, normality and P-values

T-Test, chi-square test, correlation test Pearson method are based on the hypothesis of normal samples, so at the beginning of the hypothesis test, the general will do the analysis of normality. You can use Shapiro.test () in R. To conduct a normality test. Of course there are many other options available in the Norm.test package.

The P-value is the minimum horizontal value that can be rejected for the original hypothesis.

Three or four important amount of

To synthesize the previous narrative, we know that there are four very important quantities in the study of significance tests: sample size, significance level, efficacy, and effect value.

Sample size: Obviously, the more samples, the more accurate the grasp of the sample is, but given that we can't have an unlimited sample, how many samples can be met? In today's share we can find the answer through R.

Significance level: The probability of committing the first kind of error, which we will agree before doing the test, and finally decide the trade-off according to P value.

Efficacy: This is a quantity that is generally not mentioned in the significance test but is actually very useful. It measures the probability of a real event occurring. In other words, the greater the effectiveness, the less likely the second type of error will occur. Although the significance hypothesis test does not mention it, the important indicator for measuring the quality of hypothesis tests is that the two types of errors are as small as possible.

Effect value: The amount of the effect under the alternative hypothesis

Four, using the PWR package for efficacy analysis

The following functions are available in the PWR package:

Function	Object
Pwr.r.test	Correlation
Pwr.t.test	T test
Pwr.t2n.test	T-Test with different number of samples
Pwr.shisq.test	Chi-Square Inspection
Pwr.p.test	Proportion

Let's describe the usage of some of these functions.

1. T test

Call Format:

Pwr.t.test (n = null, d = null, Sig.level =0.05, power = null, type =c ("Two.sample", "One.sample", "paired"), alternative = C ("two.sided", "less", "greater"))

Parameter description:

N: Sample Size

Statistics of D:T Test

Sig.level: Significance level

Power: Efficacy level

Type: The test types, where two samples are the default, and the sample size is the same

Alternative: The statistical test is two-sided or one-sided, where the default is two-sided

For example: a known sample size of 60, a single sample T test statistics value of 0.2 (this can be removed by t.test (data) $statistic), a significant level of α=0.1, then the effectiveness of how much?

Enter the command in R:

[Plain]View PlainCopy

1. Pwr.t.test (d=0.2,n=60,sig.level=0.10,type= "One.sample", alternative= "two.sided")

Get results:

One-sample T Test power calculation

n = 60

D = 0.2

Sig.level = 0.1

Power = 0.4555818

Alternative = two.sided

We can see that the probability of making the second kind of error is above 50%, should we believe this result (whether it is rejected or accepted according to P-value)? Obviously not, how many samples will it take to reduce the second type of error to 10%?

In R, enter:

[Plain]View PlainCopy

1. Pwr.t.test (d=0.2,power=0.9,sig.level=0.10,type= "One.sample", alternative= "two.sided")

Get results:

One-sample T Test power calculation

n = 215.4542

D = 0.2

Sig.level = 0.1

Power = 0.9

Alternative = two.sided

That is, 216 samples can be satisfied with the results, so that the second type of error probability does not exceed 0.1.

For the two samples is similar, we do not repeat, we will introduce another T test case: Two samples are not equal.

Call Format:

Pwr.t2n.test (n1 = null, n2= null, d = null,sig.level = 0.05, power = null, alternative = C ("two.sided", "less", "greater"))

Parameter description:

N1 NUMBEROF observations in the first sample

N2 Numberof observations in the second sample

D effectsize

Sig.level significancelevel (Type I error probability)

Power powerof Test (1 minus Type II error probability)

Alternative Acharacter string specifying the alternative hypothesis, must be one of "two.sided" (default), "Greater" O R "Less"

For example: Two samples of 90, 60, statistics of 0.6, one-side T-Test, α=0.05, as a big indicator.

     Commands in R:

[Plain]View PlainCopy

1. Pwr.t2n.test (d=0.6,n1=90,n2=60,alternative= "greater")

     Output Result:

T test power calculation

N1 = 90

N2 = 60

D = 0.6

Sig.level = 0.05

Power = 0.9737262

Alternative = Greater

It can be seen that the effect is very large, and α=0.05, we believe the conclusion of this test is very credible.

2. Relevance

The Pwr.r.test () function analyzes the effect of correlation analysis. The format is as follows:

Pwr.r.test (n = null, r = null, Sig.level = 0.05, power = null, alternative = C ("two.sided", "less", "greater"))

Here, unlike the T test, R is a linear correlation coefficient, which can be obtained through COR (DATA1,DATA2), but it is important to note that you do not enter spearman,kendall correlation coefficients, which are relative to the level of measurement.

Assuming that we study the relationship between depression and loneliness, our original hypothesis and alternative assumptions are:

h0:r<0.25 V.s. h1:r>0.25

Assuming a significant level of 0.05, the original hypothesis is not true, we would like to have 90% confidence to reject H0, how much do we need to observe?

The following code gives the answer:

[Plain]View PlainCopy

1. Pwr.r.test (r=0.25,sig.level=0.05,power=0.9,alt= "greater")

Approximate correlation power calculation (Arctangh transformation)

n = 133.8325

R = 0.25

Sig.level = 0.05

Power = 0.9

Alternative = Greater

Easy to see, need a sample of 134

3. Chi-Square inspection

The original hypothesis is that the variables are independent and the alternative hypothesis is that the variables are not independent. The command is Pwr.chisq.test () and calls the format:

Pwr.chisq.test (w = null, N = NULL, DF = NULL, Sig.level = 0.05, power = null)

     Where w is the effect value, can be calculated by ES.W2, DF is the column-linked table degrees of freedom

Example:

[Plain]View PlainCopy

1. Prob<-matrix (c (0.225,0.125,0.125,0.125,0.16,0.16,0.04,0.04), nrow=2,byrow=true)
2. Prob
3. ES.W2 (Prob)
4. Pwr.chisq.test (W=ES.W2 (prob), df= (2-1) * (4-1), n=200)

   Output Result:

Chi Squared Power Calculation

W = 0.2558646

N = 200

DF = 3

Sig.level = 0.05

Power = 0.8733222

Note:n is the number of observations

In other words, the probability of the second-class error is about 13% and the result is more credible.

In R there are a lot of packages related to the analysis of efficacy, which we do not introduce are listed as follows:

R Language and significance test study notes