26.6 Tests

Octave can perform many different statistical tests. The following table summarizes the available tests.

The tests return a p-value that describes the outcome of the test. Assuming that the test hypothesis is true, the p-value is the probability of obtaining a worse result than the observed one. So large p-values corresponds to a successful test. Usually a test hypothesis is accepted if the p-value exceeds 0.05.

Function File: [pval, f, df_b, df_w] = anova (y, g)

Perform a one-way analysis of variance (ANOVA).

The goal is to test whether the population means of data taken from k different groups are all equal.

Data may be given in a single vector y with groups specified by a corresponding vector of group labels g (e.g., numbers from 1 to k). This is the general form which does not impose any restriction on the number of data in each group or the group labels.

If y is a matrix and g is omitted, each column of y is treated as a group. This form is only appropriate for balanced ANOVA in which the numbers of samples from each group are all equal.

Under the null of constant means, the statistic f follows an F distribution with df_b and df_w degrees of freedom.

The p-value (1 minus the CDF of this distribution at f) is returned in pval.

If no output argument is given, the standard one-way ANOVA table is printed.

See also: manova.

Function File: [pval, chisq, df] = bartlett_test (x1, …)

Perform a Bartlett test for the homogeneity of variances in the data vectors x1, x2, …, xk, where k > 1.

Under the null of equal variances, the test statistic chisq approximately follows a chi-square distribution with df degrees of freedom.

The p-value (1 minus the CDF of this distribution at chisq) is returned in pval.

If no output argument is given, the p-value is displayed.

Function File: [pval, chisq, df] = chisquare_test_homogeneity (x, y, c)

Given two samples x and y, perform a chisquare test for homogeneity of the null hypothesis that x and y come from the same distribution, based on the partition induced by the (strictly increasing) entries of c.

For large samples, the test statistic chisq approximately follows a chisquare distribution with df = length (c) degrees of freedom.

The p-value (1 minus the CDF of this distribution at chisq) is returned in pval.

If no output argument is given, the p-value is displayed.

Function File: [pval, chisq, df] = chisquare_test_independence (x)

Perform a chi-square test for independence based on the contingency table x.

Under the null hypothesis of independence, chisq approximately has a chi-square distribution with df degrees of freedom.

The p-value (1 minus the CDF of this distribution at chisq) of the test is returned in pval.

If no output argument is given, the p-value is displayed.

Function File: cor_test (x, y, alt, method)

Test whether two samples x and y come from uncorrelated populations.

The optional argument string alt describes the alternative hypothesis, and can be "!=" or "<>" (nonzero), ">" (greater than 0), or "<" (less than 0). The default is the two-sided case.

The optional argument string method specifies which correlation coefficient to use for testing. If method is "pearson" (default), the (usual) Pearson’s produt moment correlation coefficient is used. In this case, the data should come from a bivariate normal distribution. Otherwise, the other two methods offer nonparametric alternatives. If method is "kendall", then Kendall’s rank correlation tau is used. If method is "spearman", then Spearman’s rank correlation rho is used. Only the first character is necessary.

The output is a structure with the following elements:

pval

The p-value of the test.

stat

The value of the test statistic.

dist

The distribution of the test statistic.

params

The parameters of the null distribution of the test statistic.

alternative

The alternative hypothesis.

method

The method used for testing.

If no output argument is given, the p-value is displayed.

Function File: [pval, f, df_num, df_den] = f_test_regression (y, x, rr, r)

Perform an F test for the null hypothesis rr * b = r in a classical normal regression model y = X * b + e.

Under the null, the test statistic f follows an F distribution with df_num and df_den degrees of freedom.

The p-value (1 minus the CDF of this distribution at f) is returned in pval.

If not given explicitly, r = 0.

If no output argument is given, the p-value is displayed.

Function File: [pval, tsq] = hotelling_test (x, m)

For a sample x from a multivariate normal distribution with unknown mean and covariance matrix, test the null hypothesis that mean (x) == m.

Hotelling’s T^2 is returned in tsq. Under the null, (n-p) T^2 / (p(n-1)) has an F distribution with p and n-p degrees of freedom, where n and p are the numbers of samples and variables, respectively.

The p-value of the test is returned in pval.

If no output argument is given, the p-value of the test is displayed.

Function File: [pval, tsq] = hotelling_test_2 (x, y)

For two samples x from multivariate normal distributions with the same number of variables (columns), unknown means and unknown equal covariance matrices, test the null hypothesis mean (x) == mean (y).

Hotelling’s two-sample T^2 is returned in tsq. Under the null,

(n_x+n_y-p-1) T^2 / (p(n_x+n_y-2))

has an F distribution with p and n_x+n_y-p-1 degrees of freedom, where n_x and n_y are the sample sizes and p is the number of variables.

The p-value of the test is returned in pval.

If no output argument is given, the p-value of the test is displayed.

Function File: [pval, ks] = kolmogorov_smirnov_test (x, dist, params, alt)

Perform a Kolmogorov-Smirnov test of the null hypothesis that the sample x comes from the (continuous) distribution dist.

if F and G are the CDFs corresponding to the sample and dist, respectively, then the null is that F == G.

The optional argument params contains a list of parameters of dist. For example, to test whether a sample x comes from a uniform distribution on [2,4], use

kolmogorov_smirnov_test (x, "unif", 2, 4)

dist can be any string for which a function distcdf that calculates the CDF of distribution dist exists.

With the optional argument string alt, the alternative of interest can be selected. If alt is "!=" or "<>", the null is tested against the two-sided alternative F != G. In this case, the test statistic ks follows a two-sided Kolmogorov-Smirnov distribution. If alt is ">", the one-sided alternative F > G is considered. Similarly for "<", the one-sided alternative F > G is considered. In this case, the test statistic ks has a one-sided Kolmogorov-Smirnov distribution. The default is the two-sided case.

The p-value of the test is returned in pval.

If no output argument is given, the p-value is displayed.

Function File: [pval, ks, d] = kolmogorov_smirnov_test_2 (x, y, alt)

Perform a 2-sample Kolmogorov-Smirnov test of the null hypothesis that the samples x and y come from the same (continuous) distribution.

If F and G are the CDFs corresponding to the x and y samples, respectively, then the null is that F == G.

With the optional argument string alt, the alternative of interest can be selected. If alt is "!=" or "<>", the null is tested against the two-sided alternative F != G. In this case, the test statistic ks follows a two-sided Kolmogorov-Smirnov distribution. If alt is ">", the one-sided alternative F > G is considered. Similarly for "<", the one-sided alternative F < G is considered. In this case, the test statistic ks has a one-sided Kolmogorov-Smirnov distribution. The default is the two-sided case.

The p-value of the test is returned in pval.

The third returned value, d, is the test statistic, the maximum vertical distance between the two cumulative distribution functions.

If no output argument is given, the p-value is displayed.

Function File: [pval, k, df] = kruskal_wallis_test (x1, …)

Perform a Kruskal-Wallis one-factor analysis of variance.

Suppose a variable is observed for k > 1 different groups, and let x1, …, xk be the corresponding data vectors.

Under the null hypothesis that the ranks in the pooled sample are not affected by the group memberships, the test statistic k is approximately chi-square with df = k - 1 degrees of freedom.

If the data contains ties (some value appears more than once) k is divided by

1 - sum_ties / (n^3 - n)

where sum_ties is the sum of t^2 - t over each group of ties where t is the number of ties in the group and n is the total number of values in the input data. For more info on this adjustment see William H. Kruskal and W. Allen Wallis, Use of Ranks in One-Criterion Variance Analysis, Journal of the American Statistical Association, Vol. 47, No. 260 (Dec 1952).

The p-value (1 minus the CDF of this distribution at k) is returned in pval.

If no output argument is given, the p-value is displayed.

Function File: manova (x, g)

Perform a one-way multivariate analysis of variance (MANOVA).

The goal is to test whether the p-dimensional population means of data taken from k different groups are all equal. All data are assumed drawn independently from p-dimensional normal distributions with the same covariance matrix.

The data matrix is given by x. As usual, rows are observations and columns are variables. The vector g specifies the corresponding group labels (e.g., numbers from 1 to k).

The LR test statistic (Wilks’ Lambda) and approximate p-values are computed and displayed.

See also: anova.

Function File: [pval, chisq, df] = mcnemar_test (x)

For a square contingency table x of data cross-classified on the row and column variables, McNemar’s test can be used for testing the null hypothesis of symmetry of the classification probabilities.

Under the null, chisq is approximately distributed as chisquare with df degrees of freedom.

The p-value (1 minus the CDF of this distribution at chisq) is returned in pval.

If no output argument is given, the p-value of the test is displayed.

Function File: [pval, z] = prop_test_2 (x1, n1, x2, n2, alt)

If x1 and n1 are the counts of successes and trials in one sample, and x2 and n2 those in a second one, test the null hypothesis that the success probabilities p1 and p2 are the same.

Under the null, the test statistic z approximately follows a standard normal distribution.

With the optional argument string alt, the alternative of interest can be selected. If alt is "!=" or "<>", the null is tested against the two-sided alternative p1 != p2. If alt is ">", the one-sided alternative p1 > p2 is used. Similarly for "<", the one-sided alternative p1 < p2 is used. The default is the two-sided case.

The p-value of the test is returned in pval.

If no output argument is given, the p-value of the test is displayed.

Function File: [pval, chisq] = run_test (x)

Perform a chi-square test with 6 degrees of freedom based on the upward runs in the columns of x.

run_test can be used to decide whether x contains independent data.

The p-value of the test is returned in pval.

If no output argument is given, the p-value is displayed.

Function File: [pval, b, n] = sign_test (x, y, alt)

For two matched-pair samples x and y, perform a sign test of the null hypothesis PROB (x > y) == PROB (x < y) == 1/2.

Under the null, the test statistic b roughly follows a binomial distribution with parameters n = sum (x != y) and p = 1/2.

With the optional argument alt, the alternative of interest can be selected. If alt is "!=" or "<>", the null hypothesis is tested against the two-sided alternative PROB (x < y) != 1/2. If alt is ">", the one-sided alternative PROB (x > y) > 1/2 ("x is stochastically greater than y") is considered. Similarly for "<", the one-sided alternative PROB (x > y) < 1/2 ("x is stochastically less than y") is considered. The default is the two-sided case.

The p-value of the test is returned in pval.

If no output argument is given, the p-value of the test is displayed.

Function File: [pval, t, df] = t_test (x, m, alt)

For a sample x from a normal distribution with unknown mean and variance, perform a t-test of the null hypothesis mean (x) == m.

Under the null, the test statistic t follows a Student distribution with df = length (x) - 1 degrees of freedom.

With the optional argument string alt, the alternative of interest can be selected. If alt is "!=" or "<>", the null is tested against the two-sided alternative mean (x) != m. If alt is ">", the one-sided alternative mean (x) > m is considered. Similarly for "<", the one-sided alternative mean (x) < m is considered. The default is the two-sided case.

The p-value of the test is returned in pval.

If no output argument is given, the p-value of the test is displayed.

Function File: [pval, t, df] = t_test_2 (x, y, alt)

For two samples x and y from normal distributions with unknown means and unknown equal variances, perform a two-sample t-test of the null hypothesis of equal means.

Under the null, the test statistic t follows a Student distribution with df degrees of freedom.

With the optional argument string alt, the alternative of interest can be selected. If alt is "!=" or "<>", the null is tested against the two-sided alternative mean (x) != mean (y). If alt is ">", the one-sided alternative mean (x) > mean (y) is used. Similarly for "<", the one-sided alternative mean (x) < mean (y) is used. The default is the two-sided case.

The p-value of the test is returned in pval.

If no output argument is given, the p-value of the test is displayed.

Function File: [pval, t, df] = t_test_regression (y, x, rr, r, alt)

Perform a t test for the null hypothesis rr * b = r in a classical normal regression model y = x * b + e.

Under the null, the test statistic t follows a t distribution with df degrees of freedom.

If r is omitted, a value of 0 is assumed.

With the optional argument string alt, the alternative of interest can be selected. If alt is "!=" or "<>", the null is tested against the two-sided alternative rr * b != r. If alt is ">", the one-sided alternative rr * b > r is used. Similarly for "<", the one-sided alternative rr * b < r is used. The default is the two-sided case.

The p-value of the test is returned in pval.

If no output argument is given, the p-value of the test is displayed.

Function File: [pval, z] = u_test (x, y, alt)

For two samples x and y, perform a Mann-Whitney U-test of the null hypothesis PROB (x > y) == 1/2 == PROB (x < y).

Under the null, the test statistic z approximately follows a standard normal distribution. Note that this test is equivalent to the Wilcoxon rank-sum test.

With the optional argument string alt, the alternative of interest can be selected. If alt is "!=" or "<>", the null is tested against the two-sided alternative PROB (x > y) != 1/2. If alt is ">", the one-sided alternative PROB (x > y) > 1/2 is considered. Similarly for "<", the one-sided alternative PROB (x > y) < 1/2 is considered. The default is the two-sided case.

The p-value of the test is returned in pval.

If no output argument is given, the p-value of the test is displayed.

Function File: [pval, f, df_num, df_den] = var_test (x, y, alt)

For two samples x and y from normal distributions with unknown means and unknown variances, perform an F-test of the null hypothesis of equal variances.

Under the null, the test statistic f follows an F-distribution with df_num and df_den degrees of freedom.

With the optional argument string alt, the alternative of interest can be selected. If alt is "!=" or "<>", the null is tested against the two-sided alternative var (x) != var (y). If alt is ">", the one-sided alternative var (x) > var (y) is used. Similarly for "<", the one-sided alternative var (x) > var (y) is used. The default is the two-sided case.

The p-value of the test is returned in pval.

If no output argument is given, the p-value of the test is displayed.

Function File: [pval, t, df] = welch_test (x, y, alt)

For two samples x and y from normal distributions with unknown means and unknown and not necessarily equal variances, perform a Welch test of the null hypothesis of equal means.

Under the null, the test statistic t approximately follows a Student distribution with df degrees of freedom.

With the optional argument string alt, the alternative of interest can be selected. If alt is "!=" or "<>", the null is tested against the two-sided alternative mean (x) != m. If alt is ">", the one-sided alternative mean(x) > m is considered. Similarly for "<", the one-sided alternative mean(x) < m is considered. The default is the two-sided case.

The p-value of the test is returned in pval.

If no output argument is given, the p-value of the test is displayed.

Function File: [pval, z] = wilcoxon_test (x, y, alt)

For two matched-pair sample vectors x and y, perform a Wilcoxon signed-rank test of the null hypothesis PROB (x > y) == 1/2.

Under the null, the test statistic z approximately follows a standard normal distribution when n > 25.

Caution: This function assumes a normal distribution for z and thus is invalid for n ≤ 25.

With the optional argument string alt, the alternative of interest can be selected. If alt is "!=" or "<>", the null is tested against the two-sided alternative PROB (x > y) != 1/2. If alt is ">", the one-sided alternative PROB (x > y) > 1/2 is considered. Similarly for "<", the one-sided alternative PROB (x > y) < 1/2 is considered. The default is the two-sided case.

The p-value of the test is returned in pval.

If no output argument is given, the p-value of the test is displayed.

Function File: [pval, z] = z_test (x, m, v, alt)

Perform a Z-test of the null hypothesis mean (x) == m for a sample x from a normal distribution with unknown mean and known variance v.

Under the null, the test statistic z follows a standard normal distribution.

With the optional argument string alt, the alternative of interest can be selected. If alt is "!=" or "<>", the null is tested against the two-sided alternative mean (x) != m. If alt is ">", the one-sided alternative mean (x) > m is considered. Similarly for "<", the one-sided alternative mean (x) < m is considered. The default is the two-sided case.

The p-value of the test is returned in pval.

If no output argument is given, the p-value of the test is displayed along with some information.

Function File: [pval, z] = z_test_2 (x, y, v_x, v_y, alt)

For two samples x and y from normal distributions with unknown means and known variances v_x and v_y, perform a Z-test of the hypothesis of equal means.

Under the null, the test statistic z follows a standard normal distribution.

With the optional argument string alt, the alternative of interest can be selected. If alt is "!=" or "<>", the null is tested against the two-sided alternative mean (x) != mean (y). If alt is ">", the one-sided alternative mean (x) > mean (y) is used. Similarly for "<", the one-sided alternative mean (x) < mean (y) is used. The default is the two-sided case.

The p-value of the test is returned in pval.

If no output argument is given, the p-value of the test is displayed along with some information.