Journal of Social Sciences and Humanities

Articles Information

“Statistics Gone on Holiday”: Misinterpretations of Hypothesis Tests Propagated by Internet Resources

Pages: 165-170 Views: 73 Downloads: 27

[01]
Christopher Thron, Department of Science and Mathematics, Texas A&M University-Central Texas, Killeen, Texas, USA.
[02]
Nancy Hernandez, Department of Science and Mathematics, Texas A&M University-Central Texas, Killeen, Texas, USA.

“Type I error” is a basic concept in statistical hypothesis testing. However, the term is used in two subtly different senses in statistics texts and other statistical literature. Specifically, type I error can be construed either as a conditional event (i.e. presuming that the null hypothesis is true) or an unconditional event. We explain the distinctions between the different usages of type I error, and we conduct a logical analysis of popular statistics web sites to determine their usage of the terminology. Our analysis shows that ambiguous and inconsistent usage of this terminology leads to wrong interpretations of significance level in many web pages, leading in turn to faulty interpretations of the results of experiments. We discuss the reasons for this long-standing lack of consensus in the definition of type I error. The unconditional-event definition is more intuitive and agrees with the original formulation Neyman and Pearson in 1933, but professional statisticians favor the conditional-event definition. The fact that users of statistics come from widely different fields makes it difficult to arrive at a single agreed-upon definition. We conclude that even in a rigorous technical subject like statistics, ambiguous terminology can go unrecognized and can continue to produce errors in reasoning.

Social Sciences, Hypothesis Testing, Type I Error, Significance Level, Conditional Probability, Internet, Terminology

[01]
Bar-Hillel M, Falk R. Some teasers concerning conditional probabilities. Cognition. 1982; 11 (2): 109-122. doi: 10.1016/0010-0277 (82) 90021-x.
[02]
Falk R. Misconceptions of statistical significance. Journal of Structural Learning. 1986; 9: 83–96.
[03]
Rossmo DK. Failures in Criminal Investigation: Errors of Thinking. The Police Chief 2009: 76 (10).
[04]
Thron C, Miller V. Persistent Confusions about Hypothesis Testing in the Social Sciences. Social Sciences. 2014: 4 (2); 361-372. doi: 10.3390/socsci4020361.
[05]
Khanacademy.org. Hypothesis test for difference of means [Internet]. n. d. [cited 21 July 2016]. Available from: https://www.khanacademy.org/math/probability/statistics-inferential/hypothesis-testing-two-samples/v/hypothesis-test-for-difference-of-means.
[06]
Khanacademy.org. Small sample hypothesis test [Internet]. Khan Academy. n.d. [cited 21 July 2016]. Available from: https://www.khanacademy.org/math/probability/statistics-inferential/hypothesis-testing/v/small-sample-hypothesis-test.
[07]
Khanacademy.org. Type 1 errors [Internet]. n. d. [cited 21 July 2016]. Available from: https://www.khanacademy.org/math/probability/statistics-inferential/hypothesis-testing/v/type-1-errors.
[08]
Minitab.com. What are type I and type II errors? [Internet]. n. d. [cited 21 July 2016]. Available from: http://support.minitab.com/en-us/minitab/17/topic-library/basic-statistics-and-graphs/hypothesis-tests/basics/type-i-and-type-ii-error/.
[09]
Smith MK. Type I and II Errors [Internet]. n. d. [cited 21 July 2016]. Available from: https://www.ma.utexas.edu/users/mks/statmistakes/errortypes.html.
[10]
Wikipedia.org. Type I and type II errors. [Internet]. n. d. [cited 21 July 2016]. Available from: https://en.wikipedia.org/wiki/Type_I_and_type_II_errors.
[11]
Explorable.com. Type I Error and Type II Error - Experimental Errors [Internet]. n. d. [cited 21 July 2016]. Available from: https://explorable.com/type-i-error.
[12]
Statistics.about.com. Hypothesis Test Example [Internet]. n. d. [cited 21 July 2016]. Available from: http://statistics.about.com/od/HypothesisTests/a/Hypothesis-Test-Example-With-Calculation-Of-Probability-Of-Type-I-And-Type-II-Errors.htm.
[13]
Statistics.about.com. What Is the Difference Between Type I and Type II Hypothesis Testing Errors? [Internet]. n.d. [cited 21 July 2016]. Available from: http://statistics.about.com/od/Inferential-Statistics/a/Type-I-And-Type-II-Errors.htm.
[14]
Onlinestatbook.com. Type I and II Errors [Internet]. n. d. [cited 21 July 2016]. Available from: http://onlinestatbook.com/2/logic_of_hypothesis_testing/errors.html.
[15]
Huang H. Multiple Hypothesis Testing and False Discovery Rate [Internet]. n. d. [cited 21 July 2016]. Available from: http://www.stat.berkeley.edu/~hhuang/STAT141/Lecture-FDR.pdf.
[16]
Lap.umd.edu. PSYC200_0803.pdf [Internet]. n. d. [cited 21 July 2016]. Available from: http://lap.umd.edu/psyc200/handouts/PSYC200_0803.pdf.
[17]
Jones J. Hypothesis Testing [Internet]. n. d. [cited 21 July 2016]. Available from: https://people.richland.edu/james/lecture/m113/def_testing.html.
[18]
Sigmazone.com. Roger Clemens and a Hypothesis Test [Internet]. n. d. [cited 21 July 2016]. Available from: http://www.sigmazone.com/Clemens_HypothesisTest.htm.
[19]
Investopedia.com. Type I Error Definition | Investopedia [Internet]. 2007. [cited 21 July 2016]. Available from: http://www.investopedia.com/terms/t/type_1_error.asp.
[20]
Investopedia.com. Alpha Risk Definition | Investopedia [Internet]. 2010. [cited 21 July 2016]. Available from: http://www.investopedia.com/terms/a/alpha-risk.asp.
[21]
Noortgate WV, Onghena P, Sotos AE, Vanhoof S. Students’ misconceptions of statistical inference: A review of the empirical evidence from research on statistics education. Educational Research Review 2007; 2 (2); 98-113. doi: 10.1016/j.edurev.2007.04.001.
[22]
Banerjee A, Bhawalkar JS, Chaudhury S, Chitnis UB, Jadhav SL. Hypothesis testing, type I and type II errors [Internet]. n.d. [cited 21 July 2016]. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2996198/.
[23]
Campbell R. Type I and II error. [Internet]. n. d. [cited 21 July 2016]. Available from: http://www.cs.uni.edu/~campbell/stat/inf5.html
[24]
Emathzone.com. Type 1 Error and Type 2 Error. [Internet]. n.d. [cited 21 July 2016]. Available from: http://www.emathzone.com/tutorials/basic-statistics/type-1-error-and-type-2-error.html.
[25]
Ilvento TP. Values, Type I and Type II Error [Internet]. n. d. [cited 21 July 2016]. Available from: http://www.ilvento.org/Lectures/Hypothesis%203.pdf.
[26]
King WB. Hypothesis testing [Internet]. n. d. [cited 21 July 2016]. Available from: http://ww2.coastal.edu/kingw/psycstats/mathreview/reviewfiles/hypothesis.html.
[27]
Onlinecourses.science.psu.edu. 7.2 - Terminologies, Type I and Type II Errors for Hypothesis Testing [Internet]. n. d. [cited 21 July 2016]. Available from: https://onlinecourses.science.psu.edu/stat500/node/40.
[28]
Quora.com. In statistics, do the probabilities of Type 1 and Type 2 errors have to add up to 1? [online]. n. d. [cited 21 July 2016]. Available from: https://www.quora.com/In-statistics-do-the-probabilities-of-Type-1-and-Type-2-errors-have-to-add-up-to-1.
[29]
Quora.com. In statistics, what is a type 1 and type 2 error? [Internet]. n.d. [cited 21 July 2016]. Available from: https://www.quora.com/In-statistics-what-is-a-type-1-and-type-2-error.
[30]
Stomponstep1.com. P-Value, Statistical Significance & Types of Error [Internet]. 2014. n. d. [cited 21 July 2016]. Available from: http://www.stomponstep1.com/p-value-null-hypothesis-type-1-error-statistical-significance/.
[31]
Neyman, J., & Pearson, E. S. (1933, October). The testing of statistical hypotheses in relation to probabilities a priori. In Mathematical Proceedings of the Cambridge Philosophical Society (Vol. 29, No. 04, pp. 492-510). Cambridge University Press.

Vol. 5, Issue 3, September Submit a Manuscript Join Editorial Board Join Reviewer Team

About This Journal |

All Issues |

Open Access |

Indexing |

Payment Information |

Author Guidelines |

Review Process |

Publication Ethics |

Editorial Board |

Peer Reviewers |

Copyright © 2014 - American Institute of Science except certain content provided by third parties.