Journal article · 2002 · Open access
Controlling the rate of Type I error over a large set of statistical tests
British Journal of Mathematical and Statistical Psychology, 55(1), 27–39 · Published 1 May 2002 · DOI 10.1348/000711002159680
In brief
“Controlling the rate of Type I error over a large set of statistical tests” is a 2002 journal article by Harvey J. Keselman and 2 co-authors, published in British Journal of Mathematical and Statistical Psychology. OpenAlex counts 162 citations as of 16 November 2026, which puts it in the top 7% of works published in 2002 by citations. It is open access: the full text is free to read at British Journal of Mathematical and Statistical Psychology.
- Citations
- 162
- Most cited in
- 2021 (13)
- Field-weighted impact
- 7.62
- References
- 27
- Authors
- 3
Abstract
When many tests of significance are examined in a research investigation with procedures that limit the probability of making at least one Type I error--the so-called familywise techniques of control--the likelihood of detecting effects can be very low. That is, when familywise error controlling methods are adopted to assess statistical significance, the size of the critical value that must be exceeded in order to obtain statistical significance can be extremely large when the number of tests to be examined is also very large. In our investigation we examined three methods for increasing the sensitivity to detect effects when family size is large: the false discovery rate of error control presented by Benjamini and Hochberg (1995), a modified false discovery rate presented by Benjamini and Hochberg (2000) which estimates the number of true null hypotheses prior to adopting false discovery rate control, and a familywise method modified to control the probability of committing two or more Type I errors in the family of tests examined--not one, as is the case with the usual familywise techniques. Our results indicated that the level of significance for the two or more familywise method of Type I error control varied with the testing scenario and needed to be set on occasion at values in excess of 0.15 in order to control the two or more rate at a reasonable value of 0.01. In addition, the false discovery rate methods typically resulted in substantially greater power to detect non-null effects even though their levels of significance were set at the standard 0.05 value. Accordingly, we recommend the Benjamini and Hochberg (1995, 2000) methods of Type I error control when the number of tests in the family is large.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Keselman, H. J., Cribbie, R. A., & Holland, B. S. (2002). Controlling the rate of Type I error over a large set of statistical tests. British Journal of Mathematical and Statistical Psychology, 55(1), 27–39. https://doi.org/10.1348/000711002159680
MLA 9
Keselman, Harvey J., et al. “Controlling the rate of Type I error over a large set of statistical tests.” British Journal of Mathematical and Statistical Psychology, vol. 55, no. 1, 2002, pp. 27–39. https://doi.org/10.1348/000711002159680.
Chicago
Keselman, Harvey J., Robert A. Cribbie, and Burt S. Holland. 2002. “Controlling the rate of Type I error over a large set of statistical tests.” British Journal of Mathematical and Statistical Psychology 55 (1): 27–39. https://doi.org/10.1348/000711002159680.
Harvard
Keselman, H. J., Cribbie, R. A. and Holland, B. S. (2002) ‘Controlling the rate of Type I error over a large set of statistical tests’, British Journal of Mathematical and Statistical Psychology, 55(1), pp. 27–39. doi:10.1348/000711002159680.
IEEE
H. J. Keselman, R. A. Cribbie, and B. S. Holland, “Controlling the rate of Type I error over a large set of statistical tests,” British Journal of Mathematical and Statistical Psychology, vol. 55, no. 1, pp. 27–39, 2002, doi: 10.1348/000711002159680.
BibTeX
@article{keselman2002controlling,
title = {{Controlling the rate of Type I error over a large set of statistical tests}},
author = {Keselman, Harvey J. and Cribbie, Robert A. and Holland, Burt S.},
journal = {British Journal of Mathematical and Statistical Psychology},
year = {2002},
volume = {55},
number = {1},
pages = {27--39},
doi = {10.1348/000711002159680},
url = {https://doi.org/10.1348/000711002159680}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2017 | 7 | |
| 2018 | 4 | |
| 2019 | 5 | |
| 2020 | 2 | |
| 2021 | 13 | |
| 2022 | 6 | |
| 2023 | 5 | |
| 2024 | 12 | |
| 2025 | 7 | |
| 2026 | 3 |
Its field-weighted citation impact is 7.62: it has been cited 7.62 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- type I error control
- family-wise error rate
- false discovery rate
- multiple comparisons
- statistical power
- Benjamini-Hochberg procedure
- significance level
Work with this paper
Most-cited papers in Mathematics
- Random ForestsLeo Breiman · Machine Learning · 2001 · 142,830 citations
- Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple TestingYoav Benjamini, Yosef Hochberg · Journal of the Royal Statistical Society Series B (Statistical Methodology) · 1995 · 113,923 citations
- Fitting Linear Mixed-Effects Models Using lme4Douglas M. Bates, Martin Maechler, Benjamin M. Bolker and 1 more · Journal of Statistical Software · 2015 · 100,328 citations
- The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations.Reuben M. Baron, David A. Kenny · Journal of Personality and Social Psychology · 1986 · 95,866 citations
- Using multivariate statisticsBarbara Gerson Tabachnick, Linda S. Fidell · 1983 · 79,867 citations
Questions about this paper
How do I cite this paper in APA style?
Keselman, H. J., Cribbie, R. A., & Holland, B. S. (2002). Controlling the rate of Type I error over a large set of statistical tests. British Journal of Mathematical and Statistical Psychology, 55(1), 27–39. https://doi.org/10.1348/000711002159680
How many times has this paper been cited?
162 times, according to OpenAlex on 16 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at British Journal of Mathematical and Statistical Psychology.
Who wrote this paper?
Harvey J. Keselman, Robert A. Cribbie, Burt S. Holland.
What is the DOI of this paper?
10.1348/000711002159680. The permanent link is https://doi.org/10.1348/000711002159680.
Bibliographic data from OpenAlex (CC0), last checked 16 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.