Statistical Cognition: Towards Evidence-Based
Practice in Statistics and Statistics Education
Department of Education and Psychology, The Open University, Israel
School of Psychological Science, La Trobe University, Melbourne, Australia
School of Psychological Science, La Trobe University, Melbourne, Australia
Practitioners and teachers should be able to justify their chosen techniques by taking into account research results: This is evidence-based practice (EBP). We argue that, specifically, statistical practice and statistics education should be guided by evidence, and we propose statistical cognition (SC) as an integration of theory, research, and application to support EBP. SC is an interdisciplinary research field, and a way of thinking. We identify three facets of SC—normative, descriptive, and prescriptive—and discuss their mutual influences. Unfortunately, the three components are studied by somewhat separate groups of scholars, who publish in different journals. These separations impede the implementation of EBP. SC, however, integrates the facets and provides a basis for EBP in statistical practice and education.
Keywords: Statistics education research; Statistical cognition; Statistical reasoning
Statistics Education Research Journal, 7(2), 20-39, http://www.stat.auckland.ac.nz/serj
Ó International Association for Statistical Education (IASE/ISI), May, 2008
Abelson, R. P. (1995). Statistics as principled argument. Hillsdale, NJ: Erlbaum.
Altman, D. G., Machin, D., Bryant, T. N., & Gardner, M. J. (2000). Statistics with confidence (2nd ed.). London: British Medical Journal.
Anderson, D. R., Burnham, K. P., & Thompson, W. L. (2000). Null hypothesis testing: Problems, prevalence, and an alternative. Journal of Wildlife Management, 64, 912-923.
Bar-Hillel, M. (1979). The role of sample size in sample evaluation. Organizational Behavior and Human Performance, 24, 245-257.
Bar-Hillel, M. (1980). The base-rate fallacy in probability judgments. Acta Psychologica, 44, 211-233.
Bar-Hillel, M., & Neter, E. (1993). How alike is it versus how likely is it: A disjunction fallacy in probability judgments. Journal of Personality and Social Psychology, 65, 1119-1131.
Belia, S., Fidler, F., Williams, J., & Cumming, G. (2005). Researchers misunderstand confidence intervals and standard error bars. Psychological Methods, 10, 389-396.
Bell, D. E., Raiffa, H., & Tversky, A. (1988). Decision making: Descriptive, normative, and prescriptive interactions. New York: Cambridge University Press.
Ben-Zvi, D. & Garfield, J. (Eds.). (2004). The challenge of developing statistical literacy, reasoning and thinking. Dordrecht, The Netherlands: Kluwer.
Beyth-Marom, R. (1982). Perception of correlation reexamined. Memory & Cognition, 10, 511-519.
Beyth-Marom, R. (1990). Mis/understanding diagnosticity: Direction and magnitude of change. In K. Borcherding, O. L. Larichev & D. M. Mesick (Eds.), Contemporary issues in decision making (pp. 203-223). North Holland: Elsevier.
Beyth-Marom, R., & Fischhoff, B. (1983). Diagnosticity and pseudodiagnosticity. Journal of Personality and Social Psychology, 45, 1185-1195.
Biggs, J., & Collis, K. (1991). Multimodal learning and the quality of intelligent behavior. In H. Rowe (Ed.), Intelligence, Reconceptualization and Measurement (pp. 57-76). Hillsdale, NJ: Erlbaum.
Brown, J. S., Collings, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18, 32-41
Cockcroft, W. H. (1982). Mathematics counts: Report of the committee of inquiry into the teaching of mathematics in schools. London: HMSO.
Cohen, J. (1994). The earth is round (p<.05). American Psychologist, 49, 997-1003.
Cohen, L. J. (1979). On the psychology of prediction: Whose is the fallacy? Cognition, 7, 385-407.
Cumming, G. (2007). Inference by eye: Pictures of confidence intervals and thinking about levels of confidence. Teaching Statistics, 29, 89-93.
Cumming, G., & Fidler, F. (in press). The new stats: Effect sizes and confidence intervals. In G. R. Hancock & R. O. Mueller (Eds.) Quantitative methods in the social and behavioral sciences: A guide for researchers and reviewers. Hillsdale, NJ: Erlbaum.
Cumming, G., & Finch, S. (2005). Inference by eye: Confidence intervals and how to read pictures of data. American Psychologist, 60, 170-180.
Cumming, G., Williams, J., & Fidler, F. (2004). Replication, and researchers’ understanding of confidence intervals and standard error bars. Understanding Statistics, 3, 299-311.
Davies, H. T. O. (1999). What is evidence based education? British Journal of Educational Studies, 47, 108-121.
Donovan, M. S., Bransford, J. D., & Pellegrino, J. W., (Eds.). (1999). How people learn: Bridging research and practice. Washington, DC: National Academy Press.
Eddy, D. M. (1982). Probabilistic reasoning in clinical medicine: Problems and opportunities. In D, Kahneman, P. Slovic, & A. Tversky (Eds.). Judgment under uncertainty: Heuristics and biases (pp. 249-267). New York: Cambridge University Press.
Edwards, A. L. (1968). Conservatism in human information processing. In B. Kleinmuntz (Ed.). Formal representation of human judgment (pp. 17-52). New York: Wiley.
Engel Clough, E., & Wood-Robinson, C. (1985). How secondary students interpret instances of biological adaptation. Journal of Biology Education, 19, 125-130.
Erickson, G. L. (1980). Children’s viewpoints of heat: A second look. Science Education, 64, 323-336.
Falk, R. (1986). Misconceptions of statistical significance. Journal of Structural Learning, 9, 83-96.
Falk, R. & Greenbaum, C. W. (1995). Significance tests die hard. Theory and Psychology, 5, 75-98.
Falk, R., & Konold, C. (1994). Random means hard to digest. Focus on Learning Problems in Mathematics, 16, 2-12.
Falk, R., & Konold, C. (1997). Making sense of randomness: Implicit encoding as a basis for judgment. Psychological Review, 104, 301-318.
Fidler, F., & Cumming, G. (2008). The new stats: Attitudes for the twenty-first century. In J.W. Osborne (Ed.). Best practices in quantitative methods (pp. 1-12). Thousand Oaks, CA: Sage.
Fidler, F., Cumming, G., Thomason, N., Pannuzzo, D., Smith, J., Fyffe, P., et al. (2005). Toward improved statistical reporting in the Journal of Consulting and Clinical Psychology. Journal of Consulting and Clinical Psychology, 73, 136-143.
Finch, S., Cumming, G., & Thomason, N. (2001). Reporting of statistical inference in the Journal of Applied Psychology: Little evidence of reform. Educational and Psychological Measurement, 61, 181-210.
Fischhoff, B., & Beyth-Marom, R. (1983). Hypothesis evaluation from a Bayesian perspective. Psychological Review, 90, 239-260.
Fisher, K. (1985). A misconception in biology: Amino acids and translation. Journal of Research in Science Teaching, 22, 53-62.
Gal, I. (2002). Adults’ statistical literacy: Meanings, components and responsibilities. International Statistical Review, 70, 1-25.
Garfield, J. (2002). The challenge of developing statistical reasoning. Journal of Statistics Education, 10(3).
Garfield, J., & Ahlgren, A. (1988). Difficulties in learning basic concepts in probability and statistics: Implications for research. Journal of Research in Mathematics Education, 19, 44-63.
Garfield, J., & Gal, I. (1999), Teaching and assessing statistical reasoning. In L. Stiff (Ed.), Developing mathematical reasoning in grades K-12 (pp. 207-219). Reston, VA: National Council Teachers of Mathematics.
Garfield, J., Hogg, B., Schau, C., & Whittinghill, D. (2002). First courses in statistical science: The status of educational reform efforts. Journal of Statistics Education 10(2).
Gigerenzer, G. (1991). How to make cognitive illusions disappear: Beyond heuristics and biases. European Review of Social Psychology, 2, 83-115.
Gigerenzer, G., & Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review, 102, 684-704.
Gilovich, T., Griffin, D., & Kahneman, D. (2002). Heuristics and biases: The psychology of intuitive judgments. Cambridge, UK: Cambridge University Press.
Glaser, R., & Bassok, M. (1989). Learning theory and the study of instruction. Annual Review of Psychology, 40, 631-666.
Grice, H. P. (1975). Logic and conversation. In P. Cole & J. L. Morgan (Eds.), Syntax and semantics 3: Speech acts (pp. 41-58). San Diego, CA: Academic Press.
Haller, H., & Krauss, S. (2002). Misinterpretations of significance: A problem students share with their teachers? Methods of Psychological Research, 7, 1-20.
Hargreaves, C. (1996). Teaching as a research based profession: Possibilities and prospects. London: Teacher Training Agency.
Hilton, D. J. (1995). The social context of reasoning: Conversational inference and rational judgment. Psychological Bulletin, 118, 248-271.
Hubbard, R., & Ryan, P. A. (2000). The historical growth of statistical significance testing in psychology and its future prospects. Educational and Psychological Measurement, 60, 661-681.
Huff, D. (1973). How to lie with statistics. Harmondsworth: Penguin.
Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.
Jenkins, H., & Ward, W. (1965). Judgment of contingency between responses and outcomes. Psychological Monographs, 19, 1-17.
Jones, G. A., Langrall, C. W, Thornton, C. A., & Mogill, A. T. (1997). A framework for assessing and nurturing young children’s thinking in probability. Educational Studies in Mathematics, 32, 101-125.
Jones, G. A., Langrall, C. W., Thornton, C. A., & Mogill, A. T. (1999). Students’ probabilistic thinking in instruction. Journal for Research in Mathematics Education, 30, 487-519.
Kafoussi, S. (2004).Can kindergarten children be successfully involved in probabilistic tasks? Statistics Education Research Journal, 3(1), 29-39.
Kahneman, D. (2003). A perspective on intuitive judgment and choice: Maps of bounded rationality. American Psychologist, 58, 697-720.
Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgment under uncertainty: Heuristics and biases. Cambridge, UK: Cambridge University Press.
Kalinowski, P., Fidler, F., & Cumming, G. (2008). Overcoming the inverse probability fallacy: A comparison of two teaching interventions. Manuscript in preparation.
Kitson, A., Harvey, G., & McCormack, B. (1998). Enabling the implementation of evidence based practice: A conceptual framework. Quality in Health Care, 17,149-158.
Kline, R. B. (2004). Beyond significance testing. Reforming data analysis methods in behavioral research. Washington, DC: American Psychological Association.
Lawrenz, F. (1986). Misconceptions of physical science concepts among elementary school teachers. School Science and Mathematics, 86, 654-660.
Leron, U., & Hazzan, O. (2006). The rationality debate: Application of cognitive psychology to mathematics education. Educational Studies in Mathematics, 62, 105-126.
Meehl, P. (1954). Clinical versus statistical prediction. Minneapolis, MN: University of Minnesota Press.
Meehl, P. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46(4), 806-834.
Mehlinger, H. D. (1995). School reform in the information age. Bloomington, IN: Indiana University Press.
Mooney, E. S. (2002). A framework for characterizing middle school students’ statistical thinking. Mathematical Thinking and Learning, 4, 23-63.
Nisbett, R., & Ross, L. (1980). Human inference: Strategies and shortcomings of social judgment. Englewood Cliffs, NJ: Prentice Hall.
Oakes, M. (1986). Statistical inference: A commentary for the social and behavioral sciences. New York: Wiley.
Oxford English Dictionary. (2007). Retrieved on October 15, 2007 from http://www.oed.com
Peterson, C. R., & Beach, L. R. (1967). Man as an intuitive statistician. Psychological Bulletin, 68, 29-46.
Piaget, J., & Inhelder, B. (1975). The origin of the idea of chance in children. London: Routledge & Kegan Paul.
Pinker, S. (2002). The blank slate: The modern denial of human nature. New York: Viking Penguin.
Pollatsek, A., Well, A. D., Konold, C., Hardiman, P., & Cobb, G. (1987). Understanding conditional probabilities. Organizational Behavior and Human Decision Processes, 40, 255-269.
Rosenthal, R., & Gaito, J. (1963). The interpretation of levels of significance by psychological researchers. Journal of Psychology, 55, 33–38.
Rossi, J. S. (1997). A case study in the failure of psychology as a cumulative science: The spontaneous recovery of verbal learning. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 175-197). Hillsdale, NJ: Lawrence Erlbaum.
Schmidt, F. L. & Hunter, J. (1997). Eight common but false objections to the discontinuation of significance testing in the analysis of research data. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 37-63). Hillsdale, NJ: Lawrence Erlbaum.
Sedlmeier, P. (1999). Improving statistical reasoning: Theoretical models and practical implications. London: LEA publishers.
Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 107, 309-316.
Shaklee, H., & Tucker, D. (1980). A rule analysis of judgment of covariation between events. Memory & Cognition, 8, 459-467.
Sloman, S. A. (1996). The empirical case for two systems of reasoning. Psychological Bulletin, 119, 3-22.
Smedslund, J. (1963). The concept of correlation in adults. Scandinavian Journal of Psychology, 4, 165-173.
Smith, E. L., & Anderson, C. W. (1984). Plants as producers: A case study of elementary science teaching. Journal of Research in Science Teaching, 21, 685-698.
Smith, G. (1998). Learning statistics by doing statistics. Journal of Statistics Education, 6, 1-12.
Stanovich, K.E., & West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate. Behavioral and Brain Sciences, 23, 645-726.
The American Heritage Dictionary the English Language (4th ed.). (2000). Boston: Houghton Mufflin.
Trinder, L. & Reynolds, S. (Eds.) (2000). Evidence-based practice: A critical appraisal. London: Blackwell Science.
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science 185, 1124-31.
Tversky, A., & Kahneman, D. (1982). Belief in the law of small numbers. In D. Kahneman, P. Slovic & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 23-31). New York: Cambridge University Press.
Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review, 90, 293-315.
Wallman, K. K. (1993). Enhancing statistical literacy: Enriching our society. Journal of the American Statistical Association, 88, 1-8.
Ward, W., & Jenkins, H. (1965). The display of information and judgment of contingency. Canadian Journal of Psychology, 19, 231-241.
Watson, J. & Callingham, R. (2003). Statistical literacy: A complex hierarchical construct. Statistics Education Research Journal, 2(2), 3-46.
Webster’s Online. (2007). Retrieved Oct. 15, 2007 from www.websters-dictionary-online.org
Wilkinson, L., & the Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.
Department of Education and Psychology
The Open University of Israel
108 Ravutski St. Raanana, 43107 Israel