Statistical Cognition: Towards
Evidence-Based
Practice in Statistics and
Statistics Education
Ruth
Beyth-Marom
Department
of Education and Psychology, The Open University,
Israel
ruthbm@openu.ac.il
Fiona
Fidler
School
of Psychological Science, La Trobe University, Melbourne, Australia
f.fidler@latrobe.edu.au
Geoff
Cumming
School
of Psychological Science, La Trobe University, Melbourne, Australia
g.cumming@latrobe.edu.au
Abstract
Practitioners and teachers should be able to justify their chosen techniques by taking into account research results: This is evidence-based practice (EBP). We argue that, specifically, statistical practice and statistics education should be guided by evidence, and we propose statistical cognition (SC) as an integration of theory, research, and application to support EBP. SC is an interdisciplinary research field, and a way of thinking. We identify three facets of SC—normative, descriptive, and prescriptive—and discuss their mutual influences. Unfortunately, the three components are studied by somewhat separate groups of scholars, who publish in different journals. These separations impede the implementation of EBP. SC, however, integrates the facets and provides a basis for EBP in statistical practice and education.
Keywords: Statistics
education research; Statistical cognition; Statistical reasoning
__________________________
Statistics
Education Research Journal, 7(2), 20-39, http://www.stat.auckland.ac.nz/serj
Ó International
Association for Statistical Education (IASE/ISI), May, 2008
References
Abelson,
R. P. (1995). Statistics as principled argument.
Hillsdale, NJ: Erlbaum.
Altman, D. G., Machin,
D., Bryant, T. N., & Gardner, M. J. (2000). Statistics with confidence (2nd ed.).
London: British Medical Journal.
Anderson,
D. R., Burnham, K. P., & Thompson, W. L. (2000). Null hypothesis testing:
Problems, prevalence, and an alternative. Journal of Wildlife Management, 64, 912-923.
Bar-Hillel, M. (1979). The role of sample size in sample evaluation. Organizational
Behavior and Human Performance, 24, 245-257.
Bar-Hillel, M. (1980). The base-rate fallacy in probability judgments.
Acta Psychologica,
44, 211-233.
Bar-Hillel, M., & Neter, E. (1993). How alike is it versus how likely
is it: A disjunction fallacy in probability judgments. Journal of
Personality and Social Psychology, 65,
1119-1131.
Belia, S., Fidler,
F., Williams, J., & Cumming, G. (2005). Researchers misunderstand confidence intervals
and standard error bars. Psychological
Methods, 10, 389-396.
Bell, D. E., Raiffa, H.,
& Tversky, A. (1988). Decision making: Descriptive,
normative, and prescriptive interactions. New York: Cambridge University
Press.
Ben-Zvi, D. & Garfield, J. (Eds.). (2004). The challenge of developing statistical literacy, reasoning and thinking. Dordrecht,
The Netherlands: Kluwer.
Beyth-Marom,
R. (1982). Perception of correlation reexamined.
Memory & Cognition, 10,
511-519.
Beyth-Marom,
R. (1990). Mis/understanding diagnosticity:
Direction and magnitude of change. In K. Borcherding,
O. L. Larichev & D. M. Mesick (Eds.), Contemporary issues in decision making
(pp. 203-223). North Holland: Elsevier.
Beyth-Marom, R., & Fischhoff, B. (1983). Diagnosticity and pseudodiagnosticity. Journal
of Personality and Social Psychology,
45, 1185-1195.
Biggs, J., & Collis, K. (1991). Multimodal learning and the
quality of intelligent behavior. In H. Rowe (Ed.), Intelligence, Reconceptualization
and Measurement (pp. 57-76). Hillsdale, NJ: Erlbaum.
Brown, J. S., Collings, A., & Duguid, P. (1989). Situated cognition and the
culture of learning. Educational Researcher, 18, 32-41
Cockcroft,
W. H. (1982). Mathematics counts: Report
of the committee of inquiry into the teaching of mathematics in schools.
London: HMSO.
Cohen, J. (1994). The earth is round (p<.05). American Psychologist, 49, 997-1003.
Cohen, L. J. (1979). On the psychology of prediction:
Whose is the fallacy? Cognition, 7,
385-407.
Cumming, G.
(2007). Inference by eye: Pictures of confidence intervals and thinking about
levels of confidence. Teaching
Statistics, 29, 89-93.
Cumming, G.,
& Fidler, F. (in press). The new stats: Effect
sizes and confidence intervals. In G. R. Hancock & R. O. Mueller (Eds.) Quantitative
methods in the social and behavioral sciences: A
guide for researchers and reviewers.
Hillsdale, NJ: Erlbaum.
Cumming,
G., & Finch, S. (2005). Inference by eye: Confidence intervals and how to
read pictures of data. American Psychologist, 60, 170-180.
Cumming, G., Williams, J., & Fidler,
F. (2004). Replication, and
researchers’ understanding of confidence intervals and standard error bars. Understanding Statistics, 3,
299-311.
Davies,
H. T. O. (1999). What is evidence based education? British Journal of
Educational Studies, 47, 108-121.
Donovan,
M. S., Bransford, J. D., & Pellegrino, J. W.,
(Eds.). (1999). How people learn: Bridging
research and practice. Washington, DC: National Academy Press.
Eddy, D. M. (1982). Probabilistic reasoning in clinical medicine: Problems and
opportunities. In D, Kahneman, P. Slovic, & A. Tversky (Eds.).
Judgment under uncertainty: Heuristics and biases (pp. 249-267). New
York: Cambridge University Press.
Edwards,
A. L. (1968). Conservatism in human information processing.
In B. Kleinmuntz (Ed.).
Formal representation of human judgment (pp. 17-52).
New York: Wiley.
Engel Clough, E., & Wood-Robinson, C. (1985). How secondary students interpret
instances of biological adaptation. Journal of Biology Education, 19, 125-130.
Erickson,
G. L. (1980). Children’s viewpoints of heat: A second look. Science
Education, 64, 323-336.
Falk,
R. (1986). Misconceptions of statistical significance.
Journal of Structural Learning, 9,
83-96.
Falk, R. & Greenbaum,
C. W. (1995). Significance tests die hard. Theory and
Psychology, 5, 75-98.
Falk, R., & Konold, C. (1994). Random means hard to digest. Focus
on Learning Problems in Mathematics, 16,
2-12.
Falk, R., & Konold, C. (1997). Making sense of
randomness: Implicit encoding as a basis for judgment. Psychological
Review, 104, 301-318.
Fidler, F.,
& Cumming, G. (2008). The new stats: Attitudes for the twenty-first
century. In J.W. Osborne (Ed.). Best practices in quantitative methods (pp. 1-12). Thousand Oaks, CA: Sage.
Fidler, F., Cumming, G., Thomason, N., Pannuzzo, D.,
Smith, J., Fyffe, P., et al. (2005). Toward
improved statistical reporting in the Journal
of Consulting and Clinical Psychology. Journal of Consulting and Clinical Psychology, 73, 136-143.
Finch, S., Cumming, G., &
Thomason, N. (2001). Reporting
of statistical inference in the Journal
of Applied Psychology: Little evidence of reform. Educational and Psychological Measurement, 61, 181-210.
Fischhoff, B., & Beyth-Marom, R. (1983). Hypothesis evaluation from a
Bayesian perspective. Psychological Review, 90, 239-260.
Fisher,
K. (1985). A misconception in biology: Amino acids and translation. Journal
of Research in Science Teaching, 22,
53-62.
Gal,
I. (2002). Adults’ statistical literacy: Meanings, components and
responsibilities. International Statistical Review, 70, 1-25.
Garfield,
J. (2002). The challenge of developing statistical reasoning.
Journal of Statistics Education, 10(3).
[Online:
http://www.amstat.org/publications/jse/v10n3/garfield.html]
Garfield, J., & Ahlgren, A. (1988). Difficulties in learning basic
concepts in probability and statistics: Implications for research. Journal
of Research in Mathematics Education, 19,
44-63.
Garfield, J., & Gal, I. (1999), Teaching and assessing statistical
reasoning. In L.
Stiff (Ed.), Developing mathematical
reasoning in grades K-12 (pp. 207-219). Reston, VA: National Council
Teachers of Mathematics.
Garfield, J., Hogg, B., Schau, C., & Whittinghill, D. (2002). First courses in statistical science: The
status of educational reform efforts. Journal of
Statistics Education 10(2).
[Online: www.amstat.org/publications/jse/v10n2/garfield.html]
Gigerenzer,
G. (1991). How to make cognitive illusions disappear: Beyond heuristics and
biases. European Review of Social Psychology, 2, 83-115.
Gigerenzer, G., & Hoffrage, U. (1995). How to improve
Bayesian reasoning without instruction: Frequency formats. Psychological Review,
102, 684-704.
Gilovich, T., Griffin, D., & Kahneman, D. (2002). Heuristics and biases: The psychology of
intuitive judgments. Cambridge, UK: Cambridge University Press.
Glaser, R., & Bassok, M. (1989). Learning theory
and the study of instruction. Annual Review of Psychology, 40,
631-666.
Grice,
H. P. (1975). Logic and conversation. In P. Cole &
J. L. Morgan (Eds.), Syntax and semantics
3: Speech acts (pp. 41-58). San Diego, CA: Academic Press.
Haller,
H., & Krauss, S. (2002). Misinterpretations of significance: A problem
students share with their teachers? Methods
of Psychological Research, 7, 1-20.
Hargreaves,
C. (1996). Teaching as a research based profession: Possibilities and
prospects. London: Teacher Training Agency.
Hilton, D. J. (1995). The social context of reasoning: Conversational inference and rational
judgment. Psychological Bulletin,
118, 248-271.
Hubbard, R., & Ryan, P. A.
(2000). The historical growth of
statistical significance testing in psychology and its future prospects.
Educational and Psychological Measurement, 60, 661-681.
Huff, D. (1973). How to lie with statistics. Harmondsworth:
Penguin.
Institute of Medicine. (2001). Crossing the quality chasm: A new health system for
the 21st century. Washington, DC: National Academy Press.
Jenkins, H., & Ward, W. (1965). Judgment of contingency
between responses and outcomes. Psychological Monographs, 19, 1-17.
Jones, G. A., Langrall, C. W, Thornton, C. A.,
& Mogill, A. T. (1997). A framework for assessing and
nurturing young children’s thinking in probability. Educational Studies in Mathematics, 32, 101-125.
Jones,
G. A., Langrall, C. W., Thornton, C. A., & Mogill, A. T. (1999). Students’
probabilistic thinking in instruction. Journal for Research in Mathematics Education, 30, 487-519.
Kafoussi,
S. (2004).Can kindergarten children be successfully
involved in probabilistic tasks? Statistics Education Research Journal, 3(1), 29-39.
[Online: http://www.stat.auckland.ac.nz/~iase/serj/SERJ3(1)_kafoussi.pdf]
Kahneman, D. (2003). A perspective on intuitive judgment and
choice: Maps of bounded rationality. American
Psychologist, 58, 697-720.
Kahneman, D., Slovic,
P., & Tversky, A. (1982). Judgment under uncertainty:
Heuristics and biases. Cambridge, UK: Cambridge University Press.
Kalinowski, P., Fidler, F., & Cumming, G. (2008). Overcoming the inverse probability fallacy:
A comparison of two teaching interventions. Manuscript in preparation.
Kitson, A., Harvey,
G., & McCormack, B. (1998). Enabling the implementation of evidence based
practice: A conceptual framework. Quality in Health Care,
17,149-158.
Kline,
R. B. (2004). Beyond significance testing. Reforming data analysis methods in behavioral
research. Washington, DC: American Psychological Association.
Lawrenz, F. (1986). Misconceptions of
physical science concepts among elementary school teachers. School
Science and Mathematics, 86,
654-660.
Leron, U., & Hazzan,
O. (2006). The
rationality debate: Application of cognitive psychology to mathematics
education. Educational Studies in Mathematics, 62, 105-126.
Meehl, P.
(1954). Clinical versus statistical prediction.
Minneapolis, MN: University of Minnesota Press.
Meehl, P. (1978). Theoretical risks and tabular asterisks:
Sir Karl, Sir Ronald and the slow progress of soft psychology. Journal of
Consulting and Clinical Psychology, 46(4), 806-834.
Mehlinger,
H. D. (1995). School reform in the information age.
Bloomington, IN: Indiana University Press.
Mooney,
E. S. (2002). A framework for characterizing middle school
students’ statistical thinking. Mathematical Thinking and Learning, 4, 23-63.
Nisbett, R., & Ross, L. (1980). Human
inference: Strategies and shortcomings of social judgment. Englewood
Cliffs, NJ: Prentice Hall.
Oakes, M. (1986). Statistical inference: A commentary for the social and behavioral sciences. New York: Wiley.
Oxford English Dictionary. (2007). Retrieved on October 15, 2007 from http://www.oed.com
Peterson, C. R., & Beach, L. R. (1967). Man as an intuitive statistician. Psychological
Bulletin, 68, 29-46.
Piaget, J., & Inhelder, B. (1975). The origin of the idea of chance in children.
London: Routledge & Kegan
Paul.
Pinker, S. (2002). The blank slate: The modern denial of human nature. New York: Viking Penguin.
Pollatsek, A., Well, A. D.,
Konold, C., Hardiman, P.,
& Cobb, G. (1987). Understanding conditional probabilities. Organizational
Behavior and Human Decision Processes, 40, 255-269.
Rosenthal, R., & Gaito, J. (1963). The interpretation of levels of significance by psychological researchers. Journal of Psychology, 55, 33–38.
Rossi, J. S. (1997). A case study in the failure of psychology as
a cumulative science: The spontaneous recovery of verbal learning. In L. L.
Harlow, S. A. Mulaik, & J. H. Steiger
(Eds.), What if there were no significance tests? (pp. 175-197).
Hillsdale, NJ: Lawrence Erlbaum.
Schmidt, F. L. & Hunter, J. (1997). Eight common but false objections to the
discontinuation of significance testing in the analysis of research data. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance
tests? (pp. 37-63). Hillsdale, NJ: Lawrence Erlbaum.
Sedlmeier,
P. (1999). Improving statistical reasoning: Theoretical
models and practical implications. London: LEA publishers.
Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect
on the power of studies? Psychological Bulletin, 107, 309-316.
Shaklee, H., & Tucker, D. (1980). A rule analysis of judgment
of covariation between events. Memory &
Cognition, 8, 459-467.
Sloman, S. A. (1996). The empirical case
for two systems of reasoning. Psychological Bulletin, 119, 3-22.
Smedslund,
J. (1963). The concept of correlation in adults. Scandinavian
Journal of Psychology, 4,
165-173.
Smith, E. L., & Anderson, C. W. (1984). Plants as producers: A case study
of elementary science teaching. Journal
of Research in Science Teaching, 21, 685-698.
Smith,
G. (1998). Learning statistics by doing statistics.
Journal of Statistics Education, 6,
1-12.
Stanovich, K.E., &
West, R. F. (2000). Individual differences in reasoning: Implications for the rationality
debate. Behavioral and Brain Sciences, 23, 645-726.
The American
Heritage Dictionary the English Language (4th ed.).
(2000). Boston: Houghton Mufflin.
Trinder, L. & Reynolds, S. (Eds.) (2000). Evidence-based
practice: A critical appraisal. London: Blackwell Science.
Tversky, A., & Kahneman, D. (1974). Judgment under
uncertainty: Heuristics and biases. Science 185, 1124-31.
Tversky, A., & Kahneman, D. (1982). Belief in the law
of small numbers. In D. Kahneman, P. Slovic & A. Tversky (Eds.), Judgment
under uncertainty: Heuristics and biases (pp. 23-31). New York: Cambridge University Press.
Tversky, A., & Kahneman,
D. (1983).
Extensional versus intuitive reasoning: The conjunction fallacy in probability
judgment. Psychological Review, 90,
293-315.
Wallman,
K. K. (1993). Enhancing statistical literacy: Enriching our society. Journal
of the American Statistical Association, 88, 1-8.
Ward,
W., & Jenkins, H. (1965). The display of information and
judgment of contingency. Canadian Journal of Psychology, 19, 231-241.
Watson, J. & Callingham, R. (2003). Statistical literacy: A complex
hierarchical construct. Statistics Education Research Journal, 2(2), 3-46.
[Online: www.stat.auckland.ac.nz/~iase/serj/SERJ2(2)_Watson_Callingham.pdf]
Webster’s Online. (2007). Retrieved Oct. 15, 2007 from www.websters-dictionary-online.org
Wilkinson,
L., & the Task Force on Statistical Inference. (1999). Statistical methods
in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.
RUTH BEYTH-MAROM
Department of Education and
Psychology
The Open University of Israel
108 Ravutski
St. Raanana, 43107 Israel