QUALITATIVE RESEARCH: AN ESSENTIAL PART OF STATISTICAL COGNITION research
PAV KALINOWSKI
Statistical Cognition Laboratory, School of
Psychological Science
La Trobe University, Melbourne, Australia.
p.kalinowski@latrobe.edu.au
JERRY LAI
Statistical Cognition Laboratory, School of
Psychological Science
La Trobe University, Melbourne, Australia
kj2lai@students.latrobe.edu.au
FIONA FIDLER
Statistical Cognition Laboratory, School of
Psychological Science
La Trobe University, Melbourne, Australia
f.fidler@latrobe.edu.au
GEOFF CUMMING
Statistical Cognition Laboratory, School of
Psychological Science
La Trobe University, Melbourne, Australia
g.cumming@latrobe.edu.au
ABSTRACT
Our research in statistical cognition
uses both qualitative and quantitative methods. A mixed method approach makes
our research more comprehensive, and provides us with new directions,
unexpected insights, and alternative explanations for previously established
concepts. In this paper, we review four statistical cognition studies that used
mixed methods and explain the contributions of both the quantitative and
qualitative components. The four studies investigated concern statistical
reporting practices in medical journals, an intervention aimed at improving
psychologists interpretations of statistical tests, the extent to which
interpretations improve when results are presented with confidence intervals
(CIs) rather than p-values, and graduate students' misconceptions about CIs.
Finally, we discuss the concept of scientific rigour and outline guidelines for
maintaining rigour that should apply equally to qualitative and quantitative
research.
Keywords: Statistics
education research; Mixed methods; Scientific rigour;
Qualitative analysis
__________________________
Statistics Education Research Journal,
9(2), 22-34, http://www.stat.auckland.ac.nz/serj
Ó International Association for Statistical Education
(IASE/ISI), November, 2010
REFERENCES
Beyth-Maron, R., Fidler,
F., & Cumming, G. (2008). Statistical cognition: Towards evidence-based practice in statistics
and statistics education. Statistics Education Research Journal, 7(2), 20-39.
Cohen, J.
(1960). A coefficient of agreement for nominal scales.
Educational and Psychological Measurement, 20(1), 37-46.
Coulson, M., Healey, M., Fidler, F., &
Cumming, G. (2010). Confidence
intervals permit, but don't guarantee, better inference than statistical
significance testing. Frontiers in
Quantitative Psychology and Measurement, 1(26),
1-9.
[Online:
www.frontiersin.org/quantitative_psychology_and_measurement/10.3389/fpsyg.2010.00026/abstract]
Cumming, G., Fidler, F., Leonard, M., Kalinowski,
P., Christiansen, A., Kleining, A., Lo, J., McMenamin, N., & Wilson, S. (2007). Statistical reform
in psychology: Is anything changing? Psychological
Science, 18(3), 230-232.
Elliott, R., Fischer, C., & Rennie,
D. (1999). Evolving guidelines for
publication of qualitative research studies in psychology and related fields.
British Journal of Clinical Psychology,
38(3), 215-229.
Faulkner, C. (2005).
Randomized controlled trials in clinical
psychology: Towards better understanding of research results using confidence
intervals and other statistics (Unpublished doctoral dissertation). La
Trobe University, Melbourne, Australia.
Fidler, F., & Loftus, G. R. (2009). Why figures with error bars should replace p
values: Some conceptual arguments and empirical demonstrations, Zeitschrift für Psychologie / Journal
of Psychology, 217(1),
27-37.
Fidler, F., Thomason, N., Finch,
S., & Leeman, J. (2004). Editors can lead researchers to confidence
intervals, but can’t make them think: Statistical reform lessons from medicine.
Psychological Science, 15(2),
119-126.
Fisher, R. A. (1960). The design of experiments (7th ed.).
Kalinowski, P.
(2010). Identifying misconceptions about confidence intervals. In
C. Reading (Ed.), ICOTS-8 Proceedings: Towards an evidence based society. Voorburg,
The Netherlands: International Association for Statistical Education,
International Statistics Institute.
Kline, R. B. (2004). Beyond significance testing: Reforming data analysis methods in
behavioral research. Washington, DC: American Psychological Association.
Schmidt, F. L.
(1996). Statistical significance testing and cumulative knowledge in
psychology: Implications for training of researchers. Psychological Methods,
1(2), 115-129.
Schmidt, F. L., & Hunter, J. E. (1997). Eight common but false
objections to the discontinuation of significance testing in the analysis of
research data. In L, L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.).
What if there were no significance tests?, (pp. 37-64). Mahwah, NJ: Lawrence
Erlbaum.
Thompson, B., & Snyder, P. A. (1997). Statistical significance testing practices
in The Journal of Experimental Education.
The Journal of Experimental Education,
66(1), 75-83.
Tukey, J. W.
(1969). Analyzing data: Sanctification or detective work? American Psychologist, 24(2),
83-91.
PAV KALINOWSKI
La Trobe University Victoria, 3086
Australia