On Conceptual Analysis as the Primary Qualitative Approach to
Statistics Education Research in PsychologY
Agnes Petocz
University of Western Sydney, Australia
a.petocz@uws.edu.au
Glenn Newbery
University of Western Sydney, Australia
g.newbery@uws.edu.au
ABSTRACT
Statistics education in psychology often
falls disappointingly short of its goals. The increasing use of qualitative
approaches in statistics education research has extended and enriched our
understanding of statistical cognition processes, and thus facilitated improvements
in statistical education and practices. Yet conceptual analysis, a fundamental
part of the scientific method and arguably the primary qualitative method
insofar as it is logically prior and equally applicable to all other empirical
research methods--quantitative, qualitative, and mixed--has been largely
overlooked. In this paper we present the case for this approach, and then
report results from a conceptual analysis of statistics education in psychology.
The results highlight a number of major problems that have received little
attention in standard statistics education research.
Keywords: Scientific method; Critical inquiry; Qualitative and
quantitative research; Statistics for psychology
__________________________
Statistics Education Research Journal,
9(2), 123-146, http://www.stat.auckland.ac.nz/serj
Ó
International Association for Statistical Education (IASE/ISI), November, 2010
REFERENCES
Allport, G. (1966). Traits revisited.
American Psychologist, 21(1), 1-10.
Australian Psychology Accreditation
Council (2010). Rules for accreditation and
accreditation standards for psychology courses.
Barnett, R. (2007). A
will to learn: Being a student in an age of uncertainty. Buckingham, UK:
Society for research in Higher Education, Open University Press.
Bassham, G., Irwin, W., Nardone, H., & Wallace,
J. M. (2008). Critical thinking: A
student's introduction (3rd ed.).
Boston: McGraw Hill.
Belar, C. D., & Perry, N. W. (Eds.). (1991). Proceedings: National conference on
scientist-practitioner education. Sarasota, FL: Professional Resource
Exchange.
Bennett, M. R., & Hacker, P. M.
S. (2003). Philosophical
foundations of neuroscience. Oxford: Blackwell.
Bickhard, M. H. (1992). Myths of science. Theory
& Psychology, 2(3), 321-337.
Bijker, M., Wynants, G., & van Buuren, H.
(2006). A comparative study of the effects of motivational
and attitudinal factors on studying statistics. In A. Rossman & B. Chance (Eds.), Working cooperatively in statistics education:
Proceedings of the Seventh International Conference on Teaching Statistics,
Salvador, Brazil. [CDROM]. Voorburg,
The Netherlands: International Statistical Institute.
[Online: http://www.stat.auckland.ac.nz/~iase/publications/17/5H2_BIJK.pdf]
Bowker, G., & Star, S.
(2000). Sorting things
out: Classification and its consequences. Cambridge, MA: The MIT
Press.
Bryman, A. (1988). Quantity and quality in social
research. London: Unwin Hyman.
Carnap, R. (1966). Philosophical foundations of
physics. New York: Basic Books.
Chiesi, F., & Primi, C. (2010).
Cognitive and non-cognitive factors related to students’ statistics
achievement. Statistics Education Research
Journal, 9(1), 6-26.
[Online: http://www.stat.auckland.ac.nz/~iase/serj/SERJ9(1)_Chiesi_Primi.pdf]
Chomsky, N. (1964). A review of B. F.
Skinner’s Verbal behavior. In
J. A. Fodor & J. J. Katz (Eds.), The structure of
language: Readings in the philosophy of language (pp. 547–578). Englewood
Cliffs, NJ: Prentice-Hall. (Reprinted from Language,
1959, 35, 26-58)
Cohen, M. R., & Nagel, E. (1934).
An introduction to
logic and scientific method. London: Routledge
& Kegan Paul.
Crombie, A. C. (1994). Styles of scientific thinking in the
European tradition. London: Duckworth.
Denzin, N. K., & Lincoln, Y. S. (Eds.). (1994).
Handbook of qualitative research.
Thousand Oaks, CA: Sage.
Denzin, N. K., & Lincoln, Y. S. (Eds.). (2000). Handbook of qualitative research (2nd
ed.). Thousand Oaks, CA: Sage.
Flick, U. (2002). An introduction to qualitative research
(2nd ed.). London: Sage.
Freud, S. (1975). Lecture XXXV. The
question of a Weltanschauung.
In J. Strachey (Ed. &
Trans.), The standard edition of the complete
psychological works of Sigmund Freud (Vol. 22, pp. 158-182). London:
Hogarth Press. (Original work published 1933)
Friedman, M. (1991). The re-evaluation of logical positivism. Journal of Philosophy, 88(10), 505-519.
Friedman, M. (1999). Reconsidering logical
positivism. Cambridge, MA: Cambridge University Press.
Furedy, J. J. (1978).
Negative results: Abolish the name but honour the
same. In J. P. Sutcliffe (Ed.), Conceptual analysis and method in psychology. Essays in honour of W. M. O’Neil.
Sydney: Sydney University Press.
Gal, I., & Ograjenšek,
I. (2010). Qualitative research in the service of
understanding learners and users of statistics. International Statistical Review, 78(2), 287-296.
Gigerenzer, G. (1987). Probabilistic thinking and the fight against subjectivity.
In L. Krüger, G. Gigerenzer,
& M. Morgan (Eds.), The probabilistic revolution (Vol. 2, pp.
11-36). Cambridge, MA: MIT Press.
Gigerenzer, G. (1993). The superego, the ego, and the id in statistical reasoning.
In G. Keren & C. Lewis (Eds.), A handbook for data analysis in the behavioral sciences: Methodological
issues (pp. 311-339). Hillsdale, NJ: Erlbaum.
Gigerenzer, G., & Murray, D. J. (1987). Cognition as intuitive statistics.
Hillsdale, NJ: Lawrence Erlbaum.
Good, J. M. M. (2007). The
affordances for social psychology of the ecological approach to social knowing.
Theory & Psychology, 17(2), 265-295.
Grayson, D. A. (1988). Limitation on the use of scales in psychiatry. Australian & New Zealand Journal of
Psychiatry, 22(1), 99-108.
Grayson, D. A. (1998). The frequentist façade and the flight
from evidential inference. British Journal of Psychology, 83(2),
325-345.
Grayson, D. A., & Oliphant, G. W. (1996). Multiple
inference procedures: Problems and solutions. In C. R. Latimer & J. Michell (Eds.), At
once scientific and philosophic: A Festschrift for John Philip Sutcliffe
(pp. 23-46). Brisbane: Boombana Publications.
Green, C. D. (1992). Of immortal
mythological beasts. Theory &
Psychology, 2(3), 291-320.
Haack, S. (1996). Towards a sober sociology of
science. In P. R. Gross (Ed.), The flight from
science and reason (pp. 259-265). New York: Annals of the New York Academy
of Sciences.
Haack, S. (2003). Defending
science--within reason: Between scientism and cynicism. New York: Prometheus
Books.
Halpin, P. F., & Stam, H. J. (2006).
Inductive inference or inductive behavior: Fisher and Neyman-Pearson
approaches to statistical testing in psychological research. American Journal of Psychology, 119(4), 625-653.
Hammersley, M. (1989). The
dilemma of qualitative method: Herbert Blumer and the
Chicago tradition. London: Routledge.
Henwood, K. (1996).
Qualitative inquiry: perspectives, methods and psychology. In J. T. E.
Richardson (Ed.), Handbook of qualitative
research methods (pp. 25-40). Oxford: BPS Blackwell.
Herschel, J. F. W. (1987). A preliminary discourse on the study of
natural philosophy. Chicago: University of Chicago Press.
Hibberd, F. J. (2005). Unfolding social constructionism.
New York: Springer.
Hubbard, R., Parsa,
R. A., & Luthy, M. R. (1997). The spread of statistical significance testing in psychology.
The case of the Journal
of Applied Psychology, 1917-1994. Theory & Psychology, 7(4),
545-554.
Hubbard, R., & Ryan, P. (2000).
The historical growth of statistical significance testing in
psychology--and its future prospects. Educational
and Psychological Measurement, 60(5),
661-681.
Janesick, V. J. (2000). The choreography of qualitative research design. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed.)
(pp. 379-399). Thousand Oaks: Sage.
Kalinowski, P., Lai, J., Fidler, F., & Cumming,
G. (2010). Qualitative research: An essential part of statistical
cognition. Statistics Education Research
Journal, 9(2), 22-34.
[Online: http://www.stat.auckland.ac.nz/~iase/serj/SERJ9(2)_Kalinowski.pdf]
Kerr, N. L. (1998). HARKing:
Hypothesizing after the results are known. Personality
and Social Psychology Review, 2(3),
196-217.
Kirk, R. E. (1996). Practical significance: A concept
whose time has come. Educational and
Psychological Measurement, 56(5),
746-759.
Lalonde, R. N., & Gardner, R. C. (1993). Statistics
as a second language? A model for predicting
performance in psychology students. Canadian
Journal of Behavioural Science, 25(1), 108-125.
Lambie, J. (1991). The misuse of Kuhn in psychology. The Psychologist: Bulletin of the British Psychological Society, 1, 6-11.
Leahey, T. H. (2004). A history of psychology. Main currents in psychological thought
(6th ed.). New Jersey: Pearson Education.
Levitt, N. (2001). The sources and dangers of
postmodern anti-science: Do these intellectual popguns matter? Free Inquiry, 21(2), 44-47.
Lincoln, Y. S., & Guba, E. G. (2000). Paradigmatic
controversies, contradictions, and emerging confluences. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp. 163-188).
Thousand Oaks, CA: Sage.
Lissitz, R. W. (Ed.). (2009).
The concept of validity. Revisions,
new directions, and applications.
Charlotte, NC: Information Age Publishing Inc.
Machado, A., & Silva, F. J.
(2007). Toward a richer view of the scientific method.
The role of conceptual analysis. American Psychologist, 62(7),
671-681.
Mackay, N., & Petocz,
A. (Eds.). (in press). Realism and psychology: Collected essays. Leiden: Brill.
Medawar, P. B. (1969). Induction and intuition in scientific
thought. London: Methuen & Co Ltd.
Michell, J. (1990). An introduction to the logic
of psychological measurement. Hillsdale, NJ: Erlbaum.
Michell, J. (1997). Quantitative science and the definition of measurement in
psychology. British
Journal of Psychology, 88(3), 355-383.
Michell, J. (1999). Measurement in psychology: A critical
history of a methodological concept. Cambridge: Cambridge University Press.
Michell, J. (2000). Normal science, pathological science and psychometrics. Theory & Psychology, 10(5), 639-667.
Michell, J. (2001). Teaching and misteaching measurement in
psychology. Australian
Psychologist, 36(3), 211-217.
Michell, J. (2002). Stevens’s theory
of scales of measurement and its place in modern psychology. Australian Journal of Psychology, 54(2), 99-104.
Michell, J. (2003). The
quantitative imperative: Positivism, naďve realism and the place of qualitative
methods in Psychology. Theory & Psychology,
13(1), 5-31.
Michell, J. (2004a). Item
response models, pathological science and the shape of error: Reply to Boorsboom and Mellenbergh. Theory & Psychology, 14(1), 121-129.
Michell, J. (2004b). The place of qualitative research in psychology. Qualitative Research in Psychology, 1(4), 307-319.
Michell, J. (2008). Is
psychometrics pathological science? Measurement:
Interdisciplinary Research and Perspectives, 6(1 & 2), 7-24.
Michell, J. (2009a). The psychometricians’ fallacy: Too clever by half? British Journal of Mathematical and
Statistical Psychology, 62(1), 41-55.
Michell, J. (2009b). Invalidity in validity. In R. W. Lissitz
(Ed.), The concept of validity. Revisions,
new directions, and applications (pp. 111-133).
Charlotte, NC: Information Age Publishing Inc.
Michell, J. (2010). The quantity/quality interchange. A blind
spot on the highway of science. In A. Toomela
& J. Valsiner (Eds.), Methodological thinking in psychology: 60 years gone astray? (pp.
45-68). Charlotte, NC: Information Age Publishing Inc.
Neyman, J., & Pearson, E. S. (1933). On the
problem of most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal
Society of London, A, 231, 289-337.
Nussbaum, M. (2010, April 30). Skills
for life. Why cuts in humanities teaching pose a threat to democracy
itself. Times Literary
Supplement, pp. 13-15.
Oliphant, G. W., & Grayson, D. A.
(1996). Some misconceptions concerning post hoc
statistical hypothesis testing. In C. R. Latimer & J. Michell (Eds.), At
once scientific and philosophic: A Festschrift for John Philip Sutcliffe
(pp. 47-56). Brisbane: Boombana Publications.
Osborne, J. W. (2010). Challenges for
quantitative psychology and measurement in the 21st century. Frontiers in Psychology, 1(1), 1-3.
Petocz, A. (2004). Science, meaning and the scientist-practitioner model of treatment.
In D. Jones (Ed.), Working with dangerous people (pp. 25-42).
Oxford: Radcliffe Medical Press.
Petocz, A. (in press). Re-thinking the place of semiotics in psychology and its implications
for psychological research. In S. C. Hamel (Ed.), Semiotics: Theory and applications. New York: Nova Science.
Petocz, P., & Reid, A. (2010). On becoming a
statistician: A qualitative view. International
Statistical Review, 78(2), 271-286.
Petocz, P., & Sowey, E. (2010). Statistical diversions. Teaching Statistics, 32(2), 60-64.
Porter, T. M. (1995). Trust in numbers: The pursuit of objectivity in science and public life.
Princeton: Princeton University Press.
Potter, W. J. (1996). An analysis of thinking and research about
qualitative methods. Mahwah, NJ: Lawrence Erlbaum Associates.
Resnick, M. D. (1997). Mathematics as a science of
patterns. Oxford: Oxford University Press.
Richardson, J. T. E. (1996). Handbook of qualitative research methods.
Oxford: BPS Blackwell.
Rosenthal, R. (1979). The “file drawer problem” and
tolerance for null results. Psychological
Bulletin, 86(3), 638-641.
Rosnow, R. L., & Rosenthal, R. (1989). Statistical
procedures and the justification of knowledge in psychological science. American Psychologist, 44(10), 1276-1284.
Rozeboom, W. (1960). The fallacy of the null
hypothesis significance test. Psychological
Bulletin, 57(5), 416-428.
Schmidt, F. L. (1992). What do data really mean? Research findings, meta-analysis, and cumulative knowledge in
psychology. American Psychologist,
47(10), 1173-1181.
Solovey, M. (2004). Riding natural scientists’
coattails onto the endless frontier: The SSRC and the quest for scientific
legitimacy. Journal of the History
of the Behavioral Sciences, 40(4),
393-422.
Stevens, S. S. (1946). On the theory
of scales of measurement. Science,
103(2684), 667-680.
Stove, D. C. (1991). The Plato cult and other
philosophical follies. Oxford: Blackwell.
Tanner, P. L., & Danielson, M. L.
(2007). Components necessary for the preparation of
the scientist-practitioner. American
Behavioral Scientist, 50(6), 772-777.
Tashakkori, A., & Teddlie, C. (2003a). The past and future of mixed methods
research: From data triangulation to mixed model designs. In A. Tashakkori & C. Teddlie
(Eds.), Handbook of mixed methods in
social and behavioral research (pp. 671-701). Thousand Oaks, CA: Sage.
Tashakkori, A., & Teddlie, C. (Eds.). (2003b). Handbook of mixed
methods in social and behavioral research. Thousand Oaks, CA: Sage.
Teddlie, C., & Tashakkori, A. (2003).
Major issues and controversies in the use of mixed methods in
the social and behavioural sciences. In A. Tashakkori & C. Teddlie
(Eds.), Handbook of mixed methods in
social and behavioral research (pp. 3-50). Thousand Oaks, CA: Sage.
Toomela, A., & Valsiner, J. (Eds.).
(2010). Methodological thinking in
psychology: 60 years gone astray? Charlotte, NC: Information Age Publishing
Inc.
Triplett, T. (1988). Azande logic
versus Western logic? British
Journal for the Philosophy of Science, 39(3),
361-366.
van Buuren,
H. (2006). Teaching statistics and research methods: An integrated approach. In A. Rossman &
B. Chance (Eds.), Working cooperatively
in statistics education: Proceedings of the Seventh International Conference on
Teaching Statistics, Salvador, Brazil. [CDROM]. Voorburg, The Netherlands:
International Statistical Institute.
[Online: http://www.stat.auckland.ac.nz/~iase/publications/17/3H1_VANB.pdf]
Wiberg, M. (2009). Teaching statistics in
integration with psychology. Journal of Statistics Education, 17(1).
[Online: http://www.amstat.org/publications/jse/v17n1/wiberg.html]
Wittgenstein, L. (1953). Philosophical investigations (G. E. M. Anscombe, Trans.). Oxford: Blackwell.
Agnes Petocz
School of Psychology, University of Western
Sydney - Bankstown
Locked Bag 1797, South Penrith
DC NSW 1797
Australia