Statistical Literacy:

A Complex HIERARCHICAL Construct

Jane Watson

University of Tasmania

Jane.Watson@utas.edu.au

ROSEMARY CALLINGHAM

University of New England

rcalling@pobox.une.edu.au

SUMMARY

The aim of this study was, first, to provide evidence to support the notion of statistical literacy as a hierarchical construct and, second, to identify levels of this hierarchy across the construct. The study used archived data collected from two large-scale research projects that studied aspects of statistical understanding of over 3000 school students in grades 3 to 9, based on 80 questionnaire items. Rasch analysis was used to explore an hypothesised underlying construct associated with statistical literacy. The analysis supported the hypothesis of a unidimensional construct and suggested six levels of understanding: Idiosyncratic, Informal, Inconsistent, Consistent non-critical, Critical, and Critical mathematical. These levels could be used by teachers and curriculum developers to incorporate appropriate aspects of statistical literacy into the existing curriculum.

Keywords: Statistical literacy; school students; Rasch analysis; conceptual hierarchy

__________________________

Statistics Education Research Journal, 2(2), 3-46, http://www.stat.auckland.ac.nz/serj

Ó International Association for Statistical Education (IASE/ISI), November, 2003

References

Adams, R. J., & Khoo, S. T. (1996). Quest: Interactive item analysis system. Version 2.1 [Computer software]. Melbourne: Australian Council for Educational Research.

Anastasi, A. (1988). Psychological testing. Macmillan: New York.

Australian Association of Mathematics Teachers (AAMT) (1997). Numeracy = everyone’s business. Report of the Numeracy Education Strategy Development Conference. May 1997. Adelaide: Author.

Australian Education Council (1991). A national statement on mathematics for Australian schools. Carlton, Vic.: Author.

Australian Education Council (1994). Mathematics - A curriculum profile for Australian schools. Carlton, Vic.: Curriculum Corporation.

Batanero, C., Estepa, A., Godino, J. D., & Green, D. R. (1996). Intuitive strategies and preconceptions about association in contingency tables. Journal for Research in Mathematics Education, 27, 151-169.

Biggs, J. B., & Collis, K. F. (1982). Evaluating the quality of learning: The SOLO taxonomy. Academic Press: New York.

Biggs, J. B., & Collis, K. F. (1991). Multimodal learning and the quality of intelligent behaviour. In H. A. H. Rowe (Ed.), Intelligence: Reconceptualization and measurement (pp. 57-76). Hillsdale, NJ: Lawrence Erlbaum.

Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah, NJ: Lawrence Erlbaum.

Cai, J. (1995). Beyond the computational algorithm: Students’ understanding of the arithmetic average concept. In L. Meira & D. Carraher (Eds.), Proceedings of the 19th Psychology of Mathematics Education Conference (Vol. 3, pp. 144-151). São Paulo, Brazil: PME Program Committee.

Cai, J. (1998). Exploring students’ conceptual understanding of the averaging algorithm. School Science and Mathematics, 98, 93-98.

Campbell, K. J., Watson, J. M., & Collis, K. F. (1992). Volume measurement and intellectual development. Journal of Structural Learning, 11, 279-298.

Castles, I. (1992). Surviving statistics: A user’s guide to the basics. Canberra: Australian Bureau of Statistics.

Cockcroft, W. H. (1982). Mathematics counts: Report of the Committee of Inquiry into the Teaching of Mathematics in Schools. London: HMSO.

Collis, K. F., Romberg, T. A., & Jurdak, M. E. (1986). A technique for assessing mathematical problem solving ability. Journal for Research in Mathematics Education, 17, 206-221.

Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.

Cureton, E. E. (1965). Reliability and validity: Basic assumptions and experimental designs. Educational and Psychological Measurement, 25, 326-346.

Department of Education Tasmania. (2002). Essential learnings framework 1. Hobart: Author.

Dossey, J. A. (1997). National indicators of quantitative literacy. In L. A. Steen (Ed.), Why numbers count: Quantitative literacy for tomorrow’s America (pp. 45-59). New York : The College Board.

Education Queensland (2000). New Basics Project technical paper. Retrieved January 11, 2002, from http://education.qld.gov.au/corporate/newbasics/html/library.html

Eisner, E. W. (1993). Reshaping assessment in education: Some criteria in search of practice. Journal of Curriculum Studies, 25(3), 219-233.

Fischbein, E. (1975). The intuitive sources of probabilistic thinking in children. Dordrecht: D. Reidel.

Fischbein, E., & Gazit, A. (1984). Does the teaching of probability improve probabilistic intuitions? An exploratory research study. Educational Studies in Mathematics, 15, 1-24.

Fisher, W. P. (1994). The Rasch debate: Validity and revolution in educational measurement. In M. Wilson (Ed.), Objective measurement: Vol. 2 (pp. 36-72). Norwood, NJ: Ablex.

Frankenstein, M. (2001). Reading the world with math: Goals for a critical mathematical literacy curriculum. In Mathematics shaping Australia (Proceedings of the 18th Biennial Conference of the Australian Association of Mathematics Teachers, Inc.). [CDROM] Canberra: AAMT.

Friel, S. N., Curcio, F. R., & Bright, G. W. (2001). Making sense of graphs: Critical factors influencing comprehension and instructional implications. Journal for Research in Mathematics Education, 32, 124-158.

Gagatsis, A., Kyriakides, L., & Panaoura, A. (2001). Construct validity of a developmental assessment on probabilities: A Rasch measurement model analysis. In M. van den Heuvel-Panhuizen (Ed.), Proceedings of the 25th Conference of the International Group for the Psychology of Mathematics Education (Vol. 2, pp. 449-456). Utrecht, The Netherlands: Freudenthal Institute.

Gal, I. (2000). Statistical literacy: Conceptual and instructional issues. In D. Coben, J. O’Donoghue, & G. E. Fitzsimons (Eds.), Perspectives on adults learning mathematics: Research and practice (pp. 135-150). Dordrecht: Kluwer.

Gal, I. (2002). Adults’ statistical literacy: Meanings, components, responsibilities. International Statistical Review, 70, 1-51.

Gal, I., & Wagner, D. A. (1992). Project STARC: Statistical reasoning in the classroom. (Annual Report: Year 2, NSF Grant No. MDR90-50006). Philadelphia, PA: Literacy Research Center, University of Pennsylvania.

Garfield, J. B. (2003). Assessing statistical reasoning. Statistics Education Research Journal, 2(1), 23-38.

Glaser, R. (1963). Instructional technology and the measurement of learning outcomes: Some questions. American Psychologist, 18, 519-521.

Glaser, R. (1981). The future of testing: A research agenda for cognitive psychology and psychometrics. American Psychologist, 36, 923-936.

Green, D. R. (1982). Probability concepts in 11-16 year old pupils. Loughborough, UK: Center for Advancement of Mathematical Education in Technology, University of Technology.

Green, D. (1983a). Shaking a six. Mathematics in Schools, 12(5), 29-32.

Green, D. R. (1983b). A survey of probability concepts in 3000 pupils aged 11-16 years. In D. R. Grey, P. Holmes, V. Barnett, & G. M. Constable (Eds.), Proceedings of the 1st International Conference on Teaching Statistics (Vol. 2, pp. 766-783). Sheffield, England: Teaching Statistics Trust.

Green, D. (1993). Data analysis: What research do we need? In L. Pereira-Mendoza (Ed.), Introducing data analysis in the schools: Who should teach it and how? (pp. 219-239). Voorburg, The Netherlands: International Statistical Institute.

Holmes, P. (1980). Teaching statistics 11-16. Slough, UK: Schools Council and Foulsham Educational.

Izard, J. (1992). Patterns of development with probability concepts: Assessment for informative purposes. In M. Stephens & J. Izard (Eds.), Reshaping assessment practices: Assessment in the mathematical sciences under challenge. Proceedings from the First National Conference on Assessment in the Mathematical Sciences (pp. 355-367). Melbourne, VIC: Australian Council for Educational Research.

Jacobs, V. R. (1999). How do students think about statistical sampling before instruction? Mathematics in the Middle School, 5(4), 240-263.

Jones, B. (1982). Sleepers wake: Technology and the future of work. Melbourne: Oxford University Press.

Jones, G. A., Langrall, C. W., Thornton, C. A., & Mogill, A. T. (1997). A framework for assessing young children’s thinking in probability. Educational Studies in Mathematics, 32, 101-125.

Jones, G. A., Thornton, C. A., Langrall, C. W., Mooney, E. S., Perry, B., & Putt, I. J. (2000). A framework for characterizing children’s statistical thinking. Mathematical Thinking and Learning, 2, 269-307.

Keeves, J. P., & Alagumalai, S. (1999). New approaches to measurement. In G. N. Masters and J. P. Keeves (Eds.), Advances in measurement in educational research and assessment (pp. 23-42). Oxford: Pergamon.

Kirsch, I. W. (1997). Literacy performance on three scales: Definitions and results. In W. McLennan, Aspects of literacy: Assessed skill levels Australia 1996 (pp. 98-124). Canberra: Australian Bureau of Statistics.

Kolen, M. J. (1999). Equating of tests. In G. N. Masters & J. P. Keeves (Eds.), Advances in measurement in educational research and assessment (pp. 164-175). New York: Pergamon.

Konold, C., & Garfield, J. (1992). Statistical reasoning assessment: Part 1. Intuitive Thinking. Scientific Reasoning Research Institute, University of Massachusetts, Amherst, MA.

Konold, C., & Higgins, T. L. (2002). Working with data: Highlights related to research. In S. J. Russell, D. Schifter, & V. Bastable (Eds.), Developing mathematical ideas: Collecting, representing, and analyzing data (pp. 165-201). Parsippany, NJ: Dale Seymour Publications.

Linacre, M. (1997, August). Judging plans and facets. MESA Research Note 3. Retrieved January 8, 2003, from http://www.rasch.org/m3.htm

Lokan, J. Ford, P., & Greenwood, L. (1997). Maths and science on the line: Australian middle primary students’ performance in the Third International Mathematics and Science Survey (TIMSS). Melbourne: Australian Council for Educational Research.

Luke, A., & Freebody, P. (1997). Shaping the social practices of reading. In S. Musprati, A. Luke, & P. Freebody (Eds.), Constructing critical literacies: Teaching and learning textual practice (pp. 185-225). St. Leonards, NSW: Allen & Unwin.

Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.

McLennan, W. (1997). Aspects of literacy: Assessed Skill Levels Australia 1996. Canberra: Commonwealth of Australia.

Messick, S. (1989). Validity. In R. Linn (Ed.), Educational measurement (3rd ed.) (pp. 13-103). New York: American Council on Education and Macmillan Publishing Company.

Metz, K. E. (1998). Emergent understanding and attribution of randomness: Comparative analysis of the reasoning of primary grade children and undergraduates. Cognition and Instruction, 16, 285-365.

Mevarech, Z. R., & Kramarsky, B. (1997). From verbal descriptions to graphic representations: Stability and change in students’ alternative conceptions. Educational Studies in Mathematics, 32, 229-263.

Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. (2nd ed.). Thousand Oaks, CA: Sage Publications.

Ministry of Education (1992). Mathematics in the New Zealand curriculum. Wellington: Author.

Mokros, J., & Russell, S. J. (1995). Children’s concepts of average and representativeness. Journal for Research in Mathematics Education, 26(1), 20-39.

Moore, D. S. (1990). Uncertainty. In L. A. Steen (Ed.), On the shoulders of giants: New approaches to numeracy (pp. 95-137). Washington, DC: National Academy Press.

Moritz, J. B. (2000). Graphical representations of statistical associations by upper primary students. In J. Bana & A. Chapman (Eds.), Mathematics education beyond 2000: Proceedings of the 23rd Annual Conference of the Mathematics Education Research Group of Australasia (Vol. 2, pp. 440-447). Perth, WA: MERGA.

Moritz, J. B., & Watson, J. M. (1997). Graphs: Communication lines to students? In F. Biddulph & K. Carr (Eds.), People in mathematics education: Proceedings of the 20th annual conference of the Mathematics Education Research Group of Australasia (Vol. 2, pp. 344-351). Rotorua, NZ: MERGA.

Moritz, J. B., & Watson, J. M. (2000). Reasoning and expressing probability in students’ judgements of coin tossing. In J. Bana & A. Chapman (Eds.), Mathematics education beyond 2000: Proceedings of the 23rd annual conference of the Mathematics Education Research Group of Australasia (Vol. 2, pp. 448-455). Perth, WA: MERGA.

Moritz, J. B., Watson, J. M., & Collis, K. F. (1996). Odds: Chance measurement in three contexts. In P. C. Clarkson (Ed.), Technology in mathematics education: Proceedings of the 19th annual conference of the Mathematics Education Research Group of Australasia (pp. 390-397). Melbourne: MERGA.

Moritz, J. B., Watson, J.M., & Pereira-Mendoza, L. (1996, November). The language of statistical understanding: An investigation in two countries. Paper presented at the Joint ERA/AARE Conference, Singapore. [Online: http://www.swin.edu.au/aare/96pap/morij96.280]

National Council of Teachers of Mathematics (1989). Curriculum and evaluation standards for school mathematics. Reston, VA: Author.

National Council of Teachers of Mathematics (2000). Principles and standards for school mathematics. Reston, VA: Author.

Nisbett, R. E., Krantz, D. H., Jepson, C., & Kunda, Z. (1983). The use of statistical heuristics in everyday inductive reasoning. Psychological Review, 90, 339-363.

Pollatsek, A., Well, A. D., Konold, C., Hardiman, P., & Cobb, G. (1987). Understanding conditional probabilities. Organizational Behavior and Human Decision Processes, 40, 255-269.

Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press (original work published 1960).

Reading, C. (2002). Profile for statistical understanding. In B. Phillips (Ed.), Proceedings of the Sixth International Conference on Teaching Statistics: Developing a statistically literate society, Cape Town, South Africa. Voorburg, The Netherlands: International Statistical Institute.

Reading, C., & Shaughnessy, M. (2000). Student perceptions of variation in a sampling situation. In T. Nakahara & M. Koyama (Eds.), Proceedings of the 24th Conference of the International Group for the Psychology of Mathematics Education (Vol. 4, pp. 89-96). Hiroshima, Japan: Hiroshima University.

Reading, C., & Shaughnessy, M. (in press). Reasoning about variation. In J. Garfield & D. Ben-Zvi (Eds.), The challenge of developing statistical literacy, reasoning, and thinking. Dordrecht: Kluwer.

Romberg, T. A., Jurdak, M. E., Collis, K. F., & Buchanan, A. E. (1982). Construct validity of a set of mathematical superitems. Madison, Wisconsin: Wisconsin Center for Education Research.

Shaughnessy, J. M. (1997). Missed opportunities in research on the teaching and learning of data and chance. In F. Biddulph & K. Carr (Eds.), People in mathematics education Proceedings of the 20th annual conference of the Mathematics Education Research Group of Australasia (Vol. 1, pp. 6-22). Rotorua, NZ: MERGA.

Shaughnessy, J. M., Watson, J., Moritz, J., & Reading, C. (1999, April). School mathematics students’ acknowledgment of statistical variation. In C. Maher (Chair), There’s more to life than centers. Presession Research Symposium, 77th Annual National Council of Teachers of Mathematics Conference, San Francisco, CA.

Statistics Canada and Organisation for Economic Cooperation and Development (OECD) (1996). Literacy, economy, and society: First results from the International Adult Literacy Survey. Ottawa: Author.

Steen, L. A. (Ed.) (1997). Why numbers count: Quantitative literacy for tomorrow’s America. New York: College Entrance Examination Board.

Steen, L. A. (Ed.) (2001). Mathematics and democracy: The case for quantitative literacy. Washington, DC: National Council on Education and the Disciplines.

Stocking, M. L. (1999). Item response theory. In G. N. Masters & J. P. Keeves (Eds.), Advances in measurement in educational research and assessment (pp. 55-63). New York: Pergamon.

Tognolini, J. (1996). Rasch modelling: Advantages and limitations. In Session notes National Meeting on Assessment and Reporting 25-26 November 1996. Manly: NSW Department of School Education.

Torok, R. (2000). Putting the variation into chance and data. Australian Mathematics Teacher, 56(2), 25-31.

Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgement. Psychological Review, 90, 293-315.

Wallman, K. K. (1993). Enhancing statistical literacy: Enriching our society. Journal of the American Statistical Association, 88, No. 421, 1-8.

Watson, J. M. (1994). Instruments to assess statistical concepts in the school curriculum. In National Organizing Committee (Ed.), Proceedings of the Fourth International Conference on Teaching Statistics. Volume 1 (Vol. 1, pp. 73-80). Rabat, Morocco: National Institute of Statistics and Applied Economics

Watson, J. M. (1997). Assessing statistical literacy using the media. In I. Gal & J.B. Garfield (Eds.), The Assessment Challenge in Statistics Education (pp. 107-121). Amsterdam: IOS Press and The International Statistical Institute.

Watson, J. M. (1998a). Numeracy benchmarks for years 3 and 5: What about chance and data? In C. Kanes, M. Goos, & E. Warren (Eds.), Teaching mathematics in new times. (Proceedings of the 21^st annual conference of the Mathematics Education Research Group of Australasia, Vol. 2, pp. 669-676). Brisbane: MERGA.

Watson, J. M. (1998b). The role of statistical literacy in decisions about risk: Where to start. For the Learning of Mathematics, 18(3), 25-27.

Watson, J. M. (2000). Statistics in context. Mathematics Teacher, 93, 54-58.

Watson, J. M., Collis, K. F., & Moritz, J. B. (1994). Assessing statistical understanding in Grades 3, 6 and 9 using a short answer questionnaire. In G. Bell, B. Wright, N. Leeson, & G. Geake (Eds.), Challenges in Mathematics Education: Constraints on Construction Proceedings of the 17th Annual Conference of the Mathematics Education Research Group of Australasia (pp. 675-682). Lismore, NSW: MERGA.

Watson, J. M., Collis, K. F., & Moritz, J. B. (1997). The development of chance measurement. Mathematics Education Research Journal, 9, 60-82.

Watson, J. M., & Kelly, B. A. (2002). Can grade 3 students learn about variation? In B. Phillips (Ed.), Proceedings of the Sixth International Conference on Teaching Statistics: Developing a statistically literate society, Cape Town, South Africa. Voorburg, The Netherlands: International Statistical Institute.

Watson, J. M., Kelly, B. A., Callingham, R. A., & Shaughnessy, J. M. (2003). The measurement of school students’ understanding of statistical variation. International Journal of Mathematical Education in Science and Technology, 34,1-29.

Watson, J. M., & Moritz, J. B. (1998). Longitudinal development of chance measurement. Mathematics Education Research Journal, 10(2), 103-127.

Watson, J. M., & Moritz, J. B. (1999a). The development of concepts of average. Focus on Learning Problems in Mathematics, 21(4), 15-39.

Watson, J. M., & Moritz, J. B. (1999b). The beginning of statistical inference: Comparing two data sets. Educational Studies in Mathematics, 37, 145-168.

Watson, J. M., & Moritz, J. B. (2000a). Developing concepts of sampling. Journal for Research in Mathematics Education, 31, 44-70.

Watson, J. M., & Moritz, J. B. (2000b). Development of understanding of sampling for statistical literacy. Journal of Mathematical Behavior, 19, 109-136.

Watson, J. M., & Moritz, J. B. (2002). School students’ reasoning about conjunction and conditional events. International Journal of Mathematical Education in Science and Technology, 33, 59-84.

Watson, J. M., & Moritz, J. B. (2003). The development of comprehension of chance language: Evaluation and interpretation. School Science and Mathematics, 103, 65-80.

Wild, C. J., & Pfannkuch, M. (1999). Statistical thinking in empirical enquiry. International Statistical Review, 67, 223-265.

Wilson, M. (1990). Investigation of structured problem-solving items. In G. Kulm (Ed.), Assessing higher order thinking in mathematics (pp. 187-203). Washington, DC: American Association for the Advancement of Science.

Wilson, M. (1992). Measuring levels of mathematical understanding. In T. A. Romberg (Ed.), Mathematics assessment and evaluation: Imperatives for mathematics educators (pp. 213-241). Albany, NY: State University of New York Press.

Wright, B. D., & Masters, G. N. (1982). Rating scale analysis: Rasch measurement. Chicago: MESA Press.

Zawojewski, J. S., & Shaughnessy, J. M. (2000). Data and chance. In E. A. Silver & P. A. Kenney (Eds.), Results from the Seventh Mathematics Assessment of the National Assessment of Educational Progress (235-268). Reston, VA: NCTM.

Jane Watson

Faculty of Education

University of Tasmania

Private Bag 66, Hobart

TAS 7001 Australia