ASSESSING STUDENTS’ CONCEPTUAL UNDERSTANDING AFTER A FIRST COURSE IN STATISTICS

ASSESSING STUDENTS’ CONCEPTUAL UNDERSTANDING

AFTER A FIRST COURSE IN STATISTICS

ROBERT DELMAS

University of Minnesota

delma001@umn.edu

JOAN GARFIELD

University of Minnesota

jbg@umn.edu

ANN OOMS

Kingston University

a.ooms@kingston.ac.uk

BETH CHANCE

California Polytechnic State University

bchance@calpoly.edu

ABSTRACT

This paper describes the development of the CAOS test, designed to measure students’ conceptual understanding of important statistical ideas, across three years of revision and testing, content validation, and realiability analysis. Results are reported from a large scale class testing and item responses are compared from pretest to posttest in order to learn more about areas in which students demonstrated improved performance from beginning to end of the course, as well as areas that showed no improvement or decreased performance. Items that showed an increase in students’ misconceptions about particular statistical concepts were also examined. The paper concludes with a discussion of implications for students’ understanding of different statistical topics, followed by suggestions for further research.

Keywords: Statistics education research; Assessment; Conceptual understanding; Online test

__________________________

Statistics Education Research Journal , 6(2), 28-58, http://www.stat.auckland.ac.nz/serj

Ó International Association for Statistical Education (IASE/ISI), November, 2007

REFERENCES

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (1999). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association.

Bakker, A., & Gravemeijer, K. (2004). Learning to reason about distribution. In D. Ben-Zvi and J. Garfield (Eds.), The Challenge of Developing Statistical Literacy, Reasoning, and Thinking (pp. 147-168). Dordrecht, The Netherlands: Kluwer.

Ben-Zvi, D. (2004). Reasoning about data analysis. In D. Ben-Zvi & J. B. Garfield (Eds.), The Challenge of Developing Statistical Literacy, Reasoning, and Thinking (pp. 121-146). Dordrecht, Netherlands: Kluwer.

Biehler, R. (1997). Students’ difficulties in practicing computer-supported data analysis: Some hypothetical generalizations from results of two exploratory studies. In J. B. Garfield & G. Burrill (Eds.), Research on the Role of Technology in Teaching and Learning Statistics: Proceedings of the 1996 IASE Round Table Conference (pp. 169-190). Voorburg, The Netherlands: International Statistical Institute.

Chance, B., delMas, R., & Garfield, J. (2004). Reasoning about sampling distributions. In D. Ben-Zvi and J. Garfield (Eds.), The Challenge of Developing Statistical Literacy, Reasoning, and Thinking (pp. 295-323). Dordrecht, The Netherlands: Kluwer.

Cobb, G. (1992). Teaching statistics. In Heeding the Call for Change: Suggestions for Currricular Action, MAA Notes, Vol. 22, 3-33.

delMas, R. & Bart, W. M. (1989). The role of an evaluation exercise in the resolution of misconceptions of probability. Focus on Learning Problems in Mathematics, 11(3), 39-54.

delMas, R., Garfield, J., & Chance, B. (1999). A model of classroom research in action: Developing simulation activities to improve students’ statistical reasoning. Journal of Statistics Education, 7(3).

[Online: www.amstat.org/publications/jse/secure/v7n3/delmas.cfm]

delMas, R., & Liu, Y. (2005). Exploring students’ conceptions of the standard deviation. Statistics Education Research Journal, 4(1), 55-82.

[Online: www.stat.auckland.ac.nz/~iase/serj/SERJ4(1)_delMas_Liu.pdf]

Garfield, J. (2001). Evaluating the impact of educational reform in statistics: A survey of introductory statistics courses. Final Report for NSF Grant REC-9732404.

Garfield, J. (2003). Assessing statistical reasoning. Statistics Education Research Journal, 2(1), 22-38.

[Online: www.stat.auckland.ac.nz/~iase/serj/SERJ2 (1).pdf]

Garfield, J., & Chance, B. (2000). Assessment in statistics education: Issues and challenges. Mathematics Thinking and Learning, 2, 99-125.

Garfield, J., delMas, R., & Chance, B. (2002). The Assessment Resource Tools for Improving Statistical Thinking (ARTIST) Project. NSF CCLI grant ASA- 0206571.

[Online: https://app.gen.umn.edu/artist/]

Garfield , J. & Gal, I. (1999). Assessment and statistics education: Current challenges and directions. International Statistical Review, 67, 1-12.

Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309-334.

Hammerman, J. K., & Rubin, A. (2004). Strategies for managing statistical complexity with new software tools. Statistics Education Research Journal, 3(2), 17-41.

Hodgson, T. R. (1996). The effects of hands-on activities on students’ understanding of selected statistical concepts. In E. Jakubowski, D. Watkins, & H. Biske (Eds.), Proceedings of the Eighteenth Annual Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (pp. 241–246). Columbus, OH: ERIC Clearinghouse for Science, Mathematics, and Environmental Education.

Hogg, R. (1992). Report of workshop on statistics education. In Heeding the Call for Change: Suggestions for Curricular Action, MAA Notes, Vol. 22, 34-43.

Howell, D. C. (2002). Statistical Methods for Psychology (Fifth Edition). Pacific Grove, CA: Duxbury.

Konold, C. (1989). Informal conceptions of probability. Cognition and Instruction, 6, 59-98.

Konold, C. (1995). Issues in assessing conceptual understanding in probability and statistics. Journal of Statistics Education, 3(1).

[Online: http://www.amstat.org/publications/jse/v3n1/konold.html]

Konold, C., & Higgins, T. (2003). Reasoning about data. In J. Kilpatrick, W. G. Martin & D. Schifter (Eds.), A Research Companion to Principles and Standards for School Mathematics (pp. 193-215). Reston, VA: National Council of Teachers of Mathematics.

Konold, C., Pollatsek, A., Well, A. D., Lohmeier, J., & Lipson, A. (1993). Inconsistencies in students’ reasoning about probability. Journal for Research in Mathematics Education, 34, 392-414.

Lindman, H., & Edwards, W. (1961). Supplementary report: Unlearning the gambler’s fallacy. Journal of Experimental Psychology, 62, 630.

Mathews, D., & Clark, J. (1997, March). Successful students’ conceptions of mean, standard deviation, and the Central Limit Theorem. Paper presented at the Midwest Conference on Teaching Statistics, Oshkosh, WI.

McClain, K., Cobb, P., & Gravemeijer, K. (2000). Supporting students’ ways of reasoning about data. In M. Burke & F. Curcio (Eds.), Learning Mathematics for a New Century, 2000 Yearbook. Reston, VA: National Council of Teachers of Mathematics.

Meletiou-Mavrotheris, M., & Lee, C. (2002). Teaching students the stochastic nature of statistical concepts in an introductory statistics course. Statistics Education Research Journal, 1(2), 22-37.

[Online: http://fehps.une.edu.au/serj]

Moore, D. (1997). New pedagogy and new content: The case of statistics. International Statistical Review, 65, 123-137.

Pedhazur, E. J., and Schmelkin, L. P. (1991). Measurement, Design, and Analysis: An Integrated Approach. Hillsdale, NJ: Erlbaum.

Pollatsek, A., Konold, C., Well, A., and Lima, S. (1984). Beliefs underlying random sampling. Memory and Cognition, 12(4), 395-401.

Reading, C., & Shaughnessy, J. M. (2004). Reasoning about variation. In D. Ben-Zvi and J. Garfield (Eds.), The Challenge of Developing Statistical Literacy, Reasoning, and Thinking (pp. 201-226). Dordrecht, The Netherlands: Kluwer.

Reed-Rhoads, T., Murphy, T. J., & Terry, R. (2006). The Statistics Concept Inventory (SCI).

[Online: http://coecs.ou.edu/sci/]

Rubin, A., Bruce, B., & Tenney, Y. (1990, August). Learning about sampling: Trouble at the core of statistics. Paper presented at the Third International Conference on Teaching Statistics, Dunedin, New Zealand.

Scheaffer, R. (1997). Discussion. International Statistical Review, 65, 156-158.

Sedlmeier, P. (1999). Improving statistical reasoning: Theoretical models and practical implications. Mahwah, NJ: Erlbaum.

Saldanha, L. A., & Thompson, P. W. (2001). Students’ reasoning about sampling distributions and statistical inference. In R. Speiser & C. Maher (Eds.), Proceedings of The Twenty-Third Annual Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (pp. 449-454), Snowbird, Utah. Columbus, Ohio: ERIC Clearinghouse.

Shaughnessy, M. (1977). Misconceptions of probability: An experiment with a small-group, activity-based, model building approach to introductory probability at the college level. Educational Studies in Mathematics, 8, 295-316.

Shaughnessy, M. (1992). Research in probability and statistics: Reflections and directions. In A. Grouws (Ed.), Handbook of Research on Mathematics Teaching and Learning (pp. 465-494). New York: MacMillan Publishing Company

Shaughnessy, J. M., Watson, J., Moritz, J., & Reading, C. (1999, April). School mathematics students’ acknowledgment of statistical variation. For the NCTM Research Presession Symposium: There’s More to Life than Centers. Paper presented at the 77^th Annual National Council of Teachers of Mathematics (NCTM) Conference, San Francisco, CA.

Robert Delmas

157 Education Sciences Building

56 East River Road

University of Minnesota

Minneapolis, MN 55455-0364

USA