The cognitive reflection test and students’ achievements in mathematics and physics

Daniel Doz 1 * , Josip Sliško 2
More Detail
1 Faculty of Education, University of Primorska, Koper, SLOVENIA
2 Faculty of Mathematical and Physical Sciences, Benemérita Universidad Autónoma de Puebla, Puebla, MEXICO
* Corresponding Author
EUR J SCI MATH ED, Volume 12, Issue 1, pp. 85-96. https://doi.org/10.30935/scimath/13832
Published Online: 25 October 2023, Published: 01 January 2024
OPEN ACCESS   1592 Views   417 Downloads
Download Full Text (PDF)

ABSTRACT

The cognitive reflection test (CRT) assesses an individual’s capacity to restrain impulsive and intuitive responses and to engage in critical reflection on mathematical problems. The literature indicates that several factors influence students’ performance on CRT, including gender, age, and prior knowledge of mathematics. In this study, our objective was to investigate the correlation between CRT scores and students’ achievements in both mathematics and physics. We conducted our research with a sample of 150 Italian high school students, and the findings revealed a positive predictive relationship between CRT scores and students’ performance in both mathematics and physics. Furthermore, we employed an ordinal logistic regression to evaluate the impact of CRT scores, gender, and school level on students’ achievements in mathematics and physics. The results showed that both CRT scores and school level had statistically significant effects on predicting these achievements. In contrast, gender emerged as a statistically significant factor only in predicting students’ mathematics achievements.
Keywords: achievements, cognitive reflection test, mathematics, ordinal logistic regression, physics

CITATION

Doz, D., & Sliško, J. (2024). The cognitive reflection test and students’ achievements in mathematics and physics. European Journal of Science and Mathematics Education, 12(1), 85-96. https://doi.org/10.30935/scimath/13832

REFERENCES

  • Akpotor, J., & Egbule, E. (2020). Gender difference in the scholastic achievement test (SAT) among school adolescents. World Journal of Education, 10(1), 97-101. https://doi.org/10.5430/wje.v10n1p97
  • Bartlett, J. E., & Charles, S. (2021). Power to the people: A beginner’s tutorial to power analysis using Jamovi. PsyArXiv. https://doi.org/10.31234/osf.io/bh8m9
  • Blacksmith, N., Yang, Y., Behrend, T. S., & Ruark, G. A. (2019). Assessing the validity of inferences from scores on the cognitive reflection test. Journal of Behavioral Decision Making, 32(5), 599-612. https://doi.org/10.1002/bdm.2133
  • Brañas-Garza, P., Kujal, P., & Lenkei, B. (2019). Cognitive reflection test: Whom, how, when. Journal of Behavioral and Experimental Economics, 82, 101455. https://doi.org/10.1016/j.socec.2019.101455
  • Bull, R., & Lee, K. (2014). Executive functioning and mathematics achievement. Child Development Perspectives, 8(1), 36-41. https://doi.org/10.1111/cdep.12059
  • Campitelli, G., & Gerrans, P. (2014). Does the cognitive reflection test measure cognitive reflection? A mathematical modeling approach. Memory & Cognition, 42(3), 434-447. https://doi.org/10.3758/s13421-013-0367-9
  • De Smedt, B., Janssen, R., Bouwens, K., Verschaffel, L., Boets, B., & Ghesquière, P. (2009). Working memory and individual differences in mathematics achievement: A longitudinal study from first grade to second grade. Journal of Experimental Child Psychology, 103(2), 186-201. https://doi.org/10.1016/j.jecp.2009.01.004
  • Epstein, S. (1994). Integration of the cognitive and the psychodynamic unconscious. American Psychologist, 49(8), 709-724. https://doi.org/10.1037/0003-066X.49.8.709
  • Etcheverry, P. T., Ignjatov, J. S., & de Lourdes Juárez, E. (2020). Influencia de la escolaridad en el desarrollo del razonamiento lógico y la reflexión cognitiva en estudiantes de bachillerato [Influence of schooling on the development of logical reasoning and cognitive reflection in high school students]. UNIÓN-Revista Iberoamericana de Educación Matemática, 16(60), 212-232.
  • Fang, S. C., Hsu, Y. S., & Lin, S. S. (2019). Conceptualizing socio-scientific decision making from a review of research in science education. International Journal of Science and Mathematics Education, 17(3), 427-448. https://doi.org/10.1007/s10763-018-9890-2
  • Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149-1160. https://doi.org/10.3758/BRM.41.4.1149
  • Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19(4), 25-42. https://doi.org/10.1257/089533005775196732
  • Frosch, C., & Simms, V. (2015). Understanding the role of reasoning ability in mathematical achievement. In Euroasianpacific joint conference on cognitive science. In Proceedings of the EuroAsianPacific Joint Conference on Cognitive Science (pp. 633-638). https://doi.org/10.13140/RG.2.1.1107.2727
  • Gette, C. R., & Kryjevskaia, M. (2019). Establishing a relationship between student cognitive reflection skills and performance on physics questions that elicit strong intuitive responses. Physical Review Physics Education Research, 15(1), 010118. https://doi.org/10.1103/PhysRevPhysEducRes.15.010118
  • Gilmore, C., Keeble, S., Richardson, S., & Cragg, L. (2015). The role of cognitive inhibition in different components of arithmetic. ZDM, 47(5), 771-782. https://doi.org/10.1007/s11858-014-0659-y
  • Gómez-Chacón, I. M., García-Madruga, J. A., Vila, J. Ó., Elosúa, M. R., & Rodríguez, R. (2014). The dual processes hypothesis in mathematics performance: Beliefs, cognitive reflection, working memory and reasoning. Learning and Individual Differences, 29, 67-73. https://doi.org/10.1016/j.lindif.2013.10.001
  • Gómez-Veiga, I., Vila Chaves, J. O., Duque, G., & García Madruga, J. A. (2018). A new look to a classic issue: Reasoning and academic achievement at secondary school. Frontiers in Psychology, 9, 400. https://doi.org/10.3389/fpsyg.2018.00400
  • Hyde, J. S., Fennema, E., & Lamon, S. J. (1990). Gender differences in mathematics performance: A meta-analysis. Psychological Bulletin, 107(2), 139-155. https://doi.org/10.1037/0033-2909.107.2.139
  • Kryjevskaia, M., Stetzer, M. R., Lindsey, B. A., McInerny, A., Heron, P. R., & Boudreaux, A. (2020). Designing research-based instructional materials that leverage dual-process theories of reasoning: Insights from testing one specific, theory-driven intervention. Physical Review Physics Education Research, 16(2), 020140. https://doi.org/10.1103/PhysRevPhysEducRes.16.020140
  • Lem, S., Kempen, G., Ceulemans, E., Onghena, P., Verschaffel, L., & Van Dooren, W. (2015). Combining multiple external representations and refutational text: An intervention on learning to interpret box plots. International Journal of Science and Mathematics Education, 13(4), 909-926. https://doi.org/10.1007/s10763-014-9604-3
  • Liu, X., & Koirala, H. (2012). Ordinal regression analysis: Using generalized ordinal logistic regression models to estimate educational data. Journal of Modern Applied Statistical Methods, 11(1), 242-254. https://doi.org/10.22237/jmasm/1335846000
  • Louis, R. A., & Mistele, J. M. (2012). The differences in scores and self-efficacy by student gender in mathematics and science. International Journal of Science and Mathematics Education, 10(5), 1163-1190. https://doi.org/10.1007/s10763-011-9325-9
  • Ma, X., & Xu, J. (2004). The causal ordering of mathematics anxiety and mathematics achievement: A longitudinal panel analysis. Journal of Adolescence, 27(2), 165-179. https://doi.org/10.1016/j.adolescence.2003.11.003
  • McFadden, D. (1977). Quantitative methods for analyzing travel behavior of individuals: Some recent developments. https://elischolar.library.yale.edu/cgi/viewcontent.cgi?article=1706&context=cowles-discussion-paper-series
  • Primi, C., Donati, M. A., Chiesi, F., & Morsanyi, K. (2018). Are there gender differences in cognitive reflection? Invariance and differences related to mathematics. Thinking & Reasoning, 24(2), 258-279. https://doi.org/10.1080/13546783.2017.1387606
  • Riegle-Crumb, C., & Moore, C. (2014). The gender gap in high school physics: Considering the context of local communities. Social Science Quarterly, 95(1), 253-268. https://doi.org/10.1111/ssqu.12022
  • Ring, P., Neyse, L., David-Barett, T., & Schmidt, U. (2016). Gender differences in performance predictions: Evidence from the cognitive reflection test. Frontiers in Psychology, 7, 1680. https://doi.org/10.3389/fpsyg.2016.01680
  • Shtulman, A., & McCallum, K. (2014). Cognitive reflection predicts science understanding. In Proceedings of the Annual Meeting of the Cognitive Science Society.
  • Skaalvik, E. M., Federici, R. A., & Klassen, R. M. (2015). Mathematics achievement and self-efficacy: Relations with motivation for mathematics. International Journal of Educational Research, 72, 129-136. https://doi.org/10.1016/j.ijer.2015.06.008
  • Sliško, J. (2017). Self-regulated learning in a general university course: Design of learning tasks, their implementation and measured cognitive effects. Journal of European Education, 7(2), 12-24.
  • Speirs, J. C., Stetzer, M. R., Lindsey, B. A., & Kryjevskaia, M. (2021). Exploring and supporting student reasoning in physics by leveraging dual-process theories of reasoning and decision making. Physical Review Physics Education Research, 17(2), 020137. https://doi.org/10.1103/PhysRevPhysEducRes.17.020137
  • Stanovich, K. E., & West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate? Behavioral and Brain Sciences, 23(5), 645-665. https://doi.org/10.1017/S0140525X00003435
  • Stieger, S., & Reips, U. D. (2016). A limitation of the cognitive reflection test: Familiarity. PeerJ, 4, e2395. https://doi.org/10.7717/peerj.2395
  • Szaszi, B., Szollosi, A., Palfi, B., & Aczel, B. (2017). The cognitive reflection test revisited: Exploring the ways individuals solve the test. Thinking & Reasoning, 23(3), 207-234. https://doi.org/10.1080/13546783.2017.1292954
  • Toplak, M. E., West, R. F., & Stanovich, K. E. (2011). The cognitive reflection test as a predictor of performance on heuristics-and-biases tasks. Memory & Cognition, 39(7), 1275-1289. https://doi.org/10.3758/s13421-011-0104-1
  • Toplak, M. E., West, R. F., & Stanovich, K. E. (2014). Assessing miserly information processing: An expansion of the cognitive reflection test. Thinking & Reasoning, 20(2), 147-168. https://doi.org/10.1080/13546783.2013.844729
  • Travers, E., Rolison, J. J., & Feeney, A. (2016). The time course of conflict on the cognitive reflection test. Cognition, 150, 109-118. https://doi.org/10.1016/j.cognition.2016.01.015
  • Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases: Biases in judgments reveal some heuristics of thinking under uncertainty. Science, 185(4157), 1124-1131. https://doi.org/10.1126/science.185.4157.1124
  • Warner, P. (2008). Ordinal logistic regression. Journal of Family Planning and Reproductive Health Care, 34(3), 169-170. https://doi.org/10.1783/147118908784734945
  • Williamson, K. E., & Willoughby, S. D. (2012). Student understanding of gravity in introductory astronomy. Astronomy Education Review, 11(1), 10105. https://doi.org/10.3847/AER2011025
  • Wood, A. K., Galloway, R. K., & Hardy, J. (2016). Can dual processing theory explain physics students’ performance on the force concept inventory? Physical Review Physics Education Research, 12(2), 023101. https://doi.org/10.1103/PhysRevPhysEducRes.12.023101
  • Zhang, D. C., Highhouse, S., & Rada, T. B. (2016). Explaining sex differences on the cognitive reflection test. Personality and Individual Differences, 101, 425-427. https://doi.org/10.1016/j.paid.2016.06.034
  • Zhou, C., Kuttal, S. K., & Ahmed, I. (2018). What makes a good developer? An empirical study of developers’ technical and social competencies. In Proceedings of the 2018 IEEE Symposium on Visual Languages and Human-Centric Computing (pp. 319-321). IEEE. https://doi.org/10.1109/VLHCC.2018.8506577