[en] When developing models in cognitive science, researchers typically start with their own intuitions about human behavior in a given task and then build in mechanisms that explain additional aspects of the data. This refinement step is often hindered by how difficult it is to distinguish the unpredictable randomness of people's decisions from meaningful deviations between those decisions and the model. One solution for this problem is to compare the model against deep neural networks trained on behavioral data, which can detect almost any pattern given sufficient data. Here, we apply this method to the domain of planning with a heuristic search model for human play in 4-in-a-row, a combinatorial game where participants think multiple steps into the future. Using a data set consisting of 10,874,547 games, we train deep neural networks to predict human moves and find that they accurately do so while capturing meaningful patterns in the data. Thus, deviations between the model and the best network allow us to identify opportunities for model improvement despite starting with a model that has undergone substantial testing in previous work. Based on this analysis, we add three extensions to the model that range from a simple opening bias to specific adjustments regarding endgame planning. Overall, our work demonstrates the advantages of model comparison with a high-performance deep neural network as well as the feasibility of scaling cognitive models to massive data sets for systematically investigating the processes underlying human sequential decision-making.
Disciplines :
Theoretical & cognitive psychology
Author, co-author :
Kuperwajs, Ionatan; Center for Neural Science, New York University, New York, NY, USA. ikuperwajs@nyu.edu
SCHÜTT, Heiko ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences (FHSE) > Department of Behavioural and Cognitive Sciences (DBCS) > Cognitive Science and Assessment
Ma, Wei Ji; Center for Neural Science, New York University, New York, NY, USA ; Department of Psychology, New York University, New York, NY, USA
External co-authors :
yes
Language :
English
Title :
Using deep neural networks as a guide for modeling human planning.
Busemeyer, J. R. &; Diederich, A. Cognitive Modeling (Sage, 2010).
Daw, N. D. et al. Trial-by-trial data analysis using computational models. In Decision Making, Affect, and Learning: Attention and Performance XXIII, vol. 23 (2011).
Siegelmann, H. T. & Sontag, E. D. On the computational power of neural nets. J. Comput. Syst. Sci. 50, 132–150 (1995). DOI: 10.1006/jcss.1995.1013
LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015). DOI: 10.1038/nature14539
Agrawal, M., Peterson, J. C. & Griffiths, T. L. Scaling up psychology via scientific regret minimization. Proc. Natl. Acad. Sci. 117, 8825–8835 (2020). DOI: 10.1073/pnas.1915841117
Peterson, J. C., Bourgin, D. D., Agrawal, M., Reichman, D. & Griffiths, T. L. Using large-scale experiments and machine learning to discover theories of human decision-making. Science 372, 1209–1214 (2021). DOI: 10.1126/science.abe2629
Battleday, R. M., Peterson, J. C. & Griffiths, T. L. Capturing human categorization of natural images by combining deep networks and cognitive models. Nat. Commun. 11, 1–14 (2020). DOI: 10.1038/s41467-020-18946-z
Dezfouli, A., Griffiths, K., Ramos, F., Dayan, P. & Balleine, B. W. Models that learn how humans learn: The case of decision-making and its disorders. PLoS Comput. Biol. 15, e1006903 (2019). DOI: 10.1371/journal.pcbi.1006903
Eckstein, M. K., Summerfield, C., Daw, N. D. & Miller, K. J. Predictive and interpretable: Combining artificial neural networks and classic cognitive models to understand human learning and decision making. bioRxiv 6, 66 (2023).
Miller, K. J., Eckstein, M., Botvinick, M. M. & Kurth-Nelson, Z. Cognitive model discovery via disentangled rnns. bioRxiv 6, 66 (2023).
Ji-An, L., Benna, M. K. & Mattar, M. G. Automatic discovery of cognitive strategies with tiny recurrent neural networks. bioRxiv 6, 66 (2023).
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P. & Dolan, R. J. Model-based influences on humans’ choices and striatal prediction errors. Neuron 69, 1204–1215 (2011). DOI: 10.1016/j.neuron.2011.02.027
Huys, Q. J. et al. Bonsai trees in your head: how the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Comput. Biol. 8, e1002410 (2012). DOI: 10.1371/journal.pcbi.1002410
Snider, J., Lee, D., Poizner, H. & Gepshtein, S. Prospective optimization with limited resources. PLoS Comput. Biol. 11, e1004501 (2015). DOI: 10.1371/journal.pcbi.1004501
Sezener, C. A., Dezfouli, A. & Keramati, M. Optimizing the depth and direction of prospective planning using information values. PLoS Comput. Biol. 6, 66 (2019).
Kuperwajs, I. & Ma, W. J. Planning to plan: A Bayesian model for optimizing the depth of decision tree search. In Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 43 (2021).
Callaway, F. et al. Rational use of cognitive resources in human planning. Nat. Hum. Behav. 6, 1112–1125 (2022). DOI: 10.1038/s41562-022-01332-8
Ho, M. K. et al. People construct simplified mental representations to plan. Nature 606, 129–136 (2022). DOI: 10.1038/s41586-022-04743-9
van Opheusden, B. & Ma, W. J. Tasks for aligning human and machine planning. Curr. Opin. Behav. Sci. 29, 127–133 (2019). DOI: 10.1016/j.cobeha.2019.07.002
van Opheusden, B. et al. Expertise increases planning depth in human gameplay. Nature 66, 1–6 (2023).
Kuperwajs, I., van Opheusden, B. & Ma, W. J. Prospective planning and retrospective learning in a large-scale combinatorial game. In 2019 Conference on Cognitive Computational Neuroscience 13–16 (2019).
Zheng, Z. S., Lin, X. D., Topping, J. & Ma, W. J. Comparing machine and human learning in a planning task of intermediate complexity. In Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 44 (2022).
Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 2018).
Bonet, B. & Geffner, H. Planning as heuristic search. Artif. Intell. 129, 5–33 (2001). DOI: 10.1016/S0004-3702(01)00108-4
Campbell, M., Hoane, A. J. Jr. & Hsu, F.-H. Deep blue. Artif. Intell. 134, 57–83 (2002). DOI: 10.1016/S0004-3702(01)00129-1
Dechter, R. & Pearl, J. Generalized best-first search strategies and the optimality of a. J. ACM 32, 505–536 (1985). DOI: 10.1145/3828.3830
Treisman, A. M. & Gelade, G. A feature-integration theory of attention. Cognit. Psychol. 12, 97–136 (1980). DOI: 10.1016/0010-0285(80)90005-5
van Opheusden, B., Acerbi, L. & Ma, W. J. Unbiased and efficient log-likelihood estimation with inverse binomial sampling. PLoS Comput. Biol. 16, e1008483 (2020). DOI: 10.1371/journal.pcbi.1008483
Acerbi, L. & Ma, W. J. Practical Bayesian optimization for model fitting with Bayesian adaptive direct search. In Proceedings of the 31st International Conference on Neural Information Processing Systems 1834–1844 (2017).
He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 770–778 (2016).
Sutskever, I. & Nair, V. Mimicking go experts with convolutional neural networks. In International Conference on Artificial Neural Networks 101–110 (Springer, 2008).
Clark, C. & Storkey, A. Training deep convolutional neural networks to play go. In International Conference on Machine Learning 1766–1774 (PMLR, 2015).
Elo, A. E. The Rating of Chessplayers, Past and Present (Arco Pub., 1978).
Holding, D. H. Counting backward during chess move choice. Bull. Psychon. Soc. 27, 421–424 (1989). DOI: 10.3758/BF03334644
Holding, D. H. Theories of chess skill. Psychol. Res. 54, 10–16 (1992). DOI: 10.1007/BF01359218
Campitelli, G. & Gobet, F. Adaptive expert decision making: Skilled chess players search more and deeper. ICGA J. 27, 209–216 (2004). DOI: 10.3233/ICG-2004-27403
Griffiths, T. L., Chater, N., Kemp, C., Perfors, A. & Tenenbaum, J. B. Probabilistic models of cognition: Exploring representations and inductive biases. Trends Cognit. Sci. 14, 357–364 (2010). DOI: 10.1016/j.tics.2010.05.004
