Foltz, Peter W.; University of Colorado Boulder and Pearson
Greiff, Samuel ; University of Luxembourg > Faculty of Language and Literature, Humanities, Arts and Education (FLSHASE) > Education, Culture, Cognition and Society (ECCS)
Kuo, Bor-Chen; National Taichung University of Education
Liao, Chen-Huei; National Taichung University of Education
Shaffer, David Williamson; University of Wisconsin
External co-authors :
yes
Language :
English
Title :
Assessment of collaborative problem solving.
Publication date :
2017
Main work title :
Design recommendations for intelligent tutoring systems. Volume 5
Editor :
Sottilare, Robert A.
Graesser, Arthur C.
Hu, Xiangen
Goodwin, G.
Publisher :
Army Research Laboratory, Orlando, FL, United States
Bagley, E., & Shaffer, D. W. (2015). Learning in an urban and regional planning practicum: The view from educational ethnography. Journal of Interactive Learning Research, 26(4), 369-393.
Cade, W., Copeland, J. Person, N., & D'Mello, S. K. (2008). Dialogue modes in expert tutoring. In B. Woolf, E. Aimeur, R. Nkambou, & S. Lajoie (Eds.), Proceedings of the ninth international conference on intelligent tutoring systems (pp. 470-479). Berlin: Springer-Verlag.
Cai, Z., Feng, S., Baer, W., & Graesser, A. C. (2014). Instructional strategies in trialog-based intelligent tutoring systems. In R. Sottilare, A. C. Graesser, X. Hu, & B. Goldberg (Eds.), Design recommendations for intelligent tutoring systems: Adaptive instructional strategies (Vol. 2, pp. 225-235). Orlando, FL: Army Research Laboratory.
Cai, Z., Graesser, A. C., Forsyth, C., Burkett, C., Millis, K., Wallace, P., Halpern, D., & Butler, H. (2011, November). Trialog in ARIES: User input assessment in an intelligent tutoring system. In W. Chen & S. Li (Eds.), Proceedings of the 3rd IEEE international conference on intelligent computing and intelligent systems (pp. 429-433). Guangzhou: IEEE Press.
Cai, Z., Graesser, A. C., & Hu, X. (2015). ASAT: AutoTutor script authoring tool. In R. Sottilare, A. C. Graesser, X. Hu, & K. Brawner (Eds.), Design recommendations for intelligent tutoring systems: Authoring tools (Vol. 3, pp. 199-210). Orlando, FL: Army Research Laboratory.
Care, E., Scoular, C., & Griffin, P. (2016). Assessment of collaborative problem solving in education environments. Applied Measurement in Education, 29(4), 250-264. doi:10.1080/08957347. 2016.1209204
Cen, L., Ruta, D., Powell, L., Hirsch, B., & Ng, J. (2016). Quantitative approach to collaborative learning: Performance prediction, individual assessment, and group composition. International Journal of Computer-Supported Collaborative Learning, 11(2), 187-225. doi:10.1007/s11412-016-9234-6
Cesareni, D., Cacciamani, S., & Fujita, N. (2016). Role taking and knowledge building in a blended university course. International Journal of Computer-Supported Collaborative Learning, 11(1), 1-31. doi:10.1007/s11412-015-9224-0
Chi, M. T. H., Siler, S., Jeong, H., Yamauchi, T., & Hausmann, R. G. (2001). Learning from human tutoring. Cognitive Science, 25, 471-533.
Clark, H. H. (1996). Using language. Cambridge: Cambridge University Press.
Collier, W., Ruis, A., & Shaffer, D. W. (2016). Local versus global connection making in discourse. In C. K. Looi, J. L. Polman, U. Cress, & P. Reimann (Eds.), International conference of the learning sciences (pp. 426-433). Singapore: International Society of the Learning Sciences.
Dascalu, M., Trausan-Matu, S., McNamara, D. S., & Dessus, P. (2015). ReaderBench: Automated evaluation of collaboration based on cohesion and dialogism. International Journal of Computer-Supported Collaborative Learning, 10(4), 395-423. doi:10.1007/s11412-015-9226-y
Dillenbourg, P. (1999). Collaborative learning: Cognitive and computational approaches. New York, NY: Elsevier Science INC.
Dowell, N. M. (2017). A computational linguistic analysis of learners' discourse in computer-mediated group learning environments. Dissertation, University of Memphis, Memphis, TN.
Dowell, N. M., Graesser, A. C., & Cai, Z. (2016). Language and discourse analysis with Coh-Metrix: Applications from educational material to learning environments at scale. Journal of Learning Analytics, 3, 72-95.
Dowell, N. M., Oleksandra, S., Joksimovic´, S., Graesser, A. C., Dawson, S., Gaševic´, S., … Kovanovic´, V. (2015). Modeling learners' social centrality and performance through language and discourse. In O. Santos, J. Boticario, C. Romero, M. Pechenizkiy, A. Merceron, P. Mitros, J. Luna, C. Mihaescu, P. Moreno, A. Hershkovitz, S. Ventura, and M. Desmarais (Eds.), Proceedings of the 8th international conference on educational data mining (pp. 250-257). International Educational Data Mining Society.
Dzikovska, M., Steinhauser, N., Farrow, E., Moore, J., & Campbell, G. (2014). BEETLE II: Deep natural language understanding and automatic feedback generation for intelligent tutoring in basic electricity and electronics. International Journal of Artificial Intelligence in Education, 24, 284-332.
El Masri, Y. H., Baird, J., & Graesser, A. C. (2016). Language effects in international testing: The case of PISA 2006 science items. Assessment in Education: Principles, Policy, & Practice, 23, 427-455.
Erkens, M., Bodemer, D., & Hoppe, H. U. (2016). Improving collaborative learning in the classroom: Text mining based grouping and representing. International Journal of Computer-Supported Collaborative Learning, 11(4), 387-415. doi:10.1007/s11412-016-9243-5
Fiore, S. M., Rosen, M. A., Smith-Jentsch, K. A., Salas, E., Letsky, M., & Warner, N. (2010). Toward an understanding of macrocognition in teams: Predicting processes in complex collaborative contexts. Human Factors, 52, 203-224.
Fiore, S. M., Wiltshire, T. J., Oglesby, J. M., O'Keefe, W. S., & Salas, E. (2014). Complex collaborative problem-solving processes in mission control. Aviation, Space, and Environmental Medicine, 85(4), 456-461.
Fischer, F., Kollar, I., Stegmann, K., & Wecker, C. (2013). Toward a script theory of guidance in computer-supported collaborative learning. Educational Psychologist, 48(1), 56-66. doi:10.1080/00461520.2012.748005
Foltz, P. W., Kintsch, W., & Landauer, T. K. (1998). The measurement of textual coherence with Latent Semantic Analysis. Discourse Processes, 25, 285-307.
Foltz, P. W., & Martin, M. J. (2008). Automated communication analysis of teams. In E. Salas, G. F. Goodwin, & S. Burke (Eds.), Team effectiveness in complex organisations and systems: Cross-disciplinary perspectives and approaches (pp. 411-431). New York, NY: Routledge.
Funke, J. (2010). Complex problem solving: A case for complex cognition? Cognitive Processes, 11, 133-142.
Gilbert, S. B., Slavina, A., Dorneich, M. C., Sinatra, A. M. Bonner, D. Johnston, J. … Winer, E. (2017). Creating a team tutor using GIFT. International Journal of Artificial Intelligence in Education. doi:10.1007/s40593-017-0151-2
Graesser, A. C. (2016). Conversations with AutoTutor help students learn. International Journal of Artificial Intelligence in Education, 26, 124-132.
Graesser, A. C., Cai, Z., Hu, X., Foltz, P. W., Greiff, S., Kuo, B. C. … Shaffer, D. W. (2017). Assessment of collaborative problem solving. In R. Sottilare, A. C. Graesser, X. Hu, & G. Goodwin (Eds.), Design recommendations for intelligent tutoring systems: Volume 5- Assessment. Orlando, FL: U.S. Army Research Laboratory.
Graesser, A. C., D'Mello, S. K., & Person, N. K. (2009). Meta-knowledge in tutoring. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Handbook of metacognition in education. Mahwah, NJ: Routledge.
Graesser, A. C., Foltz, P. W., Rosen, Y., Shaffer, D. W., Forsyth, C., & Germany, M. (2017). Challenges of assessing collaborative problem solving. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills (pp. 75-91). Heidelberg: Springer Publishers.
Graesser, A. C., Forsyth, C. M., & Foltz, P. (2017). Assessing conversation quality, reasoning, and problem solving performance with computer agents. In B. Csapo, J. Funke, & A. Schleicher (Eds.), On the nature of problem solving: A look behind PISA 2012 problem solving assessment (pp. 245-261). Heidelberg: OECD Series.
Graesser, A. C., Jeon, M., Yang, Y., & Cai, Z. (2007). Discourse cohesion in text and tutorial dialogue. Information Design Journal, 15, 199-213.
Graesser, A. C., McNamara, D. S., Cai, Z., Conley, M., Li, H., & Pennebaker, J. (2014). Coh-Metrix measures text characteristics at multiple levels of language and discourse. Elementary School Journal, 115, 210-229.
Graesser, A. C., Penumatsa, P., Ventura, M., Cai, Z., & Hu, X. (2007). Using LSA in AutoTutor: Learning through mixed initiative dialogue in natural language. In T. Landauer, D. McNamara, S. Dennis, & W. Kintsch (Eds.), Handbook of latent semantic analysis (pp. 243-262). Mahwah, NJ: Erlbaum.
Graesser, A. C., & Person, N. K. (1994). Question asking during tutoring. American Educational Research Journal, 31, 104-137.
Graesser, A. C., Person, N. K., Harter, D., & Tutoring Research Group. (2001). Teaching tactics and dialog in AutoTutor. International Journal of Artificial Intelligence in Education, 12(3), 257-279.
Graesser, A. C., Person, N. K., & Magliano, J. P. (1995). Collaborative dialogue patterns in naturalistic one-to-one tutoring. Applied Cognitive Psychology, 9, 495-522.
Greiff, S., Wustenberg, S., Csapo, B., Demetriou, A., Hautamaki, J., Graesser, A. C., & Martin, R. (2014). Domain-general problem solving skills and education in the 21st century. Educational Research Review, 13, 74-83.
Gress, C. L. Z., Fior, M., Hadwin, A. F., & Winne, P. H. (2010). Measurement and assessment in computer-supported collaborative learning. Computers in Human Behavior, 26(5), 806-814. doi:10.1016/j.chb.2007.05.012
Griffin, P., & Care, E. (2015). ATC21S method. In P. Griffin & E. Care (Eds.), Assessment and teaching of 21st century skills: Methods and approach. Dordrecht: Springer.
Haviland, S. E., & Clark, H. H. (1974). What's new? Acquiring new information as a process in comprehension. Journal of Verbal Learning and Verbal Behavior, 13, 512-521.
Hempelmann, C. F., Dufty, D., McCarthy, P., Graesser, A. C., Cai, Z., & McNamara, D. S. (2005). Using LSA to automatically identify givenness and newness of noun-phrases in written discourse. In B. Bara (Ed.), Proceedings of the 27th annual meeting of the cognitive science society (pp. 941-946). Mahwah, NJ: Erlbaum.
Hesse, F., Care, E., Buder, J., Sassenberg, K., & Griffin, P. (2015). A framework for teachable collaborative problem solving skills. In P. Griffin & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 37-55). Heidelberg, Springer.
Hu, X., Cai, Z., Franceschetti, D., Penumatsa, P., Graesser, A. C., Louwerse, M. M. … Tutoring Research Group. (2003). LSA: The first dimension and dimensional weighting. In Proceedings of the 25th meeting of the cognitive science society (pp. 1-6).
Jackson, G. T., & Graesser, A. C. (2006). Applications of human tutorial dialog in AutoTutor: An intelligent tutoring system. Revista Signos, 39(60), 31-48.
Janis, I. L. (1982). Groupthink: Psychological studies of policy decisions and fiascoes. Boston, MA: Cengage Learning.
Jurafsky, D., & Martin, J. (2008). Speech and language processing. Englewood, NJ: Prentice Hall.
Kapur, M. (2011). Temporality matters: Advancing a method for analyzing problem-solving processes in a computer-supported collaborative environment. International Journal of Computer-Supported Collaborative Learning, 6(1), 39-56. doi:10.1007/s11412-011-9109-9
Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25, 259-284.
Landauer, T. K., McNamara, D. S., Dennis, S., & Kintsch, W. (2007). Handbook of latent semantic analysis. Mahwah, NJ: Erlbaum.
Liu, L., Von Davier, A., Hao, J., Kyllonen, P., & Zapata-Rivera, D. (2015). A tough nut to crack: Measuring collaborative problem solving. In R. Yigal, S. Ferrara, & M. Mosharraf (Eds.), Handbook of research on technology tools for real-world skill development (pp. 344-359). Hershey, PA: IGI Global.
McCarthy, P. M., Dufty, D., Hempelman, C., Cai, Z., Graesser, A. C., & McNamara, D. S. (2012). Evaluating givenness/newness. In P. M. McCarthy & C. Boonthum-Denecke (Eds.), Applied natural language processing: Identification, investigation, and resolution (pp. 457-478). Hershey, PA: IGI Global.
McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge: Cambridge University Press.
Morgan, B., Burkett, C., Bagley, E., & Graesser, A. C. (2011). Typed versus spoken conversations in a multi-party epistemic game. In G. Biswas, S. Bull, J. Kay, & A. Mitrovic (Eds.), Proceedings of 15th international conference on artificial intelligence in education (pp. 513-515). Berlin: Springer-Verlag.
Morgan, B., Keshtkar, F., Duan, Y., & Graesser, A. C. (2012). Using state transition networks to analyze multi-party conversations in a serious game. In. S. A. Cerri & B. Clancey (Eds.), Proceedings of the 11th international conference on intelligent tutoring systems (ITS 2012) (pp. 162-167). Berlin: Springer-Verlag.
Mu, J., Stegmann, K., Mayfield, E., Rose´, C., & Fischer, F. (2012). The ACODEA framework: Developing segmentation and classification schemes for fully automatic analysis of online discussions. International Journal of Computer-Supported Collaborative Learning, 7(2), 285-305.
Nash, P., & Shaffer, D. W. (2011). Mentor modeling: The internalization of modeled professional thinking in an epistemic game. Journal of Computer Assisted Learning, 27(2): 173-189.
Nash, P., & Shaffer, D. W. (2013). Epistemic trajectories: Mentoring in a game design practicum. Instructional Science, 41(4): 745-771.
National Research Council. (2011). Assessing 21st century skills. Washington, DC: National Academies Press.
Nye, B. D., Graesser, A. C., & Hu, X. (2014). AutoTutor and family: A review of 17 years of natural language tutoring. International Journal of Artificial Intelligence in Education, 24(4), 427-469.
OECD. (2010). PISA 2012 field trial problem solving framework. Paris: OECD. Retrieved from http://www.oecd.org/dataoecd/8/42/46962005.pdf
OECD. (2013). PISA 2015 collaborative problem solving framework. Paris: OECD. Retrieved from http://www.oecd.org/pisa/pisaproducts/Draft%20PISA%202015%20Collaborative%20 Problem%20Solving%20Framework%20.pdf
Olney, A., Louwerse, M., Mathews, E., Marineau, J., Hite-Mitchell, H., & Graesser, A. C. (2003). Utterance classification in AutoTutor. In J. Burstein & C. Leacock (Eds.), Proceedings of the HLT-NAACL 03 workshop on building educational applications using natural language processing. Philadelphia, PA: Association for Computational Linguistics.
O'Neil, H. F., Chuang, S. H., & Baker, E. L. (2010). Computer-based feedback for computer-based collaborative problem-solving. In D. Ifenthaler, P. Pirnay-Dummer, & N. M. Seel (Eds.), Computer-based diagnostics and systematic analysis of knowledge (pp. 261-279). New York, NY: Springer-Verlag.
Pennebaker, J. W., Booth, R., & Francis, M. (2007). LIWC2007: Linguistic inquiry and word count. Austin, TX: liwc.net. 2007. Retrieved from liwc.net
Prince, E. (1981). Toward a taxonomy of given-new information. In P. Cole (Ed.), Radical pragmatics (pp. 223-256). New York, NY: Academic Press.
Reimann, P. (2009). Time is precious: Variable-and event-centred approaches to process analysis in CSCL research. International Journal of Computer-Supported Collaborative Learning, 4(3), 239-257. doi:10.1007/s11412-009-9070-z
Rosen, Y. (2014). Comparability of conflict opportunities in human-to-human and human-to-agent online collaborative problem solving. Technology, Knowledge and Learning, 19, 147-174.
Rose´, C. P., & Ferschke, O. (2016). Technology support for discussion based learning: From computer supported collaborative learning to the future of massive open online courses. International Journal of Artificial Intelligence in Education, 26(2), 660-678. doi:10.1007/s40593-016-0107-y
Rose´, C., Wang, Y. C., Cui, Y., Arguello, J., Stegmann, K., Weinberger, A., & Fischer, F. (2008). Analyzing collaborative learning processes automatically: Exploiting the advances of computational linguistics in computer-supported collaborative learning. International Journal of Computer-Supported Collaborative Learning, 3, 237-271.
Rus, V., Lintean, M. C., Banjade, R., Niraula, N. B., & Stefanescu, D. (2013). SEMILAR: The Semantic similarity toolkit. In ACL conference system demonstrations (pp. 163-168). Sofia, Bulgaria: Association for Computational Linguistics.
Rus, V., Lintean, M., Graesser, A. C., & McNamara, D. S. (2012). Text-to-text similarity of statements. In P. McCarthy & C. Boonthum-Denecke (Eds.), Applied natural language processing: Identification, investigation, and resolution (pp. 110-121). Hershey, PA: IGI Global.
Rus, V., & Stefanescu, D. (2016). Toward non-intrusive assessment in dialogue-based intelligent tutoring systems. In State-of-the-art and future directions of smart learning (pp. 231-241). Springer Singapore.
Sacks, H., Schegloff, E. A., & Jefferson, G. (1974). A simplest systematics for the organization of turn-taking for conversation. Language, 696-735.
Salas, E., Cooke, N. J., & Rosen, M. A. (2008). On teams, teamwork, and team performance: Discoveries and developments. Human Factors: The Journal of the Human Factors and Ergonomics Society, 50(3), 540-547.
Samei, B., Li, H., Keshtkar, F., Rus, V., & Graesser, A. C. (2014). Context-based speech act classification in intelligent tutoring systems. In S. Trausan-Matu, K. Boyer, M. Crosby, & K. Panou (Eds.), Proceedings of the twelfth international conference on intelligent tutoring systems (pp. 236-241). Berlin: Springer.
Sawyer, R. K. (2014). The new science of learning. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 1-20). New York, NY: Cambridge University Press.
Shaffer, D. W. (2017). Quantitative ethnography. Madison, WI: Cathcart Press.
Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3, 9-45.
Shaffer, D. W., Hatfield, D., Svarovsky, G., Nash, P., Nulty, A., Bagley, E. A., … Mislevy, R. J. (2009). Epistemic network analysis: A prototype for 21st century assessment of learning. The International Journal of Learning and Media, 1(1), 1-21.
Shaffer, D. W., Ruis, A. R., & Graesser, A. C. (2015). Authoring networked learner models in complex domains. In. R. Sottilare, A. C. Graesser, X. Hu, & K. Brawner (Eds.), Design recommendations for intelligent tutoring systems: Authoring tools (Vol. 3, pp. 179-192). Orlando, FL: Army Research Laboratory.
Sottilare, R. A., Burke, C. S., Salas, E., Sinatra, A. M., Johnston, J. H., & Gilbert, S. B. (2017). Designing adaptive instruction for teams: A Meta-analysis. International Journal of Artificial Intelligence in Education. doi:10.1007/s40593-017-0146-z
Sottilare, R., Graesser, A. C., Hu, X., & Brawner, K. (Eds.) (2015). Design recommendations for intelligent tutoring systems: Authoring tools (Vol. 3). Orlando, FL: Army Research Laboratory.
Sperber, D., & Wilson, D. (1995). Relevance: Communication and cognition. Oxford: Blackwell.
Stahl, G. (2005). Group cognition in computer-assisted collaborative learning. Journal of Computer Assisted Learning, 21(2), 79-90. doi:10.1111/j.1365-2729.2005.00115.x
Suthers, D. D. (2006). Technology affordances for intersubjective meaning making: A research agenda for CSCL. International Journal of Computer-Supported Collaborative Learning, 1(3), 315-337. doi:10.1007/s11412-006-9660-y
Tausczik, Y. R., & Pennebaker, J. W. (2013). Improving teamwork using real-time language feedback. Proceedings of Human Factors in Computing Systems (CHI), 459-468.
Tegos, S., Demetriadis, S., Papadopoulos, P. M., & Weinberger, A. (2016). Conversational agents for academically productive talk: a comparison of directed and undirected agent interventions. International Journal of Computer-Supported Collaborative Learning, 11(4), 417-440. doi:10.1007/s11412-016-9246-2
VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227-265.
Von Davier, A., & Halpin, P. (2013). Collaborative problem solving and the assessment of cognitive skills: Psychometric considerations. Research Report No. ETS RR-13-41 (pp. 1-42). Educational Testing Service. Retrieved from http://www.ets.org/research/contact.html
Von Davier, A., Zhu, M., & Kyllonen, P. C. (Eds.). (2017). Innovative assessment of collaboration. New York, NY: Springer.
Zapata-Rivera, D., Jackson, G. T., & Katz, I. (2015). Authoring conversation-based assessment scenarios. In R. Sottilare, X. Hu, A. Graesser, & K. Brawner (Eds.), Design recommendations for adaptive intelligent tutoring systems (Vol. 3, pp. 169-178). Orlando, FL: Army Research Laboratory.
