Playing the Imitation Game: Human-AI Simulators in Pedagogic Design

Current advances in large language models (LLMs) and generative AI (GenAI) have produced both enthusiasm and concerns in the academic world, industry, and society in general. While optimistic views foresee unprecedented increase in efficiency and productivity, concerns have been expressed on the potential of these technologies to determine significant changes in most areas of human activity, which may not always have predictable or positive outcomes. One of the challenges often evoked in this context, not yet fully addressed, is the impact of the AI-powered agents on the educational sector, and especially on aspects such as student’s agency and control, creativity, and motivation in pedagogic activities that involve the use of this type of agents. The aim of the study is to address this question starting from the hypothesis that preliminary simulations of AI-based pedagogic scenarios can help instructors to better understand the inner mechanisms of these technologies and their possible impact on the learning, assignment completion and evaluation processes. The paper presents a set of experiments with simulated student-agent interactions generated by AI chatbots and proposes a formal framework for assessing this form of “imitation game” and its possible applications in real teaching-learning environments.