THE GOAL: Misalignment between the measured metrics and the nebulous concept of critical thinking in joint decision-making

The increasing adoption of Generative AI in knowledge-heavy jobs workflows raises questions about its impact on the human knowledge worker’s critical thinking (CT) skills and decision-making. Some people think we can mitigate the effects AI has on the
human decision-making process by creating AI-based interventions that would encourage more thoughtful decision-making. Creating such interventions, however, requires a precise definition of what critical thinking is and how to measure it. Additionally, encouraging certain modes of thinking in a controlled environment doesn’t necessarily translate to people applying them in a decision-making process. This position paper clarifies the problems stemming from both the definitional and the practical CT measurement issues, discusses what characteristics a good proxy measure of CT should have, and talks about human response to ’easy automation’.