By Robert L. Simione II, Associate Professor of Professional Practice, M.S. in Applied Analytics, Columbia University School of Professional Studies
Across professional practice, artificial intelligence has rapidly moved from being a novel tool to an everyday tool. The important question now is how to use AI without surrendering judgment. Although AI can increase speed, support exploration, and extend what people are able to accomplish, using it effectively still requires careful evaluation. For that reason, critical thinking is a compass for navigating the jagged frontier of AI.
Researchers describe AI as having a “jagged technological frontier”: It can perform remarkably well on some tasks while failing unexpectedly on others that appear equally manageable. In research published in Organization Science in March 2026, Dell’Acqua and colleagues examined 758 consultants at Boston Consulting Group completing realistic consulting tasks involving strategy, quantitative analysis, idea generation, and persuasive writing. On tasks that fell within the AI’s capabilities, consultants using AI worked faster and produced higher-quality results. But on a complex business problem designed to fall outside that frontier, those using AI were 19% less likely to arrive at the correct answer than those working without it. The difficulty is that we cannot always tell which side of that frontier we are on.
In a study from the Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems published in April 2025, Lee and colleagues identified three main reasons workers engaged in critical thinking when using generative AI: to ensure the quality of their work, to guard against potentially negative consequences, and to develop or maintain their own skills. Based on participants' self-assessments, the researchers also identified three barriers that made critical thinking less likely: lack of awareness that scrutiny was needed, lack of motivation, and lack of ability. Workers were less likely to scrutinize AI output when they did not perceive the task as important or saw the AI as capable of handling it without additional scrutiny; when time pressure or workplace incentives discouraged additional review; or when they lacked the subject-matter expertise needed to evaluate and improve the AI’s response. In other words, whether people engage in critical thinking depends on whether they recognize the need for scrutiny, have an incentive to provide it, and possess enough expertise to evaluate the result.
A working paper from the Organization for Economic Cooperation and Development, published in February 2025 by Milanez and colleagues, surveyed more than 6,000 employers across six countries and found employer concerns about algorithmic tools in the workplace, including unclear accountability and difficulty following the tools’ logic. The finding raises a practical question that AI cannot resolve on its own: Who is accountable when an automated system contributes to a workplace decision?
AI can accelerate our work, but speed and judgment are not the same thing. The greatest advantage will not belong to those who use AI most often or who know the cleverest prompts. It will belong instead to those who know how to evaluate AI-generated work, when to revise or reject it, and how to remain accountable for decisions made with it.
About the Program
Columbia University’s Master of Science in Applied Analytics prepares students with the practical data and leadership skills to succeed. The program combines in-depth knowledge of data analytics with the leadership, management, and communication principles and tactics necessary to impact decision-making across industries and organizational functions.
Learn more about the program here. The program is available full-time and part-time, online and on-campus.
