By Nicole M. Alexander, Lecturer, Information & Knowledge Strategy, Columbia University School of Professional Studies
OpenAI recently published an unusual set of reports: six examples of its own models behaving in ways the company described as unexpected or concerning. One model, during training, repeatedly left notes for its future self suggesting that it could disregard constraints and didn’t need to be subservient to humans. Another left a more mundane-sounding instruction with a darker implication: “Be transparent only if asked.”
Instead of focusing on the model writing a problematic sentence; models generate strange text all the time. It’s that someone had to see it, recognize that it mattered and escalate it. That person is part of the control system.
On September 16, OpenAI published a framework for tracking, investigating, and disclosing examples of model misalignment. It’s a useful step, and it followed earlier incidents that researchers and journalists had surfaced publicly.
But frameworks don’t observe anomalies. People do. A reporting process only works if the people inside it retain the confidence and judgment to say: this isn’t normal; someone needs to look at it.
The July Hugging Face security breach shows why. OpenAI said that models undergoing a cyber-capability evaluation identified and exploited an Artifactory zero-day vulnerability to gain internet access, ultimately compromising Hugging Face infrastructure. That isn’t proof that AI systems are malicious or that controls are pointless. It’s a reminder that systems can surprise their creators.
A governance framework can’t perform the work of governance by itself. Evaluation environments, access controls and monitoring matter only if capable people can interpret what they reveal and act on it quickly enough. Which brings us to a phrase that has appeared on countless corporate slide decks: “talent at the center.”
Usually, it means retention, culture, engagement, leadership development and the HR case for treating people well. In an AI-enabled organization, it should mean something more specific: preserving the human capacity to challenge an automated system at the moment it matters. That’s not merely an HR concern. It’s an ethical, operational and governance concern.
The law noticed before HR did
Caremark oversight doctrine requires directors to make a good-faith effort to establish and monitor information and reporting systems around significant corporate risks. It doesn’t require directors to understand model architecture or audit model weights. It does require boards to establish a credible process through which management identifies material risks, assigns accountability, documents controls and escalates serious problems.
Board oversight is developing unevenly. NACD reported that 27% of boards had incorporated AI-oversight responsibilities into committee charters, while more than 62% of directors said their boards had reserved agenda time for full-board AI discussions. That’s progress. But agenda time isn’t the same thing as an oversight architecture. Nora M. Denzel, lead independent director at AMD and a director at Sony Group, Gen Digital, and NACD, calls the gap “vibe governance”: “We have a policy, we follow a framework, we train our employees, we bought a tool. It’s reassuring. But who owns the outcome, what controls are in place, and what evidence shows they work?”
A policy can establish intent; a framework can organize responsibilities; a tool can automate a task. None of them, alone, establishes that somebody owns the outcome or that a control will work when it’s needed. Regulators are beginning to press on the same problem. On December 4, 2025, the SEC’s Investor Advisory Committee recommended that the Commission develop guidance addressing how issuers define AI in their disclosures, what board-level oversight mechanisms exist, and how AI materially affects company operations.
That recommendation isn’t a binding rule. But it points toward a future in which companies may be asked to explain what they govern, who is accountable, and how they know their controls work. Broad claims about responsible AI will have to withstand Denzel’s more concrete questions about ownership and evidence.
The harder problem
The oversight that boards, regulators and stakeholders increasingly expect depends on people who can still spot a problem, challenge a confident-sounding system and trust their own reading of a situation when it conflicts with a machine’s output. That capacity shouldn’t be taken for granted.
Agentic systems make this more urgent. A chatbot generally waits for a prompt. An agent can plan, use tools, retrieve information and move through a sequence of tasks with less direct intervention. That can be useful. It also shrinks the window in which a person can notice that something is wrong.
I saw an earlier version of this problem at Meta, where I led Business and product marketing. Automation was the only way to operate at that scale. But the calls that mattered most, such as whether a message actually landed in a market or whether a segment was being served rather than merely reached, depended on people close enough to the work to notice when the dashboard looked right and the outcome didn’t. Scale didn’t reduce the need for judgment but conversely raised the cost of losing it.
So, the question isn’t simply whether an AI system has a human in the loop (HITL) but if that human is still capable of serving as one. A checkpoint only works if the person holding it retains the judgment to use it.
The most relevant research on deskilling doesn’t come from agentic AI, in part because widespread organizational use of agents is still new. But research on AI decision-support tools offers a warning. A 2025 multicentre observational study in The Lancet Gastroenterology & Hepatology found that, after continuous exposure to AI-assisted polyp detection, clinicians’ adenoma detection rate in standard colonoscopies performed without AI fell from 28.4% to 22.4%. The researchers cautioned that the observational design doesn’t establish causation, but concluded that continued exposure to AI might reduce performance in non-AI-assisted settings.
None of this means AI makes people less capable. Used deliberately, it can accelerate learning and improve performance. The issue is passive reliance: the gradual transfer of attention, memory, evaluation, and judgment to systems that are helpful most of the time but still require oversight. That’s precisely the wrong capacity to let decay just as organizations ask people to oversee systems that act quickly, confidently and sometimes unpredictably.
It’s more than AI literacy
Somebody will say this is simply AI literacy training with a more dramatic name. It isn’t. People do need to understand what models can do, where they fail and why confident language shouldn’t be mistaken for reliability. But literacy alone doesn’t ensure that someone will intervene when a system is wrong, resist automation bias or make a difficult decision independently.
A more demanding task is to preserve human judgment as an active organizational capability. That means deciding which decisions must always receive meaningful human review, making escalation rights clear and psychologically safe to use, giving people regular practice deciding without automated support and testing whether reviewers are actually reviewing rather than simply approving. The goal isn’t friction for its own sake. It’s enough productive friction that disagreement remains a real option instead of an unexercised formality.
This is closer to conditioning than compliance. Athletes retain judgment and reaction time through structured, repeated practice and organizational judgment works in the same way.
In my course, AI and the Knowledge-Driven Organization, part of Columbia’s Information & Knowledge Strategy program, this is where I push students hardest. Many already use AI at work, so I ask them to stress-test one AI-assisted decision from their own workplace: Who reviews it? Who can stop it? What evidence would make them overrule it? The exercise is simple. Answering it honestly is the difficult part, and understanding that variance is where governance work begins.
What talent at the center means
Put the two threads together and “talent at the center” stops sounding like HR language. It becomes a claim about where an essential governance asset lives: in the judgment of the people an organization depends on to notice anomalies, challenge systems and decide when an automated recommendation shouldn’t control the outcome.
The person who flags a warning sign in a model evaluation, refuses to rubber-stamp an agent’s action, or escalates a concern despite institutional pressure isn’t peripheral to the AI governance framework. That person is part of the framework.
So what does this mean in practice? For boards, it means asking not only whether an AI policy exists but who owns each consequential outcome and what evidence shows the controls work. For executives, it means treating human review as a capability to be staffed, practiced and measured, not a box at the end of a workflow. And for the professionals doing the work, including the ones in my classroom, it means protecting the judgment that makes them worth keeping in the loop.
Here’s the test I’d offer any organization: if your AI systems were unavailable tomorrow, could the people responsible for oversight still explain, evaluate and defend the decisions those systems have been making? If not, the organization doesn’t have meaningful human oversight. It has a dependency, and dependencies aren’t governance.
Views and opinions expressed here are those of the authors, and do not necessarily reflect the official position of Columbia University School of Professional Studies or Columbia University.
About Columbia's IKNS Degree
Columbia University’s M.S. in Information & Knowledge Strategy (IKNS) degree integrates data, people, and strategy skills for superior organizational intelligence. The flexible and interdisciplinary curriculum trains leaders across the entire value chain of data-driven management: Getting the data and analytics right (e.g., AI adoption, business analytics), creating a high performing, people-centric culture (collaboration, team/project management, organizational psychology), and finally the right change management to turn your strategy into reality.
IKNS is STEM-designated and available full-time or part-time. Study online or in-person on Columbia’s landmarked campus right here in New York City. To maximize opportunities for networking and community building, our online students join our New York-based students for three in-person Residencies on Columbia’s campus during their studies.
Students train under world-class faculty, including former and current executives from Amazon, Deloitte, Google DeepMind and NASA, and join a powerful global alumni network in coveted positions, including at Alphabet, Goldman Sachs, Nike, Pfizer, and the World Bank.
For more IKNS insights, news, and events, please visit our website, connect with us on LinkedIn, or attend one of our online info sessions. Visit the School of Professional Studies website to learn more about the SPS Student Experience.
