By Ralph W. Poole III, Lecturer in the M.S. in Information & Knowledge Strategy (IKNS) Program, School of Professional Studies
AI has made information easier to find, while simultaneously making it harder to trust what you've found.
AI can produce an apparently complete answer before a user sees its sources or the reasoning that connects the evidence to the conclusion. The fluency of the answer can lead to premature acceptance: the searcher stops evaluating because the answer sounds coherent, whether or not the evidence supports it.
This problem sits at the center of AI-Enabled Findability and Semantic Systems, the course I teach in Columbia University’s M.S. in Information & Knowledge Strategy (IKNS) program. In the course, I define findability through three connected disciplines:
Organizing raw information resources so their meaning, context, and relationships become inspectable.
- Locating the right information to answer a specific need.
- Confirming the source and explaining why the resulting answer deserves trust.
Before generative AI, the three disciplines were easier to see and understand: Librarians, publishers, and information architects organized collections; search engines returned documents and identified their sources; users judged relevance, compared evidence, and decided what to trust. The process was imperfect, but responsibility for each step remained comparatively clear.
Generative AI compresses the user's experience of these formerly visible steps into a single interaction. A user enters a prompt, and the system returns an integrated response, often without revealing which sources informed it, how the material was selected and organized, or whether the evidence supports the conclusion. The hard work associated with findability remains, but its boundaries have become harder to inspect. And the user still needs to decide what to trust.
Examining the three disciplines separately shows what has changed in the AI age and what still requires human judgment.
The First Discipline Organizes
Organizing comes first, because everything else depends on it. Before anyone can search a collection, its resources must be selected, represented, and made available for interaction. Resources are organized for a purpose: so that people or machines can find a document, compare alternatives, answer a question, recommend a song, or trace a claim to evidence. The interactions the system is intended to support shape what enters the collection, how resources are represented, which distinctions and relationships remain explicit, and what the system can retrieve or infer.
People, software, or some combination of the two perform this work, both before and after generative AI. In every case, choices about scope, purpose, and use shape the result.
Two broad approaches carry out this organizing. The first makes meaning explicit. Librarians use controlled vocabularies, taxonomists define categories, information architects structure navigation, and ontology engineers name concepts and relationships. The various methods differ in detail, but each produces structure that people can inspect and apply consistently.
The second approach organizes by similarity. It is this approach of organizing information that generative AI relies on in particular: First, an embedding model converts a word, passage, image, or other resource into a numerical representation and places it on a mathematical map. Then a vector database stores those representations and retrieves resources located “near” one another. Nearby resources tend to share context or characteristics, even when no one has explicitly named the relationship between them.
This similarity-based approach can search very large collections without requiring people to define every concept and relationship in advance. It can retrieve resources that use different words but express similar ideas. Therefore, the second approach improves locating, but it does not yet explain why the resources are meaningful.
A music recommendation provides a familiar example for such similarity criteria: A streaming service can group songs using metadata such as artist, genre, producer, and release date, along with audio features and listening patterns. The metadata can identify specific connections between songs. A recommendation model combines those signals into vector representations, revealing similarities that no single metadata field explains.
The distinction between organizing information by explicit relationships versus similarity becomes clearer when the kind of relationship matters: In the second approach, a vector model may place the Rolling Stones close to blues because their recordings share musical features and listeners associate the band with that tradition. However, the closeness alone does not tell us whether the Rolling Stones perform blues, were influenced by blues, or belong to the blues-rock genre. In contrast, in the first approach, an ontology names those relationships explicitly so people and software can inspect them and use them in reasoning.
Used in concert, the two approaches perform different but complementary organizing work: Vector representations help locate related resources across large collections while ontologies state which relationships matter and what those relationships mean. Whenever users need both broad retrieval and an inspectable explanation, combining the two supports both discovery and reasoning.
The Second Discipline Locates
Only once resources are organized does a specific person arrive with a specific need. A user's objective remains the same: identify the information that best fits their needs. Before generative AI, users searched indexes, browsed collections, and compared lists of results. With generative AI, users express their need in a prompt and refine the request over successive iterations. The interface has changed, but the user still must define the need precisely, recognize relevant evidence, and distinguish when the response does not answer the question.
The Third Discipline Confirms
Now the objective is to determine why an answer deserves trust. Provenance tells the user where a claim came from. Confirmation goes further: the user must see how the evidence supports the conclusion, examine the assumptions linking them, and decide whether the reasoning holds.
Consider a claim about Keith Richards. In 2006, he fell from a palm tree at the Wakaya Club in Fiji, struck his head, and traveled to Auckland for treatment (The Guardian, April 29, 2006). A system can return that answer fluently. Tracing it to the source confirms the core: the fall, the resort, the hospital. Tracing also strips the embellishments that ride along with the story, including a reported jet-ski crash and a tree that one account calls towering and another calls a low branch seven feet off the ground. Provenance points to a source. Confirmation separates what the source supports from what it does not.
Keith Richards is low-stakes trivia. The same fluency carries higher costs when a system reports confidently on what it never captured. Diann Burns, a recent graduate of Columbia University's Information & Knowledge Strategy (IKNS) program, tested Zillow's output against the facts. She compared two very similar homes in the same neighborhood. They matched on every attribute Zillow's algorithm could count: square footage, room counts, asking price. On that basis the algorithm ranked them as near equivalents.
On the ground, however, she found one home built with care and the other with visibly different and inferior materials, a distinction that no field in Zillow records. Zillow’s algorithm presented its estimate as an authoritative number, with no signal of what it had not measured. To learn what she was buying, Burns visited each home to confirm the data. She interviewed the realtor and an inspector to find what the listing left out.
Generative AI makes confirmation more difficult because it presents a synthesized answer rather than a visible trail of evidence. Yes, a citation in the AI’s output can identify a source, but provenance alone does not confirm a claim. Someone still has to do the step of confirming: Determine whether the source is credible, assess whether the evidence actually supports the claim as stated, and evaluate whether the response omits any important qualifications.
Tips for using generative AI for organizing, locating, and confirming all at once
- Generative AI rarely exposes the confirmation step by default. Still, a well-designed application can preserve it by linking individual claims to source passages, retaining provenance, and showing the named relationships or rules used to derive the answer. When those features are absent, the users themselves must reconstruct the path: open the source, locate the supporting evidence, compare its scope with the generated claim, and look for omitted qualifications or conflicting evidence.
- AI typically masks relevant metadata and obscures the organizing work, even as it reduces the effort of locating an answer, and it leaves the final confirmation to the user. As a remedy, organizing the data well can make the confirmation step faster and more reliable: When sources, concepts, and relationships are represented explicitly, the application can expose the path from evidence to conclusion. However, when those connections remain implicit only, the answer may still be useful, but verification requires more work and remains less certain.
- Remember that the prompt box creates the illusion that organizing has disappeared. But even if “behind the scenes”, it can still affect AI’s answers. At model scale, pipelines determine what content to ingest, exclude, deduplicate, and weight. In retrieval applications, pipelines also determine how content is divided into chunks, embedded, indexed, retrieved, and retained. Policies, models, and parameters encode those choices, while software carries them out largely out of sight.
Returning to the definition, findability still requires organizing resources so their distinctions and relationships support retrieval, locating the right information, and confirming why the resulting answer deserves trust. AI redistributes that work among users, models, and application pipelines. It does not eliminate it.
In short: Your AI can give you an answer. You still have to decide whether it is trustworthy. The seeker who traces the claim to its source and tests the evidence has grounds for trust. The seeker who stops at the prompt box has an “answer” in name only.
For AI does not find. It guesses, fluently.
Views and opinions expressed here are those of the author and do not necessarily reflect the official position of Columbia School of Professional Studies or Columbia University.
About Columbia's IKNS Degree
Columbia University’s M.S. in Information & Knowledge Strategy (IKNS) degree integrates data, people, and strategy skills for the AI age. The flexible and interdisciplinary curriculum trains leaders across the entire value chain of data-driven management: Getting the data and analytics right (e.g., AI adoption, business analytics), creating a high performing, people-centric culture (collaboration, team/project management, organizational psychology), and finally the right change management to turn your strategy into reality.
IKNS is available full-time or part-time, online or in-person on Columbia’s landmarked campus right here in New York City. To maximize opportunities for networking and community building, our online students join our New York-based students on Columbia’s campus for three in-person Residencies during their studies. The STEM-designated Master of Science degree offers International Students (F-1/J-1 visa) an opportunity for Curricular Practical Training during their studies (CPT) and 3 years of work authorization in the US upon completing their studies (OPT).
Students train under world-class faculty, including former and current executives from Google, IBM, NASA, and Oliver Wyman, and join a powerful global alumni network in coveted positions, including at Alphabet, Goldman Sachs, Nike, Pfizer, and the World Bank.
For more IKNS insights, news, and events, please go to our website, connect with us on LinkedIn, or attend one of our online info sessions. Visit the School of Professional Studies website to learn more about the SPS Student Experience.
