Search for photos of a city online, and you'll mostly see its most iconic landmarks: spots that don't represent the city as a whole. What you'll find is a curated collection of "published" images, not a random, accurate sample of how different parts of the city actually look. What appears online is concentrated around a handful of locations that photographers find interesting or rewarding enough to shoot, while everything ordinary and unphotographed goes unseen.
The same pattern holds for published research, with one key difference. With a city, you could, in principle, go out and produce a representative sample of photos from every part of it. In research, you don't know the extent of what hasn't been studied; you don't know what you don't know.
So there are two problems: first, search results don't realistically represent the studied domain: they favor whatever has appeared the most, regardless of how sound the research behind it is. Second, the unknowns. Even assuming every study followed sound scientific method, a proper meta-analysis should be written by someone who has spent decades in the field, not an LLM acting like an intern who can read through all existing literature quickly and plot the results, without understanding the nuances. Every published study carries limitations and simplifications that only a human expert recognizes, and an intern (or an LLM) has no way of grasping the unknowns, or the weight of the assumptions baked into published research.
This is because an LLM's answer closely resembles the mode: the most common answer among everything ever written on a topic, not a rigorously verified one. Ask an LLM, "As an occupational safety expert, what are the ten most stressful jobs?" and it will surface whatever's been repeated or gone viral most (say, mining, because one widely recirculated article named it). Ask an actual expert the same question, and you may get a completely different answer.
The same holds outside of facts and figures. Ask AI to design a logo, and you'll get something polished but generic. Even if you tell it to think outside the box, its version of "outside the box" is just the most common way people have described thinking outside the box in the past, not genuine novelty.
Conclusion
When we google something or type in a prompt, we should be conscious of one thing: what we're getting is what has been said or published the most, and that could be far from reality or novelty. Searching and prompting are genuinely useful when used for “modes”:
- finding the most popular things: places, restaurants, music, advice
- finding out what has been said or published the most
- finding "best practices" and well-established methods and procedures
They're the wrong tool when what you actually need is truth, expertise, or originality: none of which are the same thing as consensus.
Javad Seif
On a Turkish Airlines flight
July 30, 2026
Comments
Loading comments…
Leave a comment