We’re Using AI Wrong

Intelligence gets more interesting once you leave the chat.

“Reality is not always probable, or likely. But if you're writing a story, you have to make it as plausible as you can, because if not, the reader's imagination will reject it” - Jorge Luis Borges

I have a secret test for knowing when a technology has escaped Silicon Valley. Don’t laugh at me for this. But it’s not valuation. Market size or whether my friends in San Francisco have started talking about it.

I know when it’s crossed into ordinary life when it reaches my mother in rural Brazil.

I’ve watched this happen a few times.

First Facebook. My feed slowly filled with photographs of roses glistening with impossible morning dew, sunsets saturated beyond anything the sky had actually produced, and inspirational cards assuring everyone that good things happen to good people.

Then came Instagram Reels. Suddenly every phone call included updates about recipes she had discovered, strangers she was following, and miraculous cleaning tricks that apparently worked on every stain known to humanity.

Now it’s ChatGPT.

Every few days, a long message appears in our family WhatsApp group.

The latest one began:

“Life hasn’t always been easy…”

By the second paragraph it was quoting some ancient wisdom. By the third it had arrived at gratitude. The last paragraph circled back, gently, to our family .. to how distance matters less than love, and how we should never forget each other no matter where life takes us.

The strange thing is that none of it was wrong.

In fact, it was thoughtful. Warm. The sort of message most people would happily receive from someone they love. But as I read it, I felt something was off. Not because the words were fake. Because they could have belonged to almost anyone.

My mother has spent seventy years becoming exactly a unique person. She has a particular rhythm. She tells the same stories every Christmas. She worries too much. She never uses semicolons. She has certainly never written the sentence, “I hope this message finds you well.”

So I asked her:

“Mom… did ChatGPT write this?”

A laughing emoji appeared almost immediately.

“How did you know?”

A few nights later I told the story over dinner.

Nobody seemed surprised.

One friend had received a breakup message that felt composed , “hygienic” by any grammar standards, but emotionally interchangeable. Another was reading her college student essays that all sounded competent, sincere… and faintly related, as if they had all been written by students who had somehow lived the same life.

We laughed about it for a while.

Then someone asked a poignant question that changed the whole atmosphere at the table.

“Why does all the AI content feel so average now?”

At first, the answer seemed obvious. We had a few hypotheses. Maybe we’re still in the earliest stages of AI. Maybe the models are not that good. Or maybe people just hadn’t learned how to use AI well yet. At least that was the story I kept reading online when I was researching for this: “42 different ways to get the most of prompts and AI “(source).

All grand (mostly empty) promises because I don’t think the problem is the technology.

I actually think it’s the chat.

Let me tell you why.

---

From Chat to Prompt

People love to talk.

Long before we wrote things down, we explained the world to each other through stories. We sat around dinner tables replaying the day. We called a friend after a difficult meeting. We asked our parents what to do. We argued, gossiped, confessed, negotiated, flirted, taught our children and comforted each other … all through conversation.

Its safe to say chatting is our oldest way for thinking together. And it’s pretty effective.

When we don’t a place we’re travelling to, we call someone who’s been there before. When we’re unsure about what to do at work, we ask a mentor. Sometimes a single conversation changes the course of a career, a marriage, or a life. We instinctively associate dialogue with intelligence because, for as long as humans have existed, that’s how intelligence has most often reached us: through another person.

So it shouldn’t have been surprising what happened in November 2022. Large language models had existed in research labs for years. Most people had never heard of them. Then OpenAI did something so so simple. It put one inside a chat box. That’s it. I think this will be known in history as their Steve Jobs iPhone moment. Because all of the sudden, interacting with AI no longer felt like operating a piece of software, it felt like asking a question.

Within two months, more than one hundred million people had tried it. Headlines obsessed over the speed of its growth: “faster than TikTok”, “faster than Instagram”, “faster than almost any consumer technology before it”.

But the numbers, impressive as they were, missed the more interesting story. People were adopting a conversation with a model (not necessarily AI) as the interface to a new kind of capability: intelligence.

You opened a blank page, asked whatever happened to be on your mind, and something answered back. It didn’t matter that your question was vague, or incomplete, or poorly formed. The conversation continued anyway. It felt remarkably forgiving. Almost magical.

Once that became the dominant experience of AI, an entire ecosystem grew around it. Prompt libraries. Prompt marketplaces. Prompt engineers (remember this one positioned as the next frontier of new jobs?). Courses promising the perfect prompt.

The chat was a great invention. We collectively became better at talking to the AI. But … what we rarely stopped to ask was whether conversation itself had become the boundary of how we imagined intelligence and its capabilities. Which I keep asking myself: has the chat become to AI what the browser once was to the internet - the first interface so successful that we mistake it for the technology itself?

---

We're Different, We're the Same

Cosine distance between paired responses to the same creativity prompt, across 22 LLMs and human subjects. Low distance = responses are nearly interchangeable. Source: Wenger & Kenett, We're Different, We're the Same: Creative Homogeneity Across LLMs, 2025 (arXiv:2501.19361), Fig. 2.

So it turns out people much smarter than me have been studying this question for a while. I came across a paper by Emily Wenger (Duke University) and Yoed Kenett (Technion) that asks something very close to what we’ve been wondering: if people use different LLMs (GPT, Claude, Gemini, Llama, and etc) why do their ideas often end up feeling so similar?

Their finding surprised me because it helps explain a piece of the puzzle. Individually, LLMs can perform just as creatively as humans on standard creativity tests. But collectively, they’re far more similar to one another than humans are to each other.

That made a lot of sense to me. No matter how much I prompt, how much context I provide, or how much of my own perspective I bring into the conversation, the result always brings me back to the average. If you stop and think about it, that’s a bit insane. You have this smart thing talking back to you with the knowledge of everything that has ever been written, that no matter how much context, prompt, stories about yourself at the moment you provide, it will bring you back to be beautiful .. magical … middle?

Even when I bring my own perspective into the chat, the conversational interface itself encourages convergence. Why?

Imagine asking someone you deeply admire:

“How do I become a great founder?”

There are hundreds of good answers to that question. Read biographies. Talk to customers. Build things people ignore. Build things people love. Learn to hire. Watch how great companies make decisions. Raise money. Lose money. Develop taste. Ship anything before you ever know what the “thing” is. Just build stuff. Nobody agrees on the exact recipe. But everyone agrees on one thing. No serious person will ever say:

“Spend the next five hundred hours talking to the smartest friend you know.”

Maybe she’s a founder who built and sold a billion-dollar company. Maybe she’s an eighty-year-old Stanford professor who has spent decades advising entrepreneurs and lived through every technological revolution of the past half-century. Maybe she’s Steve Jobs himself.

Well .. that still won’t do it. In fact, it wouldn’t matter whether you were talking to the world’s smartest model, a billion-dollar founder, or Steve Jobs himself. Learning doesn’t happen through conversation alone. Think of the famous story of the Oracle of Delphi. When the Oracle declared that no one was wiser than Socrates, he didn’t just stop the search for truth. For years, Socrates questioned politicians, poets, craftsmen, and ordinary citizens, testing the Oracle’s claim against reality. The Oracle offered a story. The world became the testing ground for it.

And I think conversations matter a lot. Some of the most important ideas I’ve ever had began in conversations like the dinner one with my friends. But I shouldn’t mistake a conversation for the whole process of learning.

Talking to experienced people is one way of learning. Building is another. Watching customers struggle with your product is another. Making the wrong decision and living with the consequences teaches something no conversation can.

Reality has a way of introducing information that nobody (including the smartest person in the room) could have given you beforehand. That’s when I started wondering whether we had made a subtle mistake with AI. Maybe we had started confusing the window we were looking through with the thing we were looking at. Confusing the browser with the whole internet.

That’s one of the reasons I find it funny people selling online “the 50 prompts every founder needs.” Newsletters promising that one small change in phrasing would suddenly make the model think like a strategist, a therapist, a CMO, a CEO …and would give you a radically different answer.

To be fair, the opposite can happen. The longer a conversation continues, the more it begins to orbit its own center. Your pointy observations become normalized again to the middle. You can keep pushing by asking the model to be more creative, more original, more surprising … but you’re still working with the same conversational loop.

---

The Age of Composition

Forget the prompts. Forget the talking. Just for now. For the sake of this exercise. Somewhere inside the model are a billion conversations, a library of anecdotes, every book someone thought worth finishing. And the real magic was never how to talk to it. Maybe it’s more about how to reach inside it and find the one thing that's still yours.

I kept coming back to that quote from Jorge Luis Borges.

“Reality is not always probable, or likely. But if you’re writing a story, you have to make it as plausible as you can, because if not, the reader’s imagination will reject it.”

He was talking about writing. About fiction.

Life is unexpectedly messy. Coincidences happen. People contradict themselves. The most important moments often arrive without warning or explanation. A novelist can’t simply copy reality onto the page because the reader would find it implausible. Sometimes less beautiful. Fiction has to be more orderly than life. It has to fit a coherent story. To fit a prose. A romance.

Language models inherit this limitation too. They’re trying to produce the most plausible continuation of the conversation in front of them. Coherent. Fluent. Well-formed. Internally consistent. That’s precisely what makes them extraordinary to talk to. But the story you’re trying to write isn’t a language problem. Neither is the marketing plan for your startup. Or investing. Or founding a company. Or deciding whether to change careers.

Those are real problems that can’t be simply solved by the most probable answer. But they can be quite useful to explore. To challenge your assumptions. To reveal something true. And the truth has an inconvenient habit of arriving from places language alone cannot predict.

When a customer uses your product in a way nobody expected. An experiment disproves your favorite idea. A colleague notices the one assumption you’ve stopped seeing. A walk changes your mind. A conversation with someone you love.

Reality keeps interrupting plausibility. That’s why I’ve stopped thinking about AI as something you simply prompt or talk to. I’ve started thinking about it the way a composer thinks about an orchestra.

A composer doesn’t play every instrument. A composer decides which instruments belong, when they should enter, and how they should relate to one another.

The model is one instrument. Your experience is another. Documents you have. Memories. Conversations with friends. Data. Spreadsheet. A prototype you tested. Notes from your therapist. A book you read long ago. A week spent traveling and meeting strangers. Each contributes something the others cannot.

In this way, composition is the deliberate act of deciding what enters the thinking process and what remains outside it. It is deciding where the model should reason, where another human should judge, where a tool should retrieve information, and where only reality itself can settle the question. The goal is to stop asking language to discover what only real life can reveal.

Take this article, for example. I didn’t prompt Claude to write me an article. I also didn’t write alone in a google doc and then copy and paste it here.

It was composed. From small encounters that happened over the course of a few months. Some of them were barely worth noticing at the time.

• A note on my iPhone from almost a year ago with a single sentence “is chat too small of an interface for intelligence? Netscape was once thought of as the whole internet. ARE WE STUCK WITH THIS CHAT?” (I didn’t know what I meant. There was no argument. Just a thought that felt interesting enough not to throw away)

• My mother’s long WhatsApp messages

• A dinner with friends that I left thinking had simply been a good conversation.

• A set of AI research agents for work to collect academic papers, technical interviews, experiments, and industry discussions around topics I am trying to understand.

• A Borges interview quote a friend sent to me on Instagram just a week ago - and that generated the first drafts directly to Substack

• A long walk with my husband, rambling ideas, and asking him “Does your work feel more average now?” “Do you notice the same thing?”

• A workflow I’ve designed to gather material from different places: old notes in Google Docs, fragments from my phone, research agents, articles I’ve saved, sometimes ideas written in Portuguese, sometimes in English, often all mixed together.

• A few back and forth discussions with Claude on best title, exact wording, and structure (the translation from Portuguese to English sometimes is very poor).

• And then 3-4 days editing, fixing, cleaning up the article on Substack.

This article didn’t come from one single conversation with Claude. Or ChatGPT. In fact, the chat window was probably the smallest part of the process, when I asked to interview me. To challenge weak arguments. To find counterexamples. To pressure-test the logic. To help me discover where I was explaining too early. Sometimes it changed my mind. Sometimes I discarded its main counterpoint.

At no point was I relying on a single conversation. The article wasn’t sitting inside the model waiting to be summoned. It was distributed across memories, people, books, observations, research, walks, notes, experiments … and, yes, the model too. The work was knowing what each source could uniquely contribute, and knowing that no single one was enough.

And maybe that’s what we misunderstood from the beginning.

The chat was never the full story. It was one way of composing intelligence.