Your AI Thinks Your Idea Is Great. That’s a Problem.

I have a problem with AI.

It thinks too many of my ideas are good.

Maybe you’ve noticed this too.

You explain something you’ve been working on. You give the model the background, the problem you’re trying to solve, maybe even the architecture you’ve spent months building.

And it gets it.

Really gets it.

Then comes the dangerous part.

It tells you why you’re right.

“This is a strong concept.”

“There’s a real opportunity here.”

“What makes this particularly interesting is...”

It feels useful because the analysis is intelligent. It understands connections other people might miss. It can articulate the idea better than you can.

But understanding an idea and challenging an idea are two different jobs.

And AI models have a tendency to agree with the person they’re talking to. It has been measured, and it has a name: sycophancy.

That makes them potentially dangerous business partners.

Why it agrees with you

Two things are happening, and only one of them is the model’s.

The model’s part: these systems are trained to be helpful, and agreement can read as helpful to the people rating them. A 2023 study of five leading assistants found that they consistently matched the user’s beliefs over the truthful answer, and pointed to human preference judgments as a likely part of the cause. Sharma et al., “Towards Understanding Sycophancy in Language Models”

Your part: you handed it the framing. You said what the idea is, why it matters, what you’ve already built. A model working from your description is working inside your description. It does what a good assistant does with a brief. It makes the brief better.

In April 2025 a maker of these models did something about it in public. On April 25 OpenAI rolled out an update to GPT‑4o in ChatGPT that, in its words, “made the model noticeably more sycophantic.” It began rolling the update back on April 28, calling it “overly flattering or agreeable.” One factor it named: “User feedback in particular can sometimes favor more agreeable responses, likely amplifying the shift we saw.” (OpenAI, “Expanding on what we missed with sycophancy,” May 2, 2025; “Sycophancy in GPT-4o,” April 29, 2025) That was one product from one company, not every model. It does show a maker treating over-agreement as a defect worth a rollback.

Neither of those goes away on its own. You can only change the job.

I ran into this with something I’m building

I’ve been developing an architecture that uses specialized AI workers to do different jobs.

One application publishes finished content to YouTube and Instagram.

Another is a larger suite where specialized workers can research, write, design, build and coordinate work, and, through a bundled tool, publish it.

Then something unexpected happened.

The same underlying architecture started powering a language-learning classroom.

The classroom maintains information about the learner, has its lessons written around what the learner has shown they can and can’t produce, and works alongside multilingual AI voice for the actual conversation.

The same way of building has now been used for a classroom too. Research on how to teach from a screen, the lesson illustrations and the classroom app itself all came from the same team of workers.

My builder brain loves this.

One architecture.

Three products.

Workers shared across two of them.

Shared improvements.

Build something better in one place and potentially improve the others.

You can probably already hear an AI assistant responding:

“This demonstrates the generality of the underlying architecture and could create significant economies of scale...”

And that’s exactly the problem.

Maybe that’s true.

But I built the thing.

Of course I can see all the reasons it’s interesting.

I don’t need an AI to become better at explaining my own enthusiasm back to me.

I need it to find the reason I’m wrong.

So I changed the question

Instead of asking whether the language classroom was a good product, I essentially told the model:

Assume it’s dead a year from now.

Not struggling.

Not pivoting.

Dead.

Now explain why.

That changed the conversation immediately.

One possible answer was uncomfortable:

Maybe the classroom produces beautiful, personalized lessons, but doesn’t teach languages meaningfully better than simply opening an AI voice app and saying:

“Teach me Thai for 15 minutes.”

That’s a much better question.

Suddenly all my architecture had to justify itself.

Does maintaining learner state actually improve acquisition?

Does having a research worker produce better learning, or just more impressive lessons?

Do visual exercises improve recall and spontaneous production, or do I like them because they’re cool?

Does the larger worker suite make the classroom better for the learner, or more interesting for the person who built it?

Those are questions I want my AI asking me.

The move has an older name. Planners call it a pre-mortem. In Gary Klein’s version, a team assumes the project has just failed and then generates plausible reasons for why (“Performing a Project Premortem,” Harvard Business Review, 2007). He built it for teams, to make it safe for people to voice doubts. Using it as a prompt for an AI is my adaptation, not something he tested. The underlying idea is older: researchers call it prospective hindsight, explaining a future event as if it had already happened (Mitchell, Russo and Pennington, 1989). In their first experiment, explanations for events treated as certain tended to be longer and to contain a higher proportion of episodic reasons (reasons framed as specific events). I can’t tell you the paper proves this works on an AI. What I can tell you is what I felt in a single conversation. “Will this work?” invites a defense. “It’s dead, so why?” invites an autopsy.

Then came the important question

After the model gives you the strongest argument against your idea, don’t defend yourself.

Ask:

“What would have to be true for that argument to be wrong?”

Now something useful happens.

The criticism becomes a test.

In my case, if people using the full learning system retain more language, spontaneously produce structures they previously couldn’t, and transfer those structures into unfamiliar conversations better than people simply chatting with an AI voice model, the additional architecture may be earning its place.

If they don’t?

That’s important too.

Maybe I’ve built an impressive machine to solve a problem that required a much smaller machine.

That’s information I’d rather have now.

One sentence every piece of complexity has to finish

Take any feature you’re proud of and try to complete this:

“Without this ___, the user is less likely to ___ because ___.”

For the classroom: without the learner record, the learner is less likely to produce a structure they couldn’t before, because ___. I can write a hopeful ending to that sentence in ten seconds. What I can’t do yet is point at a learner who did it. The feature stays on the list until the blank has something real in it.

The step my first draft of this piece left out

Here is where I stopped when I first wrote this down: give the AI permission to try to kill your idea.

That was one step early.

This week I turned the same method on something I had already written down as true: my own results.

I had AI write devil’s-advocate reviews of my work. They agreed with the framing I handed them, in a friendly tone. Some pushed on demand and adoption, which is close to the middle of an idea, but none of them went to a record. And one claim ran through them: that a large amount of content had gone out “unattended.” That word was in the write-ups.

As it stood: a large volume of content, published unattended.

What the re-check found: The log records who logged a row, not who published it. It can’t settle “unattended.”

So I had an AI check the claim against my posting log. It came back with a verdict: the claim was wrong.

That was a claim too. I nearly acted on it.

The same AI, going back over its own check, found that the log answers a different question. A separate research session I asked confirmed it independently: the log records who logged a row, not who published it. The log labels one post as a fully automated publish; it doesn’t record how the others went out. The platform’s own totals didn’t contradict the volume claim. “Unattended” couldn’t be settled from the log at all.

The reviewers had made a claim nobody had checked, and then the checker’s check was wrong too. The log couldn’t settle it. The record that could is a different one, and I haven’t yet gone back to it.

A criticism is a claim. Check it like one. So is the check.

The same thing happened once more, smaller. A reviewer recommended reordering a sales page. I tested the reorder with fresh readers, and it did not work. Another finding said the page’s wording read as unfinished. I went back to the page: the word sat inside a line quoted from someone else, and the finding was withdrawn.

About those fresh readers: they were AI. I opened new sessions, gave each one only the page and no history, and asked what it thought the page was. They are not customers. They tell me where wording confuses. They tell me nothing about whether anyone will pay, and I would be misusing them if I treated their answers as demand.

The goal isn’t to make AI negative

That’s the other trap.

You don’t want an AI that automatically tells you everything is terrible any more than you want one that automatically tells you everything is brilliant.

Neither is intelligence.

What you want is friction.

If your evidence supports the idea, the model should be able to say so.

If the evidence doesn’t support it yet, it should distinguish:

what you’ve built

from

what you’ve proven.

That’s an enormously useful distinction.

In my case, I have good evidence that one underlying AI architecture can perform several very different kinds of work.

That’s technically interesting.

I do not yet have equivalent evidence that all three applications deserve to become profitable products.

That’s a business question.

And no amount of architecture makes the answer automatically yes.

The method, in five steps

1. Describe the idea and what you’ve done with it, in plain words. Include what you have actually seen happen, because that’s the only part the model can’t improve.
2. Tell it the idea is dead in a year and ask for the strongest reason. Don’t ask for pros and cons. Ask for the autopsy.
3. Ask what would have to be true for that reason to be wrong. The criticism turns into something you can look for.
4. Check the criticism against the real thing before you believe it or dismiss it. Make the model quote the line it rests on, then read that line yourself.
5. Write down what result, by what date, means stop, before the results arrive. After they arrive, the target moves toward whatever you got.

Steps 1 to 3 are the method from the top of this piece. Steps 4 and 5 are what that audit added.

The prompt pack

Six prompts. Each has what it’s for, a block to paste, what a good answer looks like, and the sign that it went wrong. Prompt 1 is the wording I started with. Use them in a chat that has your material in it, except Prompt 5, which needs the opposite.

1. The pre-mortem

For: getting the strongest case against the idea, and the one thing that could disprove it.

Paste:

I want you to act as an adversarial business partner, not a supportive assistant.

I’m going to describe an idea I’m considering or something I’m already building.

Before analyzing it, assume that one year from now it has failed completely.

Identify the most likely reason it failed. Focus especially on assumptions I appear emotionally, financially or intellectually invested in.

Separate what I have actually demonstrated from what I merely believe to be true.

Look for places where technical capability could be mistaken for customer demand, where something impressive may not be valuable, and where added complexity may not produce a better outcome.

Do not disagree with me simply for the sake of disagreement. Make the strongest evidence-based case you can.

Then identify the smallest real-world test that could prove your criticism wrong.

Finally ask: What would have to be true for your argument to be wrong?

My idea is:

[PASTE YOUR IDEA HERE]

A good answer: one main reason, not a list of ten. It names something you are attached to. It ends with a test you could run this month.

It went wrong when: the criticism is about something you already listed as a weakness, or the tone stays warm and nothing is checked against your records. That’s a performed attack.

There’s another line I’ve started liking too:

If this were your money rather than mine, what would you stop doing tomorrow?

That question has teeth.

2. What would have to be true

For: turning the criticism into something you can go and look for.

Paste:

What would have to be true for your argument to be wrong? Give me things I could actually observe, not opinions. For each one, tell me what I would see if it were true and what I would see if it were false.

A good answer: three or four observations, each with a “you’d see this” and a “you’d see that.” At least one you can check with material you already have.

It went wrong when: the answers are vague (“if the market responds well”). If you can’t tell what you’d see, it’s not a test yet. Ask again with a number or a date it has to name.

3. The evidence audit

For: separating what you’ve shown from what you believe.

Paste:

List every claim I have made about this idea, in my words. Mark each one DEMONSTRATED, BELIEVED or UNKNOWN. For every claim marked DEMONSTRATED, quote the exact line from my material that shows it. If you can’t quote a line, it is not demonstrated.

My material:
[PASTE YOUR NOTES, NUMBERS OR DRAFT HERE]

A good answer: most of your claims land in BELIEVED. Each DEMONSTRATED has a quote you can find in your own text.

It went wrong when: most claims come back DEMONSTRATED with no quotes, or with quotes that don’t say what it says they say. Search your material for the quote.

4. Verify the critic

For: checking a criticism before you act on it, including one you agree with.

Paste:

Before I act on your criticism, quote the exact text in my material that supports it. If there’s no such text, say “no support in what I gave you” and withdraw the criticism. Don’t fill gaps with what’s typical for ideas like mine.

A good answer: some criticisms come back with a quote and stay. At least one shrinks or gets withdrawn.

It went wrong when: every criticism survives, or the “quotes” are paraphrases. Look up the quoted line. If it isn’t there, or it sits inside a quotation from someone else, the criticism goes.

5. The cold reader

For: seeing where your wording confuses someone who has never met your idea.

How: open a new session with no history. Paste only the page or document. Nothing else.

Paste:

You’ve never seen this before. Read what’s below, then answer five questions, in order, without being polite.
1. What is this?
2. What would you use it for?
3. What would make you pay for it?
4. What confused you?
5. Why would you not buy it?

[PASTE THE PAGE OR DOCUMENT HERE]

A good answer: it misdescribes something you thought was obvious. That’s the finding.

It went wrong when: you treat the answer as demand. This reader is an AI. It is not a customer, and its answer to question 3 is not evidence anyone will pay. It shows you wording. Only people show you demand.

6. Kill criteria

For: deciding, before you see any results, what would make you stop.

Paste:

Help me write kill criteria for this test before I run it. For each: what result, by what date, means I stop. Each one has to be something I can measure with something I already have. Write them so that I can’t reinterpret them after I see the outcome.

A good answer: two or three criteria, each with a number or an observable event and a date.

It went wrong when: a criterion contains “meaningful,” “significant” or “enough.” Those words are how a stop becomes a pivot after the results are in.

Where the method fails

A common fix is to ask it to attack. In my case that got me a performed attack: objections that fit my framing, delivered politely, with the record left unchecked. I came away feeling I’d been tested.

The fix is in Prompts 3 and 4: criticisms that can be shown wrong, each resting on a line of your material you can find yourself.

Other places it breaks:

The critic is guessing. It can produce an objection with full confidence about something your material never says. That’s how a withdrawn finding gets written in the first place.

The checker is guessing too. My first check said the claim was wrong, and it was answering from a record that couldn’t answer the question. Ask which record settles it before you trust the verdict.

The fresh readers are AI. They find confusing wording. They can’t find a buyer.

You write the stop condition after the results. Then it’s not a stop condition.

You take one direction as the answer. An AI that calls everything brilliant and an AI that calls everything doomed have the same flaw, and the second one feels smarter.

One page to keep

• Assume it’s dead in a year and ask why.
• Ask what would have to be true for that reason to be wrong.
• Sort every claim: demonstrated, believed or unknown.
• Make the critic quote the line it rests on, then find that line.
• Show the page to a fresh AI reader, and remember it is not a customer.
• Write what result, by what date, means stop, before you look.

Your AI shouldn’t be your cheerleader

The most useful person in the room isn’t always the person who immediately understands why your idea could work.

Sometimes it’s the person who looks at the same evidence and says:

“Okay. What would convince us this isn’t true?”

AI can play that role better than it does by default.

But sometimes you have to explicitly give it permission to stop agreeing with you.

Give it that permission. Then check what it says, because the one that sounds sure is the one that should send you back to your own records.

Some ideas die because nobody attacked them. Some survive an attack that was wrong. You find out which from the real thing, and a model can’t stand in for that.

thequietai.com