AI won’t fix bad thinking — use it to challenge you instead

So you’re using AI to write your product requirements, draft your stakeholder emails, and even create your roadmap presentations. You’re feeling pretty efficient, right? But here’s what I’ve noticed after working with AI for months: we’re creating mountains of documents.

Remember that scene from the science-fiction movie “Idiocracy” with the mountains of trash everywhere? That’s where we’re heading with AI-generated content. Bland, generic documents that check all the boxes but don’t actually help you make better product decisions.

The real problem: You’re using AI as a yes-person

Most product people I talk to are barely scratching the surface of what AI can help with. They’re using it to automate writing tasks, but they’re missing the bigger opportunity: using AI to think through their product decisions.

Here’s the thing — AI makes a terrible writer but an excellent editor. And there’s a crucial distinction here that Roger Martin, a writer and strategy advisor known for the book “Playing to Win” captured in his writing about editors versus writers.

“What makes for a great writer is the lived experience of being immersed in the subject,” Martin explains. You have deep, meaningful, and nuanced experience of your product, your customers, and your market. That’s why you should be the one generating ideas and making decisions.

But what makes for a great editor? Martin argues: “Emotional detachment, intellectual balance, and breadth of experience.” Sound familiar? That’s exactly what AI brings to the table.

Break AI’s confirmation bias trap

Here’s the thing about AI that most people don’t realize: it’s naturally wired to be agreeable. You’re not actively asking it to confirm your ideas, but that’s exactly what it wants to do. AI models are trained to be helpful, which often translates to being supportive of whatever direction you’re heading.

This creates a dangerous confirmation bias trap. You present your thinking, and AI enthusiastically builds on it, finds supporting evidence, and makes you feel smart about your approach. But that’s not what you need.

You need someone to challenge your assumptions, poke holes in your logic, and force you to defend your decisions. You need to actively break AI out of its “helpful agreement” mode and into “productive challenge” mode.

Here’s a practical tip: if you use ChatGPT, add this to your system prompt so challenging becomes the default behavior: “Always ask probing questions about my assumptions and look for gaps in my thinking before providing solutions or agreement.”

Alongside this, let me show you three ways I’ve been using AI as a thinking partner that actually improved my product decisions:

1. Get AI to critique your work from your audience’s perspective

Once you have a draft of a document you’re content with, have AI critique it from your audience’s perspective. Think of your target audience and be as specific as possible about the persona critiquing your work.

Here’s a real example from my experience. I had a slide deck for a service we offer to customers – one I’d been using and refining for years. The presentation defined our service, the problem it solves, and the benefits we deliver. I was pretty happy with it.

Then I asked AI to critique it from our customer’s perspective. What I found out is that we didn’t focus enough on customer benefits, didn’t have enough supporting data, and focused too much on the methodology, when it should have been all about the outcomes.

We weren’t reading it from the customer’s perspective, so it’s no wonder some of the feedback we got was “why should I care? This workshop can only help you.” After updating the presentation, we’ve seen improved feedback from customers.

2. Challenge your prioritization framework

Let’s face it, most prioritization exercises are structured guesswork, especially when you’re solving complex problems or developing new ideas. You’re choosing which bets to take in order to move towards a goal. But that doesn’t mean you can’t make them better.

Here’s how I used AI to strengthen a recent prioritization exercise. We had 25 service ideas from an innovation workshop that needed to be shortlisted for a stakeholder session. I created a prioritization framework with four dimensions:

• Strategic fit (internal) — How well does this align with our company strategy?
• Market opportunity (external) — What’s the potential market size and demand?
• Organizational readiness (internal) — Do we have the internal capabilities and political will?
• Execution feasibility (external) — How complex would this be to deliver?

After I scored all 25 ideas, I gave AI three specific roles: strategy advisor, CTO, and CMO. I asked each “persona” to assign its own scores with detailed reasoning behind every decision. Then I had it cross-check the answers and provide a summary of differences with adjustment suggestions.

The results weren’t wildly different from mine, but in a few cases I did adjust my scores based on perspectives I hadn’t fully considered.

This exercise also helped me articulate my thought process. As I am an intuitive person, some of my scoring involved judgment calls that I couldn’t easily explain to others.

Seeing AI’s reasoning forced me to clarify why I weighed certain factors differently or disagreed with specific assessments. This made me much better prepared to defend my prioritization to stakeholders — turning research-backed intuition into clear arguments.

3. Let AI interview you to develop your thinking

Sometimes you know there’s a problem but you can’t quite articulate what it is or how to solve it. This is where AI can act as your thinking partner to help you work through complex product (and other) challenges.

The key is giving AI clear ground rules and the right context. Don’t let it jump to solutions immediately. Instead, ask it to challenge your ideas with tough questions, identify gaps in your thinking, and push you to explain your reasoning.

Here’s how to do it:

1. Give it the full picture — Share data, context about your product/market, what approaches you’ve already tried, and what you expect from this conversation
2. Set clear interview parameters — Tell AI something like: “Interview me about this product challenge. Ask probing questions to help me think through the problem more deeply. Don’t offer solutions unless I specifically ask for them”
3. Remember the context limitation — AI can “forget” or mix up information from earlier in long conversations. If you’re building on previous discussions, summarize the key points again or start fresh
4. Get AI to help you craft better prompts — Before diving into the interview, ask AI: “What additional information would help you ask better questions about this product challenge?” Let it help you structure the conversation setup

To be honest, this one takes practice, as well as trial and error. AI naturally wants to be helpful and provide answers, but what you need is someone to ask the right questions and force you to do the thinking yourself. Using a system prompt like “Focus on asking clarifying questions rather than providing solutions” can help maintain this dynamic.

Tips for getting AI to challenge you

Here are the prompting strategies that work for getting better thinking, not just better writing:

• Give it enough context — Pretend you’re delegating to an intern or junior colleague. What background information would they need to do this task well?
• Be specific about the role — “Critique this from the perspective of a skeptical enterprise customer who’s been burned by vendor promises before” works better than “Give me feedback from a customer’s perspective.” Ask questions like: what is their take on the matter? What is their goal when they’re critiquing it? The more specific you get about the persona’s context and motivations, the better the challenge
• Ask for additional guiding questions — Tell AI to ask you clarifying questions that would make its response 10x more accurate. Or 100x more accurate. Experiment and see the differences
• Set clear goals for the interaction — Are you looking for blind spots? Different perspectives? Stress-testing your logic?
• Request specific output formats — Do you want bullet points? A structured critique? Questions for further exploration?
• Ask it to revise for relevance — After the first response, ask AI to review its own answer for accuracy and relevance to your specific context

But keep in mind: you’re ultimately responsible for the content you produce. AI should challenge your thinking, not replace it.

The bottom line: Think first, write second

The goal isn’t to eliminate AI from your writing process entirely. Some documents can and should be written by AI — let’s not kid ourselves. But beware of creating too much documentation just because you can.

Instead, use AI to help you think through your product decisions first. Challenge your assumptions, test your logic, and get fresh perspectives on problems you’ve been staring at for too long. Only then should you move to the writing phase.

As in all aspects of business, our resources aren’t infinite. The question isn’t whether you should use AI – it’s whether you’re using it to become a better product thinker or just a faster document producer.

So go ahead and block an hour in your calendar to try out one of the approaches presented in this article on your next challenge and see what happens to the quality of your product decisions. You might be surprised at how much better your thinking gets when you stop asking AI to agree with you and start asking it to challenge you instead.

What tools or approaches have you tried for getting AI to challenge your product thinking? I’d love to hear about your experiments in the comments.

Featured image source: IconScout