Why “Just Ask AI” Isn’t a Research Strategy
by Shawn Reed, Sr. Solution Architect, Code and Theory
In the last year, “ask an AI what our customer would think” started showing up as a directive in marketing planning decks. It sounds reasonable enough; the tools are fast, cheap (at least for now) and fluent. Paste in a concept, describe a target audience in a sentence or two and get what reads like a thoughtful customer reaction in mere seconds.
Unfortunately, what you’re getting is a fairly well-spoken guess that can quietly shape decisions that were supposed to be driven by evidence. Although the shortcut is tempting, the cost of taking it doesn’t show up until a campaign is already in-market, so it’s worth pulling apart.
Asking a chatbot vs. building a grounded persona
By using an off-the-shelf chatbot to act as your customer, you’re asking it to create an average by scraping the entire public internet for whatever pattern best matches the words you gave it. If you say, “Imagine you’re a 42-year-old working mom in the Midwest considering a new rewards card,” it will produce a response that sounds like what a 42-year-old working mom in the Midwest might say per the language in its training data, i.e., a stereotype with a plausible vocabulary.
But it doesn’t actually know anything about your customers. It doesn’t know how they behave in your app, what they bought last quarter, which offers they ignored, how often they switch banks in your specific market, or what they told your research team in last year’s qualitative study. The model can’t see those data points. And even if you use a purpose-built synthetic persona solution on the market, you’re still making compromises when the solution uses generic audiences rather than personas aligned with your business.
Built in partnership with Adobe, the Creative Intelligence System (CIS) begins with a client’s first-party data in Adobe Experience Platform, joins it to our proprietary identity graph and layers in bespoke qualitative research. From that, we build a trait map, including generic signals (age, geography) and domain-specific signals that matter for the category, i.e., propensity to switch banks, luxury travel frequency, rewards-program sensitivity, whatever the question needs. The strategist ends up talking to a persona representative of that real data.
Why off-the-shelf LLMs flatter your strategy back to you
A more subtle problem, and one I think is the most consequential to marketers, is the sycophantic nature of general-purpose LLMs. When prompted with a strategic angle (“We’re positioning this card as the go-to choice for busy parents who value simplicity.”) and asked to react as a busy parent, it tends to produce a version of your own pitch, lightly rephrased in the imagined customer’s voice.
In engineering, we call this strategy leakage; in practical terms, it’s the AI flattering you. It feels like validation, which makes it dangerous. The exercise inspires an unearned, unsubstantiated sense of confidence because you didn’t actually test anything.
CIS is designed at the architectural level to mitigate the AI’s overly agreeable disposition. The persona is constrained to only “know” what’s in a tightly controlled evidence set drawn from segment data. It has to admit when it doesn’t know something, and every response is audited before it reaches the strategist. If the answer echoes the marketer’s framing without supporting evidence, it gets flagged and either corrected or held back. By design, the system is allowed to disappoint you; that’s what makes its encouragement meaningful.
What “fidelity” actually means in plain English
Fidelity is the degree to which a persona’s response is faithful to the audience it represents. Not what sounds reasonable, nor what the marketer hopes is true, but what the data and research actually support.
High-fidelity personas do three things that low-fidelity ones don’t. They cite what led them to an opinion, so you can see the evidence trail. They flag uncertainties, instead of filling the gap with fiction. And they stay consistent; the same persona, asked a related question an hour later, doesn’t contradict itself.
Those three behaviors make a synthetic persona worth listening to. Without them, you’re running focus groups with a confident hallucination.
The differentiator
For marketing teams, the practical payoff is simple. You can stress-test creative against representative audiences before spending valuable marketing budget, and the resulting feedback is tied to real customers. That means fewer expensive rewrites post-launch, stronger pre-flight confidence and a research signal you can defend in the room.
For Adobe, CIS answers a question the market is asking loudly right now: What does responsible, accurate AI for marketing look like when woven throughout the Adobe stack?
For us at Code and Theory, it’s why we built CIS the way we did. “Just ask AI” will always be faster, but what’s the point of being faster if you’re just arriving at an untrustworthy conclusion? Fidelity makes all the difference, and it’s what makes a true synthetic persona a formidable research instrument instead of an eloquent mirror.