Concept testing is too often designed to confirm a decision that has already been made, which turns research into a confirmation exercise. By the time consumers see the concept, it has cleared budget allocation and stakeholder approval, and pulling it back is politically expensive whatever the data shows.
The pattern shows up in three places. The stimulus written is to sell an idea rather than test it. Polished copy and framing replace a neutral presentation of the concept, in anticipation of a yes. The survey questions lead the respondent too. Prompts like “How excited are you” or “How highly would you recommend this product” produce biased answers. Most damaging, the test is sanctioned too late to alter any decisions, so it becomes a formality that feeds numbers into a deck instead of shaping the build or launch.
The result is confidence without evidence. A test designed to return high purchase scores does exactly that, with no genuine market pull behind them. This false confidence lets a failed product be dismissed as a fluke while the real culprit, a design flaw, goes unaddressed. The cost shows up in production runs, media plans and shelf commitments built on a score that was never allowed to disappoint.
Before funds are committed to an idea, concept testing research answers a single question. Is there any real reason a customer would actually use this specific product over one they already do? A good test settles it with evidence.
A good concept test reads four signals. Relevance shows whether the concept solves a problem your audience actually has. Differentiation tells you if your product is actually novel or just a variation of existing options. Credibility shows whether your claims are generic or hard to believe. Done early, this protects development spend, ranks a pipeline on evidence and gives leadership a shared reference point before capital moves.
A trustworthy test also diagnoses. It should pinpoint what is dragging a concept down, whether the core idea, the price, or the positioning. It must also be able to return a no early on, because a test that cannot fail is simply confirming a pre-made decision.
Confirmation testing has a legitimate place, provided everyone knows that is what it is. When the go decision rests on other evidence, a test can still lower execution risk. For example, the core idea of a product before its launch may be viable, but peripheral elements like unrealistic claims, ambiguous messaging, or an unsuitable product name may emerge as major flaws that confirmation testing can flag before launch. It also settles choices between two ad routes, concept testing for package designs or price points, where a calculated answer beats a debate.
The same testing builds internal alignment. Testing can help give functions like marketing, sales and leadership a consolidated, data-driven reference point, eliminating opinion-driven debates that may otherwise stall a launch. It also suits budgeting. A full evaluative study on a lower stakes call, such as seasonal execution or a product variant, can eat up cost unnecessarily, and confirmation testing is the proportionate measure. Even when the outcome is highly expected, confirmation testing leaves a documented record of customer reaction alongside internal opinion, which supports post-launch marketing and accountability.
Confirmation only works when the decision has support elsewhere, such as in prior research, earlier launches or existing market evidence. A test alone cannot substitute for evaluating a decision that genuinely needs evaluation.
A concept test does not have any real meaning until it is set against a benchmark. Purchase intent and value perception act as relative metrics, and a score reported in isolation gives teams no real way to judge whether it is good, average, or weak.
Three reference points help judge a score. Category norms, built from earlier tests run using the same method in the same market, show typical performance for that product category. A brand’s own launch history shows whether the concept testing methods stronger or weaker than the products that reached the market, with real outcomes as a guardrail. Competitor concepts tested under similar conditions show relative appeal directly instead of assumed appeal.
Without that context, a score of, for instance, 70% purchase intent could be strong, or merely average, depending on how comparable concepts have scored in that category and market. Arbitrary internal thresholds, such as assuming a score above 60% is good enough, have no factual basis and differ greatly by survey method, category and country.
Benchmarking also settles whether a small difference between concept scores is meaningful.
Claimed purchase intent is merely a stated measure and not one which can be observed and it often exaggerates real world stakes, especially for newer, unfamiliar concepts, or when the respondents are asked to answer based on a behavior they have not performed before.
There is a structural reason why this happens. Respondents are usually asked to answer imagined and hypothetical questions in an environment that has lower stakes such as the lack of money or competing options involved, so saying “I would surely buy this product” has no real friction which is an important part of a real-life purchase. Stated intent can therefore run well above real adoption. One industry practitioner’s analysis found many competing brands earning top-two-box interest from the same respondents, which cannot all convert.
New and one-of-a-kind products suffer most. Respondents have no reference point and little basis for predicting their own behavior, so optimistic, agreeable answers become the default.
None of this makes claimed purchase intent useless, but it should be read comparatively and directionally. A stated purchase intent of 70% does not simply mean that 70% of the market will purchase the product. Ideally the score must be tested against a behavioral proxy such as a small-scale pre-order with real money involved, which may pivot the choice closer to a real-world scenario of how people behave after launch.
Teams are often taken aback by the underperformance of a product post launch because they had treated purchase intent as forecast. The cost is asymmetric: a false positive commits production, media and shelf space to a product the market does not pull, while a false negative only delays a good idea.
Design decides whether a test can be trusted, and diagnostics decide whether this particular result can be. Four questions separate a prediction from a guess. Has the method been calibrated? Score it against past launches with known in-market results, because a test that could not have told you which earlier launches would succeed has no claim on the next one. Is the result repeatable? A second wave or a split sample should return the same ranking, and if it does not, the gap between concepts is sampling noise. Who sits behind the number? At 15% appeal, a 200-person sample leaves about 30 likely buyers, so the forecast rests on far fewer people than the headline sample suggests. Are the respondents real? Bots, duplicates and speeders inflate positive scores, which makes panel verification and in-survey quality checks part of the reliability question.
A result that fails any of these should be read as direction, not forecast.
A reliable concept testing framework rests on three elements: blinded comparative sets, benchmarks set against category norms, and behavioral proxies that check stated intent against how customers respond in real-life.
Blinded comparative sets and neutral stimuli remove the largest source of distortion, a stimulus written to sell an idea instead of testing it. Every concept goes to a separate group of respondents, one at a time, with no signal about which version is internally preferred, and persuasive or aspirational language is stripped from the stimulus. This approach isolates realistic reactions to each concept rather than employing a comparative bias shaped by which concept is revealed first.
Setting benchmarks against category norms turns an isolated score into something useful. A reference point built from earlier testing in the same category, market and method stops teams judging scores against random thresholds decided internally.
Behavioral proxies close the gap between what people say and what they do. Stated intent overstates real-world stakes, so the read needs a second signal, whether a small real-money commitment or an implicit measure of emotional response. Borderless Access built InnoSight 2.0 around this gap. Each concept goes to its own group of verified, well-profiled consumers, and facial coding and voice tonality sit beside the ratings to show where a claim lands with delight and where it draws quiet skepticism . AI-captured country and category codes then show whether the concept speaks in the right register for that market, so the score is read against what is working there now. Researchers interpret the pattern and say what to change, which lowers the cost and time of repeated concept revisions. In a packaging test, that combination can reveal a design that looks appealing yet reads as less premium once emotional response is factored in, a gap that stated intent would miss.
This layered reading matters most in packaging and design decisions, where stated intent often cannot separate two options.
Conducting a concept test that actually provides you with action-driving results requires a framework that links the test to the decision before writing the stimulus, allocating the test earlier in the timeline and adding in a comparison set as a headstart.
A strong framework depends on sequence: the decision first, an early test and a built-in comparison.
AI-enhanced concept testing shortens the time between writing a stimulus and getting a viable read, without displacing the judgement needed to analyze the result. Three applications drive the shift.
Synthetic data remains a screening aid, and treating it as a substitute for real respondents is where teams get burned. InnoSight 2.0 keeps AI on the supporting side: the read comes from verified consumer panels, and researchers who know the category and market interpret the result. BA’s recent piece on AI in concept testing covers facial coding and voice tonality in more depth.
Explore how AI and human intelligence help brands test concepts, uncover local consumer preferences, and build products that resonate across diverse markets. Download the eBook
Concept testing and A/B testing are often used interchangeably, but they sit at different stages of product development and answer different questions. Concept testing survey evaluates whether a product should exist at all and whether the idea deserves to move ahead on prospective market performance, using mockups, prototypes or descriptions tested with a sample of customers. A/B testing evaluates different versions of an already built product to determine which one performs best. It uses measures of real behavior like conversion and click-through rate instead of stated purchase intent.
The final data that is produced by both also differs. Concept testing uses metrics of stated intent, like purchase intent, perceived value or appeal that are collected through surveys. A/B testing uses observed behavior from live traffic: real users see different variants, and what they actually do is measured, instead of what they report.
Timing also separates the two. Product Concept testing sits earlier in the development timeline, which is also why is cheaper: there is still room to revise or scrap the idea. A/B testing happens much later, once a testable form of the product exists, and refines the existing idea instead of forcing an overhaul.
Both methods matter in product development, and they are complimentary, not interchangeable. Concept testing decides whether a product gets built, and A/B optimizes it afterward.
Three published examples, each anonymized, show what changes when a test is built to decide.
A CPG company needed to align the messaging and product concept for its dishwashing liquid with consumers’ day-to-day challenges. Concepts were scored on uniqueness, relevance and understandability through online surveys with a pre-profiled panel across South Africa. Most of the brand’s consumers used it for fresh fragrance and easier, faster washing, and they wanted a strong natural-ingredients association in both pack and product. The company aligned its messaging and concept to those needs, building brand associations that hold attention and support purchase.
A CPG major weighing two product design options for a craft beer needed a clear read on product-market fit in the UK. Both were evaluated on likability, uniqueness and purchase intent with a UK panel. Taste and premium quality drove brand choice, and one option came out ahead, its package design, color and style creating an impression of clarity that made it the preferred choice. Purchase intent worked as a relative read between options, and it led to the most preferred pack for launch.
A travel and tourism major wanted to test new and existing concepts against key performance metrics to shape its product mix and pricing. Among affluent leisure travelers in markets including the US, UK, KSA and the UAE, 35 concepts were tested in a design where each participant saw three. Preferences varied by country. Sports, adventure and heritage offerings travelled well, art-related offerings scored low, and the optimal price band was wider in the US and UK than in KSA and the UAE. The company used the read to bundle offerings into packages built for high-value experiences.
Each test had room to change the answer. It ranked options, showed which local signals mattered and told the team what to build, fix or bundle. That is the standard worth holding every concept test to: it can say no, it explains what to change, and it gives leadership a reason to trust a yes. Companies that build that discipline stop using research to decorate decisions and start using it to make them.
Stop validating assumptions. Start identifying winning concepts with evidence-backed consumer insights. Talk to our concept testing experts.
What is the timeline for concept testing?
A standard online study typically fields in days and reports shortly after, while multi-market designs, added qualitative work or richer emotional measures take longer. Sample size and audience complexity drive the schedule, and testing early costs less time overall than a rushed test at the end.
What is an adequate sample size?
Many teams work with roughly 150 to 300 respondents per concept for a reliable directional read. Niche or B2B audiences may run smaller, and sub-group analysis needs larger samples. What matters most is the interested subset: at 15% appeal, a 200-person sample leaves about 30 likely buyers behind any conclusion.
When should purchase intent be flagged as misleading?
Flag purchase intent as misleading when there is no benchmark to compare it against, when a unique concept has almost no reference point, or when the stimulus is non-neutral and persuasive.
How do you combine qualitative and quantitative methods?
Run the qualitative round first. A handful of interviews will show how consumers actually talk about the category, and that wording goes into the survey. The quantitative wave then puts numbers on it, showing how widely those views hold and how the concept scores against benchmarks.
When should you skip concept testing?
Skip concept testing when the decision is irreversible whatever the results, when there is no time for findings to change anything, or when a similar analog already has reliable performance data.
© 2026 Borderless Access Pvt Ltd. All Rights Reserved.
