When Discretion is Significant...

The Marketing Challenge: Management has three alternative positionings for a new product and you need to determine which one is going to get you the desired results - sales without cannibalization, frequency of purchase or similar goals. Research Results: The research scores on the three individual positionings are barely indistinguishable! The preference data to make this decision are 67, 68, and 69, each. You realize that, despite a large sample size, these differences are not statistically significant?

As researchers, we expect our numbers to be exact and would prefer 67 to be different from 69. In actuality, the 67 score is statistically representative of a range from 64 to 70 (with a robust sample size).

What To Do:
This problem resembles the situation juries face in courtrooms. Juries are asked to make a determination of guilt or innocence - a binary choice. But, after hearing the evidence, it's uncommon for a jury to be 100% sure of either choice. They may be 90% sure that the individual is innocent and 10% sure that they are guilty. And yet they are instructed to be biased toward innocence.

In research, we have a similar conservative bias. We researchers would rather say there is no difference when one actually exists than to say that a difference exists when none does. Specifically, we don't want to say that one position is better than another without clear statistical evidence.

In business, choices have to be made. Early in the development cycle we have to choose a single position to carry through. Positioning a product in multiple ways is the surest route to failure, unless your messages are well-targeted to specific segments.

The best resolution to this issue is to use more sophisticated tools during research design. First, we can use a more sophisticated measurement model that relies on additional measures. So instead of just measuring purchase intent, we also measure expected satisfaction, intent to recommend, value or several other key global scales. This approach reduces the amount of measurement error in our scales and increases reliability.

An even more preferred design is to increase the sophistication of our methodology. Conducting a Max Diff analysis will force more discretion and make the differences between positions much more clear.  Decision Insight has used Max Diff successfully in hundreds of studies with very satisfying results. Alternatively, we can conduct a conjoint or discrete choice experiment, putting the respondent in a context similar to the actual decision space. The most sophisticated method is to use these higher-end models in a shopping context or other buying environment, asking respondents to make choices among real-life competitive products.

Relying on these three methods to increase data clarity, researchers can now provide clients with greater data discretion and strong guidance toward decisions while maintaining the high research standards required of our trade.

For more information call or e-mail Alex Sodek (800.800.2124 x232).