Our opinion on... ...why quant research is in the dark ages We view market research in four buckets: 1. Probative, where the object is to learn what we don’t know 2. Evaluative, where the object is to confirm what we do know or have done 3. Metric, where the object is measure (snapshot) attitudes and behaviours 4. Predictive, where the object is to determine a future outcome (including anything that asks respondents what they will do or think in the future) While great advances have been made in the first three buckets, especially in terms of ethnography, psychoanalytics, neuro-research, metric research has taken a step backwards and predictive research lives for the most part in a fantasy world stuck in the dark ages.
Why a step backwards for metrics? Mostly because the fundamental principle on which statistical research is based — randomness — has been discarded in favour of on-line panels and similar non-random mechanisms. In spite of what they say: if it’s not random it’s not right. The fact that some say they have proven that there is no difference in outcome between random and on-line panels, says more about the inadequacies of random sampling execution, than the accuracy of on-line panels!
Why is predictive research in the dark ages? Aside from the sampling flaws discussed above, most predictive market research does not take into account the modern understanding of consumer behavior. Most studies
416.967.3337 www.proteanstrategies.com
Protean Strategies is a Toronto based management consulting firm. Since 1997 we have been helping large and small companies convert brand value into higher margins and bottom line profits by understanding their stakeholders needs; building powerful strategies; and aligning business practices with marketing and sale to a common goal.
ask the same questions that were asked in 1952 when consumers lived in a much different world: their lives were more stable and the world around them was more predictable. The housewife knew pretty much exactly what would be on the grocery shelves and how much it would cost. This meant that she might be able to say what she would buy under a single changed circumstance. Today there are very few ―housewives,‖ and it is almost impossible to know what will be on the grocery shelf tomorrow, let alone in three months time. It is utterly impossible for a person to say with any certainty what they will do or how likely they will be to do it. Furthermore, we have learned how people process information, and that deconstructing intentions (as we do in surveys) is not an effective way to speak to today’s consumers who think holistically and make pre-linguistic determinations. People can effectively answer about an idea, but have trouble honestly answering about parts or components of an idea. The sum of their answers when asked to evaluate parts of an idea do not equal the answer when asked to evaluate the holistic idea. This is just the tip of the iceberg. Modern neuro-marketing theories and behavioural economic findings support the idea that people have a very difficult time of predicting their future behavior and hence the choices they may make. However, there is considerable data to suggest that people are pretty good at predicting the behaviour of a crowd of people. The trick is to be able to aggregate the predictions of many people. The answer has been shown to be prediction markets. And Protean strategies is at the forefront of using prediction market theory for consumer research. 1 In case you’re wondering, a viral expansion loop is a state which grows by itself because, to use the social network example, each member invites other members who invite other members, and so on. And while this is exciting in the context of the Ethernet, it differs only in speed, from the growth of telephone networks and later mobile phone networks. It is interesting to point out that telephone growth came to a rapid and unfortunate halt when a new technological paradigm was introduced – will this happen with cell phones or the internet itself?
Call or email us today for more information on our suite of brand and business consulting services Protean Strategies 80 Cumberland Street, Suite 1503 Toronto M5R 3V1 Canada 416.967.3337 Bernstein@proteanstrategies.com www.proteanstrategies.com Page 2