I was wondering ... where are those research results coming from. The BCG has conducted a research, the exact wording is "BCG research confirmed ..." - no mention of the research design, the sample size, the way the online panel (and I am sure that its was online) has been constructed. As if it would matter when a big name such as BCG is behind the research.
Well, it does. If any experiences from Japan would show only a glimpse of what we could expect in China, where the concept of luxury is much younger, if underdeveloped and overly status focused (and no, they do not aim at individualism, the same way that Japanese do not even after 40 years), then China can be expected to be much less transparent when it comes to validating the relevance and representativeness of a sample. Validation and representative samples, relevance. Yes these words matter when it comes to research design. If they do not matter anymore, then research becomes a meaningless reproduction of signs, a simulation of meaning. Something that marketing departments do to confirm what they have done so far, do get KPIs (key performance indicators, .... how this word has distorted so many minds that were once able to think logically and with reason), to make sure budgets are approved, to make sure that no one finds out that the decision makers approving a research design are in no way able to judge the validity of the data, nor the vendors, nor able to judge the complexity of the issue.
There is an issue. Explanations and a deep analysis would be too much. Let us just recapture some of my favorite "symptoms", things that are far too common to be ignored, yet no one is doing anything about them, as the KPIs are in order.
- Online research panels in China where hordes of Chinese middle class consumers are faking IP-addresses using special software to fill out survey forms of research companies to be sold to major luxury companies. Faking their location, faking results, "yes, of course we buy 5 LV bags a year, yes our income is above this, yes we love this brand, yes we live in Shanghai (in truth deep country side, no money, sitting in front of a laptop)."
- Research companies that are painfully aware of the fact, yet do not care, as it enables them to come up with the right cost/survey sheet ratio. not one thought about the quality or the reasonable doubt "would a person with this high an income really spend their times on online surveys?", or the more apparent doubt "If I know I can make money with faking results, why would I not?"
- Research companies in Japan that research topics such as fashion and luxury that they know nothing about, asking the wrong questions, interpreting the answers in the wrong way, yet never questioning their own ability to do so.
- Research budgets that are limited and ask each vendor to come up with solutions fitting the budget that is decided already, without knowing if the budget is enough to produce any quality results. It is like going to Louis Vuitton and H& M asking each to come with a proposal for a handbag, and telling both, the budget was approved, we have 100 Euros. What we want is a high quality bag that holds for 10 years, made out of leather. LV's response is, sorry guys, we have what you want, but at 700 Euros and above, and we doubt anyone can produce what you are asking for at that price. The gross thing is to EXPECT that LV might somehow magically provide better quality at the same price.
- And to conclude the very short list (that could go one endlessly), my personal favorite. Taking what the consumer says in online research at face value. Asking very creative questions such as "What is the reason you bought this handbag?" - options: price, design, quality, brand recognition ... how about giving this option for a change: "I bought this handbag from Chloé because I like the brand since high school after I realized that it is higher in reputation than LV that almost everyone has in high school and in university, but only if they are not too oshare. I realized that office ladies with taste have the brand, but basically it was because all my friends have been talking about the brand for years, and because I see it in advertorials for the last 3 years in ViVi, Sweet and also in Vogue Girl, and my favorite models have featured it on their blogs. I feel safe with the choice and no one will criticize me for choosing the bag. Also, everyone at my school had See by Chloé and I thought that by buying the real thing, they would respect me and look up to me a little." - please cross: I somewhat agree, I agree, totally agree, not agree, not agree at all??? How can I somewhat agree with this? How about giving choices like social anxiety, safety of choices, reputation of the brand to be associated with a style group that reads certain magazines. Well, they did not teach these questions at business school. They did not mention during the MBA that in Asia consumption is more about social conformity, social values, social pressure, anxiety of making choices that one is critcized for, than about presonal preference.
the list goes on.
What you need is expertise. Expertise in the subject matter and involvement with actual consumers. Its like investigating a crime scene. You need an expertise in criminology to understand the problems coming with the research. You need expertise, experience, knowing how to ask the right questions and why. How to we judge bias? How can we check the quality of research in Japan. There are ways. There are simple principles. In the next weeks we will introduce them, breaking down the complex market into simple arguments, that make sense. Research is not complocated. All you have to do is use your brain and ask simple questions such as "Do I really believe this?" "What if she is lying?" "How do I know she is telling the truth?" or the most important thing: "Does her answer matter? Is she the one that we should ask this question?" - Think, does asking the shop staff of a convenient store in Yokohama who fills out a survey at night, faking her income and location, and who fills out the survey with 30 questions in 2 minutes or less, does this really interest me? Do I want to pay money for that?"
If it does not, why pay for it?
A research company should be able to prove and validate each data point, and explain how the sample was constructed in detail, meaning, publishing the incentives given to the consumers filling out the survey, and how the incentives might bias 1) their inclination to tell the truth and 2) the bias in selecting the sample, meaning, do certain groups NOT participate because of the missing incentives or missing access to them, and how does this distort the sample, the answers, and most of all, the value for the company commissioning the research.
Test your research agency. Ask them how they keep this from happening. As the BCG, McKinsey, and all the other big names, how they make sure. And ask them what they think about this problem in the first place. Ask for transparency and for their own assessment of the QUALITY of the data.
