The Differences Between Mystery Shopping and Marketing Research

Date Published May 13, 2001
Author Mark L. Michelson

Both mystery shopping and marketing research are long-established research services that help businesses and organizations operate more effectively. These research services share common goals in providing businesses with information critical to their success. However, mystery shopping and marketing research vary widely in technique and process. With this in mind, mystery shopping should not be used to replace marketing research, but rather to compliment an organization’s marketing and operational knowledge.

This essay will attempt to compare and contrast mystery shopping and marketing research services and offer some insights into how mystery shopping can be used effectively to compliment marketing research efforts. First, let’s start with some basic definitions of these services:

Mystery Shopping is a long-established research technique that uses shoppers who are given guidelines to anonymously evaluate and monitor customer service, operations, employee integrity, merchandising, and product quality. Mystery shopping fills in a gap of critical information between operations and marketing. Mystery shopping is used on the front line to collect data that helps determine what happens to customers and prospects when they visit or call your company.

Marketing Research is the process of obtaining knowledge and gaining an understanding about what people think, feel and do in relationship to meeting their needs, desires and preferences related to buying products and services. Marketing research is used to identify and define marketing opportunities and problems; generate, refine, and evaluate marketing actions; monitor marketing; and improve understanding of marketing as a process. In plain English, it is determining what real customers, real prospects and other specific groups of people think about companies, services, products, and marketing communications.

Though many marketing research firms conduct mystery shopping, technically, mystery shopping is not marketing research. It is research, but it is not marketing research. It is more closely related to operations research, Mystery shopping compliments marketing research, but it is different in critical ways. If mystery shopping data is used for marketing research purposes - then certain rules would apply, such as the guidelines established by ESOMAR (see inset).

How is Mystery Shopping Different From Marketing Research?

Mystery shoppers must follow specific guidelines on what to do during an evaluation and shop at specified locations they may not normally visit. Marketing research study participants are not given evaluation guidelines in advance

Mystery shopping is typically more operational in nature than marketing research and is most often used for quality control, training and incentive purposes. Marketing research is used most often to determine real customer and prospect opinions, perceptions, needs, and wants

Mystery shoppers are recruited based on specific profiles that closely match a companies’ real customers. Marketing research study participants are sampled at random from a qualified population to represent a larger population

Mystery shoppers are asked to be objective and explain observations. Marketing research study participants are encouraged give their subjective opinions freely

Mystery shopping reports on specific visits or calls – each evaluation can be used independently to make improvements to operations and training. Mystery shopping is not predictive of every customer’s experience unless sufficient samples are taken and data analyzed in aggregate.

Mystery shopping should not be used alone to determine customer satisfaction - it can compliment, but not replace traditional customer satisfaction research. You can't predict or measure customer satisfaction using mystery shopping because customer satisfaction is a subjective topic based on what real customers think. Mystery shoppers are not real customers - they know what to evaluate before entering the store and they may not typically visit the store they are evaluating.

Types of Mystery Shopping Methods

As with marketing research, there are many different types of data collection methods for mystery shopping. Some of the common mystery shopping data collection methods include:

• In person/on-site shops
• Telephone shops
• E-Commerce web site shops
• Hidden video/audio recording
• Full narrative shops (qualitative)
• Checklist shops (quantitative)
• Purchase & return shops
• Discrimination (matched-pair) testing.

Designing Mystery Shopping Questionnaires/Evaluation Forms

Questionnaires for mystery shopping evaluations should be designed to provide objective, observational feedback with a system to allow for checks and balances. Criteria to be evaluated must be objective rather than subjective. Typical retail mystery shopping questionnaires cover: greeting, customer service, facility cleanliness and orderliness, speed of service, product quality, and employee product knowledge.

Unlike marketing research questionnaires that employ likert scales for ratings, mystery shopping questionnaires ideally use only binary ("yes" and "no") questions. For certain questions, shoppers may be required to provide open-ended narratives for clarification of observations. Multiple response questions are used to allow shoppers to check off the features and benefits that are mentioned during the shop. Most shopping questionnaires include a "general comments" section that encourages shoppers to remark on anything they find significant or interesting during the shop.

For mystery shopping questionnaires, some questions may be more important than others - a point/scoring system for questions can emphasize the most important issues. If using a scoring system, which is often recommended, appropriate weighting of questions is critical. Some questions may not need to have points allocated to them at all, but may be necessary for background of the shop experience. Shoppers’ evaluations may be questioned and/or appealed once the facility knows that a mystery shop has occurred.

What Are The Benefits of a Mystery Shopping Program?

• Monitors and measures service performance
• Improves customer retention
• Makes employees aware of what is important in serving customers
• Reinforces positive employee/management actions with incentive-based reward systems
• Provides feedback from front line operations
• Monitors facility conditions - asset protection
• Ensures product/service delivery quality.
• Supports promotional programs
• Audits pricing & merchandising compliance
• Provides data for competitive analyses
• Compliments marketing research data
• Identifies training needs and sales opportunities
• Educational tool for training & development
• Ensures positive customer relationships on the front line
• Enforces employee integrity (The Mystery Shopping Provider’s Association strongly recommends using licensed private investigators for integrity related shops)

How to Make The Most of A Mystery

With a mystery shopping program, companies can establish customer service guidelines, monitor and reward excellent performance. As management guru Tom Peters says, "What gets measured gets done."

Once shopper reports are compiled, sharing those results with operations, training and other key personnel is the important next step in a program’s success. Make it a positive, motivating experience that rewards people for a job well done while identifying areas where training may improve customer service and sales.

Mystery Shopping can be used as a marketing and training tool to help ensure a company’s communications, service, and operational objectives are being carried out on the front line. An established, ongoing program, where employees know that any customer may be the mystery shopper, is more effective and objective than sporadic audits.