A survey is a set of questions posed to a random sample from a target audience for the purpose of defining the options of the entire market segment.
What it is: A Survey is not just a list of open-ended questions to learn what people think. Because of the scientific statistical analysis required to project the sample to the entire market segment, the questions in most cases must be quantifiable and not be just open-ended. A market or social science research survey is a scientific quantitative analysis of specific questions within a targeted population. The survey is completed so that the researcher can project how the target market segment would respond to the questions posed in the survey. Because statistical analysis will be used on a survey population made up of a random sample from within the target segment, the results can be defined within a certain range with a 95% confidence level.
What does it do: The survey lets you identify the average opinions and activities of the target segment you are researching. In the case of determinant analysis, you can learn how they make decisions even when they do not know or can not describe upon which factors they base their decisions.
Uses:
How is it used: A survey most commonly follows secondary research, expert interviews, and often a focus group. The secondary research is used to identify trends and past research done to characterize the target segment you are studying. Expert interviews narrow down the segment issues and give insight into the whys and wherefores of the segment trends and activities. In some cases, expert interviews can be used to create the factors, ranges, and lists that will be part of the survey. But in many cases, it is necessary to have a focus group to help narrow the factors, questions, lists, and ranges that will be employed in the survey. The most difficult problem with a survey is obtaining a random sample. Surveys can be conducted in person, in writing, online, telephone, email requests, etc. But you have to have access to the target population which can be difficult and expensive. Historically mailing lists, subscriber lists, mall intercepts, and many other methods were used to obtain the survey candidates. Most surveys are done today by email with some form of a list. The lists can come from customer lists, conference attendees, user files, industry directories, etc. Where a list is difficult to obtain you can use paid panels where you pay for only the completed surveys by your qualified candidates.
Where: Surveys are used for a wide variety of applications where there is value in knowing how a target segment will respond to advertising, product features, changes in service, etc.. See some examples below:
New Product introductions (what features, benefits and pricing will optimize the sales of the product
Measure the elements of customer satisfaction and track how the company or product is doing overtime
Advertising analysis to measure target market response to different advertising approaches
Gather public opinion on legislative issues and potential candidates for elected office
Social Scientists use surveys to see how people respond to specific stimuli
Polling data is most often based on scientific statistical analysis from survey data
Measure the satisfaction with and effectiveness of training programs
The response of a population to different flavors, textures, and consistency of food products or recipes
Why: A survey is not looking for broad general responses. A survey is used to project the findings of the random sample upon the larger target population. Surveys are for the purpose of identifying with confidence the opinions and likely actions of the total population you are researching.
Limitations:
Where it shouldn't be used: Many people start their discovery of questions, issues, and opinions of a target population with a survey. Of necessity, this survey would be general and open-ended. The answers would in most cases not be quantifiable and not projectable upon the target population. That is why you complete secondary research, expert interviews, and possibly focus groups before a survey.
Any restrictions: To have statistical relevance for the findings of the survey you often need a minimum of 50 responses to even perform some of the statistical analysis. In many cases, it requires 100 responses to narrow the ranges of the answers at the 95% confidence level. So analyzing a small population can be a problem. One resolution to this problem is to use conjoint analysis (see sawtooth software for more information)
Warnings: Surveys are expensive, resource-intensive, time-consuming, and potentially exhaust the target segment, so they can only be given once or at most twice to the target segment. That is why it is so important to complete the pre-work prior to initiating the survey. Also survey fatigue is a real issue, so you should make the survey as short as you can and still meet your desired outcome. If your survey is longer than the interest of the survey participant they will stop before the end. You can increase the likelihood that the participant will complete the survey by making sure the survey is of interest to them or incentivizing them to complete. Incentives can include payment, entrance into a drawing, access to the results of the survey, etc.
Define the Decision to be made or Action to be taken
First, decide who will use the survey and what question they are trying to answer
How will the results of the survey help them make their decision or take their action
How will the survey results need to be structured to answer their questions
How precise will the results need to be (therefore how large will the sample need to be)
Gathering data
Friends and relatives (probably not a random sample, and may not even be people in the target segment)
Try and get the survey to spread by posting it on social media (more random, but still probably has selection bias and not focused on target population)
Identify Ensign College alumni or other people in the target population and send the survey to them and ask them to forward it to others in their field that fit the criteria
Purchase a list of people to send the survey link to (there is no guarantee that any of them will respond and the normal response is 1% to 3% for most lists)
Purchase a panel where people who meet your demographic choices are paid to participate in the panel provider. You will only be charged for the completed surveys by people who meet your required demographics. Submit your demographic parameters to the following link for a discounted BYU rate from Qulatrics panels: http://survey.qualtrics.com/SE/?SID=SV_eas6U2OdIcJCwTjLinks to an external site.
From the determination of what the results of the survey will need to provide, determine what questions should be asked
What have you learned from the Secondary research, expert interviews, and focus groups
Which of those topics do you need to have quantifiable assurance of the answers
Select a survey source like Qualtrics.com, Survey Monkey (some free elements), Google Surveys or other sources of your choice
Write specific questions that will bring back a specific response from the participant (no double-barreled questions that contain 2 difference questions in one: e.g. do you like blond and blue-eyed people?)
Keep the survey as short as possible and still meet your needs (3 to 5 minutes unless you have some incentive to provide or the participants are motivated on their own to complete the survey, but 20 to 30 minutes max even for a paid survey)
Utilize a variety of survey questions types being sure to have measurable results as an outcome of the responses
See additional resources below for additional tips on writing survey questions
Identify how you will find participants to complete the survey
Analysis of data
Analysis can be as simple as measuring what % of the respondents selected each answer (called Cross Tabs). This requires no sophisticated statistical analysis and gives some valuable information. This type of information is often called the stated importance responses because the answers are what the respondent said overtly. There is always a risk of the lie factor where the respondent is lying for a purpose, or they do not really know the answer to your question and are guessing. If you ask them which features most influenced their last DVD player purchase they may not know and just guess.
The analysis that looks for determinants can identify the most important determinants of purchase by using regression analysis to identify which features or factors most influenced the decision. To do this you must have ratings of the individual factors by competitor/product, then an answer to what decision they actually made. Using this data you can regress the factors against the decision to identify the determinants. Often the stated importance by the respondent is called qualifiers (factors necessary to make the sale, but does not differentiate the seller) and the determinants are calculated as the factors that differentiate between competitors and products.
There are many forms of statistical analysis like perceptual maps, gap analysis, factor analysis (to remove multicollinearity ), correspondence analysis, etc. You will need to learn how to do each form of analysis, but most statistical packages will create these analyses by the push of a button.
Conjoint Analysis uses paired comparisons or choices to find the underlying utility of each factor and how it impacts the decision by the target person. Conjoint analysis can be used to find the statistically measured decision criteria for a single person or an entire population. Conjoint requires a specialized questionnaire, so you must plan to use conjoint from the very start to be sure you have data that is available for this kind of analysis.
Interpretation of results
The uncertainty ranges and confidence level (e.g. 95%) are critical to being measured to make sure that one answer is truly statistically different than another.
You should complete the factor analysis to be assured that there is no multicollinearity where 2 or 3 different factors are really measuring the same thing.
Do not assign causality unless you have run the test for causality. You can not assume that when 2 factors move together one is causing the other to happen. They may both be affected by a third factor. As an example, statistics tell us that men have more accidents than women, but when you measure accidents per 1,000 miles driven, then the accident rate is the same for men and women. Men just drive more miles, and that is why they have more accidents. The predictive measurements are miles driven, not whether they are male or female.
Look for trends and patterns in the data and the analysis to find relationships you can utilize in your decisions and actions.
Presentation of results: A full range of charts, graphs, and tables are available to show relationships and make recommendations. There is no one right approach. Each decision and statistical analysis will need a unique presentation to show the recommended outcomes.
Template for capturing data:
Use free survey services like Qulatrics, Survey Monkey, and Google Forms, Microsoft Forms (Free to Ensign College students) as a template. There is no need to create your own template. See examples below for a few specific topic templates.
Output representation and recommendations:
A full range of charts, graphs, and tables are available to show relationships and make recommendations. There is no one right approach. Each decision and statistical analysis will need a unique presentation to show the recommended outcomes.
