Choose a Sign-in Option
A survey is a set of questions posed to a random sample from a target audience to define the options of the entire market segment.
What it is: A survey is not just a list of open-ended questions to learn what people think. Because of the scientific statistical analysis required to project the sample to the entire market segment, most questionsmust be quantifiable. A market or social science research survey is a scientific quantitative analysis of specific questions within a targeted population. The survey is conducted ton project how the target market segment might respond to the questions in the survey. Because statistical analysis will be used on a survey population made up of a random sample from within the target segment, the results can be defined within a certain range with a 95% confidence level.
What it does: The survey lets you identify the average opinions and activities of the target segment you are researching. In the case of determinant analysis, you can learn how they make decisions even when they do not know or cannot describe how they make their decisions.
How it is used: A survey usually follows secondary research, expert interviews, and often a focus group. The secondary research identifies trends and past research to characterize the target segment you are studying. Expert interviews narrow down the segment issues and give insight into the principles of the segment trends and activities. In some cases, expert interviews can be used to create the factors, ranges, and lists that will be part of the survey. A focus group is necessary to help narrow down the factors, questions, lists, and ranges to employ in the survey. The most difficult problem with a survey is obtaining a random sample. Surveys can be conducted in person, in writing, online, telephone, by email request, etc. But you must have access to the target population, which can be difficult and expensive. Historically, mailing lists, subscriber lists, mall intercepts, and many other methods were used to find survey candidates. Today, most surveys are completed by email, with some form of list. The lists can come from customer lists, conference attendees, user files, industry directories, etc. When a list is difficult to obtain, you can use panels where you only pay for the surveys completed by your qualified candidates.
Where: Surveys are widely applied when there is value in knowing how a target segment will respond to advertising, product features, changes in service, etc. See some examples below:
Introduce new products (what features, benefits and pricing will optimize the sales of the product.
Measure customer satisfaction and track how the company or product does over time.
Analyze advertising to measure the response to different approaches.
Gather public opinion on legislative issues and potential candidates for elected office.
Explore how people respond to specific stimuli (social scientists).
Polling data is most often based on scientific statistical analysis from survey data.
Measure satisfaction with and effectiveness of training programs.
Analyze the response of a population to different flavors, textures, and consistency of food products or recipes.
Why: A survey is not looking for broad, general responses. A survey is used to project the findings of the random sample upon the larger target population. Surveys are for confidently identifying the opinions and likely actions of the total population you are researching.
Where it shouldn't be used: Many people start their discovery of questions, issues, and opinions of a target population with a survey. Of necessity, this survey would be general and open-ended. The answers would, in most cases, not be quantifiable to or projectable on the target population. That is why you complete secondary research, expert interviews, and possibly focus groups before a survey.
Any restrictions: To have statistical relevance for the results of the survey, you often need a minimum of 50 responses to even perform the statistical analysis. In many cases, you need 100 responses to narrow the ranges of the answers at the 95% confidence level. So analyzing a small population can be a problem. One solution to this problem is to use conjoint analysis (see Sawtooth software for more information).
Warnings: Surveys are expensive, resource-intensive, and time-consuming, and they may exhaust the target segment, so they can only be given to the target segment once or, at most, twice. That is why it is so important to complete research prior to initiating the survey. Also, survey fatigue is a real issue, so you should make the survey as short as you can and still meet your goals. If your survey is longer than the interest of the survey participant, they will stop before the end. You can increase the likelihood that the participant will complete the survey by making sure the survey is of interest to them or incentivizing them to complete. Incentives can include payment, entrance into a drawing, access to the results of the survey, etc.
Define the decision make or action to take
Who will use the survey? What question they are trying to answer?
How will the results of the survey help them make their decision or take their action?
How will the survey results need to be structured to answer the questions?
How precise will the results need to be (how large will the sample need to be)?
Gather data
Ask friends and relatives (probably not a random sample, and may not even be people in the target segment).
Try to get the survey to spread by posting it on social media (more random, but still probably has selection bias and not focused on target population).
Identify Ensign College alumni or other people in the target population, send the survey to them, and ask them to forward it to others in their field who fit the criteria.
Purchase a list of people to send the survey link to (there is no guarantee that any of them will respond and the normal response is 1–3% for most lists).
Purchase a panel where people who meet your demographic choices are paid to participate in the panel. You will only pay for the surveys completed by people who meet your required demographics. Submit your demographic parameters to the following link for a discounted BYU rate from Qualtrics panels: Qualtrics Panels.
After determining what the results of the survey will need to provide, determine what questions to ask.
What have you learned from the secondary research, expert interviews, and focus groups?
Which of those topics do you need to have quantifiable assurance of the answers?
Select a survey source like Qualtrics.com, Survey Monkey (some free elements), Google Surveys, or other sources of your choice.
Write specific questions that will bring a specific response from the participant, not double-barreled questions that contain two questions in one (e.g., do you like blond and blue-eyed people?).
Keep the survey as short as possible while still meeting your needs (3–5 min. unless you provide an incentive or the participants are motivated on their own to complete the survey, but 20–30 minutes max even for a paid survey).
Utilize a variety of survey question types. Be sure to have measurable results as an outcome of the responses.
See additional resources below for more tips on writing survey questions.
Identify how you will find participants to complete the survey.
Analyze the data
Analysis can be as simple as measuring what percentage of respondents selected each answer (called cross tabs). This requires no sophisticated statistical analysis and gives some valuable information. This type of information is often called stated importance responses, because the answers are the overt responses. There is always a risk of lies, when the respondent lies for a purpose or they do not really know the answer to your question and guess. If you ask them which features most influenced their last DVD player purchase, they may not know and just guess.
An analysis can identify the most important determinants of purchase by using regression analysis to identify which features or factors most influenced the decision. To do this, you must obtain ratings for each factor, by competitor or product, and an answer as to what decision participants made. Using this data, you can regress the factors against the decision to identify the determinants. Often, the respondent’s stated importance is called a qualifier (a factor necessary to make the sale, but does not differentiate the seller), and the determinants are calculated by the factors that differentiate between competitors and products.
There are many forms of statistical analysis, like perceptual maps, gap analysis, factor analysis (to remove multicollinearity), correspondence analysis, etc. You will need to learn how to do each analysis, but most statistical packages will create these analyses with the push of a button.
Conjoint analysis uses paired comparisons or choices to find the underlying utility of each factor and explore how it impacts the decision of the target person. Conjoint analysis can be used to find the statistically measured decision criteria for a single person or an entire population. Conjoint analysis requires a specialized questionnaire, so you must plan from the start and to be sure you have data available for this kind of analysis.
Interpret the results
The uncertainty ranges and confidence level (e.g., 95%) are criticalto make sure one answer is truly statistically different from another.
You should complete the factor analysis to ensure there is no multicollinearity where 2–3 different factors really measure the same thing.
Do not assign causality unless you have run the test for causality. You can not assume that when two factors move together one is causing the other. They may both be affected by a third factor. As an example, statistics tell us that men have more accidents than women, but when you measure accidents per 1,000 miles driven, then the accident rate is the same for men and women. Men just drive more miles, and that is why they have more accidents. The predictive measurements are miles driven, not whether they are male or female.
Look for trends and patterns in the data and the analysis to find relationships you can utilize in your decisions and actions.
Presentation of results: A full range of charts, graphs, and tables are available to show relationships and help make recommendations. There is no one right approach. Each decision and statistical analysis will need a unique presentation to show the recommended outcomes.
Use free survey services like Qulatrics, Survey Monkey, Google Forms, Microsoft Forms (Free to Ensign College students) as a template. There is no need to create your own template. See examples below for a few specific topic templates.
A full range of charts, graphs, and tables are available to show relationships and make recommendations. There is no one right approach. Each decision and statistical analysis will need a unique presentation to show the recommended outcomes.
This content is provided to you freely by Ensign College.
Access it online or download it at https://ensign.edtechbooks.org/projectbased_internships/survey_development.
