Selecting the right questions when building an online survey
There are many types of questions and answers available when you create an online survey. Sometimes it is difficult to choose the right type of question; this is why we are going to study in further details the kind of questions in this article, with all the pros and cons.
In order to present the questions better, we created 3 categories, which include subcategories. So, start discovering questions with us now!
Open-ended questions let the respondent answer freely. When someone asks an openended question, he/she does not wait for specific answer, unlike closed-ended questions, which reduce the range of possibilities. On the contrary, open-ended questions aim to collect varied information (facts, anecdotes, points of view, feelings).
Here is an example of an open-ended question:
What happened this morning?
The user ignores the event which occurred this morning. He is looking for practical information brought by the respondent.
As part of an online survey, this question provides the information brought by a respondent exactly, and it could be a customer, a classmate etc. The respondent will be able to explain his feeling with personal words. Using this type of question can make your surveys more human and encourage freedom of expression of your audience, which will perceive that his opinion really matters for you.
If you interview an audience who is lacking in time, this type of question could discourage from answering to your questions, thus creating a high rate of survey abandoned. It is important to find a good compromise between the number of open-ended questions and closed-ended questions in your survey. In this case, everything is about knowing the target audience before choosing the number of open-ended questions you want to include in your survey.
If you are not sure about the structure of your survey, test it and answer as if you were in your respondents shoes. Do not hesitate to ask to your friends to test it too. It is always positive to get an external opinion on your works.
Here are some examples of open-ended questions:
Probing questions aim to get precision about specific information.
Example of probing questions:
“What satisfies you the most?”
These questions enable to collect the respondent's feeling accurately. Potentially, they can provide interesting possibilities of feedback because the answer is going to explore an accurate target, often accompanied with an argument. The mere fact of asking a probing will give the respondent the impression that you care about his opinion. Consequently, he will answer sincerely. The data collected during probing questions has a strong credibility.
Multiple choice questions:
They ask the respondents what choice they prefer better without being forced to choose among answers.
What cities do you prefer?
Through these questions, you can analyse data and group them according to redundancy. These type of questions require the intervention of the respondents’ memory. In the example, he can remember the cities in which he travelled and select those he liked. It will put him into good conditions to continue the survey. The user can take advantage of this situation to ask also a open-ended question and continue collecting new data precisely.
Creating lists and adding a field “Other” could classify this question in the category of mixed questions, as they offer possibilities of both open-ended and closed-ended answers.
Unlike open-ended questions, closed-ended questions limit the kind of data you collect. They need an accurate information and filter the possibilities of answers. The user already knows the answers he can get, he only ignores those that will be preferred by the respondent.
Example of closed-ended question:
Do you want to come with us tonight? YES / NO
The user knows all the possible answers and he only proposes 2 choices to the respondent. The respondent cannot argue or provide other choices. In order to ask additional information, a good possibility would be asking later “Why?”.
Closed-ended questions are more intuitive. They enable to process a question quickly and effectively. They provide the possibility of being organised in a playful way (with emoticons, rating etc…) and they are a credible solution to awake the interest of the respondent. It will have as a consequence to give him the impression to go along with it.
It is difficult to include an exhaustive group of adequate answers. The proposed answers can influence the respondents. However, they remain ideal when we look for strict and centred data to analyse.
As part of an online survey, you can also add the option "I don’t know", if you want to leave a way out to the respondent.
Examples of closed-ended questions:
Matrix questions enable to evaluate several elements on the same column. This type of question enables to grade a question from 1 to 5 on a single line.
The questions can be different types:
- Drop-down menu
- Numerical text
Example of matrix question:
What did you prefer in our restaurant?
- Quality of the food (to grade on 5)
- Kindness of the staff (to grade on 5)
Matrix questions can have various forms. They provide a wide panel of presentation and can be quickly handled. Be careful not to overload them with unnecessary data. Your question could be uninteresting. Be sure about your target audience and the relevance of your matrix questions data, if you want to insert them in your surveys.
If you add redundant data, you run the risk of bothering your respondent, who could depreciate your work. Do not take this risk and propose only questions which you absolutely want to analyze.
Do not hesitate to review your questions aloud. This allows you to detect errors when it is difficult to perceive them during a classic review.
The Likert scale
The Likert scale enables to answer according to your level depending on a question as follows:
The quality of our services matched your expectations :
- I strongly disagree
- I disagree
- I don’t know
- I agree
- I strongly agree
This type of question is perfect for all the surveys because it has a lot of advantages. The Likert scale is very speed and it enables to collect accurate information thanks to a question asked in the form of a suggestion. The respondent does not need much time to think. He has to share immediately in one click. However, even if it is ideal for your surveys, you should not abuse of this question in order not to make your surveys too monotonous.
The index of answers is characterised by a 1 to 5 scale.
It enables you to evaluate a service, or indicate your satisfaction in a playful way.
Example of a rating question:
In general, are you pleased with the quality of our services?
To answer, stars need to be marked according to your level of satisfaction
This type of question is one of more popular; its playful aspect is very appreciated by the public in general. Intrigued and seduced by the display of the stars, the respondent tends to hurry to answer them. The time for reflection is less and therefore the collected data can lose precision.
If the main objective of your survey consists in getting the target public interested, it can be useful to choose these questions. For example, it could be coherent to integrate it into your customer satisfaction online survey.
The drop-down list takes the form of a drop-down menu to answer to a question raised.
Example of a drop-down menu questions:
This question also belongs to closed-ended group and provides the opportunity to select an answer in a drop-down menu. Simple and quick, this type of question stimulate the curiosity of the respondent, because he does not see the answer directly. This represents a challenge for the user, who intervenes with 2 clicks: the first is to make the decision to show the answers, the second is to follow his procedures of reflection and answer.
It's better to use this type of question at the beginning or at the end of the survey. First, it enables to awaken the curiosity of the respondent about the survey, and then to collect an accurate data. Remember to use it with a audience having time and patience.
The purpose of the advanced questions is to gather detailed information. They rely more on the respondent's thinking abilities than other types of questions. In particular, they make it possible to collect data that can be analyzed more accurately than a closed question.
Here is an example of advanced question:
Organise coulours according to your preference:
In this example, the respondent has to rank his favourite colours. This question pushes him to carry out a process of reflection in order to know the colors he prefers better. Moreover, this type of question groups the characteristics of open and closed questions. In fact, the respondent is free to provide the answer he wants, even if this freedom of response will have been preconditioned by previously defined choices.
Thanks to the reflection needed, the question allows to highlight a precise feeling. The data collected is reliable because the type of question encourages a dynamic response.
The response time can be quite long. Adding too many advanced questions to one questionnaire can quickly lengthen the total response time to your questionnaire.
Remember: no one wants to spend too much time on a questionnaire. Use only this type of question in case of real need. When answering an advanced question, the respondent needs to know its usefulness. For example, they can be useful for quizzes but should be used with common sense in an online questionnaire.
Here are a few examples of open questions:
They allow you to rank between several possible answers according to your preferences
Here is an example of a ranking question:
Please rank these cities in order of preference: (provide grades of 1 to 3 for each city)
Ranking questions require a minimum of involvement. However, they make it free expression possible and highlight preferences. The respondent must think carefully before indicating what he prefers. This will eventually push him to argue internally in order to know what criteria he decides on his results. This type of question can be interesting because it allows to better define a personality.
Organized as a percentage, distribution questions ask you to organise preferences into a percentage of different types of responses.
Here is an example of a distribution question:
Distribute the percentages according to the criteria that you prefer during an online purchase. Divide your choices so to get 100%.
- Site reputation:
- Product need:
The distribution questions call for a stricter and streamlined approach. They show up a certain aspect of the respondent's personality. The answer he will bring aims to highlight a fact, a trend or opinion...Here, he must distribute as a percentage the factors that favours the online purchase according to him, and therefore potentially, for other customers. His answers are therefore more likely to correlate with those of other users, unlike a ranking question, more subjective.
The most relevant use of a distribution question can occur as part of a personality test or an online market research.
These type of questions are the keys to create a fully customised, complete questionnaire. Now, the choice is up to you!
With a Marketing Master from Aix-en-Provence IAE school, Karine Peyrichou is in charge of Marketing for EVALANDGO in charge of QWESTEO and EVAL&GO. The company is specialized in collecting big data for their customers.