Surveys and Questionnaires
Most people will be familiar with surveys of different types. Both familiar and conventional, questionnaires can be used and administered in a variety of ways (at events, pre-, during, and post-participation, online, etc.).
Used well, surveys can help to gather quantitative data and reach relatively large numbers of respondents. In addition, surveys ask the same questions so information is collected from respondents in a standardised and systematic way.
As in all forms of research, the way in which you ask your questions impacts the type of information you will receive. For more general information on designing questions, see the Asking Questions section of the toolkit.
Having well-designed questions is particularly necessary with surveys, since you will usually not be physically present to clarify questions or expand on your meaning. You must, therefore, take great time and care in designing your questions to ensure they are accurate, clear, unbiased, and valid.
The type of question you ask will largely depend on what information you are trying to gain.
Yes/No and True/False
You could, for example, as a simple Yes/No or True/False question if you are asking a very straightforward question. These are particularly relevant for:
- Attendance (‘Did you attend the most recent meeting?’)
- Ownership ('Do you have your own bed?’)
- Completion of a task (‘Did you vote in the last election?’)
- As a gateway to determine whether more questions are necessary.
This type of question can be helpful in large-scale surveys because it allows for easy statistical analysis. True/False or Yes/No questions are, however, very limited in the amount of information they provide, and will likely need to be followed up with other questions.
You could also ask people to rank their experience on a Likert Scale. These scales generally run from 1 - 5 or 1 - 7, and are best suited for asking about attitudes, beliefs, or behaviors. Usually you would provide a statement and then ask respondents to mark how true that statement is for them, how often they do something, or how much they agree or disagree.
Questions which provide the respondent a blank area in which they can construct their own response are considered open-ended. These questions, while often part of surveys, provide qualitative data. As such, they will need to be thematically analysed or coded before the analysis of the survey could be complete. They are very good for discovering aspects of a situation you may not have thought to ask about, but are time consuming for both the respondent and the researcher, so should be used sparingly. If you find yourself wanting to ask a lot of open-ended questions, consider using an interview instead of a survey.
Multiple choice questions should offer four or five distinct choices for the respondent. It is important to ensure that the responses are clear and have no overlap in order to avoid confusion. For example, if a question offered two responses, one of which was “late afternoon” and one of which was “early evening”, the respondent would likely be confused as to where one time period ended and another began. As such, any data you received from that question would be suspect. In addition, keep in mind that if you offer a space for the respondent to fill in “other”, you are essentially creating a new open-ended question to analyse. There will be circumstances in which this is a necessary option, but use it sparingly. If an “other” box is truly necessary for a question, it will often come to light during the piloting period of the survey.
Selected methods and techniques
a) Postal surveys. In a postal survey, the data collection is based on self-completion questionnaire, which the selected target group has received by mail.
b) Online surveys. You can now create and publish online surveys in minutes, and view results graphically and in real time. This is a method that is increasingly accessible to even the smallest community and voluntary organisations through tools such as Survey Monkey.
c) Telephone surveys. In this method telephone numbers are used to contact potential respondents, either from the general population or from a known sample. Information is collected using a standardised questionnaire.
d) Face to Face surveys. Surveys can be administered face to face using a standard planned set of questions.
- Quick, easy, and rather inexpensive
- Provides the ability to get feedback from a large number of people
- Reasonably familiar, simple and flexible to administer
- Can combine open and closed questions to good effect
- Enables simple, standardised feedback and analysis
- Unobtrusive data collection method
- Respondent confidentiality or anonymity is easy to maintain
- Existing survey templates are available and can be easily adapted
The main challenges involve bias. Bias can have a major impact on thevalidityof any evaluation research.
- Sampling or weighting bias: is generally caused by faulty collection practices and results in survey respondents not being a representative group.
- Response bias: is generally caused when the survey questions are purposefully or accidentally leading or loaded, or when people want to respond a certain way because it is socially acceptable.
- Nonresponse bias: is caused when certain members of your sample do not respond to your survey, thus causing a difference between the average of your respondents and the average of your actual sample. For example, in an online survey, you will only reach those in your sample who have ready access to the internet.
- Response rates: Not everyone who receives your survey will complete it. In general, you can expect response rates of between 10 and 40%, but if you are working with marginalised or hard-to-reach groups, expect your response rate to be considerably lower.
- For organisations working with a smallpopulation,or those who only have access to a small sample size, it may not be worth the time and effort to create a survey. It may be more cost-effective and efficient to conduct focus groups or interviews instead.
- Survey research is one of the most common methods available and as such there is an array of further guidance widely available. A useful point of entry to this topic is the Research Methods Knowledge Base. This site includes a very helpful section on Constructing the Survey, which contains information on wording, types of questions, content, and more.
- The UK Data Service has a large Variable and Question Bank that allows you to explore how other surveys have worded their questions.
- Survey Monkey and Smart Survey are two of a number of online survey tools with options for free use. The free options usually have question limits or limited analysis options. You can pay for additional functionality as necessary.