Creating, administering, and interpreting a survey is much, much trickier than you might expect. (In fact I completely mangled an argument in an earlier version of this blog entry.) Let me describe some pitfalls when you want to use the simplest type of survey that has a multiple choice format. Imagine you want to ask end users how the user interface of a new software application compares with the user interface of a current system. So, you create a survey which has instructions for respondents to select the statement which best describes the extent to which they agree or disagree to each statement from a set of 100 statements. For example, one of your statements might be, "The search feature of the user interface of the new system is easier to use than the search feature of the current system." And you give survey respondents the four choices, "Strongly Disagree", "Disagree", "Agree", "Strongly Agree" to each statement. Well, let me suggest that you may have already committed several mistakes in survey design. This is an example of a Likert scale design. The first issue is that with a Likert design you should as a rule of thumb, except in rare cases, have five responses plus an explicit n/a response. The five responses are generally some closely related form of, "Strongly Disagree", "Disagree", Neutral", "Agree", "Strongly Agree", and then slightly physically to the right of these first five responses, you should have a sixth "Not Applicable" response. You should give survey respondents a neutral option because otherwise you are forcing a positive or negative opinion when they may be neutral on a statement. And you should give respondents an explicit not-applicable choice rather than assuming or guessing that no response at all means not applicable in some way (as opposed to an invalid response because the respondent just forgot to answer). A second, almost certain mistake is that your survey has too many (100) statements. Survey respondents are going to get bored and quickly launch into an auto-complete mode just to finish your survey.
These are just a few of the dozens of issues with survey design. Analyzing the results of surveys is also very difficult. The moral of the story is that survey design is a very tricky task and you cannot simply use common sense. When I was a university professor, I taught an entire semester class on survey design; this is probably the bare minimum knowledge you need to create and interpret surveys in a software testing environment.