Questions
The other day, somebody asked me why the questions I asked seemed to be rather pointed and/or double-edged. Quite apart from the difficulty of continuing in the same strain of metaphorical weaponry, I chose to reply by asking, "Don't all questions have points? And surely, if the questions are open-ended, the answers should be likewise..."
The problem of course is an old one. It is the problem of anticipation. A few things come to mind when a man is faced with a question such as "What are your politics?" He would tend to ask himself, "How much can I safely say?" or "What is the intention of this question?" These questions are metacognitive; they are not about the answer to the question, but about the kind of response it should appear to be to the questioner. If the man can anticipate the required kind of answer (or worse, the answer required), it makes his job a lot easier.
It is not difficult to rig a survey to give good answers or bad. You can propose a five-point Likert scale assessing some quality X as 'excellent, very good, good, average, below average' for example. This is positively biased; if the distribution is even and/or random, then you will get 60% of the respondents saying that X is good or better than good. In fact, since in most cases some respondents tend to give random answers, you will probably have a positive bias no matter what.
You can also propose a four-point Likert scale, with something like 'very good, good, bad, very bad'. This has the virtue of disallowing the 'sitting on the fence' strategy. However, it tends to polarise answers into good/bad without allowing for 'undecided' or 'genuinely average'. And if there is a bias as in the first example, you will get at a 75% to 25% split, worse than 60/40.
There are of course many other strategies, and much more subtle ones as well. It is good for the researcher to declare these possible sources of bias and how they might have been overcome; not to do so is an impediment to sound conclusions and appropriate peer criticism.
But the root of finding out is the asking of questions. And the root of the asking of questions is the crafting of questions. And there is no end, no end at all to the making of books and the finding out of things – as many have found out to their sorrow.
The problem of course is an old one. It is the problem of anticipation. A few things come to mind when a man is faced with a question such as "What are your politics?" He would tend to ask himself, "How much can I safely say?" or "What is the intention of this question?" These questions are metacognitive; they are not about the answer to the question, but about the kind of response it should appear to be to the questioner. If the man can anticipate the required kind of answer (or worse, the answer required), it makes his job a lot easier.
It is not difficult to rig a survey to give good answers or bad. You can propose a five-point Likert scale assessing some quality X as 'excellent, very good, good, average, below average' for example. This is positively biased; if the distribution is even and/or random, then you will get 60% of the respondents saying that X is good or better than good. In fact, since in most cases some respondents tend to give random answers, you will probably have a positive bias no matter what.
You can also propose a four-point Likert scale, with something like 'very good, good, bad, very bad'. This has the virtue of disallowing the 'sitting on the fence' strategy. However, it tends to polarise answers into good/bad without allowing for 'undecided' or 'genuinely average'. And if there is a bias as in the first example, you will get at a 75% to 25% split, worse than 60/40.
There are of course many other strategies, and much more subtle ones as well. It is good for the researcher to declare these possible sources of bias and how they might have been overcome; not to do so is an impediment to sound conclusions and appropriate peer criticism.
But the root of finding out is the asking of questions. And the root of the asking of questions is the crafting of questions. And there is no end, no end at all to the making of books and the finding out of things – as many have found out to their sorrow.
Labels: Research
etched with acid at 12:21:00 pm
0 Comments:
Post a Comment
<< Home