The way a question is asked. Framing can have an important effect upon subjects’ responses, so it is important to ensure that questions are worded properly. The first influential treatment of this issue was by Payne (1951). Much useful work followed, summarized by Sudman and Bradburn (1982). Knowledge of this work is important in conducting intentions studies, eliciting expert opinions, and using methods that incorporate judgmental inputs. Consider the effect of the wording in the following example provided by Norman R. F. Maier: “A man bought a horse for $60 and sold it for $70. Then he bought it back again for $80 and sold it for $90. How much money did he make in the horse trading business?” Almost half of the respondents answered incorrectly. Now consider this question: “A man bought a horse for $60 and sold it for $70. Then he bought a pig for $80 and sold it for $90. How much money does he make in the animal trading business?” Almost all respondents get the correct answer to this version of the question ($20). Tversky and Kahneman (1981) demonstrated biases in peoples’ responses to the way that questions are framed. For example, they asked subjects to consider a hypothetical situation in which a new disease is threatening to kill 600 people. In Program A, 200 people will be saved, while in Program B, there is a one-third chance of saving all 600 people, but a two-thirds chance of saving none of them. In this case, most respondents chose Program A (which is positively framed in terms of saving lives). However, when the question was reframed with Program A leading to 400 deaths, and Program B as having a one-third chance that nobody would die and a two-thirds chance that that all would die, then the majority of respondents chose Program B (this alternative is negatively framed in terms of losing lives). This negative way of framing the question caused people to respond differently, even though the two problems are identical. This example implies that framing could play a role in writing scenarios. The discovery of biases due to framing seems to outpace research on how to avoid them. Unfortunately, telling people about bias usually does little to prevent its occurrence. Beach, Barnes and
Beach, L. R., V. E. Barnes & J. J. J.
Christensen-Szalanski (1986), “Beyond heuristics and biases: A contingency model
of judgmental forecasting,” Journal of Forecasting, 5,
143-157. (1986) concluded that observed biases may arise partly because subjects answer questions other than those the experimenter intended. Sudman and Bradburn (1982) provide a number of solutions. Two procedures are especially useful: (1) pretest questions to ensure they are understood, and (2) ask questions in alternative ways and compare the responses. Plous (1993, chapter 6) provides additional suggestions on framing questions.