AP Stat Survey Design
Survey presentation document found here.
Survey proposal and survey question bank.
Slide 2- introduction
Starting off with the introduction. For my observational study, I investigated whether manipulating the wording of a question could create a response bias. I chose to study wording due to the upcoming 2020 election, with more polls and surveys being conducted than ever before in the history of US politics. With that, a lot of the results of these polls and surveys could be manipulated by changing the wording of the questions either into a positive or negative context. After preliminary research on the proper and improper way to word certain questions, I decided upon a neutral and then negative wording for my last question in a five-question survey. With the negative connotation, I expected a reduced support for a certain taxation policy, even though both terms mean the exact same thing. I expected for the responses of the second version including the death tax to have a higher opposition rate when compared to the estate tax.
Slide 3- Study Design
For my survey, I had two versions that varied by one question at the end. I conducted my survey by asking people if they’d be willing to participate during the afternoon while on school campus. I asked every third person to make it random, and if they weren’t willing to participate, I would count until the next third person. I had 60 subjects surveyed, 30 with each version making sure to alternate between the two versions each time someone was surveyed. The explanatory variable is the last question regarding the death/estate tax, and the response variable will be their support or opposition. The treatment is the wording of the last question difference. Random assignment is achieved via every three people, comparison from the two separate survey versions, control via the regular/neutral worded survey, and replication via a large sample size. In order to circumvent any bias on my end, I conducted the study in a written medium giving each person a copy of their version of the survey. This made it easy to later collect and export the data.
Slide 4- Examples
Here are two examples of actual surveys completed by subject participants.
Slide 5- Data Analysis
On this slide you’ll find the two finished sets of data painstakingly transcribed into an excel spreadsheet and analyzed. This wasn’t actually too hard due to the length and multiple-choice nature of the survey.
Slide 6- Graphs #1
As you can tell by this slide, the distribution of each set of data for the first question is similar. I set the first interval to include both 1 and 2 corresponding to negative responses since few people responded this way. I found that most people chose 3, or neutral, as majority of high schoolers at pine view aren’t actually being taxed at a federal level.
Slide 7- Graphs #2
Moving on, analyzing the last question we can see that in the left column, those who oppose the tax, much more people oppose when asked about the death tax when compared to the estate tax. This shows the possible response bias created by the question wording.
Slide 8- Conclusions
From this data and survey, we can infer a correlation between a negative question wording and a reduced response positivity. This can be applied in general to surveys and polls across the board, with companies and pollsters able to prefer what data they might want by changing the wording of the questions. In politics, wording is often changed in reference to questions on healthcare. While much more subtle, words like “state-run” elicit a negative connotation in reference to healthcare, while Medicare has a positive one. In truth, they mean the same thing. This is why many attempt to cite different polls that try and paint a picture about what American’s truly want, but in reality they are just manipulating how the questions are asked to get the answer they are looking for. In the future, this type of wording bias can be alleviated by using neutral wording along with other things.
Slide 9- Reflections
Looking back on my survey, I can see a lot of things I could do better. Since I chose 60 people out of a high school population of around 800, it was a little bit difficult to count out 180 three at a time and make sure I hadn’t given someone the survey previously. This is where the paper surveys came in handy. Another thing is the population being surveyed. Many people at pine view don’t actually know much about taxation, which I thought was interesting. In a more knowledgeable population, I can assume that the responses regarding policy would be more to the ends of choices. In the future, this type of research can be extended to nearly any and all survey questions, with polls attracting a great deal of attention for their ability to predict the outcome of elections and future policy. Often times, people throw their vote behind the candidate/idea with the most ‘support,’ but in truth the most popular thing might not be as easy to find out as one might expect.
