Thursday, 24 October 2013 00:00
Days before the September primary, a coalition of Rochester media organizations published an opinion poll conducted by Siena College’s Siena Research Institute on that city’s Democratic mayoral primary. The Siena poll had the incumbent mayor leading the challenger, 63 to 27 percent. The challenger won with 58 percent.
The theoretical margin of error in the Rochester poll was plus or minus 4.4 percent, which means that if, say, 50.0 percent give a particular answer to a question, the “real” result is reliably somewhere between 45.6 and 54.4 percent. Almost all published polling is designed to have a “confidence level” of 95 percent, which means that if you conducted the same survey over and over among randomly selected pools of people, one time out of 20 the results will fall outside of the expected margin of error. With piles of polls published every week during the Presidential election, it’s easy to spot these “outlier” poll results. When you have only one or two newspaper polls, like the one recently published by Newsday and Siena on the Nassau County races, you don’t know what you’re getting.
In addition to the Rochester poll, Siena polled using some of the same questions in the Buffalo and Albany municipal races and polling results more closely resembled actual voting results. This indicates that the problem was not so much in the wording of the questions but “in a sample that was off, through no fault of our own,” in the words of the institute’s director.
Siena and some other college-
based institutes jockey for media attention by running these polls on local races. Siena’s polling methodology seems no worse than the others. However, because of their ubiquitous deals with newspapers, radio stations and television stations around the state, and their well-worn media contact lists, Siena has made itself a go-to source for lazy journalists and editors who think that reporting on poll results is meaningful or useful analysis.
Poll results can become self-fulfilling. We see it all the time at the national level of politics. Rochester, Albany and most other places where these polls are being published have media markets that are cheap enough or neighborhood-based political cultures that are still intact enough to break through after a setback. Not Nassau County. Not without vast resources.
The wording of any question helps form the response. By the time this is published, the courts will decide if the statewide proposition on casino gambling will be thrown off the ballot because a last-minute wording change pushed through by the Cuomo administration is skewed to increase the chances of voter approval. Phrasing, word order and unintentional bias in how questions are asked all affect respondent choices.
I shouldn’t have to ask a survey caller, as I asked one from Siena last October, if I was allowed to say, “I don’t know”?
Limiting choices makes a big
difference, and the published answers effectively limit the choices most candidates are willing to make in offering any solutions.
Good political campaigns poll to continually test the assumptions behind their strategies. Publishing a momentary snapshot of opinion on issues without context and without trend data raises questions about timing and motivations.
Even if it were true, and it isn’t, that between 77 and 89 percent of some pool of Nassau County voters had some meaningful opinion on “down ballot” county races, as suggested by the recent Newsday poll, what does that really mean? How does it inform voters? Does it make voters pay more or less attention, and to which candidates and issues? Will the next poll be portrayed as an exciting tightening of the races? Are you feeling manipulated?
Are you? Yes or No?
Michael Miller is a freelance writer, designer and strategic consultant who has worked in state and local government. Email: firstname.lastname@example.org