Ask for Evidence infographic
If you’re not sure about something you’ve read or seen, follow these simple steps to #AskforEvidence? [...]
Surveys collect information from a specific group of people, or a random sample of a wider population.
A well-conducted survey can provide useful insights into people’s opinions or behaviour, but there are several things to watch out for that can make the results of a survey unreliable.
Who commissioned the survey?
Commercial companies can commission good quality surveys, but look out for PR spin. PR agencies will sometimes commission a survey just to get a product or survey into the newspapers.
They are often on trivial subjects designed to create a “news hook”. For instance, 80% of British families will be eating British turkeys this Christmas, says a survey by the UK association of turkey farmers. There are plenty of reputable companies that design and run high-quality surveys, for example IPSOS MORI.
Who did they interview?
Some surveys involve a group that doesn’t properly represent the wider population. For instance, if we wanted to find out what time most Londoners wake up, interviewing commuters passing through Underground stations at 7am wouldn’t tell us anything useful about Londoners’ sleeping habits as a whole.
How many people did they ask, and how many responded?
Surveys reported in adverts will often include a statement like “78% of people agree that the product did …” But if we don’t know how many people were asked the question, or what proportion of them actually responded, the results may be meaningless. There is no magic number that tells you how large a sample should be to give valid results. But as a rough guide, a political poll in the UK usually involves at least a thousand people. Tens of thousands of respondents would be needed for a survey to assess quality of life in different European countries.
How were the questions asked?
How people respond to a survey can be influenced by the person asking the questions - whether this is deliberate or not. People can be asked leading questions (questions worded or ordered such that they’re more likely to produce a certain answer), or the person being asked the questions may not give an honest reply because they’re embarrassed, don’t remember properly, or feel that the truth would cast them in a negative light.
Read Making Sense of Statistics to learn more about surveys.
(Note: some fields of science commission ‘surveys’ of land, water or the night sky – these are technical surveys which aren’t the kind we mean here).
If you’re not sure about something you’ve read or seen, follow these simple steps to #AskforEvidence? [...]
I’ve been Asking for Evidence for a couple of months now and it’s interesting how organisations respond to it. I’ve had an array of responses: Polite but dismissive….. First up [...]
Every month there are dozens of news reports about medical breakthroughs and wonder drugs. The internet is cluttered with adverts and chat-room conversations testifying to ‘amazing’ benefits. [...]
Politicians like to claim a lot of things, from how they’ve reduced unemployment to how they plan on investing in renewable energy, but how prepared are they to provide the [...]
There is a system used by scientists to decide which research results should be published in a scientific journal. This system, called peer review, subjects scientific research papers to independent [...]
If someone asks you something and you don’t know the answer, what do you do? You Google it. The internet is one the most powerful tools at our disposal, and [...]