Baird's CMC | A Unique Global Communications Management Consultancy

Global Health Discourse 2023

Tel: +44 (0) 1495 828300
Email: team@bairdscmc.com
  • E-mail
  • LinkedIn
  • Twitter
Menu
  • Home
    • Sexual and Reproductive Health
    • Stakeholder Mapping & Strategy
    • New Services
    • Qualitative & Policy
    • Communications and Management Consultancy
    • Close
  • About us
    • Privacy Policy
    • Our specialist areas
    • Case studies
      • EDCTP sub-Saharan Africa policy research
      • Emerging Democracies and Green Issues
      • Gauging Attitudes Toward an HIV Vaccine
      • Increasing Immunization Coverage in Central India
      • Close
    • Close
  • Network
    • Network map
    • Associates
      • Michael Acott
      • Ashoek Adhikari
      • Francois Baird
      • Gerhard Butschi
      • Steve Bowers
      • Mark Chataway
      • Lenore Cooney
      • Paul Dillon
      • Marta Dourado
      • Martina Dörmann
      • Dirk Van Eeden
      • Alexandra Fullem
      • Frederick Fussi
      • Ingrid Gavshon MBA
      • Denise Gee
      • Denise Gray-Felder
      • Bert Griesel
      • Aman Gupta
      • Darren Jones
      • Danie Kok
      • Andrzej Kropiwnicki
      • Angelle Kwemo
      • Nikolay Kudryashov
      • Mari Lee
      • Simba Makoni
      • Steve Mallach
      • Terry Mandel
      • Matshidiso Masire
      • Valeria McFarren
      • Hugh McKinney
      • Chris Opperman
      • Ken Rabin
      • Mark Rittenberg
      • Simon Russell
      • Jacob Sesinyi
      • Melinda Shaw
      • Cormac Smith
      • James Snodgrass
      • Hema Viswanathan
      • Mina Volovitch
      • Gillian Waddell
      • Joseph Makwata Wambia
      • Chris Ward
      • Gysbert J Wessels
      • Marion Zibelli
      • Close
    • Senior consultants
      • Simon Hardie
      • Close
    • Close
  • Services
    • New Services
      • Media Training
      • Presentation Skills
      • Staff Engagement
      • Reputation Management
      • Close
    • Services
      • Communications counsel
      • New media strategy
      • Advocacy and coalition building
      • Crisis management
      • Market research
      • Strategic planning
      • Staff training and development
      • Risk management and mitigation
      • Close
    • Close
  • Sectors
    • Vaccine Hesitancy
    • Pharma in top 15 markets
    • Pharma in emerging markets
    • Pharma in underdeveloped markets
    • Pharma headquarters planning teams on market assessment
    • Global Development: Health Issues
    • Global Development: Environmental Issues
    • Health equity and access to care
    • Reproductive health and population
    • Vaccines
    • International Donors
    • Government Relations
    • Political Consulting
    • Telecom Sector
    • Financial Services
    • Close
  • News
  • ShopTalk
  • Get in Touch

Why even good market research is becoming less reliable

There is a widely held belief that views all over the world are becoming more polarised because market research says they are. This is most obvious in politics — people seem to be taking more and more extreme positions. But we have seen this phenomenon in qualitative and quantitative reputation research too: respondents seem more likely than they used to be to say that they have a very high opinion or a very low opinion of an organisation.

The picture is obscured because the world is full of market research companies doing bad market research: my favourite is asking if the respondent is aware of an organisation and then asking those who have said they are unaware of it what they think of it — this falsely keeps the sample size high but produces junk results. In qualitative research, more and more consultancy companies are doing market research with no professional training or oversight (we named and shamed one on this website a few months ago — look at “1. Bad Research”) and usually feed back to clients what they think the client wants to hear.

Even without the charlatans, we seem to be more divided than ever in our opinions. New research (for the US Bureau of National Economic Research) suggests that this may just be that people now understand better how market and policy research is used and try to manipulate the process. The researchers asked subjects to answer a set of questions that had factual but politically contentious answers (e.g. since President Obama took office, how has the unemployment rate changed? Or how many American casualties were there in Iraq in 2007 and 2008?). Some of the subjects were given a cash reward for answering correctly; some were not. Those who knew they would get paid for the right answer (or for admitting that they did not know)  were much more likely to be accurate — the gap between Republican and Democratic voters shrunk by up to 80 percent in the paid group.

In other words, respondents often knew the right answer but chose to give a wrong answer. They were, the researchers hypothesise “cheerleading”. Respondents know how polling results are used and they want to use their answers to change the actions of others. There are probably two components to this. First, respondents may, for example, think that if they inflate their perception of the number of casualties in Iraq it will dissuade policymakers from risking further casualties in future because the policymakers will over-estimate the political cost of allowing casualties. Second, respondents may want to signal to other participants in other polling that answers which serve a particular point of view are widely held by society as a whole — dragging opinion in the respondent’s direction. The researchers call all of this, “expressive utility that [respondents] gain from offering partisan- friendly survey responses.”

“Extending beyond political science, our results also inform understandings of contemporary public opinion. Scholarly and popular analysts alike frequently take survey responses at face value …  our results suggest that the concern should be far more widespread,” the researchers write. It should indeed. Let’s say that we (or another group of qualified specialist researchers and consultants) are asking a government official what he thinks of a multilateral organisation funded in part by the official’s government. The official may say that she has a less favourable opinion if she thinks that it will reduce the chance of more requests for extra contributions or she may bias the response to be more favourable if she thinks that it will send a signal to her own government that more support is required.

Here are four things we can all do immediately

  • follow good practice and never reveal the identity of a client before an interview. It makes recruitment much harder but results far more accurate. On the other hand, agreeing to reveal the client during the interview reassures the respondent that there will be an opportunity for targeted signalling at the end thus — we think but we cannot show — reducing the risk of biased answers in the early part of the interview
  • ask a few simple but controversial factual or semi-factual questions during the warm up that should show you whether the respondent is using the research to signal or to cheerlead
  • insist on long-form, semi-structured interviews by subject experts in a language in which the respondent is fully comfortable. Good questions, good follow up (off the script) and good analysis increases the chance that you will spot signalling or cheerleading and spot real trends and ideas
  • forget forever the idea that quantitative research is inherently more reliable than good qualitative studies

If you want the painless summary of this research, listen to a five minute interview on National Public Radio. To read the whole paper (Bullock, John G et al – a team from Yale and UC San Diego ), you will have to pay $5 but the summary is free here (and there’s a link to pay for the 100-page PDF. It is an excellent read and well worth the $5).

11949810_m

Share this:

  • Click to share on Twitter (Opens in new window)
  • Click to share on Facebook (Opens in new window)
  • More
  • Click to share on LinkedIn (Opens in new window)
  • Click to email a link to a friend (Opens in new window)

Related

Filed Under: News

Contact Us

    Your Name (required)

    Your Email (required)

    Subject

    Your Message

    Please enter the letters below and press Send

    captcha

    Latest Blog Posts

    Welcome

    An astonishing decline in child immunisation numbers in Africa amid the rage of COVID-19, yet another victim of the pandemic

    Global Elite are Bored with Health, Just When It’s Getting Really Exciting

    Copyright © 2023 All rights reserved. Sitemap