Search results: research bias

I get letters like this from food trade associations all the time.  Here is the latest:.

The U.S. Highbush Blueberry Council has issued a request for research proposals (RFP) for which it offers grants ranging from $75,000 to $300,000 (or larger).

Here’s the get-go bias point (my emphasis):

The goal of our research funding is to provide initial funds, or additional funds, to explore blueberry health benefits.

The Council wants research to demonstrate benefits.  Of course it does.  These will be useful for marketing.

A priority for funding will be given to human clinical studies however the committee is also interested in further investigation of possible health benefits for pet or performance animals including dogs, cats and horses.

If the proposal is unlikely to demonstrate benefit, it won’t be funded.

That’s why I consider industry-funded research to be about marketing, not science.

Last week, my industry-influenced study of the week involved kombucha, although the involvement appeared minimal.

All kombucha and placebo drinks were donated by Craft Kombucha. Craft Kombucha did not have any access to data reported in this study. No author has any financial ties with Craft Kombucha. SD was employed by MedStar Health. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

One of the authors of the study, Daniel Merenstein, wrote to object to the way I characterized it (quoted with his permission).

…But you do make it very clear in the article that all industry did was donate free drinks and had no access to data. Not sure how that really deserves being called influence. But my much larger point is this statement, ” It’s easy to find claims for its health benefits if you search for them, but much harder to find science to back them up.  IIf you can demonstrate benefits, you can sell more products.  Hence, this study.”

I think it is exactly the opposite. It is much easier to just say your product works or even better yet to get an influencer to drink your product.

But to actually put your product into scientist’s hands and have no access to data or publication is a huge step forward in food science. Look at JAMA every week, almost all the drug studies are -. The kombucha maker should be applauded for their bravery.

We didn’t going looking for a + outcome but registered our trial and stated a priori exactly what we would be looking at and reporting. This study has many limitations but not the ones you mentioned.

Dan

I appreciate thoughtful and respectful letters like this .  This one especially deserves a response.  Dr. Merenstein implies that this is an investigator-initiated study designed to test an investigator-initiated hypothesis.  Such things do happen.  Unfortunately, they are not the norm.

Here’ what concerned me about the study:

• It involved a kombucha company, even if lightly.  Much evidence demonstrates that company involvement in research ia highly correlated with positive outcomes, so much so that it has a name, the “funding effect.”
• Funding influence is thorougly demonstrated to occur at an unconscious level; investigators do not intend to be influenced, are unaware of the influence, do not recognize it, and deny it (even in the face of much research to the contrary).  The unconscious influence usually shows up in the way the research question is asked or in the interpretation of the results.
• Statements that funders have no involvement in the research have coften been shown to be false.  Exceptions do occur; this may well be one of them.
• This is a one-food study.  It is impossible to control such studies for dietary and other lifestyle confounders unless done in a locked metabolic ward.

On this last point, I am always suspicious of one-food studies because I find it hard to believe that a single food can make a measureable difference in chronic disease outcome.

I would like to know a lot more about how the microbiome works before being convinced that kombucha has any special health benefits (I do think it is delicious).

To their great credit, these authors fully disclose the limitations of their study (it was small) .

Dr. Merenstein says this study is really about the science.  In this case then, the bias is one shared by all scientists—a belief and the desire to prove it,–in this case that kombucha has particular health benefits .  If scientists didn’t have such beliefs and desires, no science would ever get done.

Such personal biases are indeed quite different from bias induced by financial interests with a company making a product.

Perhaps I misjudged this one.  If so, I owe Dr. Merenstein and his colleagues a sincere apology, here offered.

I thank him for writing and giving me the opportunity to discuss these issues again.

(For detailed discussion and references on issues related to industry research funding, see my book, Unsavory Truth: How the Food Industry Skews the Science of What We Eat).

I posted several examples of industry-funded studies this week in part to reduce my backlog but also because of charges that (1) doing so constitites ad hominem (personal) attacks on authors, (2) I should be focusing on the science, not who paid for it, and (3) I have my own ideological biases.

To the first point:

I do not see industry funding of research as a personal matter.  I see it as a systematic problem.

If I see a study titled “Effect of food product X on disease Y,” I can often guess that

• The food’s manufacturer or trade association paid for it
• The study outcome will be favorable to the funder’s commercial interests

This phenomenon is so systematic that it has a name: The Funding Effect.

To the second point

Researchers who study funding effects, and there are many, note that the scientific conduct of the studies is not usually an issue.  Instead, the influence of the funders shows up in the way the research question is framed or the results are interpreted.

The easiest way to explain the research question bias is to cite the requests for research proposals I often receive from food trade associations.  These say: “we have (this much money) for research to demonstrate the benefits of our product on (one or more of these conditions).”

These groups will not fund research proposals unlikely to show benefits.

As for interpretation, industry-funded studies tend to report null results as positive; I posted several such examples this week.

To the third point

Yes, I have ideological or opinion biases and I try to be as clear as I can about them.  All investigators have such biases; otherwise they wouldn’t be doing science.  We all have something we believe in that we would like to prove.  Such biases are not discretionary; everyone has them.  In contrast, industry funding is about selling products, not science and is completely discretionary; investigators can do science without it.

The evidence?

I review the evidence for what I’ve just said here in my book, Unsavory Truth: How Food Companies Skew the Science of What We Eat.

In it, I cite many other books and papers addressing these points.

We all come to the same conclusions:

• Industry funding biases research.  But funded investigators do not recognize the influence, and deny it.
• The statement that accompanies many disclosure statements—“The funder had no influence on the design, conduct, interpretation, or publication of the results,”—is often untrue and must be taken with some degree of skepticism.

I see industry funding of food and nutrition research as a serious problem for public perception.  Even when the research is not conflicted, it appears conflicted.  That alone is a systematic problem.

The workshop covered the ways that funding sources (especially corporate funders) influence research quality and outcomes (precisely the topic of my book, Unsavory Truth: How Food Companies Skew the Science of What We Eat.

The report is a succinct yet thorough summary of the research on the effects of funding on research design, outcome, and interpretation.

Some of the speakers:

• Lisa Bero, PhD, Chief Scientist, Center for Bioethics and Humanities, at the University of Colorado Anschutz Medical Campus, Overview of the Evidence on Sponsor Influence
• David Michaels PhD, MPH, epidemiologist and professor at the George Washington University School of Public Health; former Assistant Secretary of Labor for the OSHA. How to Protect Public Health in the Face of Disinformation.
• Gary Ruskin, Executive Director, U.S. Right to Know,  Where are the points of influence on scientific research?
• Martin McKee, CBE, MD, president of the British Medical Association and Professor of European Public Health at the London School of Hygiene and Tropical Medicine. What You Don’t See That Counts: A Peek Behind the Smokescreen
• Laura A. Schmidt, PhD, Professor Health Policy in the School of Medicine at the University of California at San Francisco. Industry Funding Bias in the Case of Nutrition Science

This report is a great place to begin if the topic is new to you.

And so is this, of course.

I could not keep up with the number of e-mails I got last week with the subject line “Have you seen this?”

“This” referred to an opinion piece in JAMA entitled Backlash Over Meat Dietary Recommendations Raises Questions About Corporate Ties to Nutrition Scientists.

Definitely up my alley.

The article describes the outraged reaction to a series of papers in the Annals of Internal Medicine last year arguing that recommendations to eat less meat are unfounded (I posted about this at the time).  Some of the investigators involved in the meat papers turned out to have undisclosed ties to meat industry interests.

The JAMA article points out that the most outraged objections to the Annals papers came from investigators who have plenty of industry ties (plant-based) of their own.

My book, Unsavory Truth: How Food Companies Skew the Science of What We Eat, provides loads of evidence for a basic observation: research funded by food companies, regardless of the particular food studied, tends to produce results favorable to the sponsor’s interests.

It also describes how industry influence occurs at an unconscious level; recipients are unaware that they are being influenced.

How the influence gets expressed is also the subject of research.  Bias most frequently turns up in the framing of the research question.  It also turns up frequently in the interpretation of results.

Here is an example from a study of a healthy plant food—walnuts.

The study: Effect of a 2-year diet intervention with walnuts on cognitive decline. The Walnuts And Healthy Aging (WAHA) study: a randomized controlled trial.  Aleix Sala-Vila, Cinta Valls-Pedret, Sujatha Rajaram, Nina Coll-Padrós, Montserrat Cofán, Mercè Serra-Mir,  Ana M Pérez-Heras, Irene Roth, Tania M Freitas-Simoes, Mónica Doménech, Carlos Calvo, Anna López-Illamola, Edward Bitok, Natalie K Buxton, Lynnley Huey, Adam Arechiga, Keiji Oda, Grace J Lee, Dolores Corella, Lídia Vaqué-Alcázar, Roser Sala-Llonch, David Bartrés-Faz, Joan Sabaté, and Emilio Ros1.  Am J Clin Nutr 2020;00:1–11.

Conclusions: Walnut supplementation for 2 y had no effect on cognition in healthy elders. However, brain fMRI and post hoc analyses by site suggest that walnuts might delay cognitive decline in subgroups at higher risk. These encouraging but inconclusive results warrant further investigation.

Conflicts of Interest: AS-V, SR, JS, and ER have received research funding through their institutions from the California Walnut Commission, Folsom, CA, USA. JS and ER were nonpaid members of the California Walnut Commission Scientific Advisory Council. ER was a paid member of the California Walnut Commission Health Research Advisory Group. JS has received honoraria from the California Walnut Commission for presentations.  AS-V has received support from the CaliforniaWalnut Commission to attend professional meetings. All other authors report no conflicts of interest.

Comment: This looks to me like a classic example of interpretation bias.  Although the study showed no effect of walnut supplementation, the authors interpret its overall results as encouraging.  This is putting a positive spin on null results.

Walnuts—and other kinds of nuts—are plant foods with healthy fats.  Why does the California Walnut Commission need to do this?  So you will buy walnuts rather than hazelnuts, pecans, or macadamia nuts (all of which are sponsoring their own positive-result studies).  Studies like these are mostly about marketing.

Questions of whether meat is healthy or unhealthy—and, if unhealthy, at what level of intake—are about much more than marketing.   They need to be studied as objectively as possible.  It’s best to keep industry influence far away from such studies.

Even if the science is done well from start to finish, researchers’ ties to food company sponsors give the appearance of conflicted interests.  Such ties are best avoided from the get-go.

Follow-up

• A detailed analysis of the JAMA piece by the anonymous CarbSane blog
• Naomi Oreske’s comment in Scientific American
• Texas A&M’s letter to Harvard requesting an investigation of faculty belonging to True Health Initiative (who complained about the meat guidelines)
• Texas A&M’s Chancellor Calls on Harvard to Look Into Faculty’s Beef-science Ethics (AgDaily)
• Beef with Harvard (Inside Higher Ed)
• From Beef to Bots? Harvard Professors Mired in Debate over Spam Emails, Industry-Funded Research (Harvard Crimson)
• Conflict of Interest in the Post-Truth Era: What Is It, Now? (David Katz)
• Processed Meat “Guidelines:” Why the Exceptional Reaction? (David Katz)
• Tobacco, Soda, and the Beef with Truth (David Katz)
• Massive Meat Mayhem, Part 3 (ConscienHealth)
• Perspective: Red and Processed Meats and Health Risks: How Strong Is the Evidence? (Frank Hu et al)

The reasons behind the Trump Administration’s efforts to relocate the USDA’s Economic Research Service out of the Washington DC area are becoming increasingly transparent: the administration views the conclusions of ERS research on SNAP and other key issues as incompatible with its agenda.

I wrote about the real reasons behind the proposed transfer in a post on September 21 last year: 

Former [ERS] officials were unanimous in arguing that the proposal to relocate the agency outside of Washington DC and reorganize it into the USDA Secretary’s office was “ill-conceived,” made no sense, was done without appropriate consultation, was potentially illegal, would politicize the agency, and would damage, if not destroy, an agency that is the jewel of USDA. The USDA says the reasons for doing this are easier recruitment, cheaper rent, closer alignment with the Secretary’s policy initiatives, and getting the agency closer to stakeholders.  None of these bears up under even the most casual scrutiny. So what is this really about? I’m guess that this is about getting political control over—silencing—an agency that conducts independent, unbiased, nonpartisan research that risks leading to inconvenient truths.

Now, ERS employees are coming forward to state publicly that they view the transfer “as a crackdown on research that’s unflattering to White House policy priorities.”

According to Politico, ERS personnel have done an analysis of which researchers are being told to stay in Washington, DC, and which ones are being told to move.  The ones being transferred out

are largely specialists in issues like climate change, food stamps and trade policy, where economic assessments often clash with White House policies.  The list, shared exclusively with POLITICO, shows a clear emphasis was placed on keeping employees in Washington whose work covers less controversial issues like crop planting over those whose research focused on areas sensitive to the administration….USDA released a written statement from Perdue arguing the relocation is not connected to the work being carried out by ERS economists. The secretary has long said the move will save taxpayer money and bring the agencies closer to major farming regions.

Some members of the House of Representatives are trying to block the relocation but even if they can, it will be too late.  By the time any legislation gets passed, the move will already have been accomplished and key ERS researchers long gone.

Eight Democratic senators led by Sen. Chris Van Hollen (D-Md.) co-sponsored a measure that would prevent ERS and NIFA from being moved.  Van Hollen is also trying another route: Politico reports that he has  put a hold on the nomination of Scott Hutchins to be USDA’s chief scientist…”to indicate the seriousness of Van Hollen’s concerns about the proposed relocation.”

In the meantime, everyone is waiting to see where USDA intends to move the ERS.  The options are Kansas City, the Research Triangle in North Carolina and multiple spots in Indiana.

Whatever happens, the damage has been done.  Many ERS researchers have retired or resigned.

This agency produced authoritative analyses of food and nutrition issues that I relied on for much of my work.  This is an incalculable loss. and nothing less than a national tragedy.

In response to my commentary in JAMA Internal Medicine late last year, “Food Industry Funding of Nutrition Research: The Relevance of History for Current Debates, the journal published an objecting letter from Morton Satin titled  “Incorrect Impressions Concerning Industry-Sponsored Research.”

Mr. Satin works for The Salt Institute, which promotes the idea that “everything’s better with a little salt.”

Here’s my reply to his letter:

In Reply Mr Satin raises several points in response to my recent Invited Commentary¹ about how food companies fund research for marketing purposes: (1) I give the impression that all industry-funded research is inherently tainted; (2) I ignore the industry’s triumph in fortifying foods with nutrients; (3) I fail to mention intellectual conflicts of interest; and (4) I should consider such issues before stereotyping.

First, my commentary was about research sponsored by food companies specifically to demonstrate the health benefits or lack of harm of a product, or to cast doubt on evidence to the contrary. It referred to a particularly egregious example—the sugar industry’s attempt to manipulate research results.² Although some industry-funded research does produce results contrary to the sponsor’s interests, such instances are rare.³ Most ends up useful in some way to the sponsors’ commercial objectives; it is marketing research, not basic science.

The point by Mr Satin about nutrient fortification has merit, but most of the basic research on nutrients used in fortification was conducted by independent scientists. Mr Satin’s own Salt Institute credits independent scientists for promoting iodization and convincing the industry to cooperate with public health authorities to iodize salt.⁴Pasteurization kills pathogens; iodide and fluoride address geographical deficiencies; and niacin, folic acid, and fiber replace amounts removed from foods by processing in the first place. Once public health authorities recognized the need, they demanded milk pasteurization or the addition of nutrients to flour. When dental researchers discovered that fluoride prevents cavities, Procter & Gamble recognized its marketing potential and funded research on fluoridated toothpaste.⁵

All scientists have intellectual biases—that is how science gets done and why science works best when researchers with different views of science repeat each other’s experiments. But the goals of scientists pursuing intellectual hypotheses differ markedly from those of companies seeking to sell food products.

Questioning food industry funding raises sensitive issues, not least because its influence on researchers occurs unconsciously, is usually unintentional, and is difficult for recipients to recognize.⁶ Food companies are not public health agencies and should not be expected to be; their first priority is to provide profits to owners and shareholders. Funding research helps with that effort. My purpose in writing the Invited Commentary was to bring the contradictions of food industry research funding to the attention of readers.

Conflict of Interest Disclosures: Dr Nestle’s salary from New York University supports her research, manuscript preparation, and website at https://foodpolitics.com. She also earns royalties from books and honoraria and travel from lectures about matters relevant to the initial Invited Commentary and this Letter in Reply.

References

1. Nestle  M.  Food industry funding of nutrition research: the relevance of history for current debates.  JAMA Intern Med 2016;176(11):1685-1686.  PubMedArticle

2.  Kearns  CE, Schmidt  LA, Glantz  SA.  Sugar industry and coronary heart disease research: a historical analysis of internal industry documents.  JAMA Intern Med. 2016;176(11):1680-1685.PubMedArticle

3.  Lesser  LI, Ebbeling  CB, Goozner  M, Wypij  D, Ludwig  DS.  Relationship between funding source and conclusion among nutrition-related scientific articles.  PLoS Med. 2007;4(1):e5. doi:10.1371/journal.pmed.0040005PubMedArticle

4.  The Salt Institute. Iodized salt. http://www.saltinstitute.org/news-articles/iodized-salt/. Published July 13, 2013. Accessed January 17, 2017.

5.  Ksander  Y. The invention of fluoride toothpaste. Indiana Public Media. http://indianapublicmedia.org/momentofindianahistory/the-invention-of-flouride-toothpaste/. Published July 10, 2006. Accessed January 17, 2017.

6.  Lo  B, Field  MJ.  Conflict of Interest in Medical Research, Education, and Practice. Washington, DC: National Academies Press; 2009.

I wrote a commentary for a study published this morning in JAMA Internal Medicine: “Food industry funding of nutrition research: The relevance of history for current debates.”

The study, by UCSF investigators Cristin Kearns, Laura Schmidt and Stanton Glantz, is based on their archival research.  They found documentary evidence of shocking manipulation by the sugar industry of a Harvard review of studies on dietary factors and heart disease published in the New England Journal of Medicine in 1967.

Kearns et al. discovered that the sugar industry trade association paid investigators at Harvard an impressive amount of money ($48,000 in today’s dollars) to produce research demonstrating that saturated fat—not sugar—raises the risk of heart disease.

In my commentary, I reproduced a figure from the sugar-funded 1967 reviews.  This summarizes the epidemiology showing that both sugar and saturated fat intake were then indistinguishably associated with increased mortality in 14 countries.

Nevertheless, the reviews exonerated sugars and blamed saturated fat.

Yes, I know that association does not necessarily mean causation, but I’m guessing that the epidemiology still shows that both sugars and saturated fats are associated with increased heart disease risk.

My interpretation: We would all be healthier eating less of sugary foods and fatty meats.

Here are the relevant documents for your reading pleasure:

• The press release from JAMA Internal Medicine
• The press release from UCSF
• The article by Kearns et al.
• Kearns et al.’s online tables analyzing the studies (look for Supplement in their paper)
• The New England Journal of Medicine articles from 1967: Part 1 and Part 2
• My commentary on Kearns et al.’s paper
• NYU’s blog post about my commentary
• JAMA Internal Medicine’s podcast interview with me and Stan Glantz (find it under Related Multimedia on the right side of the page)

The Sugar Association issued a response to today’s article by Kearns et al.:

We acknowledge that the Sugar Research Foundation should have exercised greater transparency in all of its research activities…Generally speaking, it is not only unfortunate but a disservice that industry-funded research is branded as tainted…We question this author’s continued attempts to reframe historical occurrences to conveniently align with the currently trending anti-sugar narrative, particularly when the last several decades of research have concluded that sugar does not have a unique role in heart disease.  Most concerning is the growing use of headline-baiting articles to trump quality scientific research—we’re disappointed to see a journal of JAMA’s stature being drawn into this trend.

I will post press accounts as they appear (I’m quoted in most of these):

•  Vox 
• New York Times
• STAT (a particularly detailed account)
• CSPI’s comment on Kearns et al.
• Reuters 
• Live Science
• Bloomberg News 
• Healthy Food America comment
• Business Insider
• Associated Press 
• NBC
• Time
• Quartz
• CNN
• Forbes
• United Press International
• Good Food
• Fortune
• Daily Mail
• Helio
• The Guardian
• US News
• CBS News
• Glamour
• Smithsonian
• Huffington Post
• Union of Concerned Scientists