Credibility in science is dependent on a process of peer review. Scientific studies and their conclusions must be written, submitted to a journal, reviewed by peers, and deemed valid and worthy of sharing with the community. There is a lot of pressure on reviewers to critically evaluate the work to ensure that only legitimate studies with matching conclusions are published. This creates an inherent difficulty when work does not jive with previously published data, since it creates a larger hurdle to overcome when convincing reviewers of the legitimacy of the new work. As with any field, there is also preference given to work done by someone who is already well known (call it street cred) or to someone who has friends at the journal. Occasionally, a poorly controlled study will slip through the cracks or an unforeseen variable crops up afterward. The peer review process isn’t perfect, but it is worlds beyond anecdotes and conclusions drawn from people lacking a scientific background.

This is a process that is not present in blogs and news periodicals. A recent article from Authority Nutrition and republished by Business Insider shows the danger of scientific conclusions that would probably not pass peer review muster. The article spread quickly among people lacking the scientific background to critically analyze the piece, allowing many people to consider themselves “experts” on a topic. Unfortunately, the knowledge those people now possess is faulty and more difficult to overcome. Although this happens occasionally with peer review, perpetuating faulty information is far more common in blogs and articles written without a basic scientific background.

As with anything that goes against conventional wisdom, it is best to consider the aforementioned piece carefully. There is always a chance that conventional wisdom is wrong, but it is irrational to make that assumption blindly. Fortunately, there are some simple ways to determine whether something is worth your consideration or if it is some form of misinformation.

In this example, nearly every reference pointed to blog posts on the same website. This should immediately raise alarms about questionable credibility. There are countless scientific studies and peer-reviewed manuscripts on nearly every topic, and nearly every opinion can be backed up without self-reference; if the author cannot find a legitimate study to support their conclusions, then you should continue with increased skepticism as to their credibility and motives. Even dissenting opinions can usually be found through credible sources. PubMed covers all of science, while NLM, Mayo Clinic, CDC are good places to find anything specifically related to health and medicine.

Since the article in question touts several charts that “clearly” support their conclusion, I initially skimmed down to the first graph with a large heading geographically linking saturated fat to reduced heart disease. Although counter-intuitive, I was open to the idea until I saw that the graph plots saturated fat relative to deaths attributed to heart disease by country. Having a disease and dying from it are highly dependent on where you live and the availability of medical care. For a disease that can be treated with surgery and medication, this is especially relevant. Furthermore, wealthier countries consume more meat (with the exception of some poorer meat producing countries) and have lower mortality rates from treatable illnesses, which makes me strongly question the conclusion that has been affixed to this graph. This alone does not negate the premise of the article, but read through it and check the occasional reference to determine whether this piece is something you would be willing to put any stock in. As an added bonus, until the comment section is closed, there are often useful insights provided by concerned readers. This is as close as we get to peer review in blogs and articles.

As an exercise in critical reading as happens in peer review, what conclusions did I draw from each of the six graphs used in this piece?

1. One is more likely to die of heart disease in the poorer parts of Europe than the wealthier nations.
2. Obesity rates have been getting steadily worse for over three decades despite the release of low-fat guidelines.
3. Caloric restriction diets in women have their shortcomings, but restricting carbohydrates seems to be effective.
4. Shortenings and oils have replaced many other sources of added fats since around 1950.
5. Remember those low-fat guidelines from chart (2)? It appears that women have listened.
6. Trans fats (which were a major component of margarines back in 1997) are more dangerous than other fats.

My overall conclusion based on these graphs is that obesity is still on the rise and reducing saturated fat consumption in women has not curbed the epidemic. From these graphs, it is unclear if men still consume as much saturated fat, although the amount of total added fats has gone up. The combination of charts used makes it hard to draw any meaningful conclusions, despite the author’s very strongly stated interpretations. If I was a reviewer, I would strongly insist that this article not be accepted for publication.

Put simply, correlation does not show causation, but a well controlled study can. Using specific points in time (such as the publication of dietary guidelines) as the reason for the continuing rise in obesity is not sound science. The reason a controlled study is telling is that it takes into consideration the other factors at play such as other changes in diets over time (e.g. processed meats, salt, carbohydrates) along with changes in lifestyle (shifts from active to sedentary lifestyles that have occurred over time).

When reading any interpretation of data picked from various sources (or very few sources) to make a point, proceed with caution and draw your own conclusions as well. If you find yourself in an area in which you don’t have the necessary background for the exercise, then educate yourself or consult with someone you trust that is better suited to the topic. Don’t just turn to your friend or TV personality that pretends to know everything and assume that are always right. There are plenty of questionable sources of information out there, and they are a danger to knowledge and science everywhere. Don’t just listen with an open ear, but consider with a critical mind.