Experts – How to Tell
- Admin
- Jul 1, 2023
- 7 min read
Updated: Dec 8, 2023
Deferring to expertise is not being a ‘sheeple’. It is not a bad thing, and it’s not an argument from authority to trust expertise where necessary. It’s a common trope of course to call it out as an argument from authority, by science deniers.
But what about when science communicators reference an expert to explain something – that’s just the same as an expert with a differing view, doing the same thing? There are important differences, ways to tell. But they are not all that easy to spot.
Perhaps the biggest question here is ‘why is ‘your’ expert saying one thing more trustworthy than ‘my’ expert saying something else?’ I’ll try to answer this with a series of explanations and examples.
The Scientific Consensus
Why would you trust scientific institutions – this expert is being suppressed and why isn’t his opinion as valid as any other doctor? He thinks outside the box. He isn’t scared to ‘have a different opinion’.
Well, why would you listen to anyone else outside of the scientific consensus? The rational answer can’t be because they are an outlier – this would be committing the Galileo Fallacy. It can’t be merely that they have relevant expertise, as this would be trumped by the overwhelming consensus of said expertise. It can’t be because they don’t have expertise, and it can’t be because they are expressing claims that don’t hold up to established scientific knowledge.
A ‘consensus of opinion’ can mean ‘what the majority of people think’. A scientific consensus of opinion is not to be thought of as ‘opinion’ though; it is actually an overwhelming majority of strong evidence. A rounding up of the most reliable data, to form the most likely conclusion. Just as importantly, a reliable way to exclude the least likely conclusions.
Scientific consensus is not a selective in-group of people of the same institute. It is a globally open endeavour. They are mostly typical, working people from all kinds of backgrounds, cultures, and countries. To argue against the consensus of expertise is to argue against the fruits of many decades of sound reasoning, logic, and scrutiny of the truth from many differing viewpoints. This is because science as a method done well, is the intellectually honest pursuit of truth.
This snippet from the below episode from 13.02 mins – 14.58 mins perfectly describes the value of the scientific consensus, and why it works, why it’s fair, and why it’s reliable:
What changes scientific consensus? A LOT of higher quality evidence and reasoning. How is that achieved? Through globally collaborative and accurately replicated scientific work – i.e. the scientific consensus of evidence. You can think of the scientific consensus as the natural emergence of truth over falsehoods.
So, next time you catch a viral clip of a maverick or ‘ground breaking’, ‘suppressed’ or ‘silenced’ ‘expert’ on Joe Rogan’s podcast, I want you to remember this quote from the late Robert L. Park:
“It is not enough to wear the mantle of Galileo: that you be persecuted by an unkind establishment. You must also be right.”
Refutations to Common Flawed Arguments
“A PhD means nothing these days.”
The sentiment here is that just because a person has a PhD, doesn’t mean they are right or telling the truth. This is correct, and any critical thinker will tell you that. There are plenty of once-trusted experts with integrity in their fields who turn away from truthful discourse, for all kinds of reasons.
However, it absolutely means something to have a PhD. If I’m going to cite a source or talk about an opinion by an expert, or self-proclaimed expert on anything, you better believe I’m going to require that confirmation of a high level of dedication to their expertise at the very least. The inverse of that is of course the YouTuber with nothing but web traffic to show or gain from it.
It is not the be all and end all to have a PhD, and if you were to think so, this is where it becomes an argument from authority. But it’s a first, necessary, and important step in determining whether you can trust the expert in question with what is being claimed.
“All views deserve to be heard.”
The statement alone is arguably true in a simplistic, preliminary context. But really, the question should be ‘when is it relevant, necessary, or useful for all views to be heard, and therefore which views are more deserving to be heard?’.
Championing the minority opinion and elevating it to the same level of balance as settled science is not only of little to no value – it is wrong. The size of the minority opinion on thoroughly studied scientific topics, is proportional to its legitimacy.
Without the logical understanding, this thinking can quickly become conspiratorial. Taken from Rational Wiki, where it’s not a conspiracy, then:
“An appeal to the minority is inherently limited. If someone successfully persuades other people that they are right, then their opinion would increasingly lose its minority status — and eventually would become majority opinion.”
Views that have been debunked, when spouted by the next ‘free thinker’, are in fact not worth hearing. Certainly not deserving. Why? Because they’ve been heard and discredited hundreds of times, by strong enough scientific evidence that it becomes a waste of time and a hindrance of progress to hear it all over again. Not only that, but it becomes a tool for science deniers to use to gain attention and disrupt progress.
To lay audiences though, this entitlement can make an old, debunked idea sound novel. It becomes a powerful rhetorical tool of persuasion, under the banner of the erroneously used ‘freedom of speech’.
“Science tells us to ‘trust the experts’ but silences experts ‘it’ disagrees with.”
This also falls under the Galileo fallacy.
Furthermore, ‘it’ meaning ‘science’ is not a dogmatic, single entity – it is a method.
To say ‘trust in experts’ doesn’t mean only one expert per say, but the scientific consensus of experts. So, when you have 100 experts saying one thing, and 1 expert going against that consensus, to trust experts is to trust the 100.
Science deniers will use the rhetoric that their 1 or 2 experts are experts in X subject, to falsely equivalise their flawed viewpoints so now they can say that they are listening to experts. However, this is when it becomes an argument from authority: they are hinging on the fact that they are a qualified expert alone, and not on what the overwhelming majority of evidence and consensus finds. The fact is that the vast majority of ‘outlier’ opinions turn out to be, or have already been established to be, false.
Checklist for Evaluating Experts
It’s important to note that these are only a set of the most obvious or telling traits to watch for, and that even someone who check all the positives can change and begin to double down on a wrong idea or personally held belief! This said, the table below is a pretty solid starter guide for a lay person.
Things to Look for | Explanation | Positive or Red Flag? |
|---|---|---|
Language: ‘Suppressed’ ‘Establishment’ ‘Silenced’ ‘Cancelled’ ‘Mainstream Science’ | Conspiratorial in nature. You will often find a doctor ‘being silenced’ translates more truthfully to ‘repeatedly making claims which have been shown to be scientifically false; therefore not being entertained any more by the scientific community’. This is something you can verify, as honest science is a transparent endeavour. | Red Flag |
Source: Science Journalists, science communicators, Scientific Journals | These are your best bet for reliable and unbiased information (although be sure to verify and evaluate each individually on their merits). They are often long-form, less exciting and harder to read – but that doesn’t matter if you want truth over entertainment. Isn’t truth itself entertainment? Maybe that’s just me. | Positive |
Source: YouTubers, TikTok, Joe Rogan, Russel Brand, Politicians etc | These (and more) tend to be really sketchy sources for information, specifically chat host formats and shorts. If you don’t know the expert, be wary of the platform, and consider it less reliable. Chat shows are entertainment and content farms, not academic journalism – and the host plays a huge part. | Red Flag |
Credentials: PhD, DrPH, MS, BS Degrees | These are high quality scientific degrees. It’s not an argument from authority to first verify that the person in question knows what they’re talking about. Are they highly qualified in the relevant areas? | Positive |
Credentials: BA, Certificates, College Level, non-relevant | A Bachelor of Arts is not a scientifically literate degree. Lower level college degrees or diplomas are not very useful, and be wary of fake credentials too like ND, DC. Pay attention also to the relevance of that degree to the subject in discussion. | Red Flag |
Language: Uncertainty, Emotionally Neutral, Focused on explaining clear talking points without tangents | This is a sign of honesty, and a genuine desire for truth based on what we know. Being intellectually honest with oneself often means abstaining from taking a standpoint until the science is strong enough to confidently do so. It can appear cold and uncaring and indecisive, but it is necessary to have an honest point of view and assess claims accurately. | Positive |
Language: Certainty, Black or White thinking, Non Sequiturs, assumptions, and tangential tendencies | Opinions and biases come commonly in the form of dichotomous thinking and ignoring nuances. They take sides as an absolute certainty and ignore inconveniences to their position. They use irrelevant points such as attacks of character and emotional leverage as the core of their arguments, instead of discussing each small point on its merits. | Red Flag |
Conflict of Interest | Is there a sale involved nearby? A product that the person/source is also promoting? Are they endorsing someone else’s product on their site? Be wary, especially with novel sounding, cleverly marketed health products. | Red Flag |
Debates | Do not be fooled by anyone brazenly challenging scientists to ‘debate’ the science. This is a challenge of charisma and social duelling, and never achieves a better understanding of any technical topic for the audience. They are designed to persuade audiences using any technique possible, not to settle science. | Red Flag |
Language: Technobabble, Jargon and Pseudo-babble, Word Salad | A common technique used by perceived experts with an agenda to push is to dazzle audiences or readers with technical language, but be especially wary of this when related to consumer products/services. Language should be clear to consumers, and sounding science-y isn’t the same as being science-based. A little googling can sometimes reveal nonsense phrases, or expose pseudoscience. Wikipedia is your friend. | Red Flag |
Checklist: What Can YOU Do?
Establishing a good source of information takes a lot of careful thought. What, you thought it would be easy? Unfortunately, the easy part is just sharing the unreliable and bonkers stuff out there being thrown about. Brandolini’s Law would have it that refuting nonsense takes phenomenally more work than producing it. Nonetheless, here are some starting points a lay person can practice.
UPDATE 28/07/2023
TIP: Always begin your investigations mentioned in the table below, from an incognito mode browser to avoid contamination of your prior search algorithms, this can help somewhat in achieving a less biased search result.
Things you can do | Explanation |
Wikipedia | Contrary to what you may have heard, this is generally a very accurate resource and an easy, first-hand check. Wikipedia has citations for verification of knowledge. Search the expert in question. It will very quickly show you any controversies or red flags on that person, plus give you an overview of their expertise. This is an invaluable tool. Use this to begin to evaluate all sources in the previous table (sites, verified/relevantly qualified journalists, experts, podcasts, news channels etc). |
Seek Citations | Where just about any reference to data or a testable claim is made, a credible communicator or journalist should be able to back that up. It’s not always practical to make a citation for every single claim, but look for them for the main points of the piece and where any stand-out claim is made. If it doesn’t contain one, you can always attempt to search for and verify it yourself but be wary of the sources you find, and always evaluate them individually. |
Check the Source | Linked to the last point, verify citations are of high quality (first-hand source, relevant and well-trusted science institution). A poor quality example would be a journalism piece linking to another journalism piece like The Sun or another 2nd-3rd hand account, and never getting to the source material. Journalism from a general, non-science qualified source is prone to making mistakes and misinterpreting the topic, in favour of simplification or sensationalism. |
Seek the Scientific Consensus | To verify whether an expert is making a reasonable claim or just sounding reasonable because of their credentials, look for several independent scientific sources and cross check them. Where they might differ may indicate a difference of opinion, but where facts are concerned, they should line up pretty consistently. Go for a Science Journal like the New England Journal of Medicine, a science journalism site like Science.org, an independent science media source like BBC Inside Science, or a long form critical thinking and scientific skepticism podcast like The Skeptics Guide to the Universe, where the hosts are practicing scientists and logic enthusiasts. |
Search any mentioned Institutions, Credentials, and do a Background Check | As well as checking sources of information, specifically searching the names of impressive or authoritative sounding institutions is necessary. You might find immediately that they are linked with pseudoscience, conspiracies, or are notoriously unreliable. For example, Children's Health Defense is a real nonprofit activist group. Sounds legitimate and well-meaning. It is in fact one of the largest anti-vaccine organisations in the world and has caused children far more harm than good. Another example, Ted Kaptchuk is a Harvard professor who was given credence on one science podcast, which is astounding when you learn that his only qualifications are in pseudoscience – verified nonsense. He does not have a medical degree. He is an outlier with more credit than is due. This sometimes happens but isn’t the norm. |
Keep YOURSELF in Check | Lastly, you have to understand that the reasons most of the tricks and red flags work, is that they are designed to exploit the human brain, and they work on ALL of us. Ask yourself if you are emotionally invested in an idea and try to leave that bias out. Try to recognise the line between fair and logical reasoning and motivated reasoning, which can blind you to evidence against what you’d prefer to be true. If you don't care emotionally, then you should have no problem changing your mind when corrected. |
In addition to the above point on checking the source, there's a fantastic resource you can check called Ad Fontes Media, which is a transparent, highly analytical, and unbiased media-rating organisation. They have:
Media literacy tools for educators and individuals
Actionable data for advertisers, publishers, and tech platforms
They rigorously and regularly parse media sources and rate them in a fine tuned chart to help you distinguish what the most reliable sources are from the most politically or socially polarised, and from the factually reliable to the downright trash sites.
They also have lots of information on their methodology, so you can be sure their diligence is done.
Don’t Decide Too Early
An important take away message from the above tables is that you should never just establish one red flag, or one positive attribute of a piece and call it conclusively good or bad. This is not a dichotomous exercise; it is a continuum.
To take the first point that confirms your prior assumptions as conclusive would be confirmation bias. The way to go about avoiding confirmation bias is to look for as many of the pros and cons as possible, and weigh it all up together, against a different point of view from another expert. You might become less confident in the expert as you go, or more, or up and down – but that is correct, and not ‘indecisive’. It shows you are attempting to treat each piece of information fairly and thinking critically about conflicting positions.
As well as not damning an expert too early, don’t be too quick to trust a particular expert either. Do the necessary checks to help you establish an appropriate level of trust or skepticism.
This Skeptoid episode is another good 15 minute primer on how to go about these methods yourself, and what to look for in spotting misinformation and pseudoscience:
Conclusions
If someone’s strongest defense for not trusting in scientific expertise is that you can’t trust ‘people’, or that they have some unproven ulterior motive, or unsubstantiated claims about a huge conspiracy, then they should ask how they can never see to turn that around on their own claims.
If they can’t trust scientific communities who dedicate their life to do their job of approximating objective truths, then how can they trust a group of non-expert youtubers, or a stoner with the biggest platform in the world who chooses to promote verifiably anti-vax politicians (and his own supplements)? It is an absolutely astounding case of cognitive dissonance and confirmation bias. Pure ignorance in its most brazen form, to put it harshly. Charitably, a case of missing the fundamental values of evidence, logic, and reasoning; all of which critical thinking encompasses.
Does it not occur to people that despite the constant accusations of scientists having ‘financial interests’ (usually the opposite), that Russel Brand might no longer do TV because he makes an easy living off YouTube traffic by playing into conspiracy thinking? That Joe Rogan became so huge on various platforms by fuelling misinformation and controversy, earning a multi-million dollar deal on Spotify and direct traffic to sell his bogus supplements online? After all, and as pointed out in the above Skeptoid episode, much content online is designed for high engagement, not for accuracy.
Does it not occur to people that discredited doctors who now don’t practice because they lost their medical license years ago, only weigh in on these subjects that they are behind in (and often not even qualified in that field) to plug their Substack, where they make an easier living selling fake treatments and producing tonnes of deceptive and predatory content?
How can you establish what counts as relevant expertise? There is no airtight answer to that, but there are plenty of reliable aids, as outlined in this piece. We must defer to the most sensible options. We must make more effort to decipher these questions. Then again, maybe I’m just a ‘shill’ for the truth.
Comments