Spider-Man in a brick window reading a book.
Spider-Man in a brick window reading a book.
Spider-Man learning to make decisions. Photo by Road Trip with Raj on Unsplash

Don’t Be Fooled by Fake Statistics

How to think about studies, experts, and bias

In response to Vijay Dubey’s very well thought out story, “Don’t Be Fooled by Fake News”, I want to add some thoughts of my own based on training and experience.

Bias and lies and lies.

Bias is when we come to a conclusion before we have all of the available information. The strength of your bias influences how early you jump to conclusions, but so do time constraints.

Skyline view from rooftop infinity pool.
Skyline view from rooftop infinity pool.
Sinking city? Platinum Fashion Mall Phetchaburi Road, Thanon Phetchaburi, Ratchathewi, Bangkok, Thailand. Photo by Sua Truong on Unsplash

This is important because our minds do not like to change their beliefs when confronted with new information. Once you decide OJ is guilty, it takes far more evidence to overwhelm your existing belief to convince you he might not be guilty before you even get to the evidence required to convince you that he isn’t.

Instead, your brain tries to make new evidence fit your existing beliefs. That’s called “confirmation bias.” A letter saying, “I expect to be billed for every nail” looks like a slick coverup if you’ve already decided the writer is a fraud.

Attorneys and judges work to mitigate confirmation bias by making sure jurors are neutral at the start of trial. In addition, jurors are forbidden to talk about the case until all of the evidence is heard and they are deliberating.

Consider the following facts.

  1. Yesterday, Anne stole some cookies.
  2. Today, more cookies are missing.
  3. You find cookie crumbs in the dog’s bed.

You can see how your view changes depending on when you reach a conclusion. If you decide at 2 that you know Anne stole the cookies, then 3 is a ruse and you will yell at poor Anne. Otherwise, you would continue investigating.

Unlike in trials, journalists have short deadlines to put together a story for timely publication. The reporter with the purest possible motives will unconsciously select for information that supports his bias. The short deadline means the story may be decided long before all of the facts are in. Once the new facts are in, the story doesn’t change due to confirmation bias.

The political views of the reporter determine how a story is biased, not whether it is. That’s why when you view a news source that doesn’t match your politics, it looks like lies.

Studies, experts, statistics, and more lies.

I often say that numbers aren’t people. In many cases, study conclusions merely reflect the values of the studies’ authors.

Chart showing murder rate decreases as Internet Explorer use decreases.
Chart showing murder rate decreases as Internet Explorer use decreases.
Internet Explorer Vs Murder Rate. Source unknown.

A school has a bullying problem. The principal decides to implement a new anti-bullying program. He gathers bullying statistics for a year before implementing the program. The next year, bullying decreases by 80% according to statistics. A student survey shows that more students believe the school has a bullying problem. The principal proudly announces that awareness is up and bullying is down in the school.

Is the principal right? If he talked to the students he might find out that bullies are taking advantage of the new policies to get their victims in trouble. Reports are down because the wrong people are getting suspended and kids are intimidated from saying anything. More students believe the school has a bullying problem because it is now worse.

That is the risk of ruling by numbers. It is easier to think of people as communities where we can measure inputs and outputs instead of as individuals, but that doesn’t always work.

In court, we have competing experts. The plaintiff hires expert A who testifies about his qualifications and what the data means. The defense hires expert B who reaches the opposite conclusion from the same information. The lawyers cross examine each expert on qualifications, biases, and methodologies. The jury then decides which expert is more credible.

In many cases, juries completely and correctly ignore the experts. They may be unconvinced by the expert’s testimony, or they might reach a different conclusion because they have different values. Part of the reason we have juries is to represent the values of the community.

News and politicians are like one side of the case. Instead of hearing expert A and expert B and weighing their opinions against our values, they say, “do what expert A says.”

That is bad. As citizens, we have the right to weigh expert opinions against our values and come to our own conclusions. That is why we have voting, a constitution, and limited government.

Average Joe stuck for a month in an apartment that could fit in Joe Celebrity’s closet has different values than Joe Celebrity. That’s why Mr. Celebrity sitting by his pool sipping chardonnay calls Average Joe “selfish” for wanting to go to the beach. We’re not in this together because we are not making the same sacrifices.

Policy decisions are made by the people in mansions.

Fool me once. Then do it again.

There is a certain shame to being fooled which makes us stick to our guns and hold on to our beliefs longer than we should. If you don’t like saying, “I was wrong,” try saying, “I was right given the information available at the time.”

A close up macro shot of a bottle of fabulous gold glitter.
A close up macro shot of a bottle of fabulous gold glitter.
Gold glitter. Photo by Sharon McCutcheon on Unsplash

We’re all going to be misled by the press, intentionally or not. That is the natural and unavoidable consequence of being human.

What we need is simple self-confidence. I hope I’ve shown you that you can have an opinion, even if you are not an expert, just like jurors across the country do every day. Don’t be afraid to listen to people with whom you disagree passionately. You are intelligent enough to sift through a pile of dirt to find the nuggets you won’t find in your own pile.

Good luck!

Written by

Attorney. Author of Train Your Iguana — Think Past the Emotional Barriers to Success in Family Law. https://twitter.com/pjparkjd

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store