The peril of evidence based obsessors

Posted on October 18, 2017


In the 1980s, healthcare made a concerted effort to move towards evidenced based medicine. That the treatment a doctor prescribed should be backed by research, meaning (ideally) randomized controlled trials, placebo groups, large sample sizes, independent researchers.

Yes, incredibly, this was not already standard. Not that everybody is doing it these days, but the idea was to move away from intuition and anecdotal experience as one’s primary basis for treatment.

Again, it’s true not everybody is big on this. The personal training world is still HEAVILY influenced by n = 1, that is, a single person’s experience. Many personal trainers train their clients solely based on how they’ve trained themselves. It’s the biggest mistake trainers make, and it’s voluntary. The average trainer recoils upon seeing an abstract, much like a craft beer drinker recoils when seeing a Miller Lite can. “Just eat less and move more.” Or “Just kick the client’s ass. It’s not complicated.”

Older practitioners are notorious for this too. What was evidenced based when a doctor was in his 30s has become outdated by the time they’re in their 60s. But it’s too hard to change. Too much effort to alter your habits when seeing a patient. Too much energy to keep up with the literature.

-> When looking for a doctor, part of your criteria should be picking one who is old enough to be experienced, but not old enough to be jaded. All else being equal, on average you want your say, surgeon, 45 years old.

This is how many parents are. The most common phrase of old parents to new parents is,

[new parent states new approach]

“Oh, we did [old approach], and my kids turned out fine.”

It’s terrible logic. Just because someone turned out fine does not mean what you did was ideal. We’re rarely dealing with right or wrong here. We’re dealing with trying to get less and less wrong.

On the other end of the pendulum, you have those who OBSESS over needing a research citation for every decision. They completely discount anecdotal experience and human intuition. Declaring any person with a contrary opinion or experience to a study as fooling themselves, falling into some Daniel Kahneman bias, or being an idiot. I call these people research nazis.

One reason I disdain the research nazi is because they act as if science is flawless. Yet most research is not randomized, has small sample sizes, doesn’t account for the difference between statistical and clinical significance, biased by funding or researcher influence, has statistical sections math majors can’t even follow.

If you’re going to accept humans are flawed when it comes to decision making, you have to accept research is flawed, because it’s done with human decisions. Sure, perhaps one is less flawed than the other, but we can’t just dismiss the other end of the spectrum because there isn’t a peer reviewed meta review fully backing it up.

MATH ain’t fail-proof either

I’ve had a lot of clients who are engineers. They’re some of my favorites. However, one trait of theirs, along with many of their ilk, is they love numbers. Sometimes too much. You may have heard some version of “People lie; numbers don’t.” Or “I don’t like to deal with opinions. I like data.”

Anybody who is a slave to data truly does not know what they’re talking about. 

I was a mathematics major for two years, eventually deciding to stop to get a minor. You learn funny things about numbers when you get to that level.

The biggest of which is math is created by…humans. While in a sense obvious, many don’t truly understand this facet of mathematics. It’s common thinking math is some hidden language of the universe. Like it’s electricity. That it’s always been there, but one day we discovered it.

We did not discover math. We invented it. It just happens to be we’ve invented it in a way where we more often than not see no fault with it. Where we’ve ironed out all the problems, where the equation is flawless. Until you get to a higher level.

You’ve likely heard of Isaac Newton. He created the “laws” of motion. Yet the reason Einstein is famous is because he found some scenarios where these laws weren’t true. For instance, Newton’s world says time is absolute. It always moves in the same manner. Einstein discovered that’s wrong. Time is relative. The faster you go, the slower time goes.

Can special relativity explain any of the life extending benefit of exercise?

Einstein thought the world was fully predictive. If you do X, Y will happen. When what you’re dealing with is really big, like planets, Einstein’s approach works. When what you’re dealing with is really small, like electrons, then quantum mechanics shows if you do X, Y or Z will happen, with respective probabilities. Einstein was wrong.

These are the most acclaimed minds of human history. Yet their inventions are not fail-proof. Their theories break down in specific situations.

Here is one to truly jack your mind up: You’ve probably heard there are infinite points on a line, right? That is, take a line, and you can make any given point infinitely small, causing all the points on that line to add up to infinity.

Alright, now take a bigger line. Say it’s twice the size. How many points are on that line? Also infinity!

In math, two different sized lines have the same number of points!

Math is not fully rigorous. Human research, besides its obvious human flaws, is thus not either, because it is based on a discipline, mathematics, which itself is not fully rigorous. I mean, how would you do a research study without math??

We can’t be pretentious with our judgment, but neither can we with research. Yes, on balance, give me research over opinion, because research is often less wrong than a single person’s intuition. But it’s nowhere near as “right” as mathematical theory is, which itself is not always consistent.

Research is not perfect, and once in a while a person comes along who shows exactly how and why that research we all thought was largely correct, is in fact correct, yet still wrong. Newton’s notion of time is “correct”, that is, it works, the majority of the time. For 99.99% of experiences, we don’t need Einstein’s interpretation. Newton is good enough. You want GPS though? Where a satellite is going around the Earth at thousands of miles per hour? Then we need Einstein.

Many times when someone is shitting on either research or a given person’s experience, they’re not fully understanding the context a theory is being given in. Yet the context is as important as the theory. And regardless, it is guaranteed a theory will break down in some context. It is impossible a given theory is fully correct…it’s also unlikely a given theory is fully wrong.

With math, we tend to be talking in the 99 percentiles of being correct. It’s why math is the most amazing invention in the history of humankind.

-> Some may debate language is more important, but seemingly all species have invented their own language. Only one species has invented mathematics, with results speaking for themselves, as virtually everything around you is rooted in math.

Or you could simply view math as a specific language.

But math is not fully provable, fully consistent, fully complete.

-> For more of a mind bend, check out Godel’s Incompleteness Theorems. Where ironically, math is used to prove math is not, and can never be, complete. These are some solid layman’s explanations. (I like Edwin Chen’s best.)

With most human modeling, many of us are ecstatic if a theory works 60% of the time. Ten percent of smokers get lung cancer. Ten percent! But that’s enough for us to castigate smoking. Yet there are those out there who bitch not every study backs up a theory. Or a theory is flawed, so it’s useless.

  1. If math is flawed, does any modeling involving humans even have a chance of not being flawed?
  2. Even if you think math should be the barometer, good luck finding mathematical probabilities with humans!

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Posted in: Miscellaneous