–Cornell Food Researcher’s Downfall Raises Larger Questions For Science
If you need to catchup, Brian Wansink was, I think a case can easily be made, the most famous nutrition researcher in the world. Primarily because he wrote a book called Mindless Eating. I wrote, glowingly, about this book many years ago:
I distinctly remember publishing that post because, at the time, it was by far the most popular thing I had written. At that point most of my readers were my in-person clients. I’d send an email out once a month with new articles. That one got a lot of attention from them. Many bought his book, because of my recommendation.
Turns out Wansink broke god knows how many sacred research practices in finding coming (making?) up his research. To the point he was fired resigned. Remember, this is academia. You can practically throw a student off a building and still have a job #TenureBitch.
–More in-depth look at what he did, with emails
First, fuuuuuuuuuuuuck this guy.
Second, this only adds to the never-ending dilapidated labyrinth of bullshit that is the world of nutrition research.
–Why are we so confused about how and what to eat?
I repeat, this was a top researcher in the field. From CORNELL. Too many of us hear an Ivy League affiliation and think it’s reputable. This is what we should really think:
The weight of the school’s name is way, way too heavy. You just throw Harvard on something and it’s automatically credible. Gary Taubes is a perfect example. He has been eviscerated by those who actually know what they’re talking about (Alan Aragon), but because he has Harvard behind his name, everyday people keep going along with him.
For what it’s worth, I place ***no*** extra credence in research from “prestigious” schools. If anything, I’m more skeptical. I’ve been around enough prestigious school graduates to know plenty engage in stupidity at the highest level, but I also know, for better or worse, they are more motivated, so they know how to navigate the system better, if they wish to.
Third, we have our yearly reminder of how nobody in any realm of health research seems to comprehend or care what a p-value actually is.
-> Past reminders,
–Is there about to be a revolution in ACL surgery?
–Is an obese mom as bad as a drunk one?
I’m at the point now I can hardly read
1) anybody trying to do a research review because there is a 99% chance they do not have a background in basic statistics / don’t know what the word “significant” means statistically <- this causes probably 95% of the issues
-> This is not a lack of intelligence. It is ignorance or laziness.
2) any mainstream article about research because much of it is dictated by the researchers who have proven enough times they do not care so much about science as they do publicity
-> Also not lack of intelligence. It is fraud.
I used to read and read various accounts of research, thinking I was “staying up to date.” After being fooled enough times, it now takes me a legit two damn weeks to interpret a study. That’s what happened in this one,
–Schoenfeld’s new volume and muscle strength and size study
–The futility of averages in exercise science research (new info on Schoenfeld’s new study)
I have to,
- Read the press release / author’s mainstream account to see what they’re publicizing
- Read the abstract to see what they know many will only read
- Get the full study (not always feasible)
- Read the full study multiple times
- Read citations in the study because the study often doesn’t want to detail its full methods
- Seriously, many study’s now, when you read their methods, will say “Methods can be found in [X].” Making you read another study, which, wait for it, may also say “Methods can be found in [Y].” So now you have to read three studies to see what one did.
- Contact statisticians who often can’t even guarantee me what the study really did
- If they don’t know, how the hell am I supposed to?
- Do a background check on the author(s)
- and sometimes find out they’re in jail for writing studies they never did
- engaged in such reprehensible behavior they have been executed
- they’re hiding where their funding is really coming from
- Contact the authors who also can’t really tell you how they got what they got, refuse to answer certain questions, see them backtrack their language, or hey, once in a while they outright tell you they knowingly published a mistake (IN THE JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION!!!)
Fourth, for all you evidenced based elitists who want a citation for every word a person writes, because you think health science is so pure, go chew on Wansink’s 27,000 Google Scholar citations. You and Shooter can eat breakfast together.
asgag
October 2, 2018
Andrew Gelman has very interesting points on this: https://andrewgelman.com/2018/09/27/people-are-missing-the-point-on-wansink-so-whats-the-lesson-we-should-be-drawing-from-this-story/
b-reddy
October 2, 2018
One of Feynman’s most famous quotes is “The first principle is that you must not fool yourself – and you are the easiest person to fool.”
I agree with that general point. Many of us have been there. But I think Andrew is painting way too rosy of a picture here. While humans are no doubt the hardest thing to study, other branches of science do not have these kinds of problems, at least not at the scale we do. The concern for p-hacking when you have to deal with five sigma isn’t nearly the same. That might be too ambitious for the non-physics crowd, but the issues with .05 are readily apparent at this point, yet it continues to be the benchmark.
I’m also not sympathetic to any argument revolving around “well, it’s really hard.” Sure, but that doesn’t mean e.g. small sample sizes are then acceptable. It’s not like quantum physics or the large hadron collider are easy. Nor does the “it’s hard” argument doesn’t pass in business. When a cellphone starts spontaneously combusting, Samsung doesn’t get a free pass because lithium ion batteries are hard.
Lastly, while I get the replication crisis is fairly recent, the general issues in human / health science have been around for a long time. Can see Feynman go after social science and nutrition ~35 years ago: https://www.youtube.com/watch?v=tWr39Q9vBgo
asgag
October 9, 2018
Ah, I get what you’re saying and I agree. It’s true that the “it’s hard” argument doesn’t pass elsewhere. Maybe the thing with social/human/health sciences is that it was getting overcrowded, unlike quantum physics, I guess. The replication crisis might be one more step towards professionalization in those fields, and many people will have to go if they can’t meet the standards, while at the same time those standards (peer review etc) should also be scrutinized.
b-reddy
October 12, 2018
That’s a very good point.
There probably are too many of us working in healthcare. Whether the number of employees (1 out out 8 Americans!) or the amount of money (nearly 20% of our budget).
There have also been some good looks at the amount of PhDs, and how that’s likely oversaturated.
More specifically to your point about narrowing down due standards- I’ve seen a great tidbit from Jeff Bezos about why he dropped out of theoretical physics. It’s good to remember Bezos is uniformly considered an incredibly intelligent person.
The short version of it was
“I had trouble doing this one problem. A classmate and I spent hours and hours on it. We eventually went and asked another classmate. He looked at it for a minute or two, then said the answer. He practically did it in his head. That’s when I knew I’d never be a good theoretical physicist. There’s really only like 50 of them in the world, and the writing was on the wall I wouldn’t be one of them.”
He recently told the full story here: https://youtu.be/xv_vkA0jsyo?t=1335
I don’t think there is anywhere near enough of that in healthcare. Per Wansink’s approach, we kick the can down the road, always finding someone who will accept the paper, doctor, nurse, trainer, whatever. We don’t even have any semblance of what a surgeon’s complication rate is. The surgeon’s themselves rarely know! (I’ve asked some.)
I do think Americans are waking up to this though. I find healthcare costs to be nearly an inevitable regulation at this point. California has a very interesting prop on the ballot this year regarding dialysis. Google recently hammered health and fitness sites for even remotely not appearing credible, my bet being because they’re terrified of more EU fines.
And that’s my glass half full of optimism :).