9 comments
I've long thought that scientists should make their money betting on the outcome of experiments run by 3rd parties. This would allow for small steady flows of cash to establishment ideas, while breakthroughs would result in huge one time payouts. And the establishment ideas holding those breakthroughs back would be appropriately punished.
In order for this to be possible, conducting the experiment would need to be done by an impartial 3rd party separate from the researchers.
Considering sociology and similar field's complete lack of correlation between data and conclusion, I don't believe they should be considered "science". They're more similar to astrology and phrenology.
The term you are probably searching for is "political manipulation." They should be considered political manipulation.
Maybe data and hypothesis should be published independently from conclusions, such that the conclusions of the researchers are not privileged over others. Without the conclusion, the peer review of the data can be more neutral and focused. Peers can review conclusions on their separate merits.
If the experimenter is in a better position to judge their own data than others, then they have not included sufficient data.
Maybe hypothesis and methodology should be published before seeking a grant, before any data is collected at all. The methodology could perhaps benefit from peer review.
Should tag 2022. Fun article but familiar to many after making the rounds back then.
Research Objective:
"Does Immigration Undermine Public Support for Social Policy?"
ISSP Data (immigration, welfare attitudes) for 17 countries.
73 independent research teams come to different conclusions, nobody knows why.
https://xkcd.com/3101/ "Good science"