Fixing science via a basic income

I ran across Ed Hagen’s article “Academic success is either a crapshoot or a scam”, which pointed out that all the methodological discussion about science’s replication crisis is kinda missing the point: yes, all of the methodological stuff like p-hacking is something that would be valuable to fix, but the real problem is in the incentives created by the crazy publish-or-perish culture:

In my field of anthropology, the minimum acceptable number of pubs per year for a researcher with aspirations for tenure and promotion is about three. This means that, each year, I must discover three important new things about the world. […]

Let’s say I choose to run 3 studies that each has a 50% chance of getting a sexy result. If I run 3 great studies, mother nature will reward me with 3 sexy results only 12.5% of the time. I would have to run 9 studies to have about a 90% chance that at least 3 would be sexy enough to publish in a prestigious journal.

I do not have the time or money to run 9 new studies every year.

I could instead choose to investigate phenomena that are more likely to yield strong positive results. If I choose to investigate phenomena that are 75% likely to yield such results, for instance, I would only have to run about 5 studies (still too many) for mother nature to usually grace me with at least 3 positive results. But then I run the risk that these results will seem obvious, and not sexy enough to publish in prestigious journals.

To put things in deliberately provocative terms, empirical social scientists with lots of pubs in prestigious journals are either very lucky, or they are p-hacking.

I don’t really blame the p-hackers. By tying academic success to high-profile publications, which, in turn, require sexy results, we academic researchers have put our fates in the hands of a fickle mother nature. Academic success is therefore either a crapshoot or, since few of us are willing to subject the success or failure of our careers to the roll of the dice, a scam.

The article then suggests that the solution would be to have better standards for research, and also blames prestigious journal publishers for exploiting their monopoly on the field. I think that looking at the researcher incentives is indeed the correct thing to do here, but I’m not sure the article goes deep enough with it. Mainly, it doesn’t ask the obvious question of why researchers have such a crazy pressure to publish: it’s not the journals that set the requirements for promotion or getting to the tenure track, that’s the universities and research institutions. The journals are just exploiting a lucrative situation that someone else created.

Rather my understanding is that the real problem is that there are simply too many PhD graduates who want to do research, relative to the number of researcher positions available. It’s a basic fact of skill measurement that if you try to measure skill and then pick people based on how well they performed on your measure, you’re actually selecting for skill + luck rather than pure skill. If the number of people you pick is small enough relative to the number of applicants, anyone you pick has to be both highly skilled and highly lucky; simply being highly skilled isn’t enough to make it to the top. This is the situation we have with current science, and as Hagen points out, it leads to rampant cheating when people realize that they have to cheat in order to make the cut. As long as this is the situation, there will remain an incentive to cheat.

This looks hard to fix; two obvious solutions would be to reduce the number of graduate students or to massively increase the number of research jobs. The first is politically challenging, especially since it would require international coordination and lots of nations view the number of graduating PhDs as a status symbol. The second would be expensive and thus also politically challenging. One thing that some of my friends also suggested was some kind of a researchers’ basic income (or just a universal basic income in general); for fields in which doing research isn’t much more expensive than covering the researchers’ cost of living, a lot of folks would probably be happy to do research just on the basic income.

A specific suggestion that was thrown out was to give some number of post-docs a 10-year grant of 2000 euros/month; depending on the exact number of grants given out, this could fund quite a number of researchers while still being cheap in comparison to any given country’s general research and education expenses. The existence of better-paid and more prestigious formal research positions like university professorships would still exist as an incentive to actually do the research, and historically quite a lot of research has been done by people with no financial incentive for it anyway (Einstein doing his research on the side while working at the patent office maybe being the most famous example); the fact that most researchers are motivated by the pure desire to do science is already shown by the fact that anyone at all decides to go to academia today. A country being generous handing out these kinds of grants also has the potential to be made into an international status symbol, creating the incentive to actually do this. Alternatively, this could just be viewed as yet another reason to just push for a universal basic income for everyone.

EDIT: Jouni Sirén made the following interesting comment in response to this article: “I think the root issue goes deeper than that. There are too many PhD graduates who want to do research, because money and prestige are insufficient incentives for a large part of the middle class. Too many people want a job that is interesting or meaningful, and nobody is willing to support all of them financially.” That’s an even deeper reason than the one I was thinking of!

4 comments

  1. I would strongly context the points made herein.

    A.
    In the natural sciences, Grad school and post-doc pay you a living wage (post doc a little better); the former 20-36k (36k only in like Silicon Valley etc., it’s all by cost of living), the latter 35-50k per year
    In return for unreasonable working hours (60-80 per week is typical) and stimulating but physically and/or intellectually demanding work

    The people who are seeking research positions, especially in academia, are those going through grad student and then into post-doc; the total process in the biosciences can last twelve years or more- the system you describe already exists and is the root cause of excess demand for academic positions. People already spend all of their time in post-docs waiting to luck/excel enough to be a professor while being paid government grant money to be post docs .

    B. i.e. In fact, it is the ‘glut of people seeking a research career’ that is the only reason academia survives at its present scale.
    Labs are run on a shoestring budgets and funding gets tighter and tighter; only by having lots and lots of cheap high-skill grad student and post doc labor does academic science function at all.

    Most of the grind of modern laboratory research is not something that is especially feasible part-time, especially if people want to design experiments and keep up with the literature; otherwise we’ll see even more methodological and intellectual balkanization than we already have in an academia overburdened by vastly varying standards and overproduction of literature relative to anyone’s ability to read it even often within a specialty.

    Anyone with basic training can be a pair of hands in a lab- such work is not especially rewarding or stimulating. This is what most B.S. science majors do in the pharma industry- only those with a masters or PhD actually engage in a significant intellectual part of the science, with rare exceptions. The rewarding or stimulating work requires an ability to read and synthesize the literature and systematically design good experiments, addressing multiple hypotheses. These are skills that require considerable exertion to ingrain and extensive expertise.

    ‘Classical’ research by amateurs was largely done by individuals of independent means and wealth; Einstein is a terrible comparison due to A. doing theory and thus having no materials cost and B. being himself a genius prodigy and thus a total outlier.

    That is to say, doing ‘real science’ requires dedication, high intelligence, and curiosity- the structure of grad school and post-doc already drives many promising individuals out of science, or at least into less intellectually free and rich industrial applications (Biotech, etc.). ‘You get to work a minimum wage for the rest of your life in an intellectually demanding job’ is not going to attract an especially high caliber of talent, and the work is not incredibly amenable on an institutional level to large-scale part-timing.

    C, Also a serious lack of standardization of rigor and core skills is a problem in many fields; in addition to perverse incentives, I routinely read papers, from the US and abroad where incentives are different, that are just not done well. I argue that you overestimate the contribution of perverse incentives vs. the tendency of output to fall along a distribution in accord with the training, intellect, creativity, talent pool, and resources of those responsible (although I have seen papers from brilliant people in top labs that nonetheless make amateur mistakes for want of attention to rigor).

    • Thanks for the thoughtful comment! You may be right, though some of the details also probably vary by field; you’re critical of the Einstein example because of him mostly doing theory and having no materials cost, but many fields are largely about theory or let you do experiments without major material costs. Math is the most obvious one, but also many kinds of experiments and work in psychology, computer science, etc. (These are also the kinds of fields that I’m most familiar with and which I was mostly thinking in terms of; you’re probably right in that this wouldn’t work for the biosciences in the same way.)

      Re: A, you seem to suggest that people are already relatively fine doing a long string of postdoc positions, but I’ve heard from a lot of people that they spend about as much time trying to secure the next grant/position after their current one ends, as they spend doing research. That sounds like the kind of long-term unsustainable thing that may drive people into doing questionable stuff in the hopes of getting a more secure position.

      • A fair point- but the limiting factor for ‘indefinite post-docs’ (which can already last 6 or 7 years…) is by constraints on lab funding. Some labs do maintain permanent staff scientists and in many cases also a semi-permanent lab tech or two (these positions are often temporary by choice), but few have the funding to support more than one or two staff scientists plus the normal post-docs/grad students. I’m not sure how much retention of talent in academic science you can reasonably expect if the end goal is ’50k per year, 60 hours a week, for life’ for most aspirants (staff scientists make a bit more into the genuinely comfortable range; I think 70-80k is typical?). The arrangements that make grad/post-doc labor affordable and efficient only survive because it is understood that those terrible conditions are temporary- stepping stone to a more intellectually rewarding career (or a generally less intellectually rewarding but better paying job in the chemical/biotech/etc. industry).

        The only solution there is just to increase funding- whether directly to grant agencies (decreasing the pressure to exaggerate results/potential as high as plausible to compete for preciously limited resources) or via very long-term (multidecade?) individual fellowship grants (post-docs and grad students can already apply for and receive individual funding fellowships from the US government, for a few years at a time). That is to say, the core solution just boils down to more money either way. There are other inefficiencies in academic structures (i.e. a lack of systematic overarching direct collaborations as a norm rather than an exception in most fields, resulting in extensive duplication of effort, failures of access to expertise that would save problems with time/rigor/replication, obfuscation of information necessary for replication due to competition, and wasteful competition in general- although the competition does of course have some benefits as it creates pressure to refine and justify opposing models). The model of ‘labs-as-professorial fiefdoms’ comprised largely of student-scholars under usually a single PI (the norm in the US, less so in Europe) is I’d argue quite inefficient and impractical in an age where many fields mandate multidisciplinary approaches and where the production of new scientific literature is so rapid that staying on top of one field, not to mention several allied fields, is often untenable even for a veteran academic.

  2. While it’s true that many forms of research have lower materials cost, the biggest source of loss of return on investment is going to be inefficiencies and rigor-failures in experimental physics, chemistry, biology, etc. all of which have high costs for instrumentation, time on regional/national facilities (particle accelerators, high-field NMR, etc.), and supplies/reagents.

Trackbacks/Pingbacks

  1. Rational Feed – deluks917 - […] Fixing Science Via A Basic Income by Kaj Sotala – The Root cause of bad science is the pressure…

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.