My Ph.D. was Pointless, Noisy, and Exploitative–and I would do it again.
A systems-level look at a California-based STEM Ph.D.
Positionality: I am writing from the angry but grateful perspective of someone who was lucky enough to finish their Ph.D., recognizing that many others weren't this fortunate.
Maturity Level of This Essay: Budding [0]
When I was four, I declared my desire to become a scientist. At age eight I played Beyond Good and Evil which kickstarted my desire to create games. From then onwards, I worked towards becoming a game scientist to improve how games are developed. I was lucky enough to be admitted to a Ph.D. program in Computational Media at UC Santa Cruz. After almost five years of playing, thinking, experimenting, and writing, I became a Doctor of Games.
Armed with the experience of the past five years in the academic system, I can confidently say that my corner of the Academia, and my Ph.D. within it, was exploitative, pointless, and noisy–and yet, I would do it all over again.
In this essay, it is not the people in academia I am criticizing. Instead, I am arguing against the organizations, incentive structures, and traditions in which people are trying to survive. Ph.D. students have around five times increased depression risk compared to the general populace, with 1 in 4 Ph.D. students showing clinically significant signs of depression. [1] It doesn't take an expert to see that there are deeply rooted failings in the academic system.
While reading this critique, it is also important to note that there are hundreds of different academic fields, with their own priorities and traditions. Context matters. I am writing with my perspective gained from a US-based technical Ph.D. Yet, I believe all of academia shares some aspects of the systems I critique.
I hope that this essay will contribute to the conversations we have about the systems we create and will be a helpful data point for those who are just now deciding to step into academia.
Academia is Exploitative
Every first Friday of the month, our Graduate Student Association would host a gathering. The cheap beer was a good reason to go. After the event closed down, we often migrated to the bars in downtown Santa Cruz. I remember loaning my friend a twenty-dollar bill one of those Friday nights so he could join us. He literally had no money left after using a significant amount out of pocket to attend an academic conference that was required for his work.
In 2016, at UC Santa Cruz, 11% of the graduate students were homeless, and around 40% of them were suffering from food insecurity. [2] I started intermittent fasting to avoid racking up lunch bills as I spent more than half of my income on rent. As a response to this economic pressure, the graduate students went on strike for a living wage, and over 100 of us got fired. Others have told the story of the UCSC wildcat strike, with its financial hopelessness and the militaristic police response in much more detail. [3] What I want to highlight, instead, is how money is moved around and earmarked which creates this economically oppressive situation.
In the US, most Ph.D. students earn their livelihood either through teaching, fellowships, or doing funded research. Only half of the money students earn goes into their pockets. For me, this was around 2000$ monthly, which was lower than the "minimum liveable amount" found by the University of California themselves [4]. The other half is used to pay for the tuition of the Ph.D. student.
This structure is somewhat reasonable at the start of the Ph.D. journey. A first-year Ph.D. student has to take classes. These classes are taught by professors and facilitated by admin that need to be paid–tuition in this context makes sense. However, what doesn't make any sense is that this structure stays the same even after the Ph.D. student has stopped taking any classes and has effectively become a researcher doing work for their lab and institution.
In the last two years of my Ph.D. I did not take a single class. Yet I was, due to creative waiving and accounting, paying 2000$ per month for tuition. During this time, I had a single weekly one-on-one meeting with my advisor. If my tuition was directly given to him, I would have paid my supervisor 500$ an hour every time we met. Even worse, maybe before COVID, the tuition would have afforded more amenities, but since 2020 I exclusively worked from home, a significant amount of the time using my own laptop and resources.
After several years of work, the Ph.D. student stops being primarily a student that takes classes and needs hands-on guidance. We grow into someone who teaches classes, creates knowledge, and guides others. In my last year of my Ph.D., I was meeting multiple junior Ph.D. students to support them. Yet the system is blind to this change because it is financially convenient. Can you imagine a context where, after half a decade of experience and a direct change in job description your compensation barely grows? It is academia. I wish I could say that the exploitation stopped when you earned your Ph.D. Unfortunately, this is far from the truth, and the system is masterfully crafted to put pressure at every level.
Many who fail to secure a tenure track position continue their academic career as Adjunct Faculty. They have almost no job security and earn very little money [5,6]. These overworked and underpaid folks teach most of the classes in our "world-class institutions." Often their contracts have to be renewed quarter to quarter, and they have to teach at multiple institutions to make ends meet.
The lucky few who have achieved a tenure track position have seven years to prove their merit by publishing as many papers as possible. Academic institutions have an up or out system: you either get promoted, or you get fired. You can imagine how much pressure this puts on junior faculty, which gets passed down to Ph.D. students, which gets passed down to undergraduates.
Unfortunately, the systematic exploitation doesn't end there either! Let's briefly look at the bread and butter of academics–academic papers. Your mandate as an academic is to collect as many citations as possible. The citation count is the one single metric that is primarily optimized for. Apart from citing Goodhart's Law [7] I won't delve deeper into the effects of having citations count as a broken incentive here (as it will take a whole essay). However, I can suggest Anarchistic Fragment 22/23 of James C. Scott [8] if you want to read more on this topic. Instead, I want to briefly trace how labor and money flow through this broken system.
Let's say, through some means, you as an academic are funded and have been able to write a paper. Great! Now the paper is sent off to a peer-reviewed conference. This means that a group of "peers," other academics in your field, are asked to take time off from everything else they have going and decide whether this paper is worthy of being accepted to the conference. Are these reviewers compensated? Of course not.
Let us assume the reviewers decided to accept your paper into the conference. Now you have to register for the conference; otherwise, your paper will not be published. If you are lucky, your department will pay for the registration and all the accompanying travel resources; it likely won't if you are not in a STEM field.
Thankfully now that the paper is published, everyone can read it, right? Of course not. To read this highly selective academic research, you either need to purchase the right to access the document or be a part of a university that has an agreement with the publishers.
So one would expect that you, the academic who has done the research, has reviewed the work, and has paid the publication and conference fees, would get some portion of the journal money, right? Of course not. All the money stays with the publishers.
And yet the free labor doesn't even stop there! A Ph.D. student has to have their dissertation read by several faculty members to graduate. On the side of the faculty member, this situation means that they commit to reading, understanding, and responding to a document that is usually around 200-300 pages. Of course, no one is compensated for this significant time and intellectual investment. This is considered "service to the institution," the most unappreciated pillar of academic performance.
Rereading what I just wrote, it sounds ridiculous. Academic labor is unpaid at every level, for multiple actors, throughout the production and dissemination of human knowledge. Just thinking about it makes me exasperated. However, this isn't even the part that I despise about academia the most! Let's talk about the pointlessness.
Academia is Pointless
In academia, there is a whole class of research called "research for research's sake." I want to be clear here. I believe this type of research is incredibly important and valuable. It is a separate question if this research should be done in a university that prioritizes publishing in conferences and extracting labor from its graduate students.
At the very least, however, the "research for research's sake" variety openly displays its intentions of not providing any use towards productive output. Instead, it focuses on contributing to the general well of human knowledge or providing the structure for future more "useful" research.
Unfortunately, however, there is a much larger body of work that purports to be productive but isn't. I call this the "simulacra of productivity." [9] A large portion of academic research claims to be solving real-world problems, yet these solutions never find themselves used in practice. To be a productive part of the research landscape, you often have to play a very specific and small game of producing papers within the current conversational discourse of your field. These papers might state they are solving real-world problems, yet often require unrealistic assumptions that never would occur outside a laboratory setup.
Sometimes when there is a proposal to fix some real-world problems, the correct course of action is to just go ahead and try to fix it, rather than philosophize and write papers about it. [9] If the goal was to design a hammer, we wouldn't philosophize about the possible length of the handle, its curvature, or the angle of the head without actually building the damn hammer, feeling the heft of it, and using it to hit a nail!
If the game is to "maximize the number of citations," very reasonable behaviors become writing incremental papers that are riffs of what already is established in the field, rather than trying to make something that tries to fix the problem at hand. Furthermore, it incentivizes sticking to niches where your expertise is already established, as you are able to write papers faster; rather than learning new techniques that would have solved the problem better. I really wish I was one of these idealists in the ivory tower, but due to the pressures of "publish or perish," "finding funding," and "needing to support oneself monetarily," I had to play the game too.
Maybe if this was the worst it gets, I wouldn't be as critical. Unfortunately, on top of being exploitative and pointless, academia is also noisy.
Academia is Noisy
In the landmark book Noise: A Flaw in Human Judgment, Daniel Kahneman and colleagues make a distinction between bias and noise. [11] Any difference between the average judgment from a group and the true correct value is called bias. On the other hand, noise is the variability of judgments by members of the same group. Unfortunately, the peer review system in academia, which is supposed to be the gold standard for quality, contains immense amounts of bias and noise.
In 2014, the neural information processing (NeurIPS) conference ran an experiment. [12] The organizers took 10% of submissions and had them reviewed by two separate groups of reviewers. Only 38% of the accepted papers were accepted in both groups. Put in another way: 62% of the papers would have a different outcome if they were just assigned a different group of reviewers. How can you reasonably use such a flawed system for gatekeeping new knowledge and judging the futures of career academics?
When I was a first-year Ph.D. student, I went to a top-tier games conference and presented a poster. In the same year, we also submitted a paper to this same conference, which was desk-rejected. To "desk-reject" a paper means that the editor, a singular person, decides that the paper doesn't meet the standard of the conference to even be sent to peer reviewers. This year the desk-rejection rate was around 60%. Fast-forward a couple of years, and we submitted the same paper, making very minor changes, to the same conference again. There were some changes in the editor board. However, the same head editor was in charge of final decisions and desk-rejections. This time our paper got accepted. [13]
This situation was incredibly baffling to me. What was worse, when I wanted to discuss the situation with my larger community, the sentiment was confusion as to why I was making this big of a deal. Didn't the paper get accepted after all?
This amount of noise would have been acceptable if you had a high number of attempts. The law of large numbers states that one can get to the true underlying mean given a large enough sample size. Unfortunately, however, in research, you get a somewhat limited number of shots. Often Ph.D. students submit one to two first-author papers per year.
Even if we move past the noise contained in the human evaluators, there also is an immense amount of uncertainty in research, almost by definition. If one was able to state which research questions are worthwhile before exploring them, it wouldn't be called research! By sheer luck disguised as intuition, you could pick a very fruitful research domain or wander around for years on a barren topic. Of course, as one matures as a researcher, one's intuition sharpens, but the uncertainty never disappears. Realistically, however, starting Ph.D. students don't yet have the intuition to pick promising research fields–at least, I didn't. Thankfully the Ph.D. advisor is always there to offer guidance, which leads us to the next source of uncertainty: the compatibility of the student with the advisor.
Does the student prefer joining a pre-existing established project, or does she want to start something on their own? Does the advisor have a strict, organized approach, or does she prefer a more free-flow approach? Do the priorities and values of one align with the other? There are no correct answers to these questions, just productive matches. The answers to these questions affect the success of the research lab drastically. Unfortunately, these questions are often not considered in the application phase and are left to chance.
According to the New York Times in the US, the average length of a Ph.D. is eight years. [14] This is as long as the mean marriage length in the US! [15] When one decides to get married, hopefully, they have months, if not years, of experience getting to know each other. Often a Ph.D. student and advisor agree to work together after exchanging only a few emails. Once again, at least in certain US institutions, the admission system leaves a lot of these crucial compatibility questions to luck. For example, I decided to change labs at the end of my first year due to an incompatibility between myself and my advisor. I believe it was one of the three most important decisions I took in my Ph.D., and I probably would have not been able to finish my Ph.D. otherwise.
Most Ph.D. students aren't aware of how big of a role luck plays in their results, which can result in immense feelings of doubt and imposter syndromes. "My papers aren't getting accepted. The reason must be because I am not smart." Furthermore, research is incredibly personal and emotional. For better or worse, often, people pour their identity into the ideas they are defending and presenting, which makes the noisy rejections sting even worse and likely contribute to the increased mental health issues.
And yet, I would do it all over again
I am grateful for my time during a Ph.D. It was incredibly enriching. Most importantly, it gave me time. Time to ask questions that didn't pan out, time to talk about failure for months on, and time to contribute to a larger ongoing conversation. It got me a well-paying job. I have a cool-sounding title. It worked for me. [16]
I state that I would do it again, because: I didn't have medical bills that were prohibitive. I didn't have dependents that I need to take care of. I didn't have a sense of financial expectation that was put on me by my culture or parents. I had an incredibly supportive supervisor and cohort. I lucked out on finding a reasonable research topic. I lucked out on matching with reviewers with whom my work resonated. I lucked out to be in a situation where I didn't have to suffer from racism or sexism.
I hope I made it clear that getting a Ph.D. has an incredibly high variance, and unfortunately, I believe an overall negative expected reward. Even though most suffer, some end up extremely fulfilled and successful. Whether this is a risk you want to take is only up to you. I hope that this systems-level look at how academia is structured will help you make a more informed decision. I put all my skill points in luck in character creation, rolled the dice, and scored a critical success. If you roll the dice, I hope luck smiles upon you as well.
More importantly, however, systems can change.
Let's do better. I had said "Let's do better." in the initial version of this document. However, it was to lessen the bridges I was burning and give this whole clusterfuck of a situation a hopeful spin. After more conversations with people I trust, I realized I was being weasely and dishonest. Especially being able to write this from a position of privilege, having been able to receive the title.
I am not trying to do better. Because I got out. I am working in a high-paid job in the games industry, doing research into tools that will reach players. So maybe, let me be braver and update my call to action. Let's leave academia, and take our skills, passions, and research elsewhere, where it will be recognized and rewarded. Then maybe, the pressure will get academia to up their game.
Batu Aytemiz
Thanks to Sudikchya, Breanna, Oskar, and Grammy's House for comments!
[0] This is the first entry on my "digital garden". The posts here aren't static, but will grow as I have more conversations and thoughts around it. I am planning on three levels of maturity, seedling, budding, and evergreen. I was inspired by Maggie Appleton, read more about their gardens here: https://maggieappleton.com/garden-history
[1] https://www.nature.com/articles/s41598-021-93687-7#Sec2
[2] https://twitter.com/leyabreanna/status/1292897163719000064 / https://www.universityofcalifornia.edu/about-us/information-center/student-basic-needs
[3] https://payusmoreucsc.com/
[4] https://financialaid.ucsc.edu/cost-to-attend/graduate-costs.html
[5] https://www.theatlantic.com/education/archive/2019/04/adjunct-professors-higher-education-thea-hunter/586168/
[6] https://www.chronicle.com/article/how-to-fix-the-adjunct-crisis/
[7] Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure."
[8] https://theanarchistlibrary.org/library/james-c-scott-two-cheers-for-anarchism#toc32
[9] https://notes.andymatuschak.org/Insight_through_making
[10] https://www.econlib.org/call-it-sour-grapes/
[11] Noise: A Flaw in Human Judgment, Kahneman, 2021
[12] Inconsistency in Conference Peer Review: Revisiting the 2014 NeurIPS Experiment. Cortes and Lawrence, 2021. https://arxiv.org/abs/2109.09774
[13] I have a proposal on how to change this in our field! Moving to guidance over gatekeeping format. I will convert this thread into a post soon™. https://twitter.com/BatuAytemiz/status/1475510479825027074
[14] https://www.nytimes.com/2007/10/03/education/03education.html
[15] https://legaljobs.io/blog/divorce-rate-in-america/
[16] A repeated critique I got was that this paragraph was weak. I, uh, yeah. Decided to write separately about the nice parts of my journey later.