From the outside, peer review looks broken. To me it looks like we still might have some smart people making progress, but in a sewer of corrupt funding, paper mills, and I’m sure plenty of chatbot output. Plus, the idea of sharing your innovations with “peers” aka competition only works in a high trust environment.
The idea of these papers wasting everyone’s limited, valuable time makes me {
REDACTED}.
A
[snip]
Peer review & the academic journal system are fucked up
known issues:
- the selected 'peers' don't know what you're doing (subfield too niche) (arxiv with comments would fix this, so does academic twitter - basically your actual peers getting to have takes. also write clearer, dumbass)
- the selected 'peers' are your competitors but usually that's okay bc if they're reviewing something they're working on you've basically scooped them already if you resubmit fast enough. It's not too bad if you're the reviewer bc then you're ahead of the knowledge curve of where to pivot your work, but if it's groundbreaking or in a breakneck field, L for u.
- the selected 'peers' hate you and want to tear you apart (actually a W for peer review, that's what they're for)
- academic journals are a business. we pay to access papers, pay to publish them, and review for free, in exchange for them basically being field filters, which is a scam
- paper mills
- tbh the ai stuff is great imo because its a very convenient way to identify what fields are hot garbage and what writers don't know english BUT running your work though openAI/etc which has next to no security against corporate espionage is not great and something we're going to have to face sooner than later
benefits:
- organize work into fields
- the reputable ones filter for quality, you just have to ignore bullshit ones. people know the reputable ones in their field. for an outsider, usually the number of citations is a clue, but nature/science are the gold standard & you can google field specific ones (aside from nature "field", science "field")
alternatives:
-arxiv - downside: nearly no filter - they have the endorsement system but it only takes one bad egg to ruin that - also it's very hard to tell what's going to be impactful without reading it (which is the benefit of nature/science/etc, you can gauge impact by how many people they've managed to convince) - arxiv refuses to do a rating system unfortunately, with valid concerns but it's annoying
-conference-centered publication (you have to be selected by people running the conference and if your work is shit you'll be torn into by everyone at the conference, but niche conferences are word of mouth)
-there's also open source/open access data quality reviewers which I want to give a shout out to
essentially we all hate having to pay + do work for publishing companies, but arxiv doesn't provide the filtering/structure & conference-centered is kinda niche. Open Access enjoyers do both. good news is yeah these are 100% problems being worked on and I expect scientific publishing as we know it to be undermined in my lifetime. good because fuck those guys
