42Degrees strives to openly rank and publish all research regardless of perceived quality or impact. The goal of this approach is to address existing challenges with peer review (Table 1).
Many of science's groundbreaking discoveries were initially dismissed by the majority. Two notable examples include Lynn Margulis's endosymbiotic theory of the origin of eukaryotic cells, and Rachel Carson's detailed investigation of pesticide bioaccumulation. Rejected manuscripts, when eventually published, are often highly cited and influential1.
Table 1. Solutions proposed by 42Degrees to address peer review challenges.
Peer Review Challenges
42Degrees Solution
Biased reviewers.
Balanced input from editor author(s) and AI using a standardized review form and rubric.
Time required by editors and reviewers is unsustainable.
A system of reviewing that includes the author soliciting peer reviews and the editor in addition to AI reviewing the science. The use of a standardized review form and rubric.
Supply of papers outruns demand.
Creating true impact will require more effort such as that required for creating scientifically rigorous videos, minipods, and games.
Gatekeeping stifles innovation.
All research is ranked and published. Authors also evaluate their own paper; their unique expertise can provide context regarding significance.
Cheaters game the system to improve their success measures (Goodhart's Law).
All authors given equal opportunity to publish, removing impetus to cheat. Rankings provide quality control.
Editorial decisions that are arbitrary or based on limited knowledge can stifle vital discourse and delay publication.
All research is ranked and published. Caveats section of publon includes dialogue regarding quality concerns.
Is it wise to eliminate gatekeeping? What about cheaters? This is a valid concern; however, current rules and measures have not prevented some individuals from gaming the system. Inventing new rules is unlikely to solve these problems in the long-term; according to Goodhart's Law, when a measure becomes a target it is no longer a good measure2. Thus, the measures used to quantify a scientist's success have shifted the focus from science and its contribution to humanity to maximizing often arbitrary measures of productivity. Eliminating gatekeeping will also enable publication of solid research that has little impact, for example, summer high school student projects or a unique observation that warrants sharing.
1
Siler, K., K. Lee, and L. Bero. 2014. Measuring the effectiveness of scientific gatekeeping. PNAS. 112:360.
Cooke, S. J. et al. 2024. A harm reduction approach to improving peer review by acknowledging its imperfections. FACETS. 9:1.
2
Biagioli, M., 2016. Watch out for cheats in the citation game. Nature. 535:201.
Fire, M. and C. Guestrin. 2019. Over-optimization of academic publishing metrics: observing Goodhart's Law in action. GigaScience. 8:1.
Example of Publon Evaluation Rubric
The 42Degrees publon evaluation system and rubric will be adapted through time as we identify which publons are best at communicating, explicitly looking at publon influences on AI and human learning. Initially, each publon's scoring calculations will not be released in order to prevent individuals from gaming the system.
Evaluators will use this review form to assess publons based on two key criteria: Generational Intellect Impact (GII) and Scientific Rigor. Each criterion is scored on a scale from 1 to 5 by five evaluators: AI, the Editor, two peer Reviewer(s), and Author(s). Each score is given equal weight; this results in the combined Reviewers score carrying more weight (20% + 20% = 40%) compared to the AI, Editor, and Author consensus, each of which contributes 20%.
The scores are interpreted as follows:
1: Minimal or trivial contribution, no novel ideas.
2: Minor contribution, limited significance.
3: Moderate contribution, some new insights but lacking depth.
4: Substantial contribution, significant insights, and innovative ideas.
5: Exceptional, groundbreaking work with transformative insights.
1: Poorly defined methodology, superficial or flawed data analysis.
2: Weak methodology, significant flaws in data analysis.
3: Adequate methodology, competent analysis with some flaws.
4: Robust methodology, thorough and appropriate analysis.
5: Exemplary methodology, exceptionally rigorous and reliable conclusions.
Tier I (Top 1%): Scores of 10 indicating exceptional groundbreaking work.
Tier II (Top 10%): Scores of 8-9 indicating high-quality significant contributions.
Tier III (Solid Working Paper): Scores of 5-7 representing competent well-conducted research.
Tier IV (Fundamentally Wrong): Scores of 2-4 suggesting serious methodological or conceptual flaws.
