DAOrayaki DAO Research Bonus Pool:

Funding address: DAOrayaki.eth

Voting progress: DAO Committee /7 passed

Total bounty: 120 USDC

DAOrayaki DAO Research Bonus Pool:

Voting progress: DAO Committee /7 passed

Contributors: Demo, DAOctor @DAOrayaki

Total bounty: 120 USDC

Types of research: Peer Review; DAOs

Original Author: outofpocket house

Contributors: Demo, DAOctor @DAOrayaki

Original article: Decentralizing Journals and Peer Review DAOs: the evolution of legitimacy in scientific publishing

Peer Reviewed Journals and Peer Plugs

I'm tired of begging my doctor friends for years to send me articles in certain journals, and I'm tired of using Sci-Hub (especially if I don't use my work computer as a hassle). The journal arouses my ire, and I feel the rage of the corresponding author. I'm sure journals are also shuddering at the hassle of paying.

I think it's worth talking about peer review and how journals work today, because COVID has actually shown us how important speed and quality of papers need to be in this age of the internet and information changing rapidly. What is even more interesting when we simultaneously see the rise of new community structures, such as DAOs, are that these structures have features well suited to rethinking journals and peer review.

OG peer review and journals as a business

Publishing in journals is important for anyone looking to make a name for themselves in their field, especially in healthcare. Same with tech legitimacy - if you're on a list of people under a certain age cutoff, healthcare looks at your publication history in journals to assess your social capital. Once you have 3 papers published in Nature, you get appointed to a position like a chair in healthcare, or you have to tell aspiring medical students that publication is a requirement to get into their dream program.

To publish a paper, you have to go through many steps after completing your experiment/analysis.

First, you write the manuscript of your dissertation and submit it to a potentially relevant publication. In most cases you will have to confirm that the paper has not been reviewed anywhere else - having to wait for the full review process to complete before submitting it elsewhere can slow down the process significantly.

The publication's editors will assess the article's suitability for their publication. Every publication has different rules about what they care about, some combination of whether it's original, whether it's important/promotes the inflation of the field, reader readability, conflict of interest issues, etc.

Editors send manuscripts to a network of peers in the same field, the peer review process. Apparently some people call this "refereeing", I think it's like "football vs. soccer" in academia.

Reviewers do it very altruistically to stay at the cutting edge of research, along with some societal pressure from others in the field. Reviewers can be new to the field or very experienced. It's really just a matter of who is available and whether they have the right expertise to review.

The number of reviewers will vary according to the nicheness of the journal and subject, but is usually at least two. The process can be single-blind - where the reviewers are anonymous, or double-blind - where both reviewers and authors are anonymous, or open - where they know each other. From here suggestions can be made on how to improve the manuscript, which would otherwise be rejected/accepted.

At this point, if the article is published in a journal, the authors are praised, they can relax, they get a lot of money from the journal, and the journal becomes more valuable by virtue of their contribution.

1. Wait, sorry, I'm told the author doesn't actually get paid. Wait, peer reviewers aren't paid either? But how do journals sell seven-figure subscription contracts to institutions when they essentially provide free labor for peer review? Only one journal with an operating profit of more than $2B and a profit margin of more than 35%? ? ? ?

2. You can actually email any author and they'll provide a pdf of their paper, since they're not going to get anything from the journal anyway. I do this all the time, and I've managed to meet some super smart authors!

3. Peer review and publication systems have worked this way for a long time, and we've actually relied on them to handle the physical distribution of journals, manuscripts, etc. Today, journals use industry's reliance on peer-reviewed publications as a proxy for quality to coordinate massive amounts of unpaid labor. Authors depend to some extent on these journals for reputation points in their field. This is still viewed heavily in terms of access to certain positions/scholarships, access to grants and funding, etc.

1. The current program has a number of problems:

2. Peer review as a process takes a long time. PNAS is a highly selective publication and says it takes an average of 6.4 months from submission to publication. It will take 6 months just to stop making fun of publication names (sorry, I'm only a kid).

3. Papers are at the discretion of editors, who may tend to be more conservative in the kinds of viewpoints they approve. This can easily lead to bias based on what the editor considers "important" issues, especially when it comes to issues affecting marginalized/lower socioeconomic groups. In addition, this also means that if the findings of the paper are not interesting enough, even if it is beneficial to society, it will rarely be published (eg, meta-analysis of other studies, studies with negative findings, etc.).

4. Your paper is subject to the peer reviewers you happen to find, i.e. 2+ random people in your field. There isn't a ton of training on how to be a good peer reviewer, which can lead to variance in reviews/approvals. They might just disagree philosophically with the general point of view, they might have some different form of bias depending on who the author is, etc.

Since peer review is largely altruistic, reviewers will often favor other academics rather than people actually working in industry, who may view research from a more practical perspective than theory.

Reviewers also have their jobs and other things to do. Reviewer fatigue is becoming more and more real due to a higher proportion of papers being submitted than new reviewers. More importantly, journals are happy to spin off new journals that are increasingly niche. There is a journal devoted to mitochondria, and authors are hired based on how many times they use the phrase "the powerhouse of the cell." More journals = more subscription fees institutions pay = more reviewer work.

There seems to be an open debate about whether the current peer review system is actually weeding out papers correctly. This experiment resubmitted 12 approved papers, and 89% of the peer reviewers said the paper should not have been published. Recently, hundreds of bogus papers managed to get through the process thanks to scammers posing as guest editors.

Many feel that keeping this kind of information behind a paywall is bad for society, and even the institutions that pay for these large subscriptions are starting to loathe them. Elsevier has been in very public spats with several universities, such as the University of California, arguing that the journals charge more without adding any value, and that the universities basically have to pay.

In general, people in the field generally agree that peer review is important as a concept and have fairly good experience with it. Below is a more thorough and larger survey of those involved in peer review.

However, in the internet age, there may be new ways to do peer review and disseminate information. In fact, it's happening either way.

The Rise of Open Source Papers, Preprints, and Sci-Hub

Over the past decade, the Internet has largely eliminated the need for print media as a means of communication, and this includes periodicals as well. This led to the rise of a few different phenomena.

The first is Sci-Hub. In 2011, Alexandra Elbakyan, a 22-year-old graduate student in Kazakhstan, launched Sci-Hub, a way to pirate papers behind a paywall. Elsevier was furious and sued Sci-Hub, and now Alexandra has to keep moving her current location, and Sci-Hub's servers are constantly changing. It reminds me of the moment when the labels in the music industry went against Napster — they finally found a better business model, only now against each other.

Second is the rise of open source journals. These journals allow readers free access to published articles and generate most of their income from charging authors article processing fees. Some prestigious journals also have a hybrid model where authors can pay to make their research open access. The price can be high, such as "Nature" charges more than 5,500 US dollars. The paid nature of open access raises questions about whether these journals have undergone rigorous peer-review standards, or are only focused on the number of manuscripts published. PLOS seems to be the most well-known/respected open access journal. For federally funded research, an open access version must be available within a year of publication in a journal - perhaps the best of both worlds?

The obvious benefit of open journals is that anyone can access them, which in theory also means that anyone can comment on them after publication. This is good for the spread of ideas, and any problem is more likely to be noticed by more people. If you take this idea to its logical conclusion, it might suggest that we should publish things before peer review, and peer review should happen in the public domain.

That's the theory behind the preprint papers you can read in repositories like bioRxiv and arXiv. Why go through peer review before publishing if you can... directly publish online? "I've been researching this topic for months and they just...tweeted..."

Why do peer review with a small group of people when it can happen in public? People can contribute opinions, other people can contribute to those opinions, and so on. Ideas can be published faster, iterated faster, and become more of a group review instead of a 6 month wait.

This is especially necessary during COVID, where the speed at which information spreads is key. In the early days of COVID, many experimental treatments being tested around the world would quickly become preprint papers, and other institutions could implement those interventions that seemed to work.

However, this also highlights the problem of making these papers extremely accessible - some groups will use them as a weapon to suit their own agenda. In fact, an early preprint surrounding the myocarditis risk of COVID vaccines ended up being a talking point for anti-vaccine groups, and was eventually retracted because they got the denominator wrong by 25 orders of magnitude. However, the paper had spread like wildfire by then. It's like you accidentally put two 0's at the end of your Venmo request and you and your friends laugh at the mistake, only this time the mistake was to confuse the entire scientific community and spread the word about vaccines during a global pandemic Risk of misinformation.

1. So, clearly, between closed journals, open access journals, and preprints, each of the different models has advantages and disadvantages. Again, the web is how some of these modes are unlocked, and perhaps new technologies will create other new ones.

2. secondary title

3. Peer-reviewed DAO?

Two of my roommates are self-proclaimed "crypto degenerates." I'm learning something from them.

If you get the right tips from a twitter account with an anime character profile, you can increase your income by 8000 times in a week;

It's easier to sell a pixelated jpeg for $3 million than it is to raise $3 million for a chronic disease management company;

1. Our hosts do not accept any Shiba Inu related monetary rentals.

2. But they also talked about more interesting stuff. In particular, the idea of ​​decentralized autonomous organizations (DAOs) is being experimented with by the cryptocurrency community. It's just getting started, but the general idea is if you can build trust and incentive systems that enable a community of people who don't necessarily know each other to coordinate and get something done.

3. Currently, for journals, this is done through centralized top-down decision-making. The CEO of the journal determines the direction of the company, the rules are propagated through the company, and the authors/peer reviewers don't have much say or ownership over the direction of the journal. The question DAO poses is whether you can do this without any single entity making these decisions, that is, these decisions are discussed and guided by the participants of the DAO.

4. There are many good articles about DAOs, such as Mario Gabriele's post and Linda Xie's post. In my head, the main thing a DAO needs is.

5. There is some means of voting and governance on issues, where participants have some form of weighted vote. Weights can be the reputation of voters, how much ownership they have of the network via tokens, and more.

6. There are means of discussion where members can chat about what the DAO wants to do. A lot of this is happening today on Discord, Twitter, etc.

DAO membership may or may not have a threshold. You may need to buy tokens to participate, be accepted by existing members, etc. Or it can be completely open.

There is some form of reputation scoring. For example, Reddit has Reddit coins and badges etc. that travel with an individual's avatar. In the DAO's version, these rewards can travel with one to any other DAO built on the same protocol (aka interoperability, a word you all know!).

The DAO has a system of organizing tasks that need to be completed and rewards for those tasks. This could be a monetary incentive like a bounty or a reputational incentive to get other benefits in the DAO like more voting power.

There are rules for how things are executed and smart contracts that make it actually happen. There are basically no middlemen in building and enforcing these (which can also perpetuate decisions, creating other problems).

1. DAOs today are used in many different ways. They might focus on making upgrades to the blockchain protocol, making decisions about where the community should choose to spend their money, pooling capital to buy things, etc. Below is a cool example of a vote for SushiSwap, which at a high level makes it easier to trade different cryptocurrencies. Community members proposed building analytics and dashboards to create activity reports on different blockchain protocols. There is a budget request, a discussion board on how to use it, and a vote to secure it (along with a history of which wallets voted for the proposal).

2. As I learn more about DAOs, I feel like this has the potential to be a great model for peer review. Today's journals serve a coordination + reputation role, but they charge prohibitively high fees for it. Perhaps DAOs can do better, especially since those involved in peer review want more recognition, funding, transparency, and interoperable peer review contributions.

3. same survey as above

4. This is way outside my comfort zone, but I'd like to know what a journal DAO actually looks like.

5. A journal DAO that focuses on a specific topic exists and only lets people in based on certain inclusion criteria (eg proving they are part of a university, having previously published research, passing tests to demonstrate their knowledge, etc.). A little friction here will keep the giants from gaming the system.

6. Papers are submitted by authors who already have a reputation score based on their previous contributions to peer review.

7. Any reviewer can leave a comment, but highly reputable authors can offer some reputation currency by inviting specific reviewers to take a look.

8. Reviewers are rewarded with some form of non-monetary reputation token, which can be used as currency between different journal DAOs. Reviewers earn reputation tokens based on review quality, possibly determined by staking (perhaps people with high reputation can give more stakes?) and/or people in the community get a fixed amount of badges to denote specific things ( For example, a badge indicates timeliness of review, or thoroughness).

Reviews are public, so reviews are transparent, and while reviewers themselves may be blind, reputation scores are still available.

Reputation tokens provide greater weight to votes to determine different future actions of the journal, which will be voted on by the community.

Maybe you can't even submit a manuscript for peer review until you make some comments/ideas yourself.

Maybe there will be some kind of central funding system for people to apply, and use a reputation system. So, even if you're not getting paid directly from reviewing papers, you can build your reputation and make it easier for you to get funding for your own projects.

This is just random thinking on my part and obviously not well thought out, but my thought is that community members should have a way of building a reputation and using that reputation to gain input into the work of other members of the community.