Skip to content

can human effect be minimized or degraded to zero in peer review system?

Featured Replies

I have conducted more than several reviews in sci-journal for two years or more and wonder the answer for this question both to expect / ensure the faster process and the less subjective assessment?

Edited by ahmet

I don't see how much, if anything, in science can be completely objective since it is being done by humans and even AI is subject to its programming. That being said, peer review has been how we weed out unworkable ideas. Whether or not AI actually will improve the process is anyone's guess but I can see ways it could help, especially with speed.

It also the question whether you want to. Few papers are just merely a series of statements of facts, but rather they try to push the boundary of knowledge by providing a particular perspective, which are then underpinned by evidence. Whether that evidence is convincing or not (or require further discussion) is ultimately a value judgement. Fundamentally, you can think of scientific papers as a discussion among peers and if AI takes over, it is going to be a discussion among algorithms.

Regarding the use of AI as support, I could imagine that they could help in finding additional literature, given the exploding (and also deteriorating) literature landscape. However, I can see that backfire spectacularly, as traditionally experts in the field will know the relevant lit they want to see cited. AI would move the discussion more likely to an amalgamated consensus, with no discerning voices.

  • Author
On 3/2/2026 at 5:49 AM, npts2020 said:

I don't see how much, if anything, in science can be completely objective since it is being done by humans and even AI is subject to its programming. That being said, peer review has been how we weed out unworkable ideas. Whether or not AI actually will improve the process is anyone's guess but I can see ways it could help, especially with speed.

from the point of my view, I think we have of course completely objective scientific contexts. However, again to me, even the existing version of "scientific laws" are not in fact so much strong and we currently do not have so much effective / useable "completely objective scientific contexts".

I am almost sure that we would be able discuss or even maybe we might particularly be able to refute at least some of the existing scientific laws.

Edited by ahmet

3 hours ago, ahmet said:

I am almost sure that we would be able discuss or even maybe we might particularly be able to refute at least some of the existing scientific laws.

It’s always possible, in principle, to do so, but I must ask: What is your understanding of what “scientific laws” are?

  • Author
22 hours ago, swansont said:

It’s always possible, in principle, to do so, but I must ask: What is your understanding of what “scientific laws” are?

if you want to learn what "scientific laws" mean or check what I knew on it, then just googling will better help you. Check please.

if you want to ask "how would they be" in future or "how would they change" according to my opinion, then well, I imply that the definition of "scientific law" would regrade/reduce to theorem or lower definition (e.g. proposition, yes this is even probable) in future.

Edited by ahmet

50 minutes ago, ahmet said:

if you want to learn what "scientific laws" mean or check what I knew on it, then just googling will better help you. Check please.

That’s not what I asked. I want to know what your understanding is, and Google won’t give me that.

The reason I ask is that your phrasing suggests that it’s not what the generally accepted definition is.

50 minutes ago, ahmet said:

if you want to ask "how would they be" in future or "how would they change" according to my opinion, then well, I imply that the definition of "scientific law" would regrade/reduce to theorem or lower definition (e.g. proposition, yes this is even probable) in future.

Case in point. A law is not superior to a theory, it is a subset of it. (and not theorem; that’s math).

So I ask again: What is your understanding of what “scientific laws” are?

On 3/3/2026 at 2:42 PM, ahmet said:

from the point of my view, I think we have of course completely objective scientific contexts. However, again to me, even the existing version of "scientific laws" are not in fact so much strong and we currently do not have so much effective / useable "completely objective scientific contexts".

I am almost sure that we would be able discuss or even maybe we might particularly be able to refute at least some of the existing scientific laws.

We shall have to disagree, then. My understanding of what a scientific law is says that it is simply something shown to be true through repeated experimentation. This IMO is as close to objectivity as we are likely to get but even laws are subject to modification or even outright rejection (extremely rare) if better experiments or data show them to be wrong. What may happen here is we may figure out a way of testing any given law but pixels on a screen will not change it. Only completing the test (experimentation) might do that.

Edited by npts2020
context

  • Author
1 hour ago, swansont said:

That’s not what I asked. I want to know what your understanding is, and Google won’t give me that.

And I do not know why you do want to know my personal understanding. Be sure, there are tons of scientists who just rely on "published materials (i.e. reference)" so, when there is no reference to support your claim,again then, you will find tons of academics who would just believe that you were just making an illogic allegation/assertion.

so, as said , I do not understand why you do want to know my personal understanding. ah, simply forget it now. :)

1 hour ago, swansont said:

The reason I ask is that your phrasing suggests that it’s not what the generally accepted definition is.

so, you believe or think I did not know what the generally accepted definition was.

1 hour ago, swansont said:

Case in point. A law is not superior to a theory, it is a subset of it. (and not theorem; that’s math).

this does not make any excitement for someone like me. :)

1 hour ago, swansont said:

So I ask again: What is your understanding of what “scientific laws” are?

then, forget everything for now. But returning to the issue, while in our country even being defined as "scientist" would differ than somewhere else (maybe in U.S.) ,I do not define myself as a traditional scientist, I just wondered whether machine tools (ah yes, for instance AI) would be helpful in peer review system. Because my time is important and I do not want to spend it for a thing which will not bring me money back in return.

Edited by ahmet

I find the term traditional scientist a bit confusing. One of the earliest formal lessons I learned is never rely on any given theory. Its one of the reasons I strive to learn as many different theories as I can. However no matter how outstanding tried and true any theory is all it takes is sufficient evidence to overturn any theory. Regardless if its considered a law or not. The other lesson I learned is any viable theory has some likely-hood of being correct. So if flexibility is traditionalist then I for one will continue in that manner as it maintains an open mind to other possibilities.

2 hours ago, ahmet said:

so, I do not define myself as a traditional scientist,

11 hours ago, ahmet said:

this does not make any excitement for someone like me. )

I fail to see why this matters, other than having a misunderstanding of the structure means any criticism of it is likely flawed, as you would be attacking a strawman

11 hours ago, ahmet said:

then, forget everything for now. But returning to the issue, while in our country even being defined as "scientist" would differ than somewhere else (maybe in U.S.) ,I do not define myself as a traditional scientist, I just wondered whether machine tools (ah yes, for instance AI) would be helpful in peer review system. Because my time is important and I do not want to spend it for a thing which will not bring me money back in return.

AI, specifically LLMs, are trained on existing information, so how are they going to evaluate an investigation into a new phenomenon or a new technique? How will you be sure the LLM didn’t just make up the evaluation of a paper, as they are prone to do?

  • Author
1 hour ago, swansont said:

AI, specifically LLMs, are trained on existing information, so how are they going to evaluate an investigation into a new phenomenon or a new technique? How will you be sure the LLM didn’t just make up the evaluation of a paper, as they are prone to do?

Currently I do not closely interest to software contexts (AI or any related programming). However to me by successful programming and significant effort, it does nıt seem impossible really.

like this:

if the article/paper contains a new phenomenon or technique, then basically the programmer will define

1) what a phenomenon and / or a technique means. (first step)

2) the checker then will scan all the previous documents whether they contain a phenomennon/technique. (if , do , for loops may function helpful there)

3) if yes, then the program will compare the content of (that phenomenon / technique of) existing paper(s) with the submitted paper/article.

else, the program will simply pass to another document. and so on until to the last document that was available.

4) then you will indirectly have defined what a new phenomenon / technique means. (i.e. if at the end of all loops the comparison brings no result , then you may reach to say:

"this content includes new phenomenon or technique."

I think it works succesfully in general

say; why not?

That does not seem to make any real sense. Fundamentally a review is to some degree a value judgement on the persuasiveness of an argument or finding presented. Yes, one can use software to figure the basis and whether others have already reported on it. But if it is something new, you will have to provide evidence supporting whatever hypotheses you propose. And there will be degrees in judgement as folks might disagree on the value of a particular approach, especially when not applied in a standard way.

So in a normal review, folks will form an argument regarding why they think this approach is sufficiently strong or not. This, again, is ultimately a value judgement and requires folks with different perspectives to agree (or disagree) on whether a publication has sufficient merit. Flawed as it is, I don't see how loop searches are going to add anything to it.

15 hours ago, ahmet said:

And I do not know why you do want to know my personal understanding. Be sure, there are tons of scientists who just rely on "published materials (i.e. reference)"

Because that is fundamental to the whole topic. A review is based on the perspective and knowledge of the review, and it is not an enumeration of facts or factoids. As such, the reviewer's understanding of the topic, as well their specific background and perspective is immensely important.

3 hours ago, ahmet said:

Currently I do not closely interest to software contexts (AI or any related programming). However to me by successful programming and significant effort, it does nıt seem impossible really.

like this:

if the article/paper contains a new phenomenon or technique, then basically the programmer will define

1) what a phenomenon and / or a technique means. (first step)

2) the checker then will scan all the previous documents whether they contain a phenomennon/technique. (if , do , for loops may function helpful there)

3) if yes, then the program will compare the content of (that phenomenon / technique of) existing paper(s) with the submitted paper/article.

else, the program will simply pass to another document. and so on until to the last document that was available.

4) then you will indirectly have defined what a new phenomenon / technique means. (i.e. if at the end of all loops the comparison brings no result , then you may reach to say:

"this content includes new phenomenon or technique."

I think it works succesfully in general

say; why not?

If it’s new, why would there be a previous mention of it?

A new phenomenon or technique has to be evaluated ; you can’t just let an algorithm allow all new stuff through - that’s quite contrary to the purpose of peer review. We have too many scam journals already; this would just add to the problem

And importantly, the authors coming up with a new concepts are supposed to convince skeptical folks. From what I can tell, current LLMs are really bad at that. But to me, it seems that OP sees peer review as a sort of box ticking exercise, which it really shouldn't be.

1 hour ago, swansont said:

We have too many scam journals already; this would just add to the problem

Anecdotally in my field it seems that beyond scam journals, there is also a bit of fragmentation in science publishing and a seeming decline in article quality. Most notably I found that quite a few new papers, unknowingly replicate finding from somewhat older articles (the magic number seems about 15 years, I suspect it has something to do with changes in keywords as well as accessibility). Theoretically, AI which are well-trained could assist here. But then it seems that for some reasons they tend to coagulate around the same articles and tend to have issue to find relevant papers.

  • Author
21 hours ago, CharonY said:

That does not seem to make any real sense. Fundamentally a review is to some degree a value judgement on the persuasiveness of an argument or finding presented. Yes, one can use software to figure the basis and whether others have already reported on it. But if it is something new, you will have to provide evidence supporting whatever hypotheses you propose. And there will be degrees in judgement as folks might disagree on the value of a particular approach, especially when not applied in a standard way.

So in a normal review, folks will form an argument regarding why they think this approach is sufficiently strong or not. This, again, is ultimately a value judgement and requires folks with different perspectives to agree (or disagree) on whether a publication has sufficient merit. Flawed as it is, I don't see how loop searches are going to add anything to it.

Because that is fundamental to the whole topic. A review is based on the perspective and knowledge of the review, and it is not an enumeration of facts or factoids. As such, the reviewer's understanding of the topic, as well their specific background and perspective is immensely important.

nope. do not just make random reading about COPE or reading it straight only.

you speak like someone who does not know the bad side of peer reviewing. Be sure that I have many bad fact of it in my hand. Anyway. I do not want to spend my time to freely make my valuable evaluation. :) (I am planning to reject some forthcoming invitations (if any), but in my aside not really for the case that I believe I was unable to make a judgement. I just do not want to spend my time for a thing whicever will not bring me money in return. ) Say, what do I miss or what do I lose??

19 hours ago, swansont said:

If it’s new, why would there be a previous mention of it?

I tried to tell the comparison between all of the published materials and the submitted material. I did not say if the submitted material was new,then there would be a previous mention of that. check please I will also check.

19 hours ago, swansont said:

A new phenomenon or technique has to be evaluated ; you can’t just let an algorithm allow all new stuff through - that’s quite contrary to the purpose of peer review. We have too many scam journals already; this would just add to the problem

Although, I am not a good programmer, I disagree with great ration. The existing profile of AI may be not as good as to expect to carry out such an evaluation in any content. However, two things stand available in my opinion: 1) I do not think the same thing for future. 2) not every content cannot be evaluated.

Edited by ahmet

11 hours ago, ahmet said:

you speak like someone who does not know the bad side of peer reviewing.

I speak like someone whose professional responsibilities includes being on both sides of peer reviews.

11 hours ago, ahmet said:

I just do not want to spend my time for a thing whicever will not bring me money in return. ) Say, what do I miss or what do I lose??

I am not sure why you bring it up. Serving as reviewer is a voluntary service and if you do not have interest in it, then just don't do it. Likewise, if you are not an expert in a particular area, don't review in that area. It is really simple, actually. What you shouldn't do is to offload it to AI.

  • Author
11 hours ago, CharonY said:

I speak like someone whose professional responsibilities includes being on both sides of peer reviews.

to me, you are neither professional nor a knowledgeable one in making right selections and definitions. Ipresume, your decisions are problematic and you do not have sufficient knowledge in defining things so almost all of your decisions will be problematic. Anyway, this is my last comment under this thread. I won't write anything else here.

3 hours ago, ahmet said:

Anyway, this is my last comment under this thread. I won't write anything else here.

Moderator Note

And on that sour note, the thread is closed. NOT the way discussion should work, imo.

Guest
This topic is now closed to further replies.

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.