Probability

[sangui]

Hi everyone,

I was looking on the post about the reliability of published research.

And I was wondering if we could know, a posteriori, if a study is dependable or not.

Do you know any statistic test to improve our understanding of already published paper?

In biology I have seen a lot of people doing three or four different tests to see if their results are meaningful. Isn't it a kind of fraud?

Thank you very much.

[CharonY]

24 minutes ago, sangui said:

And I was wondering if we could know, a posteriori, if a study is dependable or not.

Yes, often replication studies, or studies building up on the study in question could verify or invalidate the prior study. Alternatively, a statistician might spot issues in the data analysis that reviewers may have missed.

25 minutes ago, sangui said:

Do you know any statistic test to improve our understanding of already published paper?

There is none as such as data analysis is always dependent on the type of data. What is possible is to conduct meta-analysis (i.e. incorporating data from other studies) and look at the stability of the outcome.

26 minutes ago, sangui said:

In biology I have seen a lot of people doing three or four different tests to see if their results are meaningful. Isn't it a kind of fraud?

You mean running different statistical tests on the same data set to find one that indicates statistical significance ? This is often called data dredging , and while it is not outright fraud, it indicates poor understanding statistical analyses and also the data set. Essentially one runs afoul of multiple hypothesis testing issues. 

[sangui]

1 minute ago, CharonY said:

You mean running different statistical tests on the same data set to find one that indicates statistical significance ?

Yes it's what I meant.

 

[Country Boy]

 Which statistical test to use will depend strongly on what the data is.

[taeto]

On 2/13/2020 at 4:41 PM, Country Boy said:

 Which statistical test to use will depend strongly on what the data is.

That came out as possibly ambiguous. The test to use will depend on what kind of data you expect as the outcome of the experiment. Then once you can actually see the data, you are bound by your initial decision. Anything else would be cheating.