Decision-Making in Research Tasks with Sequential Testing
Author Information
Author(s): Thomas Pfeiffer, David G. Rand, Anna Dreber
Primary Institution: Harvard University
Hypothesis
How does sequential testing affect the reliability of research findings?
Conclusion
Findings tend to become more reliable over time when the most informative tests are performed.
Supporting Evidence
- Computer simulations showed that the fraction of false positives declines over several rounds of testing.
- Human subjects often performed the most informative tests, leading to expected declines in false positives.
- Sequential testing allows researchers to make more informed decisions based on previous results.
Takeaway
When scientists test ideas one after another, they get better at finding the right answers over time.
Methodology
The study used computer simulations and experiments to analyze sequential testing scenarios.
Potential Biases
Independent testing may introduce negative effects on performance.
Limitations
The study's tasks were simplified and may not fully represent real scientific research complexities.
Participant Demographics
Participants were mostly students from the Boston area, median age 21, with roughly equal male and female representation.
Statistical Information
P-Value
p<0.05
Statistical Significance
p<0.05
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website