This is FredHasselman's Typepad Profile.
Join Typepad and start following FredHasselman's activity
Join Now!
Already a member? Sign In
Recent Activity
Hi Lara, I cannot disagree with your recommendations for replication studies, they're great. However the way you build the narrative it now appears you are suggesting that RP:P does not comply with these high standards. I hope you agree that would be very wrong to suggest, one could refer to the procedure of RP:P and see virtually all of your recommendations were implemented in RP:P Also, I was wondering... can you give examples of replication studies that do not meet the high standards and 'further muddy up the literature with a bunch of false negatives'? I can't think of many studies that would qualify as such in the recent literature... there aren't that many replication studies around to choose from. What I *can* disagree with are the ideas about the role of hidden moderators in the application of the scientific method to produce scientific knowledhe Assuming that we're talking anout confirmatory studies and that research questions are always based on some deductive chain of propositions that lead to a prediction of measurment outcomes in terms of an observational constraint between at least a dependent and independent variable.... Then it is absolutely valid to claim, after a failed direct replication study, that an uncontrolled confounding variable in the replication was responsible for the failure to replicate the original effect. However, the consequence of claiming this was the case for the replication is that the original deductive chain/theory/claim was invalid as well! It means that: - the Ceteris Paribus clause is violated. There apparently were sensible/foreseeable moderators that systmaticilly vary with the effect, which were the original study did not explicitly attempt to control for. - to claim a failed direct replication was due to a hidden moderator and the original study observed a true effect implies that randomisation failed in the original study and the the hidden moderator was controlled for 'by accident' / or by selection bias. Random assignment of subjects to groups or conditions controls for non-systematic variability within / between subjects. A moderator variable implies systematic variation, something that randomisation can't resolve. So either way, the interpretation of the results of the original study is problematic if one wishes to explain a failed replication was due to hidden moderators. Luckily philosophers of science (Lakatos) have analysed these defensive strategies in great detail and here's the deal: 1. Progressive research programme: Acknowledge hidden moderator and accept there is a problem with the original study and the theory/deductive chain that predicted the effect. Amend the original claim or start from scratch. 2. Degenrative research programme: Point to hidden moderator in order to protect the perceived veracity of the original result and theory. Do not amend, or reject the original claim. All the best, Fred
Toggle Commented Sep 21, 2015 on Guest Post by Laura Scherer at sometimes i'm wrong
FredHasselman is now following The Typepad Team
Sep 21, 2015