This is Nick Brown's Typepad Profile.
Join Typepad and start following Nick Brown's activity
Join Now!
Already a member? Sign In
Nick Brown
Recent Activity
Anna: If the journal had replied with "Unfortunately, six people all sent basically the same ideas, so we can only publish one" then this might have some merit. But when six people, presumably independently, submit similar arguments, doesn't that suggest that they might, collectively, have a point?
The underwater hippo is very cute :-), but there's also an elephant in the room here, namely the relative prestige and power of the authors of the original and replication studies. It ought not to matter whether the first is a named-chair full professor and the second is a grad student. In practice, it does. We're never going to be doing "science" properly until we stop doing the "comédie humaine".
Toggle Commented Apr 9, 2015 on on flukiness at sometimes i'm wrong
Your description of nailing down methods reminds me very strongly of what happens when testing dowsers (people who claim to be able detect liquids underground). Dowsers are among the most sincere of believers in pseudoscience (compared to, say, cold readers), but when they fail to detect the effect they're looking for, they tend to blame it on perturbances caused by that car or this person's hat. So experimenters spend a very long time checking that every minor detail of their setup is to the dowser's satisfaction, and the latter agrees not to claim that any of the items they've checked was the cause of a fault in the vortex (etc). Then they run the experiment, and of course they get a null result. "Oh," says the dowser (always, always, without fail), "It must have been because the vortex (etc) was perturbed by the bird that flew past/the sun going behind a cloud/whatever". I'm very interested to see what comes out of Kahneman's "adversarial collaborations", but I don't expect much better, because researchers - especially those who have enjoyed a measure of success up to now - are not about to go back to square one. To this outsider, the failure of psychologists to appreciate that the solid stuff we know about social psychology (starting with cognitive dissonance, confirmation bias, and groupthink) also applies to them, is one of the strangest aspects of the field.
Toggle Commented Jan 14, 2015 on modus tollens bitches at sometimes i'm wrong
Simine, I agree, if all we were doing was taking articles where the PI currently decides to run six underpowered studies and replacing those with two studies with higher N, then "statistically significant" results might also mean something, and PI employment levels are unaffected. But think of all the single-study, N=60 articles that would no longer be possible because the budget, or number of grad students who can be co-opted to conduct the experiments, doesn't extend to N=200. Maybe three colleagues can amalgamate their resources and run a single study instead of three, and then all be "co-PIs" on a hybrid study that doesn't really represent any of their true interests, but I don't see that being very popular. In my extensive experience of working in bureaucracies of various sizes, I have found that giving up status is about the last thing that people are prepared to do, even if the alternative is giving up some of the resources that are needed to do the job properly. (Why that should be, even among very intelligent people, would probably make an interesting study.) A further problem is that if fewer articles are being published (hooray!) for us to wade through each month looking for the power problems, it means that fewer "scientific findings" are being made (especially if we continue to not publish and/or ignore null results). I'm not sure if the whole self-sustaining industry of funding, researchers, journals, press releases, media, and institutional prestige is ready for the implications of that. I suspect that the number of studies being conducted has a near-lawful relationship with the number of people with an economic interest in those studies taking place, such that a reduction in one implies a near-proportional reduction in the other. As someone with no dog in this fight (I'm a retired non-academic), I have no problem either describing that or thinking about its implications, but I realise that that probably doesn't apply to most people who are likely to be involved in this discussion.
Toggle Commented Oct 9, 2014 on open letter to editors at sometimes i'm wrong
When talking to experimental psychologists, I find that they are almost universally receptive to the idea of having 5x fewer psychologists and 5x more resources per study. Perhaps they all imagine that they, personally, would all make the cut (cf. Kruger & Dunning, 1999). Anyway, fewer articles and bigger samples means fewer PIs. Just sayin'.
Toggle Commented Oct 8, 2014 on open letter to editors at sometimes i'm wrong
For Brett: I (@sTeamTraen) may be the originator of the #repligate meme. Although I normally hate the way that any public controversy automatically gets turned onto something-gate, the pun with "replicate" was clearly too strong for my unconscious to resist. So I mentioned the word "repligate" in a tweet on June 2, 2014. It was immediately picked up by Brian Nosek, one of the two guest editors of the special issue of Social Psychology dedicated to replications, and it went on from there. I subsequently discovered, while searching Twitter, that someone else had come up with the same term independently a few days earlier (which is hardly surprising, as we're not talking about the creativity required to write a symphony here), but I hadn't seen that when I thought of it, and as far as I know it was "my" version that took off.
Toggle Commented Sep 27, 2014 on Guest Post by John Doris at sometimes i'm wrong
Nick Brown is now following The Typepad Team
Sep 27, 2014