If such automation is possible, I would certainly be interested in pursuing it. See my answer 16 above.

I wonder whether there might be a way around this for letters (at least at some schools), given that letters do not come from the applicants themselves, and thus might not be properly thought of as part of the application. One might argue that obtaining a letter of recommendation from a reference is more like perusing an applicant's website for further information about that applicant (and I assume HR plays no role in the latter). Perhaps, on that grounds, one might exclude the request for letters from the official application but ask applicants separately to have their letters sent in?

Joel: For applicants to use AJO to apply for a job, that employer has to sign up with AJO. Many employers don't sign up, and that's why applicants have to use Interfolio instead. So, MARGY is better than AJO only insofar as it, like Interfolio, doesn't require employers to sign up. David: I'm curious how much you learned about how Interfolio works when you were looking into this. Do you know that Interfolio pays people to upload the letters? Do you have any idea whether they've managed to automate that at all? Even if it's only 25% of jobs, MARGY might still make a significant difference. As Ben notes, this can cost $20/application. If someone applies to 60 jobs, using MARGY for 25% of those would save the applicant $300.

That all sounds right: Many schools do request that letters be uploaded rather than emailed. Ideally, this would all be centralized and make life better for everyone; but given that schools are sticking with their own systems, letters of recommendation present a particular challenge because (unlike other parts of the dossier) applicants can't just handle it themselves. MARGY is only of limited use here, insofar as it can't handle such uploads. Interfolio does better, but is (perhaps prohibitively) expensive. So, can MARGY be adapted (either for free or for less) to work with these upload systems? I honestly don't know. My guess is that Interfolio actually has people doing the uploading, at least for some schools, since I would be very surprised if there were an automated way to upload to just any site (especially since some of these systems require you to fill out a form, not just upload a letter). However, I have noticed that many of these HR systems seem to follow a particular model, which I'm guessing they purchase from some company that has largely cornered the market here. One possibility is that Interfolio has an arrangement with such companies whereby they are given access to the inner workings of the system and can figure out how to automate uploads. It is also possible that Interfolio has people who write uploading scripts for systems as they are discovered (their ability to do so would probably depend on how transparent those systems were int terms of their inner workings). If it's all manual, then a free service could only be offered with significant funding to pay the person doing the uploading. In the other cases, it would depend on how much time and coding is required, and whether the systems charge to allow you to work with them in these ways. Without knowing more about this, it's hard to say what would be feasible. I am certainly open to looking into this, but it might be a greater undertaking than I can do alone or even with my co-developer. Three further things: 1. Vitae offers a free service, but I don't really know the details of it, or to what extent schools have to sign up with them. But that might be worth looking into. 2. I know that in some cases, HR requires that people apply through them, but don't care what particular things are requested. So I know of at least one case where a department had applicants fill out a minimal HR application *and* used AJO, so they met the HR requirements but did all their internal vetting through AJO. This is a bit of extra work for applicants, but is free and easier on the letters end. 3. There is a roundabout way that MARGY can handle automated requests from HR systems, (a) if HR doesn't actually care about whether letters are uploaded, (b) the department is willing to accept letters via email and (c) if the department has any control over the contents of the request email. Since MARGY doesn't care what is in your email other than email addresses, in theory an automated system could send an email to MARGY with a relevant mailto code, and so long as the department includes (e.g.) [email protected] in the text of the request email, MARGY would email them the letters. This has two advantages. First, it means that applicants wouldn't have to both fill out the HR forms and send request emails to MARGY. Second, this could help streamline the process you mention in (1), where your administrator uploads letters received via email. My guess is that one annoying part of that person's job is wading through the mess of varied email formats and such. If your applicants use MARGY, every letter will come in looking the same, with "Letter for [Applicant Name]" as the subject heading.

I'll look into it. Unfortunately, most such reviews cost money, and at the moment I'm bankrolling this myself. But if I get sufficient donations or if I find a reviewer interested in getting some pro bono experience (I've been told this sometimes happens), I will pursue this further.

Daniel: I didn't mean to oversell the influence of a particular department on HR. But I do think that if this issue were sufficiently recognized, pressure could be exerted to avoid systems that are costly for applicants. Viktor: I entirely agree that a centralized online application system would be best. Something like AJO's system would do the trick (if it were made more user-friendly), but that requires getting schools to sign up. My point was only that *given* the wide variety of systems employed by different schools, I am tempted to think that the benefits to the school of having letters uploaded and automatically associated with a file are outweighed by the costs (temporal or financial or both) to applicants and their letter-writers. The nice thing about email, in that context, is that it's a shared standard for everyone involved, which is what allows for automated systems like MARGY. It would be much harder (perhaps even possible) to automate uploads within this heterogeneous system. And without automation, someone has to do the grunt work of uploading to each school's system, whether that be a paid dossier service, the letter-writers themselves, or someone else entirely.

It may well be more efficient *for them* because they can have everything in a centralized location that they control. But it is much less efficient for applicants, who (with respect to what we're dealing with) have to find someone to manage their letters of recommendation by hand and (more broadly with respect to the whole hiring process) have to fill out proprietary applications for each job. Arguably, this places a greater burden on applicants than a system that makes things easier for applicants would on HR departments, suggesting that it is also less efficient overall.

1. It's not clear to me that this is the general trend; I also see more and more schools moving towards systems like Interfolio every year. 2. Even if this is the trend, it's one we can buck against. The huge waste of money here is a reason to do what we can to get HR to either allow departments to handle their own searches, or to use systems that allow for email (or other automated collection) of letters of recommendation. 3. Even at schools where applications have to go through HR, there are potential loopholes. At many schools, it may be possible to simply leave out letters of recommendation when dealing with HR. Let HR collect all of the other stuff (names, CVs, etc.) and the department can handle letters separately via email. But, yes, at the end of the day, such a move towards a less efficient system might put programs like MARGY out of work. That would be a shame, and an absurd waste of time and money.

It's possible that the emails could be encrypted, but this would be significantly more secure than the current system. I don't believe Interfolio encrypts emails, and certainly individual letter-writers don't. I'm not sure the costs of doing so, especially in terms of making sure the schools are set up for it, would be worth it. We may go for a formal security review at some point. But, again, how much security do we really care about here? These are not highly sensitive documents; they are just documents that the system shouldn't make too easily available to either the applicant or the server administrators. During beta, the whitelist is unmanaged (people can add their emails freely for testing purposes). Once out of beta, I will be managing the whitelist personally, at least for a while. I will only add emails to the whitelist that I have personally verified as being managed by a hiring entity (e.g., I will see that this is the email listed on PhilJobs as being the place to send letters). Eventually, it would be nice to partner with PhilJobs and other advertising entities to have emails automatically added to the whitelist when ads are put up.

Yes. The files are encrypted before being saved on the server, and remain encrypted on the server at all times. When someone sends in an email request to have their letter sent to [email protected] using their mailto code, the system attaches a decrypted copy of the relevant file to the outgoing email to [email protected]. That decrypted copy isn't saved anywhere else.

Robert, thanks, I hadn't heard about that, and it certainly does seem similar.

I think it's impermissible. I don't think the fact that the would-be saver is an agent makes a moral difference. So, I don't think this is different from deflecting an asteroid carrying the cure for the dying. And I don't think deflecting the asteroid is (much, if at all) different from killing them yourself.

David, In an ideal world, papers would be reviewed by only those who are really qualified to review them,* and who will take that job seriously. The question is whether this ideal is better approximated by a top-down system like yours or crowd-sourcing system like the one I'm proposing. Just a couple of preliminary thoughts: First, it seems to me that the current system does far from a perfect job of this. Even top journals ask people to referee papers when they have little to no evidence that those people are experts in the relevant subfield. Nothing in your system seems prepared to improve upon this. Indeed, if Dale's worries about incentives is at all legitimate, it might turn out to be worse. In contrast, the hope with a crowd-sourced system would be that with a large number of participants, most of whom would have enough integrity to rate only those papers they are qualified to rate, and who would do their best to set aside personal biases, the trolls get swamped. There is at least some evidence (e.g., perhaps, Amazon) that this can work. Finally, the above suggests a major advantage of the crowd-sourcing system with respect to the worry about false negatives, which is that crowd-sourcing is dynamic: A paper's grade is not "locked in" by one review. *It's actually not clear to me that this should always be people who work in the relevant subfield; but I'll set that aside here.

David, I certainly think that something along these lines is a good direction to go in. I'd like to hear more about why you prefer (if you do) a centralized grading system over crowd-sourcing. We might (even more cheaply!) just let everyone post papers to a central location and institute a voting system, say either grades or Reddit-style up/down votes?

First off, you might have noticed this post vanished and then reappeared. I hadn't realized that people were having trouble with comments. At the request of the editors, I postponed briefly to make room for discussion on Ralph's original post (sorry Ralph!). If people are still having trouble commenting, I'm happy to postpone again. If not, maybe this will encourage discussion on both posts! To answer your question, Hille, I think I want to argue something like this: 1. If X is necessary-if-true, then if we believe X and X is true, we are guaranteed to be sensitive to X. 2. If our X-belief-forming mechanisms are stable across nearby possible worlds, then if we believe X and X is true, our belief in X is safe. 3. The modal security of our normative beliefs is derived from the assumption that our normative beliefs are true in combination with: (a) 1 above; (b) the metaphysical necessity of the basic normative truths; (c) 2 above; (d) the availability of stable genealogies for our normative beliefs (e.g., evolutionary stories). 4. None of (nor any combination of) (a)-(d) is relevant to the question of whether or how our normative beliefs track the normative truth at the actual world. 5. Only considerations relevant to the question of whether or how our normative beliefs track the normative truth at the actual world can establish the reliability of those truths. 6. Therefore, modal security established as above cannot be sufficient to establish the reliability of our normative beliefs. Before I stop and let people school me, let me just contrast this with the perception case Clarke-Doane appeals to by analogy: Consider the perceptual case. What we can arguably offer in this case is an evolutionary explanation of how we came to have sensitive mechanisms for perceptual belief, and a neurophysical explanation of how those mechanisms work such that they are sensitive. But these explanations blatantly assume the (actual) truth of our explanatorily basic perceptual beliefs. If the reliability challenge for D-realism requested an explanation of the reliability of our D-beliefs which failed to assume the (actual) truth of our explanatorily basic D-beliefs, then the apparent impossibility of answering it could not be thought to undermine those beliefs. I think Clarke-Doane is right that there can't be a prohibition on assuming the truth of our beliefs. But I think there's an important difference between the perceptual and normative cases. For in the perceptual case, the analogy of (3) above is something like: (3*) The modal security of our perceptual beliefs is derived from the assumption that our perceptual beliefs are true in combination with an evolutionary and neurophysical explanation of how our perceptual appartus came to and function so as to track the subjects of our perceptions at the actual world. So, obviously, I think the heavy lifting is going to be done with something like (4), since (3*) avoids my objection by establishing modal security in a way that seems relevant to how our perceptual beliefs track the truth at the actual world. But that's all very tentative.

I've been curious about this for a while, too. I have a friend who works for an economics journal, and he has encouraged me to write cover letters. His suggestion was less about explaining the value of the content and more about things like mentioning that a paper had been accepted to prestigious conferences or won awards. He claims that this happens to them all the time, and can make a difference. I've never done it; it seemed sort of gauche. But I've wondered if I should.

That's really helpful. So, is the story something like this? Sentiments aim at accurate representation of their objects just as beliefs aim at truth. There can be epistemic reasons to believe regardless of what practical reasons there are to believe or even to not believe—(roughly) reasons having to do with the truth of the thing believed. So, too, can there be fittingness reasons to feel regardless of what practical reasons there are to feel or even to not feel—(roughly) reasons having to do with the accuracy of representing the object as the sentiment does. If that's right, I think I see and understand the view. I still want to resist it, but that's because I don't think epistemic reasons are normative reasons, either, except in the case where we have reasons to think about what's true. Similarly, I wouldn't think fittingness reasons would be normative except in cases where we have reasons to represent via the sentiments. In the snake case I describe, I don't think this condition is met.

Justin (and Dan), It is rather the claim that there are reasons supporting fear, and indeed they constitute an adequate justification of a certain sort, even if you would be better off without fear. Do you find the same story plausible about language? Suppose that, for some reason, I'm set up such that when I think the word "snake," I freeze up. I come upon a snake and, recognizing it, could just turn and run. But then I think "SNAKE!!!" Do you want to say that the combination of psychology (we just tend to think words of things we see, without intending to) and representational accuracy (if it was a mouse, I'd have made a mistake) similarly provides me with a kind of justification? If yes (the cases seem the same to me, certainly), that just drives home for me the absence of normativity. I don't think I had any reason to think "snake" in that case. I just think there's a sense in which the naturalness and accuracy of my doing so inclines us away from characterizing me as making a mistake, and there's a temptation to express that in terms of a kind of justification. Nevertheless, where genuine normative reasons are concerned, I had none: Nothing that actually matters spoke in favor of my feeling fear or thinking "snake."

Dan & Justin, I'm just here to make my usual trouble, and if you'd prefer to focus on this newer stuff, please don't feel obligated to get into this. But I thought it might be worth rehashing some things I suspect you, Dan, have heard (probably from Christian) in case there are new things to say, or others want to weigh in. I take it you think the fitting and meriting relations are normative, in that they provide normative reasons: We have reason to fear the dangerous, be amused by the humerous, etc. I'm skeptical of this. I like the general picture of sentiments as representative, and I think they play an important role for us, I just don't think their fittingess provides us with reasons to feel them. Here are three quick points in favor of my view: Potential Redundancy Suppose I'm in the woods and come upon a poisonous snake. I am capable of fully representing the snake's danger to me, and being suitably motivated, without feeling fear. I can't imagine why, in that case, I would still have reason to feel fear. This is because if I did have a reason to fear, it would stem from my needing to fear the snake in order to respond appropriately to it. But ex hypothesi, I don't. (This might seem similar to Shadow Skepticism, but I take it that it's different given that we might well be able to represent even attitude-dependent values without deploying our sentiments.) Lack of Motivation from the Analogy with Language The above problem isn't surprising when you think of the sentiments as representative in something like the way words are. The word "dog" is fitting for my dog Silke. This does not provide me with any reason to think or say "dog" when I see her, unless that would help me achieve some goal. More generally, there is simply no reason to go around representing things. It's More Plausible that Unfittingness Is Normative While I don't have any reason to think or say "dog" when I see Silke, you might argue that I do have reason not to think or say "cat"—i.e., not to misrepresent. I'm not sure that's true, but it certainly sounds more plausible than the idea that I have reason to go around representing things. Of course, you could claim that both fittingness and unfittingness are normative. But assuming each sentiment is either fitting or unfitting for each object—either the sentiment accurately represents the object or it doesn't—this amounts to ruling out error theory by fiat. For instance, every time I see an animal (say) I either have reason to fear it or reason to not fear it. The possibility that I have no reasons at all has vanished. I don't think we should be comfortable with that result.

David S., Sorry, I think I must have lost the thread somewhere. I thought we were still just talking about seeking the best restaurant as an analogy for seeking the Thing To Do. Of course I agree that Subjectivism doesn't have to actually tell us what the best restaurant is. But now I fear I'm still missing your point. Darn you for creating such an interesting thread with so many issues to tackle.

Jamie, I tend to agree that we should be worried about explaining why some facts end deliberation (i.e., I take it, why they are reasons) while others don't. I'm not sure everyone else thinks we need to do this. I think non-naturalists can't, for instance, and as I read him Enoch isn't much bothered by that. That's one of the main reasons I'm not a non-naturalist! As for Humeans, I was thinking that given that I rejected a Humean theory of reasons, if I accepted a Humean theory of motivation I'd be more worried about my answers to those questions, because I'd think (I take it) that even if a general tendency to be rational could explain why I believe as I judge I ought, it couldn't explain why I act as I judge I ought, since I would still need to reference a desire somewhere.