I probably disagree with you on almost everything, but I’m very glad and grateful to see you publish this. I’ve seen in real life effective altruists go to extremes before in ways that are both self-destructive and destructive to the people around them. Just absolutely begging more public EAs to speak up more about practical and prudent limits on how far to take things.
I dont really like moral uncertainty as a solution here, because uncertainty is something you try to resolve. Such a theory would mean we should wish to be sure enough to maximise, but we dont. And if our epistemic access to ethics is ultimately intuitive, then theres no strong reason to think we ever will converge on a single theory to maximise, unless we think we should. In the event that we dont, I think it makes more sense to say that the good *just is* a certain division of resources between different causes, instead of maintaining an officially-epistemic gap that will never be resolved. This is not to rule out moral uncertainty on the way there, but I think we can allow division even for ideal agents.
You may also be interested in the theory linked below. It takes the diversification mechanism over beliefs to be the basic decision theory, and can recover maximising behaviour in "normal" cases as a mutually beneficial agreement between some of those agents.
"Safer options will reduce potential upside, but—if done well—not to the same extent as we lower the downside risk."
In the case of a typical person choosing which charities to donate to, I worry that the ensemble of subagents symmetrically lowers one's upside potential.
If we are choosing between donating to various charities, and the amount of money to be donated is small compared to each charity's annual donations, then there is (to excellent approximation) roughly constant marginal utility of each dollar. In that case, choosing to give to a mix of charities instead of going all-in on the expectedly best one symmetrically lowers your downside risk and upside payoff.
In this case, I don't see why someone who endorses expected value maximization for ideal agents would endorse worldview diversification for non-ideal agents.
Realistically, a "typical person" is not going to be open to maximizing expected value, but more may be open to the moderate pluralism of the ensemble approach, so I'd expect popularizing the idea to increase donations to high-impact causes.
As I say in the post, I wouldn't necessarily want to discourage an individual from going all-in on a neglected worthy cause. I'm more trying to support the intuition that we shouldn't want *all of society* (or a massive philanthropic foundation) to adopt an undiversified approach.
This is something I thought about for a long time before I came to the conclusion that ultimately
1. The moral assessment of persons really doesn’t matter.
2. I can come to terms with being a morally flawed person. As a utilitarian I eat meat and don’t do some of the other things I should, and that’s wrong and I work to do better but I’m ok with not being perfect.
3. I don’t actually agree with managing moral uncertainty using multiple moral theories as anything but heuristics.
There is a weird belief that seems rampant in philosophy and the wider world that a moral theory must make all people at least minimally good people, and this is why many think utilitarianism is too demanding. I think there is no reason this needs to be true.
As another point about the naive instrumentalist is that it really depends on the strength of existing norms. As we see those norms lose value and efficacy the case for breaking them on instrumentalist grounds becomes stronger. In the U.S. today where this is happening this may tempt people to break them more, possible motivating more political violence and so on.
I disagree with your claim that the moral assessment of persons doesn't matter. This is a core question of ethics and one of the main reasons people are interested in ethics. Everyone who isn't sociopath cares about moral assessment at some level. And indeed, it's arguably necessary for human society to function. I think the real problem is that we don't distinguish moral assessment from other questions in ethical philosophy.
There are multiple questions we're trying to answer with moral theorizing:
1. What actions are good or bad, and what makes them so?
2. What constitutes good or bad moral character?
3. What makes someone worthy of negative or positive moral judgement and deserving of punishment or reward (socially or legally)?
The debate between utilitarianism and deontology is mostly about 1, but most people (even philosophers) want to package 1-3 all together, so good character is defined simply as maximizing good actions. And of course we generally think people of bad moral character deserve to face consequences, even if it's just mild social shaming or feelings of guilt. So basically we come to a view that anyone who isn't maximizing good acts is a bad person. I think this gets at the heart of the demandingness objection.
So your view, where we are open to the idea that most people may not be good, that everyone/most people might be deeply morally flawed, but also that no one should be punished or feel guilty about this, is odd to most people.
While the answers to 2 and 3 are related to your answer to 1, they are separate questions and I don't think one's answer to 1 on its own entails specific answers to 2 and 3. I think moral debate would be less muddled if we could separate these out more explicitly, but also actually try to provide answers to 2 and 3 rather than dismissing them.
This is something that I often get a lot of push back for and I should clarify that what I mean when I say “it doesn’t matter” is that it’s not intrinsically important, or a first order concern. It has lots of practical importance but the justification for any practice or moral praise, blame, etc… is in its effects and has nothing to do with any notion of desert or the like.
I really think the only relevent question is 1. The third question the one that deals with moral assessment most directly requires a version of moral responsibility or desert that simply need not exist in utilitarian ethics(and introduces substantial problems besides).
I recognize that they package all three together, I simply think they are wrong. We can do as you describe but there is nothing essential about it, it’s not a first order concern of ethics. I crucially didn’t so nobody should face repercussions, it’s simply that those repercussions really need have no reason for existing except insofar as they are conducive to promoting utility. They are not a first order concern.
Treating it this way (in utilitarian ethics) gets you out of dealing with a universe of tricky problems that surround desert, free will and moral address. Since for utilitarianism it’s simply superfluous you lose nothing in my estimation.
As for how this muddies moral debate I think that’s spot on.
Great post. When you mention speculative reasoning, it seems to me like this covers a few different kinds of things, and I'm wondering if you think we should approach any of them different in our moral portfolio. Some of the speculative AI concerns depend on factual predictions/projections about the future. Concerns about things that are surprising to most people, like the moral status of shrimp, involve facts that are unremarkable, but the moral claim will be surprising. High-stakes naive instrumentalist actions (like going to see an archduke at a parade, attacking AI CEOs, stealing to give) might involve a combination of both (weighing the wrong against an expected outcome).
I could be way off based since I'm not a philosopher, but am I wrong to read this as endorsing a type of practical applied ethical intuitionism? It gives me Michael Huemer vibes.
The easy dupe versus dogmatist framing is the point I believe. Both failure modes are real, and the history of ethics is littered with people who were destroyed by both. What Richard is really arguing is that fallibility itself should shape method. Not what we conclude, but how much trust we place in the reasoning that gets us there. That’s not a retreat from rationality. It’s the most rational thing an imperfect reasoner can do.
I probably disagree with you on almost everything, but I’m very glad and grateful to see you publish this. I’ve seen in real life effective altruists go to extremes before in ways that are both self-destructive and destructive to the people around them. Just absolutely begging more public EAs to speak up more about practical and prudent limits on how far to take things.
I dont really like moral uncertainty as a solution here, because uncertainty is something you try to resolve. Such a theory would mean we should wish to be sure enough to maximise, but we dont. And if our epistemic access to ethics is ultimately intuitive, then theres no strong reason to think we ever will converge on a single theory to maximise, unless we think we should. In the event that we dont, I think it makes more sense to say that the good *just is* a certain division of resources between different causes, instead of maintaining an officially-epistemic gap that will never be resolved. This is not to rule out moral uncertainty on the way there, but I think we can allow division even for ideal agents.
You may also be interested in the theory linked below. It takes the diversification mechanism over beliefs to be the basic decision theory, and can recover maximising behaviour in "normal" cases as a mutually beneficial agreement between some of those agents.
https://www.lesswrong.com/s/4hmf7rdfuXDJkxhfg
"Safer options will reduce potential upside, but—if done well—not to the same extent as we lower the downside risk."
In the case of a typical person choosing which charities to donate to, I worry that the ensemble of subagents symmetrically lowers one's upside potential.
If we are choosing between donating to various charities, and the amount of money to be donated is small compared to each charity's annual donations, then there is (to excellent approximation) roughly constant marginal utility of each dollar. In that case, choosing to give to a mix of charities instead of going all-in on the expectedly best one symmetrically lowers your downside risk and upside payoff.
In this case, I don't see why someone who endorses expected value maximization for ideal agents would endorse worldview diversification for non-ideal agents.
Realistically, a "typical person" is not going to be open to maximizing expected value, but more may be open to the moderate pluralism of the ensemble approach, so I'd expect popularizing the idea to increase donations to high-impact causes.
As I say in the post, I wouldn't necessarily want to discourage an individual from going all-in on a neglected worthy cause. I'm more trying to support the intuition that we shouldn't want *all of society* (or a massive philanthropic foundation) to adopt an undiversified approach.
This is something I thought about for a long time before I came to the conclusion that ultimately
1. The moral assessment of persons really doesn’t matter.
2. I can come to terms with being a morally flawed person. As a utilitarian I eat meat and don’t do some of the other things I should, and that’s wrong and I work to do better but I’m ok with not being perfect.
3. I don’t actually agree with managing moral uncertainty using multiple moral theories as anything but heuristics.
There is a weird belief that seems rampant in philosophy and the wider world that a moral theory must make all people at least minimally good people, and this is why many think utilitarianism is too demanding. I think there is no reason this needs to be true.
As another point about the naive instrumentalist is that it really depends on the strength of existing norms. As we see those norms lose value and efficacy the case for breaking them on instrumentalist grounds becomes stronger. In the U.S. today where this is happening this may tempt people to break them more, possible motivating more political violence and so on.
I disagree with your claim that the moral assessment of persons doesn't matter. This is a core question of ethics and one of the main reasons people are interested in ethics. Everyone who isn't sociopath cares about moral assessment at some level. And indeed, it's arguably necessary for human society to function. I think the real problem is that we don't distinguish moral assessment from other questions in ethical philosophy.
There are multiple questions we're trying to answer with moral theorizing:
1. What actions are good or bad, and what makes them so?
2. What constitutes good or bad moral character?
3. What makes someone worthy of negative or positive moral judgement and deserving of punishment or reward (socially or legally)?
The debate between utilitarianism and deontology is mostly about 1, but most people (even philosophers) want to package 1-3 all together, so good character is defined simply as maximizing good actions. And of course we generally think people of bad moral character deserve to face consequences, even if it's just mild social shaming or feelings of guilt. So basically we come to a view that anyone who isn't maximizing good acts is a bad person. I think this gets at the heart of the demandingness objection.
So your view, where we are open to the idea that most people may not be good, that everyone/most people might be deeply morally flawed, but also that no one should be punished or feel guilty about this, is odd to most people.
While the answers to 2 and 3 are related to your answer to 1, they are separate questions and I don't think one's answer to 1 on its own entails specific answers to 2 and 3. I think moral debate would be less muddled if we could separate these out more explicitly, but also actually try to provide answers to 2 and 3 rather than dismissing them.
This is something that I often get a lot of push back for and I should clarify that what I mean when I say “it doesn’t matter” is that it’s not intrinsically important, or a first order concern. It has lots of practical importance but the justification for any practice or moral praise, blame, etc… is in its effects and has nothing to do with any notion of desert or the like.
I really think the only relevent question is 1. The third question the one that deals with moral assessment most directly requires a version of moral responsibility or desert that simply need not exist in utilitarian ethics(and introduces substantial problems besides).
I recognize that they package all three together, I simply think they are wrong. We can do as you describe but there is nothing essential about it, it’s not a first order concern of ethics. I crucially didn’t so nobody should face repercussions, it’s simply that those repercussions really need have no reason for existing except insofar as they are conducive to promoting utility. They are not a first order concern.
Treating it this way (in utilitarian ethics) gets you out of dealing with a universe of tricky problems that surround desert, free will and moral address. Since for utilitarianism it’s simply superfluous you lose nothing in my estimation.
As for how this muddies moral debate I think that’s spot on.
Great post. When you mention speculative reasoning, it seems to me like this covers a few different kinds of things, and I'm wondering if you think we should approach any of them different in our moral portfolio. Some of the speculative AI concerns depend on factual predictions/projections about the future. Concerns about things that are surprising to most people, like the moral status of shrimp, involve facts that are unremarkable, but the moral claim will be surprising. High-stakes naive instrumentalist actions (like going to see an archduke at a parade, attacking AI CEOs, stealing to give) might involve a combination of both (weighing the wrong against an expected outcome).
I could be way off based since I'm not a philosopher, but am I wrong to read this as endorsing a type of practical applied ethical intuitionism? It gives me Michael Huemer vibes.
The easy dupe versus dogmatist framing is the point I believe. Both failure modes are real, and the history of ethics is littered with people who were destroyed by both. What Richard is really arguing is that fallibility itself should shape method. Not what we conclude, but how much trust we place in the reasoning that gets us there. That’s not a retreat from rationality. It’s the most rational thing an imperfect reasoner can do.