I don't think I agree. I think you're putting too much focus on motivated reasoning (which, granted, is likely a real bias that leads people to distort their perceptions), instead of what is more likely the 'cynical view' that many people hold, i.e., morality is a socially constructed, evolutionarily useful human fiction for solving coordination problems, and our brains and all of us are kludgy, clunky rube goldberg machines that indeed, can reason well if they make a substantial effort, but are pulled asunder by our conflicting and inevitably inconsistent, inbuilt, preferences.
I don't see why abstract reasoning can lead one "to conclude that we ought to be much more impartial and altruistic than we actually are". Mine just leads me to conclude that:
-there are no oughts. They belong to the same class as Santa Claus, Pink Unicorns and assorted beliefs.
-One should follow one's preferences.
-One should try to maximize the satisfaction of one's preferences, but caveat this with the fact that most of our evolved ones will be inconsistent, and you can't build a model that just locks into one of them as the most important.
-Generally, engaging as an agent in contractual obligations is the best way to generate positive sum games and maximize your own well-being, particularly in the long run. And you're born into communities whose rules you implicitly accept by living there and being educated/indoctrinated into them.
Being more utilitarian/ impartially altruistic doesn't follow from that.
Fair. Modals have a habit of clinging to one, even when tries to do without them. 2 and 3 can be rewritten by changing the 'one should' for 'Nothing impedes one to...'
This is the same structural divide as ethics versus morality. Ethics evaluates; morality commands. And this is why “ought” and “should” are not synonyms. A moral ought derives from the religious “Thou shalt.” It carries a command structure — an authority issuing an instruction. That immediately raises the question: who or what makes the command, and why obey it? A should has no such implication. It is conditional and optional: if I want X, then I should do Y. No authority. No command. No judgement.
This is also why I am neither an error theorist nor a nihilist. Both positions are oxymorons. Both require morality to exist — either to have errors (error theory) or to be its opposite (nihilism). They remain inside the category. I reject the category in its entirety. I am outside the category; they are not.
Agree overall. The error most people make is not having a clear understanding of what morality is, or how it differs from ethics. Most debates dissolve once a few simple facts are clear. Morality is binary; binary systems are man‑made, because neither the universe nor human behaviour is binary. Morality even has a birthday — roughly 1100 CE onward — which means it cannot be normative.
Morality also claims to know, categorically, what will be good or bad in the future. That is god‑like foreknowledge — a metaphysical claim — and secular reality rejects it outright. Not‑X = Bad is treated as timeless truth, projected forward as if morality can see the future. That alone disqualifies it from any secular definition of reality.
Ethics, by contrast, predates morality by a thousand years and doesn’t tell you what to do. It is a method for evaluating actions. Morality doesn’t evaluate; it does the opposite. It tells you what to do and then judges you for it.
My own structure is ethical: if I want X, then I am required to do Y to make X possible, without disproportionate harm. Morality collapses this into a binary: X = Good, Not‑X = Bad — and then pretends it can project that judgement into the future. That’s the category error at the heart of most moral arguments
Your optimism assumes morality is normative. But morality is binary and reality is analogue. That mismatch makes normativity impossible. Morality is a medieval invention, not a discovery. Evolution gives us behaviour, not moral truth. If you think impartial altruism is metaphysically real, explain when the universe became binary. Until then, the premise isn’t coherent.
I generally agree with the optimism, but perhaps for a different reason. I think once I really started to understand moral realism better, it gave me such hope for the future of humanity, as well as the prospect of scientific fields to start working with philosophy, like with moral philosophy.
Just as mathematics looks on track to be done mostly by AIs rather than humans in the near-ish future, if/when the day comes that most moral philosophy is being done mostly by AIs rather than humans, would you still be as optimistic?
One important caution here - while it's quite possible that most proofs get carried out by AI some time soon (the way most lines of code are written by AI now), there hasn't yet been any sign of AI doing a good job of coming up with new conjectures to try to prove, or developing new concepts with which to construct these conjectures. In software engineering, AI has learned to take some sort of initiative on sub-problems, but I think even there it's still not proposing ideas for new software that is particularly worth building. So it would be a bit misleading to say that most mathematics or most software engineering is done by AIs any time soon, even if the steps that used to take most of the time are done by AIs.
Moral Ai is dangerous in the extreme. Study the Milgram experiments, or look at the trial of Alan Turing, then ask, do I want moral Ai judging me as a binary +1 or-1 or as I am, analogue, messy, nuanced?
I don't think I agree. I think you're putting too much focus on motivated reasoning (which, granted, is likely a real bias that leads people to distort their perceptions), instead of what is more likely the 'cynical view' that many people hold, i.e., morality is a socially constructed, evolutionarily useful human fiction for solving coordination problems, and our brains and all of us are kludgy, clunky rube goldberg machines that indeed, can reason well if they make a substantial effort, but are pulled asunder by our conflicting and inevitably inconsistent, inbuilt, preferences.
I don't see why abstract reasoning can lead one "to conclude that we ought to be much more impartial and altruistic than we actually are". Mine just leads me to conclude that:
-there are no oughts. They belong to the same class as Santa Claus, Pink Unicorns and assorted beliefs.
-One should follow one's preferences.
-One should try to maximize the satisfaction of one's preferences, but caveat this with the fact that most of our evolved ones will be inconsistent, and you can't build a model that just locks into one of them as the most important.
-Generally, engaging as an agent in contractual obligations is the best way to generate positive sum games and maximize your own well-being, particularly in the long run. And you're born into communities whose rules you implicitly accept by living there and being educated/indoctrinated into them.
Being more utilitarian/ impartially altruistic doesn't follow from that.
Since 'ought' and 'should' are synonyms, your first two conclusions seem straightforwardly inconsistent.
But yeah, I'm not trying to convince the nihilist/error theorist here. That's a task for another post. fwiw, I explain some of my dissatisfaction with "hypothetical imperatives" here: https://www.goodthoughts.blog/p/hypothetical-imperatives-and-normativity
Fair. Modals have a habit of clinging to one, even when tries to do without them. 2 and 3 can be rewritten by changing the 'one should' for 'Nothing impedes one to...'
This is the same structural divide as ethics versus morality. Ethics evaluates; morality commands. And this is why “ought” and “should” are not synonyms. A moral ought derives from the religious “Thou shalt.” It carries a command structure — an authority issuing an instruction. That immediately raises the question: who or what makes the command, and why obey it? A should has no such implication. It is conditional and optional: if I want X, then I should do Y. No authority. No command. No judgement.
This is also why I am neither an error theorist nor a nihilist. Both positions are oxymorons. Both require morality to exist — either to have errors (error theory) or to be its opposite (nihilism). They remain inside the category. I reject the category in its entirety. I am outside the category; they are not.
Agree overall. The error most people make is not having a clear understanding of what morality is, or how it differs from ethics. Most debates dissolve once a few simple facts are clear. Morality is binary; binary systems are man‑made, because neither the universe nor human behaviour is binary. Morality even has a birthday — roughly 1100 CE onward — which means it cannot be normative.
Morality also claims to know, categorically, what will be good or bad in the future. That is god‑like foreknowledge — a metaphysical claim — and secular reality rejects it outright. Not‑X = Bad is treated as timeless truth, projected forward as if morality can see the future. That alone disqualifies it from any secular definition of reality.
Ethics, by contrast, predates morality by a thousand years and doesn’t tell you what to do. It is a method for evaluating actions. Morality doesn’t evaluate; it does the opposite. It tells you what to do and then judges you for it.
My own structure is ethical: if I want X, then I am required to do Y to make X possible, without disproportionate harm. Morality collapses this into a binary: X = Good, Not‑X = Bad — and then pretends it can project that judgement into the future. That’s the category error at the heart of most moral arguments
Your optimism assumes morality is normative. But morality is binary and reality is analogue. That mismatch makes normativity impossible. Morality is a medieval invention, not a discovery. Evolution gives us behaviour, not moral truth. If you think impartial altruism is metaphysically real, explain when the universe became binary. Until then, the premise isn’t coherent.
I generally agree with the optimism, but perhaps for a different reason. I think once I really started to understand moral realism better, it gave me such hope for the future of humanity, as well as the prospect of scientific fields to start working with philosophy, like with moral philosophy.
Just as mathematics looks on track to be done mostly by AIs rather than humans in the near-ish future, if/when the day comes that most moral philosophy is being done mostly by AIs rather than humans, would you still be as optimistic?
One important caution here - while it's quite possible that most proofs get carried out by AI some time soon (the way most lines of code are written by AI now), there hasn't yet been any sign of AI doing a good job of coming up with new conjectures to try to prove, or developing new concepts with which to construct these conjectures. In software engineering, AI has learned to take some sort of initiative on sub-problems, but I think even there it's still not proposing ideas for new software that is particularly worth building. So it would be a bit misleading to say that most mathematics or most software engineering is done by AIs any time soon, even if the steps that used to take most of the time are done by AIs.
Moral Ai is dangerous in the extreme. Study the Milgram experiments, or look at the trial of Alan Turing, then ask, do I want moral Ai judging me as a binary +1 or-1 or as I am, analogue, messy, nuanced?