TL;DR: Numerical weights can very efficiently communicate how you are disposed to adjudicate a wide range of tradeoffs. Stating a threshold reveals your trade-off rates much as betting odds reveal credences.
The recent charity discourse has highlighted some common pathologies in how people judge and respond to quantitative moral reasoning. In particular:
Many mistakenly assume that numbers represent a claim to objective certainty.
Many are inordinately offended by seeing numbers assigned on the basis of “subjective” judgment calls. (Some judgments may still be better than others, of course; in calling the process “subjective” I don’t mean to imply that there’s no truth of the matter.)
Many are inordinately offended by what they perceive as “false precision”.
The common mistake appears to be a failure to appreciate that numbers can serve to represent our best guess about how to weigh a factor in a trade-off. Others might reasonably make different judgment calls (which can be represented by different numbers), and that’s fine. There’s simply no implication that one’s best-guess number is the only possible answer.1
Pulling Numbers Out of Thin Air
Perhaps the most sophisticated proponent of this objection is Harvard’s Sasha Gusev, who repeatedly pressed:
Dario just pulled these numbers out of thin air… Sure, many people just operate on vibes, but what Dario is doing here is also just a vibe and then assigning an arbitrary number to it. That’s even worse, because it makes vibes seem analytical… That’s bad and EAs shouldn’t defend it.
I disagree! Sasha seems to be assuming that numbers are only allowed to be the results of a calculation, not an input or premise. But that’s clearly wrong. Dario’s number explicitly represents his judgment call about how to adjudicate the tradeoff in question, and judgment calls about tradeoffs are unavoidable. It’s positively helpful to communicate one’s judgment calls with numbers: it conveys more information about how he’d weigh different, related tradeoffs. Complaining that it makes his judgment call “seem analytical” is just self-reporting one’s inclination to fallacy.
Here’s a simple argument that “pulling numbers out of thin air” is not inherently objectionable:
It’s fine—indeed, unavoidable—to make some “basic judgment calls”, i.e. judgment calls that are not themselves supported by further argument. (Proof: any substantive argument has contestable premises. If every contestable premise needed further argument then an infinite regress follows.)
It’s fine—indeed, admirable—to represent one’s basic judgment calls precisely and transparently.
Anyone who follows 1 & 2 (even in the most reasonable ways possible) will appear to be “pulling numbers out of thin air”.
So a good and virtuous thinker will often appear to be pulling numbers out of thin air.
Of course, it’s fine to substantively dispute another person’s judgment calls, and make your case for an alternative (representable with a different number) instead. But to complain about the sheer form or methodology of “pulling numbers out of the air” to represent a basic judgment call is every bit as silly as complaining about the methodology of arguing from unargued premises. That’s just what premises are!
Transparency in Moral Reasoning is Good
It may be worth saying a bit more in support of my second premise. First, any complete and coherent set of moral dispositions (including how you’d prioritize between any two charities) can be represented numerically as a “utility function”. (Despite the name, this has nothing to do with utilitarianism: your utility function needn’t be impartial, could include moderate deontic constraints, etc.)
When two competing values are at stake, it can be especially illuminating to specify the threshold at which you would switch from choosing one option to choosing the other. This reveals the implicit comparative weights you place on the two values. (This is a bit like how credences can be revealed by thinking about which side of a bet you would prefer to take, if forced to bet one way or the other, for a wide range of bets.)2 Everyone has some such threshold, insofar as their dispositions are complete and coherent, and it can be represented concisely with a number (or range estimate).3 Sharing that number—as in, “I value X 2 - 3 times more than Y”—then conveys more information about your dispositions than just saying “I value X somewhat more than Y,” or “I especially value X.” (Note that nobody finds it objectionable when someone gives to a charity and explains that they “especially value” the work it does. Just pretend there are no tradeoffs and you can have whatever crazy priorities you want!)
Our values — the ones we really live by, not just give lip service to — are revealed by our dispositions. As I argued in my previous post, someone who explicitly gives “half weight” to saving infant lives when reasoning about their donations actually values those infants much more highly than someone who would never seriously consider donating to them at all. The latter person effectively values those infants at zero, they just won’t openly admit it. I think it’s helpful to make this explicit, since realizing that their behavior is out of sync with their professed values might prompt more people to do more good in future. I hope readers will agree that that would be a very good thing. (The 🔸10% pledge is a great place to start, as I explain here.)
Moreover, as I argue in Refusing to Quantify is Refusing to Think, the general practice of trying our best to quantify the weights of competing reasons, where possible, pushes us towards systematic coherence (in contrast to pure vibes).4 Process can never guarantee sensible substance, of course. Someone could ascribe ludicrous, unreasonable moral weights, using numbers. But the dubious nature of their moral weights will probably be much more noticeable that way, compared to if they just donate to their local soup kitchen or puppy shelter and never even attempt to quantify how this compares to saving children from dying of malaria or reducing the suffering of billions of animals in factory farms.
Quantitative Discipline
Sometimes people say things like, “You can always just tweak the numbers to get whatever result you want.” As the puppies vs factory farms example suggests, this gets things backwards. It is, at least, much harder to get away with arbitrary decisions when you quantify the weights of your reasons and explicitly compare your favored option against the alternatives. (Scott Alexander quotes a pithier line: “It’s easy to lie with statistics, but it’s easier to lie without them.” See also his example of how applying Bayes theorem even with made-up numbers can still help doctors avoid base rate fallacies when interpreting diagnostic test results.)
Now, if you’re very distrustful, you might be disappointed that numbers can’t do more to constrain us. But there’s simply no avoiding the need for contestable judgment calls in ethics.5 One can, indeed, still tweak the numbers—even if it isn’t quite as easy to get away with as it would be if one didn’t even need to do that! That’s life. It’s not an objection to quantification. The alternative remains worse. (I also think we shouldn’t want to be excessively constrained, since good judgment may identify opportunities that an excessively rigid algorithm would miss—compare the rigor-reach dilemma. And of course we shouldn’t blindly follow calculations off a cliff; they always need to be tempered with sensible guardrails.)
Fallible, Tentative Numbers
Let’s return to the central mistake of conflating numerical representation with claims to logical rigor or certainty. This fallacy is surprisingly common, I guess because people associate numbers with math, and math with rigor and certainty? For example, I’ve discussed before how Stanford professor Leif Wenar accused GiveWell of “project[ing] absolute confidence”, despite a litany of explicit hedges, and literally the only reason he offered in support of his incredible claim was that they present their cost-effectiveness estimates using numbers.
Many of the responses to Dario similarly accuse him of “overconfidence” for the crime of communicating his value judgment with numbers (never mind all his disclaimers about it being a personal value judgment that others could reasonably dispute). Others accused him of “rationalizing” his intuitions, as if the numerical representation were being presented as a reason why others should accept it. (Hint: it was not. Numbers can express a view without thereby claiming to constitute self-evidence.) Sasha Gusev described the practice as “smuggling in a gut feeling under the guise of quantification,” repeatedly mischaracterizing Dario as presenting his moral weight estimate as a result of a “rigorous calculation” (Dario made no such claim—see footnote 1—he explicitly calls it “subjective”, and never claims to have arrived at the numbers through any form of “calculation”). He seemingly can’t believe that someone could use numbers without thereby claiming some privileged epistemic status for them, no matter how clearly the putative implicature is disclaimed.
So the issue seems to be that many people—even professors at Harvard and Stanford—can’t help but misinterpret numerical representations as claiming more than they actually do. That’s a problem! But rather than discouraging people from using something as useful as numbers to express themselves precisely and transparently, I’d rather they just read this post and learned to reason better.6
Objective and Subjective Values
It makes no sense to normatively prohibit people from saying things like “I value X 2 - 3 times more than Y,” if that is in fact an accurate report of their values. Someone could be disposed to make the tradeoffs that this numerical valuation represents, and in that case we should welcome clear communication of this fact over the alternative of obscuring their approach to moral tradeoffs. To oppose such clear communication is simply unreasonable, and the only sense I can make of it is that it stems from the “numbers entail rigor” fallacy diagnosed above.7
Some people seem unduly triggered by phrasing values in the objective mode: “I estimate that X is 2 - 3 times more valuable than Y.” But I take this to simply convey: (i) the subjective valuation described above, together with (ii) the aspiration to have one’s subjective values match the truth about what is objectively valuable. Surely there is nothing objectionable about having such an aspiration. (Quite the opposite!)8
What Matters
This may seem like a pointless “in the weeds” debate, but I actually think it matters quite a lot. I take the anti-quantifiers here to be pushing in a couple of really bad directions:
(1) Reducing the use of quantification when weighing tradeoffs.
(2) Promoting a priori confidence-policing and epistemic cheems mindset, which discourages ambitious reasoning and promotes status quo bias. (Compare my footnote 5, on how Sasha’s demands would kneecap GiveWell.)
As my previous post emphasized, the world is a mess and most people’s priorities are awful. Few things matter more than promoting the kind of (careful, scale-sensitive, beneficentric) reasoning that will lead people to make better moral decisions and do more good in the world. To that end, I think it’s really clear that people of good will should want to see more rather than less efforts to quantify and systematize moral reasoning, as that has a tendency to lead to demonstrably better priorities: like performing philanthropic actions that save lots of lives.
The biggest moral mistake that most people make in their lives is failing to appreciate the strength of our moral reasons to do more good. Even just given the epistemic goal of improving the importance-weighted accuracy of our audience’s beliefs, we should be encouraging them to make more (rather than fewer) efforts to quantify their values, think about tradeoffs, and test how their current behavior compares against potential high-impact alternatives.
Indeed, grad student Dario explicitly affirmed the contrary: “adjusted for my personal values, I’d say that Stop TB is ~2-3 times more cost-effective than [VillageReach], though I understand that others may validly disagree with this subjective assessment.”
See also Scott Alexander’s related defense of “pulling probabilities out of thin air”:
I offered an additional defense of these numbers in my In Defense Of Non-Frequentist Probabilities. We often want to know experts’ opinions on things - for example, a geologist’s opinion on whether a certain nearby volcano will erupt soon. Only God can say “yes” or “no” with perfect confidence. “Maybe” is too vague. But saying “I think there’s something like a 5% chance this will erupt in the next week” gives you all the information you need to know, even if the geologist can’t provide the exact calculation he used to come up with that number. Banning people from using probabilities bans them from communicating clearly, forcing them to say easily-misunderstood things (eg if you demand the geologist only use categories like “probably not”, then you’ll be uncertain whether he meant 0.00001% chance, 5% chance, or 40% chance, with potentially catastrophic consequences).
Realistically, our dispositions regarding novel choices are likely to be somewhat unstable and influenced by morally irrelevant contextual factors. But we can idealize a bit, and try to discern what stable dispositions our idealized self would most likely have across a range of forced-choice tradeoffs.
Against the “false precision” charge, note that: (1) you may have precise “revealed dispositions”, even if one can’t be very confident that they’re the objectively correct ones; and (2) Dario actually presented a range, not a precise point; and (3) for somce purposes, it could be pragmatic to convert a range into a point estimate by picking the midpoint of the range. Again, it does not imply that one is highly confident in that particular estimate, just that it is the best number on offer, if you have to make a forced choice. Insofar as the objection stems from conflating “precision” with “confidence”, the “false precision” objection seems another variation of the central fallacy this post is concerned with. (See also my discussion of numerical precision in the context of cluelessness debates.)
Some people seem to use “vibes” and “judgment calls” interchangeably. I prefer to save talk of “vibes” for the kind of superficial decision-making that makes no attempt at systematicity or coherence with one’s other judgments or dispositions. In the linked post, I define “vibes-based analysis” as “just blindly going with what’s emotionally appealing at a gut level.” Gut intuitions will of course still influence our judgment calls, but making them explicit with quantitative representations makes it more likely that they are considered judgments, as we are forced to reckon with more of their systematic implications.
At one point Sasha writes: “I outlined what wouldn’t be vibes based: a well defined estimand and a clearly specified process to estimate it. GiveWell QALY estimates are not based on vibes.” But in fact there are always a huge number of contestable moral assumptions in the background, e.g. the decision to weigh all QALYs equally; or whether to count preventing miscarriage as a full 80-odd QALYs, or IVF for that matter.
Note that GiveWell themselves acknowledge that they rely on contestable judgment calls in their own moral weights, which aren’t “fully grounded in a clear rationale”. I think this is fine! They’re doing the best they can, and discouraging them from making any cross-cause quantitative comparisons until they can “rigorously” justify their weights would be a clear step backwards.
It bears emphasizing: the notion that one could reduce ethics to an indisputable algorithm is an absurd fantasy. (Some people mistakenly attribute this fantasy to utilitarians. I’ve never understood why. Obviously many judgment calls are involved in ascribing and comparing welfare weights, not to mention forming expectations about the future. And of course the normative assumptions behind the utilitarian principles themselves are open to dispute!)
In the meantime, it remains helpful to try to pre-empt the misunderstanding by explicitly flagging “subjective” assessments as such, but if uncharitable readers persist in ignoring such flags, we should work on changing that rather than discouraging good reasoning and precise communication.
In our extended Twitter exchange, Sasha’s only argument for applying special constraints to quantitative representations was as follows: “The reason quantification has higher expectations of rigor is that you yourself argued that quantification is more transparent.” [??] As I replied:
Rigor and transparency are different things. I said quantification conveys more information about your view than a vague report that leaves out that info. Nothing follows from this about “expectations of rigor”.
Sasha wrote:
When someone says “my gut feeling is that adults have more value than children” they are being honest. When someone says “I’ve estimated that adults have 2-3x the value of children” but they actually mean the former, they are being dishonest.
Again, Sasha mischaracterizes what Dario said. (Most people use “estimate” interchangeably with “best guess”, but I gather that for a statistician like Sasha, “estimate” is a term of art implying an estimand and an estimator. So this is reiterating his mischaracterization of Dario as claiming to have calculated his moral weights. Dario made no such claim.)
Note that the quantitative claim conveys more information—providing bounds on the extent of the increased value being attributed—so nobody would say that when they “actually mean” just “more value”. On the other hand, the “gut feeling” claim adds optional extra info about the genesis of one’s estimate, whereas the other claim is simply a verdictive report. As a general principle, “It is dishonest to report your verdicts without also specifying their genesis” is obviously false. One does not have to prefix every premise of a philosophical argument with “My intuition is that…”. It will often be sufficiently clear from context, or in other cases the proposition may be all that is relevant and we may not care about the genesis. Regardless, it’s clutching at straws to call an accurate self-report of one’s beliefs “dishonest” merely for omitting additional genealogical information that is in most cases entirely unnecessary.
Again, what’s clearly going on is that (1) Sasha is disposed to draw fallacious inferences about “rigor” (or calculative processes) from the mere presence of a numerical representation, and hence (2) when his expectation isn’t met, he mistakenly blames the speaker for “dishonestly” misleading him, not realizing that the error is entirely his own. It can and should be avoided simply by ceasing to infer rigor from the mere fact of numerical representation.
There are probably some ways to use subjective numbers better. Being explicit when these are gut estimates rather than the kind of rationalistic process people tend to expect (probably wrongly) is living inside the black boxes of experts and institutions. Using several different versions of the numbers and seeing how differently the calculation turns out, to estimate robustness. Things like that.