I think it's wrong and bad for leftists to oppose near term EA stuff like public health. But I don't think it's hypocritical for them to oppose longtermist EA stuff, despite the fact that political activity and longtermism both have a similar low-probably/high-reward dynamic.
Political activism involves a kind of intermediary step between the low-probability thing and the expected very good outcome. Activists have a small chance of persuading legislators with their protests or campaigns. But once a certain intermediate goal (having people in power who want to enact your policy) is achieved, the probability of getting the desired outcome is pretty high. But with longtermism, you donate to a bunch of nerds to study AI safety ➡️ ??? ➡️ avoid end of the world?
I get what you're saying, but your claim is about what motivates people to engage in political work, and it's the expectation that if we got certain policies, it would be really terrific.
Sure, but I take the interesting question to be whether there's good reason to be more confident of that expectation than comparably optimistic expectations about x-risk mitigation.
I agree there's a psychological difference: it may be easier for political partisans to imagine that they've hit upon *the* policy ("overthrow capitalism"?) that will fix everything. With x-risk, by contrast, it's clearer that we don't yet have *the* solution, and each investment in further research is just a lottery ticket that (hopefully) slightly improves our odds. Many people find such uncertainty demotivating. But I think it is better reasoning, and more relevant to determining which actions are actually best justified.
I do wonder if there's an extent to which leftist criticism here is coming from a more process-oriented (virtue, deontological) politics than a more outcome oriented one. If you think good politics looks like "building solidarity" in the abstract (unions, riots, rallies in particular) then both giving to charity (howsoever effective) and longtermist policy design don't look like *that*. (Not to defend this, but I do suspect a thought like this is motivating.)
There's probably a good amount of Goomba twitter meme happening in these objections (favoring reach when looking at rigor and favoring rigor when looking at reach, not noticing the contradiction), but I think some might think you should round down very low probabilities of impact to zero, while still fundamentally favoring high-EV reaches above say 1 in 100,000 chances of impact. Empirically, this is really hard to maintain, but it might speak to a not-contradictory theoretical intuition.
While possible in theory, I think most advocates of systemic change support political activities, like voting in US presidential elections, which have a lower than 1 in a million chance of impact.
The costs matter a lot here, though. I think people object more to dedicating your life and large portions of your money to speculative things, vs quickly and cheaply deciding which way to vote on a long shot. Their claim would be that full time work on AI safety has a 1-in-a-million chance of paying off, but full time work in politics is the equivalent of many, many votes, bringing the odds down to something more like 1-in-10,000.
Maybe it's impossible to avoid that this really dressing up object level disagreements about the numbers and interventions rather than about weighing risks and EV though.
That turn around is the least surprising thing ever. It's like getting criticism from christians that you aren't taking seriously the potential implications of a possible afterlife and then being surprised when they are upset because you decided that some non-christian based intervention was appropriate.
Anyone who looks at politics for 5 minutes should realize that it is an area -- like religion -- where strongly held beliefs are influenced a lot more by things like community and the kind of values you want to signal than an accurate epistemic account of likely impact. Why is there virtually no one in US politics who (believably) really likes the idea of affirmative action but sadly concludes it likely does more harm than good?
And I think we need to keep in mind that EA's are no more immune to this than anyone else. I mean who could have guessed that a bunch of computer programmers and people who see intelligence as really important would tend to see the *technical* problem of unaligned AI to be the critical issue if our day rather than the political or economic issues?
Fundamentally, I think that :
1) People just aren't capable of thinking about vague social change -- an issue with weak constraints and that is exceptionally emotionally salient -- without injecting a ton of bias related to social conflict and group dynamics.
2) They also aren't capable of thinking of really long term effects without importing current conflicts and saliences.
---
If we really want to do longtermist things the first thing we need to do is extensive research into technology to limit the impact of such biasesz
where things usually unfold in ways that are orthogonal to the concerns of the present without injecting a ton of bias related to salient conflict in the present.
Certainly, but I thought I implicitly dealt with that at the end. Even supposing that an ideal being would realize they could have the most impact via these more vague long-term social interventions -- something which seems plausible -- when deciding what to actually focus on and certainly what to advocate for we need to be cognisant of our vulnerability to bias in these matters. Not to mention the fact that it is likely to be so especially tribal and triggering risks harm by bringing the community into disrepute (because of longtermist stuff I think less people probably donated to givewell). Now maybe you just do it under a different brand but:
I think that if we see strong evidence showing that it is very difficult for humans to evaluate these kind of impacts without really strong bias we should adjust our estimate of impact from our pursuit of those projects accordingly.
As such, I would argue (as I did) that a precondition of trying such interventions should be to start by trying to develop tools to manage that strong temptation to bias.
Liked this post, and the argument seems reasonable to me. I am of the rigorous frame of mind (mathematics is my most beloved subject for a reason!), but to each his own, and let's be consistent.
I think there are two related criticisms that are not in tension and that seem quite convincing to me:
1. Old school EA was wrong to neglect systemic change. Substantially more resources were directed toward RCT-supported economic interventions (e.g., unconditional cash transfers) compared to, for example, higher expected-value interventions to improve national development in poor countries. In particular, for a while, efforts to prevent engineered pandemics or manage risks from AI were very neglected compared to global development within EA, contradicting many of EA’s apparent values like “being willing to do cross-cause prioritization in an impartial, scope-sensitive way.”
2. Once EA escalated its efforts to reduce existential risk, it did a poor job in many ways. In particular, it plausibly significantly—and foreseeably—accelerated race dynamics in AI, prioritized prosaic approaches to align and control weaker AIs over ambitious approaches that seemed more like they might scale to takeover-capable ones, and encouraged an epistemic environment that took the most concerned people about AI less seriously. That said, I think a lot of these mistakes were plausibly just hard to anticipate ex ante and I’m not sure how much to fault effective altruists for making them.
(Of course, there’s some mild tension in that pursuing systemic change is harder to get right. My guess is that EAs could’ve still navigated this situation much better.)
Right, there's always room to question/critique various judgment calls on their merits. But I often see critics making more sweeping claims in a way that doesn't seem to depend much on the empirical details, so I'm interested in what principles could underlie such sweeping claims.
This is a good thought.
I think it's wrong and bad for leftists to oppose near term EA stuff like public health. But I don't think it's hypocritical for them to oppose longtermist EA stuff, despite the fact that political activity and longtermism both have a similar low-probably/high-reward dynamic.
Political activism involves a kind of intermediary step between the low-probability thing and the expected very good outcome. Activists have a small chance of persuading legislators with their protests or campaigns. But once a certain intermediate goal (having people in power who want to enact your policy) is achieved, the probability of getting the desired outcome is pretty high. But with longtermism, you donate to a bunch of nerds to study AI safety ➡️ ??? ➡️ avoid end of the world?
I think you may be underestimating the risk that enacting a desired policy won't actually result in desired outcomes.
I get what you're saying, but your claim is about what motivates people to engage in political work, and it's the expectation that if we got certain policies, it would be really terrific.
Sure, but I take the interesting question to be whether there's good reason to be more confident of that expectation than comparably optimistic expectations about x-risk mitigation.
I agree there's a psychological difference: it may be easier for political partisans to imagine that they've hit upon *the* policy ("overthrow capitalism"?) that will fix everything. With x-risk, by contrast, it's clearer that we don't yet have *the* solution, and each investment in further research is just a lottery ticket that (hopefully) slightly improves our odds. Many people find such uncertainty demotivating. But I think it is better reasoning, and more relevant to determining which actions are actually best justified.
I do wonder if there's an extent to which leftist criticism here is coming from a more process-oriented (virtue, deontological) politics than a more outcome oriented one. If you think good politics looks like "building solidarity" in the abstract (unions, riots, rallies in particular) then both giving to charity (howsoever effective) and longtermist policy design don't look like *that*. (Not to defend this, but I do suspect a thought like this is motivating.)
Is there some reason to think that expressive collective activities are more "virtuous" than actually helping people? Seems backwards to me. Compare: https://www.goodthoughts.blog/p/moral-self-indulgence
There's probably a good amount of Goomba twitter meme happening in these objections (favoring reach when looking at rigor and favoring rigor when looking at reach, not noticing the contradiction), but I think some might think you should round down very low probabilities of impact to zero, while still fundamentally favoring high-EV reaches above say 1 in 100,000 chances of impact. Empirically, this is really hard to maintain, but it might speak to a not-contradictory theoretical intuition.
While possible in theory, I think most advocates of systemic change support political activities, like voting in US presidential elections, which have a lower than 1 in a million chance of impact.
The costs matter a lot here, though. I think people object more to dedicating your life and large portions of your money to speculative things, vs quickly and cheaply deciding which way to vote on a long shot. Their claim would be that full time work on AI safety has a 1-in-a-million chance of paying off, but full time work in politics is the equivalent of many, many votes, bringing the odds down to something more like 1-in-10,000.
Maybe it's impossible to avoid that this really dressing up object level disagreements about the numbers and interventions rather than about weighing risks and EV though.
That turn around is the least surprising thing ever. It's like getting criticism from christians that you aren't taking seriously the potential implications of a possible afterlife and then being surprised when they are upset because you decided that some non-christian based intervention was appropriate.
Anyone who looks at politics for 5 minutes should realize that it is an area -- like religion -- where strongly held beliefs are influenced a lot more by things like community and the kind of values you want to signal than an accurate epistemic account of likely impact. Why is there virtually no one in US politics who (believably) really likes the idea of affirmative action but sadly concludes it likely does more harm than good?
And I think we need to keep in mind that EA's are no more immune to this than anyone else. I mean who could have guessed that a bunch of computer programmers and people who see intelligence as really important would tend to see the *technical* problem of unaligned AI to be the critical issue if our day rather than the political or economic issues?
Fundamentally, I think that :
1) People just aren't capable of thinking about vague social change -- an issue with weak constraints and that is exceptionally emotionally salient -- without injecting a ton of bias related to social conflict and group dynamics.
2) They also aren't capable of thinking of really long term effects without importing current conflicts and saliences.
---
If we really want to do longtermist things the first thing we need to do is extensive research into technology to limit the impact of such biasesz
where things usually unfold in ways that are orthogonal to the concerns of the present without injecting a ton of bias related to salient conflict in the present.
2) Because any more
Regardless of whether their complaints are "surprising", I think it's worth trying to assess whether they are *warranted*!
Certainly, but I thought I implicitly dealt with that at the end. Even supposing that an ideal being would realize they could have the most impact via these more vague long-term social interventions -- something which seems plausible -- when deciding what to actually focus on and certainly what to advocate for we need to be cognisant of our vulnerability to bias in these matters. Not to mention the fact that it is likely to be so especially tribal and triggering risks harm by bringing the community into disrepute (because of longtermist stuff I think less people probably donated to givewell). Now maybe you just do it under a different brand but:
I think that if we see strong evidence showing that it is very difficult for humans to evaluate these kind of impacts without really strong bias we should adjust our estimate of impact from our pursuit of those projects accordingly.
As such, I would argue (as I did) that a precondition of trying such interventions should be to start by trying to develop tools to manage that strong temptation to bias.
Liked this post, and the argument seems reasonable to me. I am of the rigorous frame of mind (mathematics is my most beloved subject for a reason!), but to each his own, and let's be consistent.
I think there are two related criticisms that are not in tension and that seem quite convincing to me:
1. Old school EA was wrong to neglect systemic change. Substantially more resources were directed toward RCT-supported economic interventions (e.g., unconditional cash transfers) compared to, for example, higher expected-value interventions to improve national development in poor countries. In particular, for a while, efforts to prevent engineered pandemics or manage risks from AI were very neglected compared to global development within EA, contradicting many of EA’s apparent values like “being willing to do cross-cause prioritization in an impartial, scope-sensitive way.”
2. Once EA escalated its efforts to reduce existential risk, it did a poor job in many ways. In particular, it plausibly significantly—and foreseeably—accelerated race dynamics in AI, prioritized prosaic approaches to align and control weaker AIs over ambitious approaches that seemed more like they might scale to takeover-capable ones, and encouraged an epistemic environment that took the most concerned people about AI less seriously. That said, I think a lot of these mistakes were plausibly just hard to anticipate ex ante and I’m not sure how much to fault effective altruists for making them.
(Of course, there’s some mild tension in that pursuing systemic change is harder to get right. My guess is that EAs could’ve still navigated this situation much better.)
Right, there's always room to question/critique various judgment calls on their merits. But I often see critics making more sweeping claims in a way that doesn't seem to depend much on the empirical details, so I'm interested in what principles could underlie such sweeping claims.
Do you think the criticisms are ever coming from the same person/ sources? Even within EA there seems to be noticeable disagreement about risk tolerance/ how far ahead in the future to think ahead https://nosian.substack.com/p/effective-altruism-linked-orgs-split