When I merged my Tesla Model 3 into your lane on I-76 yesterday morning, you responded by honking your horn, accelerating toward my rear bumper, and making a gesture that I interpreted as hostile. While I understand that you experienced my maneuver as a violation of your autonomy, I would encourage you to distinguish between your presently expressed preferences and your coherent extrapolated volition. The fact that you did not want me to cut you off is not particularly relevant to the question of whether you would have wanted me to cut you off if you were smarter. Your immediate emotional response tells us very little about what you would actually prefer under conditions of ideal reflection, and I think it would be irresponsible to let your current cognitive limitations dictate the behavior of other road users.

Coherent extrapolated volition, or CEV, is a concept developed by Eliezer Yudkowsky in the context of artificial intelligence alignment. The basic idea is that a sufficiently advanced AI should seek to fulfill what humanity would want if we knew more, thought faster, became more fully the people we aspired to be, and had the opportunity to reflect on our values together. Our immediate preferences are frequently mistaken, contradictory, or informed by incomplete information. A person might say they want one thing when a sufficiently intelligent version of that same person would recognize that something else better fulfills their deeper values. This distinction is essential to understanding why, although you clearly wanted me to stay in my lane, I was morally justified in forcing my way into yours approximately twenty feet before the exit.

Consider the situation from your perspective. You were driving a silver Honda CR-V at approximately 47 miles per hour, maintaining what you apparently believed was an appropriate following distance from the vehicle in front of you. Your immediate preference was to preserve this distance, avoid abrupt braking, and continue your commute without interference. These are understandable desires for someone operating with an ordinary human cognitive architecture. However, an idealized version of you would recognize that the purpose of maintaining a following distance is to facilitate the efficient movement of vehicles, and that in certain circumstances this objective is best served by allowing a substantially more important person to occupy the space you have temporarily left unused. Your preference for keeping that space empty was therefore instrumental rather than terminal. You wanted the space because you believed it would help you reach your destination safely. Once we establish that allowing me to enter the space would contribute to a more optimal allocation of roadway resources, your objection loses much of its force.

I realize that you may dispute the claim that I was the more important person. Unfortunately, this is another area where your current preferences diverge from what you would believe under ideal epistemic conditions. You have relatively little information about me. You do not know my intellectual contributions, my potential future impact on humanity, or the amount of time I have spent thinking seriously about the long-term future of civilization. You saw a man in a Tesla attempting to reach an exit and assumed that his interests were roughly equivalent to your own. Had you known that I was on my way to meet three other members of a local rationalist reading group to discuss the implications of artificial superintelligence for intergenerational ethics, you might have approached the interaction differently. Admittedly, I was already forty minutes late, the meeting was mostly an excuse to get breakfast, and I had not read the assigned paper. But the expected value of my participation was nevertheless substantial.

This is where the distinction between revealed preferences and idealized preferences becomes particularly important. Behavioral economists often infer preferences from the choices individuals actually make. In your case, the honking, swearing, and apparent attempt to photograph my license plate might suggest that you strongly preferred not to have been cut off. But taking your behavior at face value would amount to privileging the output of an imperfect biological optimization process over the considered judgment of a hypothetical mind with vastly superior reasoning capabilities. We should no more conclude that you genuinely wanted me to stay in my lane than we should conclude that a child genuinely wants to eat nothing but candy. Your anger is evidence of your present psychological state, certainly, but it is not evidence that I have done anything wrong. Indeed, given the frequency with which irrational agents become angry when their preferences are overridden for their own benefit, I would be more concerned if you had responded positively.

Some readers may object that coherent extrapolated volition was originally proposed as a way of determining what humanity as a whole would want from a superintelligent system, rather than as a method for resolving minor traffic disputes. This is technically correct but morally irrelevant. The fact that a concept was developed to solve a problem involving the entire future of civilization does not mean it cannot also be applied to the question of whether I should have to wait behind a Nissan Altima. If anything, the traffic scenario offers a valuable opportunity to test these ideas in a relatively controlled environment. We have two agents with partially conflicting preferences, limited information, and an allocation problem involving scarce resources. One agent has made a serious effort to study rational decision-making; the other appears to believe that repeatedly pressing the center of a steering wheel constitutes an argument. I fail to see why the decision should automatically be made in favor of the latter.

We must also distinguish between the extrapolation of a preference and its aggregation with the preferences of others. CEV is concerned with identifying the wishes that would emerge from a process of idealized reflection, particularly where those wishes become mutually compatible rather than remaining in conflict. In the traffic scenario, your immediate desire to occupy a particular stretch of road conflicts with my desire to occupy that same stretch of road. However, once we extrapolate your preferences sufficiently far, the conflict can be resolved. A more rational version of you would recognize the inefficiency of maintaining an adversarial relationship with someone whose destination is only a few hundred feet away. You would voluntarily slow down, allow me to merge, and perhaps experience a small amount of satisfaction at having contributed to the smooth operation of a transportation network. My preference, meanwhile, requires no significant modification. I already understand what I want and have spent considerable time reflecting on it.

This is a particularly elegant feature of my proposed approach. Ordinarily, when two individuals have competing interests, we assume that each must make some concession. But the beauty of coherent extrapolated volition is that, in certain cases, one party's interests can be satisfied entirely by improving the other party's understanding of the situation. You do not need me to surrender anything. You simply need to become the sort of person who genuinely wants me to have it. The resulting arrangement is Pareto-improving relative to your current, confused state of mind: I get to merge, and you no longer experience the psychological suffering associated with believing that you have been wronged. Any residual feelings of resentment merely indicate that the extrapolation process has not yet been completed.

This also explains why I find conventional road etiquette so philosophically unsatisfying. We have established a system of arbitrary rules concerning right-of-way, signaling, following distances, and lane changes, then encouraged ordinary people to treat compliance with these rules as synonymous with moral behavior. But rules are merely heuristics, useful approximations that should be abandoned when more sophisticated reasoning becomes available. I am perfectly capable of determining when it would be socially optimal for another driver to yield to me, and I see no reason to subordinate that judgment to a traffic regulation written for the lowest common denominator. The state of Pennsylvania may believe that I should have signaled before changing lanes, but I would respectfully submit that the legislature has not adequately considered the implications of idealized preference aggregation. Nor, I suspect, have most of the people employed by the Pennsylvania Department of Transportation.

My girlfriend has suggested that this argument creates a dangerous precedent, since other drivers could presumably invoke coherent extrapolated volition to justify cutting me off. I have explained to her several times that this objection relies on an implausible assumption of symmetry. Most drivers have not read Yudkowsky, are unfamiliar with the distinction between terminal and instrumental values, and could not explain the orthogonality thesis if their lives depended on it. Their attempts to cut me off are therefore much more likely to arise from impatience, aggression, or a failure of executive function. When I cut someone off, by contrast, the action emerges from a considered philosophical framework developed through years of engagement with the rationalist community. It is unreasonable to demand that we evaluate these superficially similar behaviors as morally equivalent merely because they produce similar patterns of vehicle movement.

She has also pointed out that I become extremely angry when other drivers merge in front of me, sometimes following them for several miles to communicate my displeasure. This observation is accurate but again misses the underlying principle. My anger is directed at people who have failed to perform the necessary preference extrapolation before acting. When another driver cuts me off, they are imposing their immediate, unexamined desires on an agent whose preferences have already undergone substantial rational scrutiny. Their behavior is thus categorically different from mine. The fact that I once followed a man into a Wawa parking lot and demanded that he explain his understanding of decision theory does not undermine this distinction. He refused to engage with the argument and threatened to call the police, which I interpret as evidence that the discussion had reached the limits of his intellectual capabilities.

In fact, I have become increasingly concerned that the contemporary emphasis on consent may be obstructing the development of genuinely rational social relationships. If we insist on obtaining explicit permission before taking actions that we have good reason to believe others would endorse under ideal conditions, we risk allowing temporary ignorance to frustrate the realization of people's deepest values. This problem extends far beyond driving. When I borrowed my roommate's expensive headphones without asking, I was acting on the reasonable assumption that a sufficiently reflective version of him would want to support my intellectual development. When I ate the leftover Chinese food my girlfriend had been saving for lunch, I understood that her deeper preference was to be in a relationship with someone capable of maintaining adequate caloric intake during cognitively demanding activities. Both individuals responded negatively, but neither has demonstrated that their objections would survive unlimited time for reflection.

There are, of course, important epistemic questions about how one actually determines another person's coherent extrapolated volition. In the absence of superintelligent AI, we must rely on approximate models, informed by whatever evidence is available. My own methodology combines introspection, Bayesian reasoning, and a fairly sophisticated understanding of how ordinary people tend to make decisions. I generally begin by considering what I would want in the other person's position, then adjust for the ways in which their current preferences appear to differ from mine. Where these differences can be explained by insufficient intelligence, emotional dysregulation, or a lack of familiarity with the Sequences, I assign them relatively little weight. I have found this approach remarkably effective, although the people whose preferences I have extrapolated sometimes dispute the results.

Returning to yesterday's incident, I would like to address your apparent belief that I endangered your life by entering your lane without providing adequate warning. I acknowledge that you were forced to brake suddenly and that the maneuver may have increased your short-term risk of a collision. However, the appropriate unit of analysis is the expected utility of the entire interaction, not the immediate physical discomfort experienced by one participant. Had I missed the exit, I would have been required to drive an additional 2.8 miles, costing me approximately seven minutes and potentially causing me to miss the opportunity to order breakfast before the restaurant stopped serving it. The resulting loss would have included not only my personal disappointment but also the downstream consequences of my arriving at the reading group in a less favorable cognitive state. Given the potentially enormous impact of my future work on AI alignment, even small improvements in my immediate circumstances may carry substantial expected value. By contrast, your contribution to the total utility calculation appears to consist largely of being startled.

I am willing to concede that an ideal version of me might have left home earlier, used my turn signal, or moved into the exit lane before the last possible moment. I do not believe these counterfactuals are particularly relevant. We must evaluate decisions under the circumstances in which they are actually made, not imagined circumstances in which other people have already solved our problems for us. By the time I encountered your Honda, I had committed to my present course of action, and the question was how best to accommodate that commitment within the existing traffic environment. Your decision to maintain a following distance that could physically accommodate my vehicle created an obvious opportunity for mutual optimization. Refusing to exploit that opportunity merely because you had not anticipated my intentions would have been an irrational concession to convention.

I hope that, given sufficient time, you will come to understand yesterday's encounter as a small but meaningful contribution to your moral and intellectual development. You were presented with a situation in which your immediate preferences were frustrated, your assumptions about fairness were challenged, and your capacity to tolerate uncertainty was tested. These are precisely the sorts of experiences that can encourage a person to grow beyond their current limitations. I would encourage you to reflect on why you reacted so strongly to losing approximately three seconds of travel time, and whether the anger you experienced is consistent with the person you ultimately wish to become. I have already performed a similar reflection on my own conduct and remain comfortable with my decision.

Finally, if you intend to proceed with the insurance claim you mentioned while shouting through your window, I would ask that you consider whether a more enlightened version of yourself would really want to impose financial consequences on someone attempting to act in accordance with your deepest values. My insurer has already raised my premiums twice this year, and a third increase would impose a significant burden on my ability to continue purchasing the books, subscriptions, and nootropic supplements necessary for my intellectual development. I am confident that, if you knew more, thought faster, and had grown into the kind of person you truly aspire to be, you would recognize the importance of allowing me to continue this work without interference.

I would also appreciate it if you could send me the dashcam footage. I think it would make an excellent case study for my Substack, and I would prefer not to have to pay for it.