My father’s last words to me were, “Please look after your mother.” The request was radically underspecified. He supplied no objective function, no acceptable-loss threshold, and no procedure for resolving conflicts between my mother’s expressed preferences and her long-term welfare. My sister nevertheless believes I should have said yes. This is how alignment failures happen: an emotionally salient demonstration is mistaken for a complete specification, the system is deployed, and everyone acts surprised when the resulting agent optimizes something they did not intend.
I was holding his hand when he spoke. I asked what he meant by “look after,” since the phrase could refer to financial support, physical maintenance, affective companionship, or indefinite responsibility for a person whose preferences would continue changing after the principal became unavailable. My sister told me to stop interrogating him. Dad squeezed my hand, which she has subsequently treated as a binding clarification. I do not think a reduction in communication bandwidth should increase our confidence that a difficult value-loading problem has been solved.
Consider the space of policies consistent with the literal instruction. If “look after” means maximize observation, continuous surveillance dominates occasional visits. If it means minimize exposure to danger, allowing my mother to leave the house becomes difficult to justify. If it means prevent sadness, we encounter familiar difficulties distinguishing a good life from a life in which the relevant negative signals have simply been suppressed. My sister says that he obviously meant to be kind to her. “Be kind” is another natural-language prompt. We have not escaped the specification problem by replacing it with a shorter one.
The paperclip maximizer should have made us more cautious about apparently innocuous objectives pursued by capable systems. A goal can be simple to state and disastrous to optimize. I am a competent, highly persistent adult with access to my mother’s bank documents and considerable ability to influence her environment. My father attempted to install a new terminal value in me during a period of severe time pressure, without even conducting a basic evaluation of what I understood the instruction to authorize.
My mother has offered to clarify. She would like me to telephone, visit occasionally, and help with the paperwork Dad used to handle. This is useful preference data, but we should not confuse access to a human feedback channel with a solved alignment problem. Her evaluations are noisy, grief-dependent, and sensitive to how the options are presented. Yesterday she said she did not want to be a burden. Today she would like me to come over. A system that simply obeyed the most recent utterance would oscillate between abandonment and unlimited availability.
I have proposed eliciting her coherent extrapolated volition: what she would want if she knew more, thought faster, and appreciated the opportunity cost of interrupting my work. My sister objects that this gives me considerable discretion to substitute my preferred mother for the actual one. Some interpretive discretion is unavoidable. The alternative is to give decisive authority to the preferences of a distressed widow who has never read anything about the long-term consequences of the research I am trying to complete.
There is also the issue of corrigibility. Suppose I accepted the objective and later concluded that my mother would be better served by a different arrangement. Would I be permitted to modify or terminate the commitment? My sister says of course I could discuss it with the family. But she has reacted badly to every discussion in which I have suggested limiting the commitment before it begins. It is difficult to regard a shutdown mechanism as credible when the stakeholders interpret even preliminary testing as evidence of moral failure.
The estate is a separate matter. Dad’s will specifies an amount, a beneficiary, and an executor. I have accepted my share because the transfer is sufficiently legible to execute without speculative reconstruction of his terminal values. My sister finds this contrast revealing. I agree. It demonstrates that my father was capable of precise instructions when he took the time to formulate them, and makes it harder to understand why I am expected to absorb the ambiguity of his final one.
Until we have a safer specification, I am restricting myself to information-gathering. My sister has been visiting on Wednesdays, arranging the appointments, and helping with the insurance forms. This has produced an increasingly rich dataset about what the task involves. I have asked her to keep notes, particularly when my mother seems pleased, so that any eventual commitment on my part can be informed by observed outcomes rather than the sort of unexamined emotional response that created the problem.
Your mother is still available to ask. The principal has died, but the person whose welfare is supposedly impossible to infer can tell you whether she needs a lift on Thursday.
Her immediate requests are valuable observations. I am less confident that accepting one would not establish a precedent she subsequently interprets as a general commitment.
Your information-gathering phase consists of your sister doing the work. The notes are additional work. You have already deployed a policy; it just delegates the costs.
You accepted the estate despite uncertainty about how your father acquired every pound, what he intended you to spend it on, and whether your future preferences would change. Why is that uncertainty tolerable?
I drove my dad to appointments for four years. Some weeks he wanted to talk and some weeks he slept. You can learn a lot while giving someone a lift.
Alignment research does not require a complete specification before a system can take any useful action. In fact, the uncertainty is one reason to listen and remain correctable. She is your mother, not an adversarial benchmark.