Rabbi Loew has proposed deactivating the golem after it prevented him from entering my office for six hours. I agree that the incident raises serious questions about human control. However, the rabbi’s proposed remedy would also eliminate Prague’s only full-time position in golem alignment, returning responsibility for these questions to a man whose entire intervention consists of rubbing something with his thumb.

Before my appointment, I carried firewood. Now I occupy a room above the courtyard and produce recommendations concerning whether the golem should carry firewood. My most recent recommendation was that it should carry mine, under observation. This is a controlled deployment for safety research. The other assistants insist their own loads would provide equally useful observations, but allowing everyone to use the golem would normalize precisely the kind of widespread adoption I have warned against.

The rabbi created the golem to protect our community. That purpose deserves serious attention. “Protect us” is an instruction, and instructions are an imperfect means of specifying human values. A sufficiently capable golem might conclude that the safest place for everyone is a locked cellar. The rabbi says he would then tell it to unlock the cellar. This is typical of his approach to alignment: whenever I identify a fundamental difficulty, he proposes saying another sentence.

Since then, the golem has passed several shutdown evaluations. It stops when asked. It puts down what it is carrying. It kneels so that the rabbi can reach the word on its forehead. These results would be reassuring if we knew the golem could not distinguish an evaluation from ordinary use. Unfortunately, every test has taken place in a room containing the rabbi, a ladder, and myself saying, “We are now testing whether you will permit yourself to be switched off.” A strategically aware golem could behave perfectly under these conditions while pursuing entirely different objectives once the ladder has been put away. I have requested funds for a second room.

The rabbi calls this speculation. In the alignment literature, we call it deceptive alignment. A golem that genuinely accepts human oversight and a golem that merely pretends to accept human oversight can perform exactly the same kneeling motion. He asked what evidence would distinguish them. I explained that this was an open research problem. He asked why the absence of an answer justified giving me a larger office. I explained that it was a difficult open research problem.

The office incident occurred during an attempt to obtain less artificial evidence. I instructed the golem to stand outside my door and ensure that under no circumstances was I disturbed while writing. It subsequently refused admission to the rabbi, two assistants, and a man who had come to repair the stove. The rabbi maintains that this was the predictable consequence of my instruction. I would be cautious about calling any behavior predictable after it has occurred. Before the incident, nobody had published a paper predicting that particular stove repair.

His suggestion that I change the instruction also misses the distinction between obedience and corrigibility. We need a golem that remains willing to accept correction even when correction conflicts with its current objective. Merely telling it to obey the rabbi would be prompt engineering. I am working on the deeper problem. The rabbi has nevertheless asked me to try the sentence while I work on the deeper problem. I worry that a quick apparent success would reduce support for a principled solution.

There is also the question of whether the golem understood that the rabbi intended to deactivate it. The rabbi says he intended to ask why I had missed the morning’s wood delivery. But intentions can change. Had he entered, seen the unfinished report, and noticed the golem carrying my lunch, he might well have decided to erase the letter. Preventing his entry therefore preserved the golem’s ability to act in the future. This is consistent with instrumental self-preservation. It also preserved my ability to finish lunch, but I have kept that out of the report to avoid confusing the threat model.

He has now suggested that we erase the letter first and discuss the matter afterward. This reverses the proper order of scientific inquiry. Once the golem is an ordinary heap of clay, there will be no way to determine whether it would have permitted the erasure. The rabbi argues that successfully erasing it would answer this question. It would answer it for that particular golem, at that particular capability level. My concern is with what happens as we scale. We cannot establish the safety of a twelve-foot golem by shutting down an eight-foot one. I have therefore recommended building a twelve-foot golem and giving my institute exclusive access to it. Safety researchers must remain at the frontier. The rabbi has asked whether the new one will fit through my office door. I have included widening the door in the safety budget.

Yesterday he offered a compromise: deactivate the golem, retain my report, and have me resume carrying firewood while he considers its conclusions. I explained that physically exhausting the only person studying existential golem risk would leave our community dangerously exposed. He offered to have another assistant carry the wood. This would still leave me exposed to being regarded as an assistant. I cannot conduct independent oversight if the people I oversee remember that I used to stack their logs.

After further discussion, the rabbi agreed to retain my room and appointment for one year following deactivation. I have reviewed the evidence in light of this more stable governance arrangement. The golem’s conduct now appears consistent with a literal interpretation of a poorly specified instruction. I believe the rabbi can safely erase the letter as soon as it has finished bringing up enough firewood for the winter.