According to the Swedish DVD release (which contains an introductory interview with Bergman himself), Ingmar Bergman wrote the movie with Victor Sjöström in mind. He and the production company agreed that there would be no movie without Sjöström. Bergman didn't dare to call his idol Sjöström himself about the movie though, so the head of the production company made the call. Sjöström was initially reluctant, due to his advanced age, but agreed to meet with Bergman to discuss the movie. So Bergman went to his apartment and talked about it, Sjöström said he'll think about it. The next morning Sjöström called and agreed to the part on one condition: that he would be able to come home and have his whiskey grog at 5 pm every day.
Ingmar Bergman has described in the interview how he came up with the idea while driving from Stockholm to Dalarna, stopping in Uppsala where he had been born and raised, and driving by outside his grandmother's old house, when he suddenly began to think about how it would be if he could open the door and inside it would be just as it had been during his childhood. "So it struck me - what if you could make a film about this; that you just walk up in a realistic way and open a door, and then you walk into your childhood, and then you open another door and come back to reality, and then you make a turn around a street corner and arrive in some other period of your existence, and everything goes on, lives. That was actually the idea behind Il posto delle fragole (1957)"
Cinematographer Gunnar Fischer says that several scenes had to be shot indoors due to Victor Sjöström's poor health. "We had to make some very bad back-projection in the car because we never knew if Victor would come back alive the next day." Nevertheless, as long as Victor was home by 5:15 P.M. each day, "and had his whiskey punctually, all went well."