William Wyler spent a whole afternoon shooting the sequence where Fran (Ruth Chatterton) burns a letter from her husband; he wanted the letter to specifically blow gently along the terrace, stop for a moment, and then continue to flutter as the scene faded to black as a metaphor for Fran and Sam's failing marriage.
William Wyler thought the characterization of Mrs. Dodsworth was too black and white and insisted on some subtleties to the performance. Ruth Chatterton vigorously disagreed with this interpretation and the two would often argue fiercely on the subject. At one point Chatterton slapped Wyler across the face and retreated to her dressing room. In her memoirs, Mary Astor observed that Chatterton's character "was that of a woman trying to hang onto her youth--which was exactly what Ruth herself was doing. It touched a nerve."
At the time of filming, Mary Astor was going through a very public and very scandalous child-custody case against her ex-husband, who used Astor's diaries to prove that she had been having an affair with playwright George S. Kaufman. (Her ex, Dr. Franklyn Thorpe also had affairs, as detailed in Scandal: The Trial of Mary Astor (2018).) With the press constantly stalking her, she sometimes slept on the set to avoid confrontation. Many people involved in the production sided with Astor throughout the ordeal, including William Wyler, Samuel Goldwyn, and Ruth Chatterton, who appeared as a character witness on Astor's behalf. Coincidentally, Astor's character in this film is a divorcée.
David Niven later said he was "bloody miserable" working with William Wyler, whom he described as a "Jekyll and Hyde" and "a sonofabitch to work with". Although conceding Wyler could be "kind, fun, and cozy" off the set, Niven said, "He became a fiend the moment his bottom touched down in his director's chair." Wyler was not terribly impressed with Niven's talent either, later noting that he was little more than "a sort of playboy around town". However, the director thought that since Niven was essentially playing himself on screen, he was perfect for the part of the charming cad Captain Lockert.
Mary Astor wrote in her memoirs that Edith Cortright was her favorite role, also reflecting that she channeled her struggle of her public child-custody hearing (She had been divorced the previous year.) into her role: "When I went into court and faced the bedlam . . . that would have broken me up completely, I kept the little pot boiling that was Edith Cortright."