Originally, producer Derek Granger asked Anthony Andrews to play the role of Charles Ryder. Andrews, however, felt he was better suited for the part of Sebastian Flyte. Jeremy Irons, Granger's first choice for Sebastian, preferred to play Ryder, so the two actors swapped roles.
Laurence Olivier was offered his choice of roles in either Lord Marchmain or Edward Ryder (which ultimately went to John Gielgud). Olivier picked Lord Marchmain, but later regretted the choice as he realized that Edward Ryder was actually a much stronger role.
Sebastian's teddy bear, Aloysius, was based on a real one called "Archie", which belonged to writer John Betjeman, one of author Evelyn Waugh's friends from his days at Oxford University.
When filming the scenes during the storm on-board the ocean liner, the small cabin sets were made to rock from side to side, but this could not be done for the much larger dining room set, so producer Derek Granger stood on a chair behind the camera and waved a stick from side to side to indicate to the cast which way to lurch and sway.