Rosemary was murdered on her 23rd birthday, December 13, 1935. The song "When Did You Leave Heaven?" is from the August 1936 movie Chante, bébé, chante! (1936). (The song later received an Academy Award nomination for Best Music, Original Song.) Jake could have at least heard the song while he was in prison.
The mansion used in this episode is the Marian Davies-William Randolph Hearst mansion in Beverly Hills. This mansion was also featured in The Godfather and Fletch, among other movies and TV shows.