Michael Dorman is a musician as well as actor, and sings all his songs on the show himself.
The "Luxembourg" scenes were actually filmed in the beautiful Czech city of Prague in the area surrounding the Charles Bridge. Constructed between 1357 and 1402 and named after King Charles IV, it is well known for its stunning Gothic Tower and the 30 mainly Baroque statues that span the length of the bridge.
In S1 E6, when Peter Icabod is being interviewed by the police in Luxembourg, he mentions that his full name is "Peter Cassius Icabod". The middle name Cassius means "hollow", so it was chosen as a sort of in-joke reference to the fictional book/folktale 'The Legend of Sleepy Hollow', since in the show Peter often gets made fun of for looking like Icabod Crane.
The CD album is titled 'Circles of Resonance' by Tyhmens & Saperstein. Tracklisting includes the songs Afternoon Spray, Diamond Dust, Supercell, I Don't Wanna Be Found, Fortunate and Can't Wait For May.
Tyhmen's & Saperstein model themselves after folk singers Simon and Garfunkel. Their CD cover is a copy of the photo on Simon and Garfunkel's Sounds of Silence album. The title songs on both albums refer to sounds, and the duos' names even have the same cadence.