In the banquet scene, they display a spoonful of eggplant caviar ("a foreign product") next to bowls of red and black caviar. This is a dig at the lack of consumer goods in the Soviet Union, where red and black caviar were almost impossible to find so Soviet citizens made do with an eggplant confection described as "caviar."
Miloslavski's quote "Gentlemen, keep your money in your bank, if you have any" is a pun to the commercial slogan of the soviet bank "Bank of the working, saving and loaning people of the USSR", also know as the "Savingbank of the USSR".
In the scene where Bunsha meets with the Swedish ambassador, the first thing he says is "Hitler kaput!" It's one of the most recognizable phrases in German for many Russians. However, the original script had him say "Peace - friendship!", but the Soviet censors thought it inappropriate. The resulting change ended up being more humorous.
The painting Ivan the Terrible is looking at in Shurik's flat is the painting "Ivan the Terrible killing his son" by Ilya Repin from 1885, describing an event that happened "on Friday, 16 November 1581". Ivan Ivanovich of Russia, Ivan the Terrible's son died actually on 19th November 1581.
Boris Godunov, who is mentioned several times in the movie, is a character from the play of the same name by Alexandr Sergejevich Pushkin. In real life Godunov succeed Ivan the Terrible in 1585 after Fyodor I of Russia ruled Russia since 1584 to 1598.