Just before the boys go to the football game, they hear a commotion in the distance. Curly Howard says, "Maybe it's the Fourth of July!" Moe Howard replies, "The Fourth of July in October?" Curly answers, "You never can tell. Look what they did to Thanksgiving." This reference is lost on most people today, but before 1939 Thanksgiving was not a fixed date, it relied on a Presidential Thanksgiving Proclamation each year. President Abraham Lincoln began the national holiday in 1863 and most people were used to Thanksgiving being the last Thursday of November. In 1939 (the year before this short was released), President Franklin D. Roosevelt changed the date of the national holiday, much to the disagreement of many states' governors and their citizens. This change added an extra week of holiday shopping, which pleased business leaders. The move was quite controversial and it wasn't until the end of 1941 that Congress passed a law to settle the dispute and establish the "fourth Thursday" of November as Thanksgiving Day.
Near the beginning when Moe Howard explains to the boys that they're working for the Census, Curly Howard asks, "Will Hays?" This is an "in-joke", as Curly is referring to Will Hays, who headed a film industry board called the Production Code Administration--commonly called the "Hays Office"--which was the film industry's censorship board from 1934 to 1968.
At one point in the short, Moe Howard and Larry Fine attempt to guess the age of the "lady having the bridge party". Larry guesses 43 while Moe guesses 50. Averaged out, this is roughly 46.5, or 46 years and six months old. The actress playing the role, Symona Boniface, was just barely over 46 years and six months old.
The "litter of three" joke was more or less an "in-joke" dealing with the fact that Larry Fine was the only one of The Three Stooges who was not a "Howard" (the other two being Moe Howard and Curly Howard, both of whose real last name was Horowitz). Curly says he was one of the ones "kept" (being a Howard), while Larry says he was the one "they threw away" (not being a Howard).
Columbia production reel #474