The film was shot in reverse order. Al Pacino began with long hair and a beard, then for each scene, his hair and beard were trimmed bit by bit until he became clean-cut.
After he decided to make the film, Al Pacino invited Frank Serpico to stay with him at a house that Pacino had rented in Montauk, New York. When Pacino asked Serpico, "Why did you do it?" Serpico replied, "Well, Al, I don't know. I guess I would have to say it would be because if I didn't, who would I be when I listened to a piece of music?"
Al Pacino considers this movie to be one of his greatest achievements as an actor. In his autobiography, Pacino explains he was given the official biography of Serpico for research but found the book dull saying it read like, "He does this, then he does that. Then he meets his mother, then he meets his mother, and so on." However, when Pacino met the real Frank Serpico face to face, immediately he knew that he could play this role.
According to Sidney Lumet, Al Pacino always needed to be in the character's state of mind in any given scene and could not shed that state off camera, so he behaved accordingly at all times, either happy, joking and laughing for a lighthearted scene or angry and lashing out at everyone if the scene they were working on called for that behavior.
After spending a lot of time with Al Pacino as he prepared for the role, the real Frank Serpico wanted to remain on the set during filming. Producer Martin Bregman said he had to hurt Serpico's feelings and ordered him to leave because he believed that his presence would prove a distraction.