This phrase emphasizes two things about the narrator. The first is that the narrator finds people, in general, to be rude or annoying. The second is that the narrator is being rude himself (in context, it emphasizes his impatience to have someone answer the door).
