Not a dumb question at all. I dont have the info to back up what im gonna tell you, i could try to google it, but.... it can.
If he is older, or has a lower sperm count to begin with, then it really can effect it. The best idea is to bd like every other day around O, and every 2 days or so when not around O if you still want some booty lol. Sperm live for 3-5 days so..... even if its every other day around ovulation, youll be ok