Actually, my DH had a low count. I don't know what it was within the past year or so, but a while ago, he had a count...he smoked heavily, drank tooooons of Diet Coke (seriously, something like 8 2-litres per day), and soaked in the hot springs at least 4 times per week--in the very hot pool. He stopped a lot of that stuff, but ended up working in construction and having a much warmer groin than he should have. We know it couldn't have been that high. Our main problem, though, was me. I'm sure that his count must have gone up significantly.
I think that your husband's count sounds pretty good, overall. It's not that he's infertile--not by a long stretch. He has a count over 20 M, which is good, it's just that on the scale of "most fertile", he's at the low end. I would try very hard not to worry too much about it. As for your progesterone on CD 21, had you ovulated? For sure, I mean? If so, then 5 is low.

When is your doc thinking about starting any sort of treatment??? (Clomid seems to be working wonders for so many women on this board! I hope you get yours soon!!!!)
Good luck, Danielle, and PLEASE keep us posted. I know that this is such a difficult time...I wish I could hug you.