People are (in general) less sensitive about secondary infertility than primary infertility?
I think they are. Granted, not a whole lot of people know at this point what we are dealing with, but from those who do know I get a lot of, "Well, you should just be thankful to have one child..." And I am. More and more every day. But DH and I have always dreamed of having a big family, and almost all of our friends/family have or are planning to have at least 4 kids. But somehow we are supposed to be the ones that have to be okay with just one?!

I mean is it really selfish to want to give our son a sibling? I think it would be more selfish to be like, "Ugh. This parenting thing is a lot of work. Let's never do this again."
And there's DH's parents. They just do not "get" how we were able to get pregnant before and now we can't. Well, we don't really get it either, but either we conceived our son miraculously quickly (and even that took 6 months) with a super low sperm count or DH's count has dropped in the last 2 years b/c of his varicocele. Either way, I'm not pregnant and don't expect to be anytime soon. I don't see why this is so impossible to grasp.
And then in general, I think people are more sensitive to a childless couple. I mean if you're married several years before you have kids, eventually it might start to dawn on people that you are having trouble getting pregnant and they'll quit harrassing you about when you plan to have kids. But if you have one child and they just keep getting older and you never announce your next pregnancy, for some reason people assume it's your choice. After all, you have one kid, surely you can have another when ever the urge strikes, right?
*SIGH*