Help!! I'm hoping someone can offer some advice, or at least tell me if I'm crazy or wrong lol!

Connor just turned 2 in May, and has co-slept/bed shared with DH and I since the day he was born. He's always been EBF'd, and until about 2 months ago, was nursing to sleep and still nursing 1-3 times during the night. We finally night weaned, set a bed time routine, etc, after reading the No Cry Sleep Solution. Thank GOODNESS! lol FINALLY....Sleeping through the night at 2 years old is AWESOME!! ha ha
So the problem is my crazy parents. Crazy as in obsessed with Connor, their first and only grandchild so far. They live about 10 min across town from us. I've always been REALLY close with my parents, they do a lot for us and have helped us a lot, financially and otherwise, in the past. As an example of crazy obsessed, when we travel to FL for a week to visit DH's family, they MUST/DEMAND to see Connor the few days before we leave, and again, as soon as we return. Because "they haven't seen him for SO LONG" (10 days is usual travel and vacation time). They must see him at least one day every single weekend, or I'm getting phone calls and requests for us to bring him to their house during the week....which in turn would mess up our evening schedules and such. It's seriously an unhealthy obsession, even my Mom has admitted that.
I finally put my foot down about bed times when he started STTN. If DH and I don't feel like hanging out over there, then we drop him off for the afternoon/evening and then have some alone time and pick him up before bed time. Now lately, I'm getting constant demands and ridicule for not allowing him to spend the night over there. My major problem with this is the fact that Connor has never slept in his own room or bed (we've tried it at different times in the past, but never worked out). And DH and I both are not comfortable with him bed sharing with anyone but us. Especially my parents, for many, many reasons. But even DH said that he wouldn't feel comfortable with Connor even sleeping in his own parents bed, and his parents are much more responsible and....normal...for lack of a better word...than my parents (for starters, my dad is an alcoholic).
Sooooo......when I went to pick Connor up this evening, we had it out again. My Dad even taught Connor to say "stay here" when I said it was time to go home. Connor LOVES his "Papa" and loves spending time at my parents house. Of course, they are his grandparents and probably spoiled rotten over there at 2 years old already! LOL Which is fine, to a point. When I explain my side of not wanting him to sleep in their bed, my Dad's alternative is to make a pallet on the floor for him to sleep. I've even suggested they make him his own room/bed....though I can't imagine he'd actually sleep in it. He doesn't' at home and he still nurses before we start our bedtime routine.
What age did your LO's start spending the night at their grandparents? Am I overreacting? Do I just give in and see how it goes? Ugh!! Thanks for reading all of this mess, I appreciate any responses, even if it's to tell me I'm crazy and need to get over it!