When I was pg with dd I would drink coffee once in a while after all my morning sickness was gone, but by that point, I didn't want it anymore.
I don't drink coffee anymore but I do drink about 8-10oz of green tea every morning and I don't plan on stopping that unless I don't like the taste or something! I have read up on it and you have to drink a seriously excessive amount of green tea for it to be harmful. And I'm pretty sure with coffee it's a limit of 2 cups a day? I could be wrong but thats what I'm remembering in my head.
Considering when I got pg (accidentally) with DD I was not only drinking lots of coffee, but I was also having a beer with dinner a few nights a week, I don't think it affects chances of conception all that much
In fact, one of the reasons I thought I might be pregnant was because all of a sudden one evening, I turned my nose up at my beer and steak dinner, which was VERY abnormal for me!
Oh and also to add, in my first trimester I had to keep a 2 Liter of sugary soda in the fridge bc I would get faint quite a bit!