Hi my name is Brandy and my daughter Emmalyn has not been labled FTT yet.. but I'm worried that her doctor will any day now

She's almost 3 months old and is only gaining about 2oz a week. Since birth she has been EBF, and at about 2 weeks I cut dairy out because she started showing signs of possible milk protien allergy. For about a month now she's had wattery poops just about everyday. Her last weigh in (monday) she was 9lbs 6oz. At this point her doctors want to me to try exclusive FF with progestimil. If still no improvment Neocate formula.
I've been down this road before with my last child. I had to give up bf and use Neocate for 12 months. However even then he stayed in the bottom 3% for a long time! So I'm weary to give up on breastfeeding as a way to fix her slow weight gain.
QUESTION for you gals... My doc told me that brain development is the biggest worry for a child not staying on a weight gain curve or staying below the 3%.. however my last child did and is just fine. I'm tempted to just stop taking her to doc appts and just bf her naturally.. but would I be putting her in danger? Does anyone else here have a small baby/child and is just fine.