Yes, 20oz a day of milk is where our LOs are to be as toddlers. You will need to increase his food and offer another source of liquid (water, juice, etc). I'm having a hard time with getting Bella around that number because she is a milk addict

We are around 24oz now but that took time to get there!
We give her mots for tots (which is watered down to begin with) and then add more water to it, so it's more or less flavored water. She hates it but we are working on it. I've even watered down her milk (if it's for snack time or something) because she's so smart...if it's not white, she won't drink it!
You can judge it by giving him milk with his three main meals (which should be about 6oz or so each) and then milk before bed. Any other time he should be drinking other liquids (is what I've been told).
Good luck!!!!