I have had the last ones close together, so I've said when they had a baby brother or sister I thought they were not a baby, though we do refer to Amber and LJ (who's 2 1/2) as "the babies" and the other two as "the big kids" or the "big 'uns."
They're a baby as long as they don't tell you otherwise