Seven months; started some foods: cereal, fruits, veggi, juice. Now he seems to not want his formula??

My son is seven months this month. He has tried some first foods; carrots, apples, banana's, oatmeal, oyster crackers, and apple juice(diluted). I go by what my doctor says, and I read alot. Yet since we started foods he seems to reject his formula more and more. I go for the formula first, and it only has a one hour life. So when that doesnt work, i get fusterated he hasnt eaten,and go for one our liked foods. Then he eats twice as much as "they" say he should. He is happy, and gaining steady weight. However he is only taking in at max 24 oz of formula a day, with the solids I give him. Should I be worried enough to talk to the doctor, or just let him take in what he can and will?