How can I get my baby to sleep through the night again?

Hello fellow moms, I need some advice on feeding a 5 month old:I’ve just started feeding my 5 month old food, such as: bananas, strawberries, sweet potatoes, oatmeal and rice. He eats about 2 oz of actual food a day, and the rest of the time I feed him formula. The problem is, even though he has just started eating food, the milk does not seem to hold him over. He wakes up during the night starving, even if I fed him a rice and milk bottle before bedtime. This baby has slept through the night since he was 2 months old without issues. Any ideas or food recommendations for a baby newly trying food? I don’t want to overwhelm his digestive system, but it really seems like he needs something more sustainable than formula.