How can I get my toddler to sleep through the night?

How can I get my 18-month-old to sleep through the night? My son has always had problems sleeping through the night, but lately, it’s leaving us exhausted and struggling to get through the day. He used to wake up once in the middle of the night crying his heart out. We would try back rubs, giving him his blanket, putting on his bedtime music, letting him cry it out, using a heating pad, half a dose of children’s Tylenol, and giving him a warmer bath for pain or gas. If all else fails, we give him 0.25 mg melatonin (the pediatrician is aware and approves). We asked for help from his pediatrician. She suggested not to pick him up, give him a bottle (he refuses bottle and pacifier anyway), or give a bath as they could cause him to wake up every night wanting the extra attention and affection. Since then, he wakes up twice a night, most nights crying uncontrollably for up to an hour each time. He weaned himself off the pacifier and nighttime bottles. The more we engage him now, the angrier he gets and longer he stays awake. Are there any successful methods you have tried and recommend?