That’s because some lip balm ingredients are just making the problem worse. “Lip balms containing ingredients like phenol, menthol and salicylic acid actually make your lips drier.